Data Ingestion for Financial Services and Fintech

Three key learnings from this article:

• As Fintech and the financial services industries grow, look to Osmos to streamline and accelerate data ingestion, data transformation, and automation.
• Financial services firms leverage Osmos to automate processes, empowering implementation teams to own the data ingestion.
• Leading fintech firms trust Osmos to help aggregate financial data across markets, accelerating processes and boosting profits.

As our personal finances become more IoT and less IRL, an abundance of digital financial data flows from the apps and services we use to the systems that keep our most sensitive information safe and secure. 

This invisible dance begins with data extraction, data ingestion, and data transformation. Customers, vendors, and partners must seamlessly and securely pass your financial data to the companies that conduct financial transactions on your behalf.  

With new apps and services announced daily, there’s no indication that these systems will get any less complex. When you think about the amount of interconnection in our digital financial lives, the orchestration is remarkable.

Imagine you are an employee at an established company offering an excellent benefits package. Your first week as an employee is filled with exciting meetings and a thorough onboarding process. On day three, you meet with your personal benefits advisor, who hands you a stack of paperwork and begins the employee onboarding process.

You’ll be opening a 401K and an investment account to handle your stock options and employee stock purchase. The company offers extensive life and health insurance. They also offer membership to a credit union, which has its own very convenient iOS app. Their healthcare plan has a generous, flexible spending account that they’ll fund annually. As a bonus, a “wellness” debit card covers your gym membership and complimentary spa days.

The whole benefits smorgasbord is funded through their employee payroll, which conveniently runs on direct deposits of your paycheck. 

This is just one very common financial data situation that requires an immense amount of secure financial data transfer across disparate systems from various vendors and partners, each with its own ideal data schema. 

In order to provide you, the customer, with timely information about the state of your retirement savings or the balance of your bank account, data must be moved digitally from your company’s payroll system to each individual vendor. Ideally, data transfer can be automated, but in most instances, the schema of one organization, say the payroll service, will not match the schema of another. Schema challenges are just the beginning of the first-mile problem of data ingestion.

Systems integration and data migration are among the most common forms of data transfer for financial services organizations, but as we illustrated earlier, financial data is constantly on the move. 

Below, we’ve outlined a few key data situations from fintech and financial services organizations that have successfully leveraged Osmos solutions to streamline and accelerate data ingestion. 

‍

Data ingestion relief for tax professionals

Service sector: Tax Preparation and Accounting ‍

Data ingestion challenge: Ingesting messy data from disparate sources

Customer need: Process automation and data cleanup

It’s no secret that accountants spend too much time manually scrubbing messy data, especially around tax season. Next to data scientists, accountants are among the most elite spreadsheet power users, but even the most seasoned accountant has their limits. A single customer’s transaction data can run into 100s of millions of records and contain information from several financial institutions, which makes manual cleaning and reconciliation a nightmare. 

Businesses supporting tax preparation and accounting must regularly ingest customer transaction data from a variety of sources. A tax business might receive information directly from another financial institution, or a consumer might extract the data themselves. The data that is received is almost always “messy.” It arrives in varying formats, schema, and quality. 

Financial accounting data must be validated and transformed into a uniform data model before it can be reconciled for tax calculations. 

This is when dev teams get involved. They’re tasked with building custom scripts to speed up the ingestion process. Once the data ingestion becomes a dev-driven process, the business loses data autonomy. The development team is now tied to those custom scripts. They’ll be responsible for their perpetual maintenance. This is an untenable solution for any financial institution.

‍Solution: Financial services firms trust Osmos to automate the ingestion of tax data into standardized data models, making the process more reliable and efficient. 

Osmos helped empower tax implementation and onboarding teams to own the data ingestion process without dependency on their dev and engineering counterparts.

‍

Forecasting market changes in a dynamic landscape

Service Sector: Currency Trading

Data ingestion challenge: Aggregating masses of data from non-uniform sources

Customer need: Speed and accuracy

Insight into changing markets can signal to a business where money can be made and what’s at risk of being lost. 

Many companies rely on the collection of trade, transaction, and pricing data from various markets. The changing of currencies and stocks also determines the exchange rate between currencies. Data providers and vendors then aggregate and package Forex data for sale.

The data is collected through sheets and is updated regularly. Global teams create price changes based on 

    (a) local market changes

    (b) where competitors take action 

Changes are submitted via sheets to Global Pricing Teams for review. Implementation Teams review the changes, validate the data, and submit the sheet for ingestion. This process is typically very manual. Spreadsheet power users work largely in Excel using Macros to get the job done.

This limited machine-enforced validation and error handling often leads to costly human error. An error like the one below significantly impacts large principal sums if updated before being caught. This single oversight will result in a sizeable downstream ripple effect on profits and may take weeks to correct and resolve.

Solution: Fintech firms rely on Osmos to quickly and accurately aggregate sensitive financial data across regions, boosting profits on FX spreads.

Firms can now rely on their front-line teams to execute complex lookups and joins thanks to tools like Omos’s Datasets. For external data sources, they look to Osmos Uploader and Pipelines to automate the data ingestion process, helping teams quickly aggregate clean pricing data from non-standard sources in hours, not weeks.