Financial Services firms have long embraced cutting-edge technology to drive competitive advantage and associated profits. Increasingly, MongoDB plays a key role as essential data infrastructure for major Financial Services firms for reference data management, risk analysis and reporting, trade repositories, portfolio reporting, and more.
Among this array of applications, banks commonly turn to MongoDB to build tick databases. Tick databases are real-time database servers for capturing, managing, and processing market data.
In a recent webinar, Matt Kalan, senior solution architect for 10gen, highlighted the reasons banks have turned to MongoDB to build tick databases, among other applications. Historically banks turn to pure tick databases, though often they have a steep learning curve and have a hefty price tag. Hence, firms are looking at easier alternatives in the NoSQL space and MongoDB in particular.
MongoDB Delivers on Tick Database Requirements
To successfully capture and analyze tick data, a database must be able to do the following:
- Capture real-time market data (multi-asset, top of book, depth of book, even news)
- Load historical data
- Aggregate data into bars, daily, monthly intervals
- Enable queries & analysis on raw ticks or aggregates
- Drive backtesting or automated signals
MongoDB delivers on and exceeds these requirements, offering a range of benefits:
- High throughput - MongoDB can capture real-time feeds for all products and/or asset classes needed
- High scalability - MongoDB can capture all data and depth for all historical time periods
- Flexible & Range-based indexing - Offers fast querying on time ranges and any fields
- Aggregation Framework - MongoDB can shape raw data into aggregates (e.g. ticks to bars)
- MapReduce capability - Using MongoDB’s native MapReduce or Hadoop Connector, MongoDB affords batch analysis as users search for patterns and opportunities in their data
- Easy to use - MongoDB comes with native language drivers, letting users write in the language of their choice, and JSON expressions that users can also apply for most operational database needs
- Dynamic schema - In a relational world, a user must predefine their schema. MongoDB provides far greater flexibility, allowing the user to change their schema as often as required. Additionally, MongoDB allows user to store data any way an application uses it. There’s no need to waste time data modeling or mapping data. Users can even index nested fields
- Low TCO - Low software license cost and commodity hardware mean users get improved performance at a significantly lower price
At a high level, this is what a MongoDB trading architecture looks like:
Instead of complex joins, developers write documents, with data modeling that is comparatively simple.
If a user prefers to process data on Hadoop, rather than employ MongoDB’s built-in MapReduce or Aggregation Framework, it’s also relatively simple to use Hadoop through MongoDB’s Hadoop Connector. The MongoDB Hadoop Connector supports MapReduce, streaming, and pig, and lets enterprises use MongoDB as the input and/or output storage for Hadoop jobs, with no need to run the data through HDFS.
But Can MongoDB Scale?
Of course it can. 10gen has Financial Services customers that store petabytes of data in MongoDB. After all, MongoDB is architected for serious scale with attributes like auto-sharding to enable seamless horizontal scalability, plus in-memory caching and better data locality to boost efficiency and, hence, performance.
Indeed, one well-known investment bank is storing 30,000 ticks per second and that is because the network is the bottleneck, not MongoDB. Another bank is handling 50,000 ticks/second on average with peaks higher. MongoDB could handle 100,000, 200,000, or any number of ticks per second just by auto-sharding and adding commodity hardware.
MongoDB: Tick Databasesâ€Â_and Beyond
The biggest banks, insurance firms, and other Financial Services companies increasingly look to MongoDB to build high-performance applications that are easy to develop and scale. Tick databases are one of the many use cases for which MongoDB is a great fit.
Fortunately, you needn’t take our word for it. MongoDB is 100% open source. Download it and try it out.
We’d love to hear how it goes, and whether we can be of help.