Whenever you cook, do you use one ingredient? You could do, but the results are going to be pretty bland. A burger, without bread, or a patty, is a bit of a non-starter. It’s all about finding the right mix of ingredients and then cooking up a storm (if you’ll excuse the pun). Analysing financial markets is no different (well, except for the question of actually cooking)! Except in financial analysis, rather than storing all your ingredients in one place, they’re scattered around random locations which makes the cooking (or analysis!) even more difficult.
Whenever we do analysis, we will frequently use more than one dataset. The difficulty is that often it can be a a time consuming process to join together datasets from different sources, because it’s not always the case that every single dataset we want can be found from a specific vendor. Different vendors will often have different tickers for one for the same entities. For example for EURUSD spot, you’ll have many different variations depending on which vendor you use. You’ll probably want to create a harmonized ticker for a specific entity that can map to multiple vendors. On top of that, each vendor will have a different API. There are cloud services which help to harmonize access from many different datasets.
However, what if you want to do it yourself and maybe would prefer not to use a data harmonization cloud service on the cloud? You’ll have to write some sort of ticker mapper, between your harmonized tickers and vendor tickers. You might also wish to create ticker grouping to easily download larger datasets (eg. all the implied vol points for a currency pair, all the stocks in an index etc.) You’ll also have to create your own API wrapper, so you can get data from many different data sources using a common wrapper. Having a common API wrapper means you don’t need to massively modify your higher level analysis code when you switch between data sources (or combine together several). Many funds are likely to have written this type of library internally to access financial data.
I started writing findatapy a couple of years ago to do the above, and because I couldn’t find something like it in open source (there might well be now?), which could handle lots of data sources which I wanted to use. It’s an open source Python library (free download from GitHub) enabling you to download data from many different data vendors using a simple common API. At the moment you can use it to download data (provided you have subscriptions) from many sources including Bloomberg, Eikon, Quandl, ALFRED/FRED, AlphaVantage, Yahoo Finance, Dukascopy, many crypto exchanges etc. You can also set a data source as a Parquet file, which is stored locally or on S3.
I use findatapy extensively for my own research work and its also used inside finmarketpy, my trading strategy backtesting library. You can create your own ticker mappings in findatapy and it already has a few for FX. I’ve written a Jupyter notebook here, to illustrate how to use it to download market data. Here I’ll give a quick preview to show how to download market data through it. Let’s say we want to download VIX Index from Bloomberg, it’s two lines of code, and map it a nickname “VIX”.
from findatapy.market import Market
df = Market().fetch_market(“raw.data_source.bloomberg.tickers.VIX.vendor_tickers.VIX Index”, start_date=’week’)
Or if we want to switch to using Quandl for my market data, in this case EURUSD sourced from ECB, we use a very similar call. Our nickname for the asset is EURUSD and the Quandl ticker is ECB/EURUSD.
df = Market().fetch_market(“raw.data_source.quandl.tickers.EURUSD.vendor_tickers.ECB/EURUSD”, start_date=’week’)
We can also create more elaborate queries using a MarketDataRequest too, which has many parameters like the field to be downloaded and combine it with the above. In this case, we are downloading all the predefined tickers for G10 FX from Quandl, which we’ve already mapped.
df = market.fetch_market(md_request_str=”fx.quandl.daily.NYC”, start_date=’year’, md_request=MarketDataRequest(quandl_api_key=”Type your key here”))
What if we want to download all the implied vol points for EURUSD, that’s going to be real hassle right, and I have to remember many different tickers (ATM etc.)? No, it’s one line of Python code with findatapy!
df = market.fetch_market(md_request_str=”fx-implied-vol.bloomberg.daily.NYC.EURUSD”, start_date=’year’)
I’m sure there are lots of other much smarter ways of harmonizing access to market data, what I’ve done is just a start. If you have any ideas on how I can improve findatapy let me know, or if you’re a data vendor who would like to sponsor the package, so I can add your own data API into it let me know. One area which I want to improve, is to provide a GUI to browse all the predefined tickers you have, and also to make it easier to manage the ticker mappings.
Ultimately, I wouldn’t class any of this functionality as “proprietary” (and that’s one reason I open sourced it) but it’s pretty useful to have such functionality if you’re analysing markets, because it makes it easier to blend together data from many different vendors and APIs. A big part of any analysis, is just loading the market data, and if that is simpler, than it means that more time can be spent on the fun bit, analysing the data itself!