So where’s the best place to get a burger? I get asked that a lot. I’ll try to give my best answer, but if you live in a place I haven’t visited, I’ll probably draw a blank. Yes, you can read reviews, but the only real way to tell if a burger joint is good, is to try it. Everyone has their own different taste, some prefer greasier burgers, others prefer lots of cheese on a burger etc.
When it comes to data, the equivalent “burger” question is about free data. Where can I get free data? In practice, if you want good data, you’ll have to pay, particularly if you want to have a very comprehensive dataset that covers many areas. Firms like Refinitiv and Bloomberg have huge data histories and many different products that cover all sorts of markets for your subscription. For example, if you’re looking for extensive tick data histories, Refinitiv have Tick History. If you’re looking for machine readable newswire data, there’s Bloomberg’s event driven feeds.
Many datasets are simply not available for free sources, such as FX volatility data. Furthermore, we need to be aware that even if we can get raw data free from some some free sources, we might need to spend extra time cleaning and normalizing it. In some cases, the API download can also be quite slow. Does this mean we should only just paid for data? I would suggest that whilst free data is unlikely to cover all your data needs, it is possible to find some decent free datasets to compliment your paid sources.
Below, I’ve listed some free data sources that I’ve found useful (and also a few paid sources too). In some cases, I’ve also put links to Python code from my open source findatapy library for some of these sources which makes it easy to download them, using a single API, so you don’t have to learn every single vendor API. I’m still working on adding new sources as we speak, so unfortunately not everything on the list below is available via findatapy! My findatapy library also supports access to many paid sources including Bloomberg’s DAPI and Refinitiv’s Eikon API.
Market data sources
Here we have data sources for market data from equities to FX. Note that for many markets, you might not be able to find free data (eg. FX implied vol). You can also download a lot of free crypto data using findatapy.
- Quandl (free/paid) – https://www.quandl.com
- Many datasets on Quandl are free, from underlying sources like IMF, FRED etc. However, there are also many paid datasets covering areas like futures and alt data (like CLS’s FX flow dataset). You need to get a free API key to access it.
- Download FX daily spot data using findatapy
- Yahoo Finance (free) – https://finance.yahoo.com
- Focused on equities based data, including historical price history for cash and also options
- Also has lots of company data, such as earnings
- Download equity price data using findatapy
- Dukascopy (free) – https://www.dukascopy.com
- Dukascopy is a Swiss based retail FX broker. They distribute free tick data gathered from the quotes they provide to their clients. There’s also a historical data downloader GUI on their website.
- Download FX tick data using findatapy
- FXCM (free) – https://www.fxcm.com
- FXCM is a retail FX broker, who also give you access to their FX tick data.
- Download FX tick data using FXCM
- Bloomberg (paid) – https://www.bloomberg.com
- Comprehensive data across markets both via API and Terminal
- Download FX vol surface data using findatapy
- Download economic events data using findatapy
- Refinitiv Eikon (paid)
- Covers many markets, accessible by API and desktop application
- Download FX daily spot and tick data using findatapy
Economic data sources
Listed below are a few data sources for economic data.
- FRED/ALFRED – https://fred.stlouisfed.org/
- The Federal Reserve of St. Louis maintains this very large database of economic data (and also a smaller amount of market data). ALFRED also has very comprehensive point-in-time data for economic time series. The datasets can be incredibly granular for US economic data.
- Download economic data using findatapy
- db.nomics – https://db.nomics.world
- Collects data from many different economic sources like BLS, ONS, IMF etc. into a single API
- IMF – https://www.imf.org/en/Data
- The IMF maintain many different datasets, covering many different time series ranging from more common economic data to more unusual (eg. COFER currency reserve data)
- UNComtrade – https://comtrade.un.org/
- Data on global trade and composition of import/exports
Alternative Data
Whilst many forms of alternative data are indeed paid datasets, it is possible to find some interesting free alternative datasets. More broadly if you’re interested in alternative data, have a look at The Book of Alternative Data, which I coauthored.
- Hedonometer Index – https://hedonometer.org/timeseries/en_all/
- The Hedonometer Index is constructed using Twitter data and measures the relative happiness of sentiment
- Google Mobility Data – https://www.google.com/covid19/mobility/
- Measures how much people are travelling by tracking Android phone usage
- Apple Mobility Trends – https://covid19.apple.com/mobility
- Using iPhone location data, Apple create metrics for traveling activity
- AlternativeData.org – https://alternativedata.org/
- This isn’t a specific dataset, but it’s a directory of alternative datasets which is freely accessible. Note that many of the datasets listed are not free.
Conclusion
Whilst in practice, you are likely to need to pay for most of your financial data, it is possible to find some free datasets which might be useful too for certain areas. The above is certainly not a comprehensive list of free datasets for markets (and mostly focused on the datasets that I’ve personally used in the past), however, I hope it provides a good start. If there are any good free datasets that you’ve used and I haven’t written it above, please do let me know.