Need historical cryptocurrency exchange data for bot testing, analytics, or just to play around with? Here's how to get it.

in #cryptocurrency7 years ago (edited)

chart_big_pixabay_steemit.jpg
Frictionless, transparent 24/7 markets are part of what makes the emerging crypto economy great. At any time of the day or night, it is very easy to find current information on any of the digital currencies being bought or sold anywhere in the world. But finding historical data on crypto exchange activity is far more challenging. Especially if you are after high resolution data on altcoin trades going back more than a couple of months. This data is out there, however, and this post describes how to obtain it in whatever form you are looking for at a reasonable price.


Step 1: Identify what specific data you need.

If you are just looking for daily average prices of BTC or ETH, these are easy enough to find from a variety sources for free. And some exchange and other website APIs are pretty good about allowing you to request data from the past for stuff in time-frames ranging from hours to a day. But for anything more detailed or obscure, fewer choices exist.


Step 2: Choose a data vendor based on your specific need, budget, and technical know-how. There are really only 3 options I could find.

Option one: Coinigy


Coinigy calls itself an "all-in-one platform for digital currency." They've got a good trading platform linked to a bunch of exchanges, and several traders I know love their service. While their main focus seems to be as a software provider, Coinigy also sells historical exchange data. For crypto pairs (like BTC/STEEM), Coinigy charges $15.00 for tick data, $9.00 for 1min OHLCV data, $4.50 for 1hr OHLCV data, and $2.40 for 1day OHLCV data ... per coin pair for each month of data requested.

If you are only interested in one or two coins, these prices aren't bad. Or if you have limited technical know-how and money to burn, Coinigy's data is formatted to make it easy to drop into an Excel spreadsheet and start generating simple, readable charts. But if you are interested in doing more comprehensive or market-wide analysis, and aren't working with the budget of an institutional investor, these prices quickly become prohibitive.

The bottom line: A year's worth of high resolution data from an exchange like Bittrex or Poloniex on all traded pairs will cost you more than $10,000 if purchased from Coinigy.


Option two: Kaiko

Kaiko sells "trade and order book data covering most crypto exchanges, all currency pairs". Digital assets data appears to be Kaiko's exclusive focus. After a ton of research, asking around, and even an appeal to the Steemit community, Kaiko is where my own recent search for comprehensive exchange data eventually led. And after buying their Bittrex and Poloniex historical trade data (each currently priced at $549.00), Kaiko's datasets seem to provide good value for the money. Their customer service is great and their weekly updates (for the subscription plan I added on to my purchase) arrive on time.

Some technical knowledge is definitely required if you plan to work with one of Kaiko's data products. Together, the two datasets I have so far purchased amount to dozens of GB (once uncompressed) spread across tens of thousands of files in thousands of folders. This is great for stuff like the venture I'm currently involved with, but it is probably overkill for the average crypto-hobbyist.

The bottom line: for raw marketwide data, Kaiko charges a fraction of what Coinigy charges. If your business needs a ton of historical trade or order book data, I highly recommend Kaiko.


Option three: find someone willing to sell you a customized dataset.

From what I can tell, this is pretty much the only option if you are looking for something more affordable than what Coinigy offers but less comprehensive than what Kaiko sells. After trying and failing for months to find such a data source to recommend to other traders/investors that I know, I finally gave up and decided to start offering some specialized crypto data products for sale myself.

Initially, these are simply useful byproducts of summarizing Kaiko's data in specific ways for an as-yet-unannounced venture and my own trading activities. Right now, I can offer OHLCV data in .csv format going back a year or more on most coin pairs traded on Bittrex or Poloniex. (Some of the Poloniex data goes back a few years.)

As a gift for my Steemit followers, here's a link to a sync folder containing a year's worth of hourly Bittrex OHLCV data on STEEM, SBD, and USDT in .csv format. This link will expire in a few days, and the folder is password protected. The password is "freeOHLCVdata" (without the quotes). Note that times are UTC, missing price data was backfilled for periods of inactivity during resampling, and volume is a sum of all buy and sell volumes for the resampling period.

The bottom line: for raw cryptocurrency exchange data, Kaiko is the best option by far. For OHLCV data on one or two coins, Coinigy is a convenient option. For a specialized slice of OHLCV data/something more esoteric, you can either try bitcointalk or send me a direct chat msg/ encrypted wallet memo indicating your interest and email contact.


Aside from being a happy customer, I am unaffiliated with Kaiko. But I did notify them that I'd be posting this review. They've said they'll happily answer any questions you have for them in this post's comments.

Sort:  

Low resolution data. Good for some uses, insufficient for others.

This comment has received a 20.83 % upvote from @steemdiffuser thanks to: @stimialiti. Steem on my friend!

Above average bids may get additional upvotes from our trail members!

Get Upvotes, Join Our Trail, or Delegate Some SP

@mada, just randomly came across you via @lovejoy. Steemit world is getting richer and richer. Happy and not surprised to see you here! Thanks for the tips ^^ :)

Thanks historic data can be very important to make complete charts for analytics - excellent and useful research so thanks⭐

Sure thing - good data is essential for decision support: )

Thanks for doing the research and reporting on your findings! I hope more people with a knack for making sense of this data will avail themselves of resources you've shared here. :)

If you have any question, feel free to reach out at [email protected] :-)
Have a great end of week!
Ambre from Kaiko

It seems to me https://bitdataset.com is cheaper (vs Kaiko)? Did you test bitdataset? I need Level-2 (full orderbook) and I don't know what to choose bitdataset or kaiko.

I did not test bitdataset, but may in the future. Thanks!

Coin Marketplace

STEEM 0.19
TRX 0.15
JST 0.029
BTC 62836.52
ETH 2558.21
USDT 1.00
SBD 2.72