Crypto Kitty Data Mining - What does it take?

I've been asked a few times what it would take to setup an application like mine. The short answer. A LOT.

Here's what I have done in the last 2 weeks.

Server Infrastructure

• DB
• Application
• Cache
• Message Broker
• Cron

Application

• Crypto Kitty API Parsing
• Crypto Kitty Core Contract Parsing
• Integrating Parsers with Message Broker (For durability)
• Integrating Parsers with Cron (Set to sync every 5 minutes)
• Build the Front-End
• Build the DB

Why Do it?

I build large scale applications professionally, I know what it takes to make something like this work. The data is needed to build other Pawesome tools and that's what I really want to do!

How much time have I spent

Building the DNA data-set and interacting with the contract has taken me easily over 20 hours of actual work. The Server infrastructure to read over 500,000 kitty DNAs was extensive. At one point I had over 10 active "Consumer Applications" pulling from different ETH nodes to read the kitty DNA and even then I was only getting around 50-60 DNA responses /per second. The sheer volume of requests required a lot up front infrastructure costs as running servers with this much data was resulting in data corruption when the servers died from maxing out.

What did it make?

It made this! http://dna.cryptokittydata.info/ This is a Kitty DNA Profile tool. You can download your kitty DNA sequences (Dominant and Recessive genes). This is only the first phase of what I plan on building. There are more things coming soon.

What's next

Currently I need to take a few days break, I might need to sure up the site for the China Launch.

I think the next big thing is going to be a Gene Search tool.

How to support my development

What really helps is ETH so I can test new theories about how to make our kitties better!
Please send Kitties as well! 0xcecddbe88359f6ecebe90b42643b002543f27fe9