Trading Assistant Bot (Bitfinex): API websocket connection, channel subscriptions and candle data parsing
Repository
https://github.com/imwatsi/trader-bfx-backend
This is the first development post for my new Trading Assistant Bot (Bitfinex) project, as I announced in a previous introductory blog post. I added the basic features upon which the app will be built: an active websocket connection and a framework for handling data from subscribed channels via Bitfinex's API.
![trader-bfx-backend.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmS4tavhYZXUAZzck6GKo5PUaTbV7fP9SgxmmA4LAhfnzM/trader-bfx-backend.png)
Commits at a glance
- basic websocket api connectivity
- bot init script
- candle subscription and basic channel handling
- parse candle data
- rename modules
- better snapshot identification
1) Basic websocket connection
Commit: basic websocket api connectivity
With this commit, I added basic functionality for the websocket connection.
Websocket client
Python has a number of websocket modules you can use. Two notable ones include:
This project uses websocket-client.
pip install websocket-client
I created a python script containing functions to connect to Bitfinex's API, authenticate and handle the interactions: t_api_socket.py
.
Using the template found on the websocket-client website, I wrapped the initialization code in a function start_api()
, which will be called from a separate init.py
script. (reasons for this explained in the Threading section below)
![start_api.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmdFqNC1v2TwmNrSnsgLhy2bTrTm8PThetA3MetZDWkHiF/start_api.png)
Bitfinex's API comes in two versions. The first is the stable version that's been around the longest and the second is an improved version, that's still in beta (at the time of this writing). I use the second version in this project, as it gives a much better experience and more features than the first.
NOTE: slight changes might occur with the API since it is in beta, as stated in their documentation.
Authentication
I nested a a function called authenticate()
within the on_open()
function of the API connection (which is called when the connection is opened).
![on_open.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmaYCJLy2EjFCujM1Af1vXyqcTGU65BCGFjNLt3dkxSUbN/on_open.png)
The authenticate()
function is called upon opening a new API connection and it authenticates the session by sending a signed message, using the API_KEY
and API_SECRET
. The 'dms': 4
parameter in the payload is a deadman's switch, which cancels all orders if the connection is dropped. It's optional so it can be removed if it's not needed.
Threading
I will use Python's threading module to run separate threads for various functions of the whole app. The first instance is for the API connection, which is initiated by calling start_api()
from init.py
.
![init.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmTfmKYRNNjtMRJn9M89YJLU7GF8WUHjmHufd8jij6dcRN/init.png)
2) Basic channel subscriptions
Commit: candle subscription and basic channel handling
The Bitfinex API offers numerous data channels that you can subscribe to, from market data to account related data. I started with the basic market data.
Candles
Candles are probably the most used data sets in trading, with most technical analysis tools using data from them to produce results.
A subscription message is sent to the API from on_open()
. For now, it sends subscriptions for 5 minutes, 1 hour and 1 day candle data for the BTCUSD market, but in the next updates, support for multiple markets will be added.
![subscribe.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmVSUuWmBirBzj4KFvv1ntiegoVRhgbSP8JSnL55PWFPUy/subscribe.png)
3) Handling API messages and parsing candle data
When the API responds to the requests, the responses come through the on_message()
function.
I wrote code to handle event messages:
- Status of authentication
- Subscription to channels (used to keep a record of open channels)
![events.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmSzNJuDDrnY4dgtQL3auitYGurgyD4FKqDMPFwUUSdcMm/events.png)
Next, I dealt with channel messages. I left a placeholder for where to handle Bitfinex account messages and then wrote code to capture and parse candle data. Commit: parse candle data
![channel data.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmYwTKk4oFjK8iFfcYLG7qxSfKVAduRzru6bqxSao1h2UR/channel%20data.png)
Bitfinex sends a snapshot of historical candles first and then periodically sends updates. This commit improved the way snapshots are differentiated from updates: better snapshot identification
Nested functions
Two additional functions were nested in the on_message()
function to extract timeframe and market info from the message header (cross-referenced with the channel list in memory). A third nested function parse_candle
creates a candle dictionary from the list type received.
![nested.png](https://steemitimages.com/640x0/https://cdn.steemitimages.com/DQmWsSRnE5fwV99R5eo6rjiDwxx6hoav2KJs1r18VoyQDno/nested.png)
Candles, once parsed, are stored in the candles
variable, located in the t_market.py
script. The prices are categorized by market and timeframe.
That's it for this update!
Thanks for the contribution, @imwatsi! I didn't realise you were a programmer as well, so it's great to see you also contributing in the development category! You've done a really great job showing what you implemented and how - it's a really great post. As for the code, since it's Python I have a couple of things I'd like to point out / suggest:
setup.py
or even arequirements.txt
so people can easily install dependencies and use your code as a proper package. Also adding a configuration file would make sense, I think.if __name__=="__main__"
tot_init.py
. This will allow you (and others) to import it so it can be both used by other programs and modules as a module, and can be run as the main program.pep8
or another linter (e.g.pylint
) withpip
and use that to keep your code consistent. Currently some things could be improved like keeping line width < 80 characters, imports on a separate line, etc. but it's not a big deal.lstData[0]
,lstData[1]
etc. more readable, since it's not clear what they actually are, but once again, that's up to you.pytest
for this, since it's really easy to set up, and really helps with development.Other than that it looks great, and I'm curious to see how the development goes. Good luck!
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Chat with us on Discord.
[utopian-moderator]
Thanks for the review @amosbastian! Yeah, I’m a programmer, but had been on a hiatus for like a decade (long story). Started with Visual Basic back in 2004.
I recently decided to start coding again (for good this time, hahaha). My Python is not ‘pro-level’ yet, learning new things everyday. Thanks for the wonderful feedback and suggestions. I’ll start working on them. Cheers!
Good luck with (continuing) your programming journey, this is a great start!
Thank you for your review, @amosbastian! Keep up the good work!
Hi @imwatsi!
Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server
Hey, @imwatsi!
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!
Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!