Websocket Trading on Bitfinex Using Python: Part 1 - Create a Basic Authenticated Connection

imwatsi (67)in #utopian-io • 6 years ago

Repository With Sample Code

https://github.com/imwatsi/crypto-market-samples
Find the Python script on GitHub: bitfinex_websocket_basic.py

My Profile On GitHub

You can find code used in this tutorial as well as other tools and examples in the GitHub repositories under my profile:

https://github.com/imwatsi

What Will I Learn

Benefits of using websocket connections to trade
Using the websocket-client module to create a websocket connection to Bitfinex's API
Authenticating the connection
Use API methods to subscribe to ticker data and maintain a state in memory
Handling messages via websocket
Placing basic orders via websocket

Requirements

Python 3.6+
External dependencies:
- websocket-client module
Active internet connection
An active Bitfinex account with API keys

Difficulty

Basic

Tutorial Content

In this tutorial, you will learn how to create a websocket connection to the Bitfinex exchange, authenticate it, subscribe to basic ticker data and place market orders. The Python module websocket-client will be used. You can find the Bitfinex API documentation here: https://docs.bitfinex.com/v2/reference

Setting up the environment

This tutorial assumes you already have Python 3.6 or later installed on your computer. To install the websocket-client module, use pip. Type one of the following commands on a terminal:

pip3 install websocket-client if using Linux
pip install websocket-client if using Windows/MacOS

Definition of terms

Below are definitions of terms used in trading that might be helpful if you're unfamiliar with them:

Ticker: a high level overview on the state of a market, usually showing current bid/ask prices, the last traded price, 24 hour volume, high/low for the day and daily percent change.
Market order: an order that's placed to execute immediately against the order book (buys at best ask price, and sells at best bid price).

Benefits of using websocket connections to trade

Realtime data streams

Websocket connections, since they remain alive throughout the session, deliver data updates faster than if you were to request the data via REST. This enables data to be pushed to your computer in realtime, so you can maintain an up-to-date state in memory.

Faster order placement

Placing orders via websocket has less latency compared to REST, again because of the live connection and smaller overhead of sent requests.

Quicker reaction to market changes

If your trading strategies rely on getting speedy updates on market conditions, then websocket is the way to go. For example, strategies based on order book depths (which are constantly changing) will need very low latency if they are to work as expected.

Writing the code

Once Python is setup and websocket-client is installed, we move on to the code.

Import necessary modules

The code below imports the modules that this script will depend on.

import websocket
import hashlib
import hmac
import json
import time
import os
from threading import Thread

websocket is used to open and maintain a websocket connection
hashlib and hmac handle encryption related operations for authenticating the connection
json helps parse incoming JSON data and format outgoing requests as well
time is used to create time delays
os is used to trigger termination of the script, when the websocket connection closes
threading is used to create new threads for synchronous operations

Define variables and constants

# INPUT API CREDENTIALS:
API_KEY = ''
API_SECRET = ''

# GLOBAL VARIABLES
channels = {0: 'Bitfinex'}
tickers = {}

Put the API Key and Secret from your Bitfinex account in API_KEY and API_SECRET respectively.
channels will store the channels opened by the websocket connection and tickers will store the ticker data from the market

Write the fundamental websocket functions

To maintain a websocket connection we need a number of functions serving different purposes. We can start off with this basic frame:

def on_message(ws, message):
    pass

def on_error(ws, error):
    pass

def on_close(ws):
    pass

def on_open(ws):
    pass

def connect_api():
    pass

connect_api() to create a websocket instance and initiate the connection
on_open() to perform task when a connection is open
on_message() to handle messages sent back by the server
on_error() to handle errors
on_close() to determine what happens when the connection is closed

connect_api()

Now, to fill in the frame...

We will use this function connect_api() to create a global object called ws, to which we will assign a new websocket instance, set its parameters and link it up with the other functions so it can run properly. As a whole, the function should contain the code shown below.

def connect_api():
    global ws
    websocket.enableTrace(False)
    ws = websocket.WebSocketApp('wss://api.bitfinex.com/ws/2',
                            on_message = on_message,
                            on_error = on_error,
                            on_close = on_close,
                            on_open = on_open)
    ws.run_forever()

The line websocket.enableTrace(False) disables debugging output. Set this to True to see print outputs of the connection process when trying to fix bugs.
A websocket instance (with the server address parameter) is created with ws = websocket.WebSocketApp("wss://api.bitfinex.com/ws/2", and our other functions are linked to their appropriate parameters (on_message...etc).
ws.run_forever() starts the connection

on_open()

Once a connection is opened, the function on_open is called. This is a good place to print out a statement saying the connection was opened successfully. It's also where we will handle authentication and subscription to channels (these are methods through which we receive data from the server).

The final code should look like this:

global API_KEY, API_SECRET
    def authenticate():
    # Authenticate connection
        nonce = str(int(time.time() * 10000000))
        auth_string = 'AUTH' + nonce
        auth_sig = hmac.new(API_SECRET.encode(), auth_string.encode(),
                            hashlib.sha384).hexdigest()

        payload = {'event': 'auth', 'apiKey': API_KEY, 'authSig': auth_sig,
                    'authPayload': auth_string, 'authNonce': nonce, 'dms': 4}
        ws.send(payload)
    print('API connected')
    authenticate()
    sub_ticker = {
        'event': 'subscribe',
        'channel': 'ticker',
        'symbol': "tBTCUSD"
    }
    ws.send(json.dumps(sub_ticker))
    # start printing the ticker
    Thread(target=print_ticker).start()

Let's break this down. We have global declarations of API_KEY and API_SECRET, pulling in our API authentication details. There is a nested function in there: authenticate() which uses the API keys to create a signed message that will be sent to the Bitfinex server to authenticate our connection, effectively granting access to account features.

Here's an explanation of the code inside that function:

nonce = str(int(time.time() * 10000)) to secure the request, Bitfinex requires "an ever increasing numeric string" as a nonce. This code creates one from the current timestamp, which will be combined with auth_string when sent.
The following code creates the signature: auth_sig = hmac.new(API_SECRET.encode(), auth_string.encode(), hashlib.sha384).hexdigest()
With all the inputs generated, we create a dictionary and send it as JSON:

payload = {'event': 'auth', 'apiKey': API_KEY, 'authSig': auth_sig,
                    'authPayload': auth_string, 'authNonce': nonce, 'dms': 4}
ws.send(json.dumps(payload))

Going back to the main function on_open(), you can see that after it prints "API connected" it calls the authenticate()nested-function and then goes on to send a message with a subscription request for the BTCUSD ticker.

Finally, it starts a new thread running a function we will define later on called print_ticker, which will print the BTCUSD ticker in realtime on the terminal.

on error()

You can use this function to handle errors that occur during the websocket session. To keep things simple, I just put a print statement printing the error.

def on_error(ws, error):
    print(error)

on_close()

Use this function to determine what happens when the connection closes. Here we just notify the user with a print statement and then terminate the script, to keep things simple.

def on_close(ws):
    print("### API connection closed ###")
    os._exit(0)

on_message()

As soon as the websocket connection is opened, messages can be sent and received. In this script, the first message that has to be received will be one notifying us of authentication status (successful or failed), from the authentication request we sent to the server.

We need code to handle this and determine whether or not it was successful. This function will also receive messages containing details of the data channels we subscribe to, as well as the data that those channels will provide. Everything will be handled by this function. The entire function's code is shown further below. Here, I will break down it's components.

    global channels, tickers
    data = json.loads(message)
    # Handle events
    if 'event' in data:
        if data['event'] == 'info':
            pass # ignore info messages
        elif data['event'] == 'auth':
            if data['status'] == 'OK':
                print('API authentication successful')
            else:
                print(data['status'])
        # Capture all subscribed channels
        elif data['event'] == 'subscribed':
            if data['channel'] == 'ticker':
                channels[data['chanId']] = [data['channel'], data['pair']]

The code above declares the global variables channels and tickers that we defined above. A combination of if statements filter out successful authentication scenarios and capture the channels that will have been successfully subscribed to and saves them to the channels variable.

The next block of code handles the data that we receive from open channels:

    # Handle channel data
    else:
        chan_id = data[0]
        if chan_id in channels:
            if 'ticker' in channels[chan_id]: # if channel is for ticker
                if data[1] == 'hb':
                    pass
                else:
                    # parse ticker and save to memory
                    sym = channels[chan_id][1]
                    ticker_raw = data[1]
                    ticker_parsed = {
                        'bid': ticker_raw[0],
                        'ask': ticker_raw[2],
                        'last_price': ticker_raw[6],
                        'volume': ticker_raw[7],
                    }
                    tickers[sym] = ticker_parsed

Channels are identified by channel ID numbers and these are the keys used to store entries in the channels dictionary. The code above extracts the channel ID from the message and looks for a match in the stored channels.

If one is found, a further check is made to make sure the channel is for a ticker with the line if 'ticker' in channels[chan_id]:. This filters out other channel types, when you have order book and candle data channels for example.

Bitfinex sends heartbeat messages at regular intervals to keep a channel open, so we ignore these with:

if data[1] == 'hb':
    pass

The rest of the code parses the ticker information and saves it to the tickers variable. Bitfinex sends these updates as sets of lists, so bid, ask , etc will be indexed; hence the ticker_raw[0] references.

These messages will be coming in frequently and when you have multiple tickers subscribed (100 market symbols for example) the function on_message will be used each time, which can be multiple times per fraction of a second. In this case, this can be a latency bottleneck and measures will need to be taken to reduce the amount of computations done in this function by deferring them to other functions and opening new threads. My next tutorial will be on this subject.

To wrap up the on_message function: it should look like this in its entirety:

def on_message(ws, message):
    global channels, balances, tickers
    data = json.loads(message)
    # Handle events
    if 'event' in message:
        if data['event'] == 'info':
            pass # ignore info messages
        elif data['event'] == 'auth':
            if data['status'] == 'OK':
                print('API authentication successful')
            else:
                print(data['status'])
        # Capture all subscribed channels
        elif data['event'] == 'subscribed':
            if data['channel'] == 'ticker':
                channels[data['chanId']] = [data['channel'], data['pair']]
    # Handle channel data
    else:
        chan_id = data[0]
        if chan_id in channels:
            if 'ticker' in channels[chan_id]: # if channel is for ticker
                if data[1] == 'hb':
                    pass
                else:
                    # parse ticker and save to memory
                    sym = channels[chan_id][1]
                    ticker_raw = data[1]
                    ticker_parsed = {
                        'bid': ticker_raw[0],
                        'ask': ticker_raw[2],
                        'last_price': ticker_raw[6],
                        'volume': ticker_raw[7],
                    }
                    tickers[sym] = ticker_parsed

new_order_market()

Next, we define a new function with which new market orders can be placed. It starts by declaring global variable ws (the websocket instance). Then it generates a "client order ID" for the new order using a timestamp.

After that, it creates a dictionary containing the order's details: the client order ID, trade symbol(e.g. BTCUSD) and the amount being transacted. This dictionary is included in a special message format required by Bitfinex.

The message must be sent as a list, to the 0 (zero) channel, with the order details as payload, as seen below. Once this is compiled it will be sent as JSON via ws.send() function.

def new_order_market(symbol, amount):
    global ws
    cid = int(round(time.time() * 1000))
    order_details = {
        'cid': cid,
        'type': 'EXCHANGE MARKET',
        'symbol': 't' + symbol,
        'amount': str(amount)
    }
    msg = [
            0,
            'on',
            None,
            order_details
        ]
    ws.send(json.dumps(msg))

To use this function to place an order, you can call it with new_order_market("BTCUSD", 0.01) for example, to buy 0.01 BTC at market price. Use negative amounts to sell.

It can also be invoked from user input by adding a function like the one below and opening a new thread for it at script init. It runs a loop listening for user commands, and executing instructions as coded.

def user_input():
    while True:
        command = input()
        if command == 'Buy':
            new_order_market("BTCUSD", 0.01)

print_ticker()

The last function we will write is a loop used to retrieve the current ticker data from memory and print it on the terminal. It uses a print statement with end="\r" to maintain the print on the same line while updating the values, so you can see the ticker change in realtime.

def print_ticker():
    global ticker
    symbol = 'BTCUSD'
    while len(tickers) == 0:
        # wait for tickers to populate
        time.sleep(1)
    while True:
        # print BTCUSD ticker every second
        details = tickers[symbol]
        print('%s:  Bid: %s, Ask: %s, Last Price: %s, Volume: %s'\
            %(symbol, details['bid'], details['ask'],\
            details['last_price'], details['volume']), end="\r", flush=True)
        time.sleep(1)

That's it! The script is done.

Debug output

Below are screenshots of output from setting websocket.enableTrace(True). The first screenshot contains header information for the request we sent and the response received as the connection is being established.

The next screenshot shows our print statements mixed with what seems to be gibberish. There are two send statements there: one for authenticating the connection and one for subscribing to the BTCUSD ticker channel. Both are encrypted, hence their illegible form. The last line is where the script prints out the realtime ticker, updated every second as we stated in the code.

You should get the following terminal output when you run the script normally: