Blackbox – the big mystery

out3-kopia

This is a screen shot of Hydra running live on 6 Bitcoin markets in parallel. I believe it’s the first time i post it public. The social media call it BlackBox – in other words – a computer running an algorithm (high frequency in this case) to find out nuances on the market and use speed to gain profits before others do. Why do they call it BlackBox? It’s obvious because nobody knows what’s going on inside that generates the profits.

At some point i got interested in financial world – but more from a tech side, not really the financial and business talks. This part of the business is called Fin-Tech and to be precise i got interested in Algorithmic Trading (Blackbox trading) and Direct Market Access.

It wasn’t just because i thought i’ll get rich in one month – i already knew playing the market is a losers approach. The thing that really got me was the Algo-trading wars that are going on behind the scenes. It’s not people who trade the markets today, it’s machines and their algorithms that fight for a penny millions times per second. And this got me, speed and competition in technology that comes with a reward.

Just to be clear, Algo Trading is not Quantitative Trading – these two are coupled together at some point, but Algo Trading is the whole automated/semi automated system that does strategy, risk, portfolio and etc.. Quantitative – is the cool name for all of these that are Masters and PhD’s in maths and physics – these guys make the best algotrading strategies in the world and earn best money around.

I picked BITCOINS as market data is freely available and the fees are still low. Additionally 95% of the Bitcoin markets work with REST api, so there’s no need to deep dive to FIX protocol in the beginning. Sounded like a quick learning curve, but believe me it’s not.

out2
A spike like this can last only 1 second, above – Hydra missed the execution, somebody was faster.

HYDRA
I really like this name – it reflects the way the blackbox works.
The initial idea to start with algotrading was to go with a risk-free strategy called Arbitrage. In few words – you find out a spread (difference) between markets and if numbers are fine (after fees, transaction costs) you Buy low (cheapest market) and Sell high (most expensive market). You should always buy the same quantity as you sell for the  arbitrage to work. This is very important, as seen later on – sometimes (due to slippage/miss) this can be very tricky.

To get things done, HYDRA required to have an access to Market Data and poll it continuously without delays. So a MarketLink’s had to be implemented – these are the classes that talk to Markets via SSL (wolfssl in my case) and HTTPS.


The data required by HYDRA consists mainly of the publicly available OrderBooks for Markets (to get best offers in BID/ASK + Quantities) and my own Portfolio (Wallet) to ensure how much money and BTC i still have at each market.

HYDRA includes a simple deviation value for every market it monitors. This deviation value tells the strength of the change since last time for a given market (ASK/BID) price. Thanks to this HYDRA knows which market moved and by how much, this is useful when a decision : WHERE TO PLACE ORDER FIRST is needed.

Imagine a situation :

13:00:00
MarketA) BID 80 ASK 90
MarketB) BID 80 ASK 90
MarketC) BID 80 ASK 90

13:00:00:133
MarketA) BID 80 ASK 90
MarketB) BID 100 ASK 103
MarketC) BID 80 ASK 90

(Forget about quantities now)
You quickly find out that there is an arbitrage possibility between MarketA and MarketB.
You buy low (ASK 90) at MarketA and Sell high (BID 100) at Market B.

Now after calculating income you find out that executing on this you’ll earn 8$ after fees.
So you start placing order in the naive approach BUY first, SELL last.

You place a BUY order – got executed.

Meantime new data comes in
13:00:00:150
MarketA) BID 80 ASK 90
MarketB) BID 80 ASK 90
MarketC) BID 80 ASK 90

You place a SELL order  on MarketB – missed (not executed)

What happened? Well, MarketB “Moved” when you’ve been placing a BUY order on MarketA. This is mostly because placing order takes time.. (ssl, connect, send(), recv()).

This is common, so HYDRA takes that into consideration by calculating strength of the move (deviation) and placing the order first on the market that moved most ( in this situation, HYDRA would place a SELL first and BUY later ). See below, HYDRA missing an order when the Bid (green) spike happened (market manipulation).

out3

So knowing that HYDRA calculates the deviation, how does actually HYDRA choose the order of the action?

It might happen that there are more than two markets at a time vulnerable for Arbitrage.
So HYDRA is calculating what is called OrderProfit by calculating a matrix between all markets and returning best profit found. This is actually pretty easy.

Hydra is calculating buy/sell profit between available markets – takes into account BTC quantities both at the current order book and in your wallets on the markets. Then profit matrix is sorted and best BUY/SELL markets are taken for execution.

Zrzut ekranu 2016-08-04 o 22.24.39

At this moment Hydra has all it needs to get the job done – so it fires up the orders.
And here comes the trouble.

 

Approach 1. Buy first (higher deviation), Sell later

What if your first order is missed?
Well nothing happens, you don’t action the second one.
No loss.

What if your first order is slipped
You need to adjust your sell order QTY to ensure you don’t oversell.
Partial loss.

What if your second order is missed?
Well nothing you can do, you bought but didn’t sell.
Loss.

What if your second order is slipped?
Well you didn’t loose but earned a bit less.
Partial loss.

 

Approach 2. Sell first (higher deviation), Buy First

What if your first order is missed?
Nothing, do not action the second one.
No loss.

What if your first order is slipped?
So you sold less than you expected? Ensure that you buy enough to cover the profit.
Partial loss.

What if your second order is missed?
Pity, you sold without a Buy.
Loss.

What if your second order is slipped?
You sold QTY but bought less – unfortunately you’re a bit lost.
Partial Loss.

So as you see, there are couple possible things that can happen and HYDRA takes them all into consideration. Bare in mind you need to include FEES in order to get out from cases like that with a profit (if possible).

Zrzut ekranu 2016-01-11 o 20.59.11

Above Hydra owning the markets.

Additionally thing like socket that hangs on recv() can happen too, so you this needs to be taken into consideration too 🙂

Ok this was very briefly, i don’t want to make a long post here – let’s treat this as an introduction to HYDRA. I want to describe more details and how did i approached them all in next posts. But by looking into this intro i hope you start to see the challenge and fun in coding a blackbox from scratch.

And if you ask, the GUI – i had to made it myself earlier before. I used it a lot for a Raytracing tool on GridMan.

Hydra was supposed to run on a dedicated (VPS) machine as close as i could get to a markets. So it is using a direct frame buffer operations to render the pixels on the screen – this later on is turned into a PNG file and displayed on the web page for monitoring. However at the beginning i ran it on Raspberry PI at home. This gave an option to click through the GUI and play a bit.

After running it for a couple of days i decided to move to VPS due to network latency issues i had on PI. Secondly Hydra became fully automated – so there was no need for a PANIC button at all :).

It all starts with a single pixel 🙂

Zrzut ekranu 2016-08-04 o 22.50.55

 

Hope you liked it, stay tuned for more!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s