Market games

Interesting sshots of BTC Markets games.

1. Forcing price to go up? (Quote) Spoofing?
zrzut-ekranu-2016-10-28-o-16-13-37

One of the market kicks in simulating trades at a very high price, trying to fire an order to catch ARB on this obviously misses the orders. Both because of latency ( BTC markets serving their API through CloudFlare cause a significant delay ) and because of the fact they do it ‘locally’ – probably to force the price to move up and speculate. Or i am not reading this correctly?

IoT – The WiFi Probe REQ an RTLS. Tracking WiFi Devices for a penny.

zrzut-ekranu-2016-09-14-o-08-35-27

Every WiFi enabled device that you have (laptop, phone, tablet, kindle, watch, … ) is sending a PROBE REQuest packets in background to figure out if there are any known AP’s.
It’s done transparently, wether you are in a shopping center, outside in a park, or in the mountains – it’s a continuous process.
If an AP is found the device automatically associates to let you stay online ASAP. This is built in the 802.11 protocol. (More)

Knowing that every single device is broadcasting a Probe packet i figured out that it might be interesting to place couple of ESP8266 around with sniffer mode enabled to listen to these Probe Requests and forward them to a central server for analysis. The idea behind that is to be able to track every device between zones (places where ESP8266 are located) and guess what, i had couple of hours free to code and try it out.

I placed couple of nodes in different places within the city and enabled sniffer mode that looks for 0x40 802.11 type packets (PROBE REQUEST). When a packet is received it is sent out to a central server where packets are stored and presented later on on a web page.


This time i used ESP8266 on a ‘Witty’ chip – it’s a cool 2cmx2cm sized board that has a single button, rgb led built in and obviously an USB power. Pretty cheap, you can get tons of them for almost ‘no money’.

Every single ESP8266 device in this system that listens for packets is called ZONE, so if you place 4 of them in different locations – you have 4 zones. I have placed 2 of them for tests, one in the Garage and one outside in the area where i live. This is to figure out if i can determine wether people (and at what time) travel between garage and main area, how often and at what time. Time for a start, ESP’s placed, kick started – analysis time.

zrzut-ekranu-2016-09-14-o-08-51-13

The above graph shows number of packets for every hour in last 24hours, there’s a quite number of packets received as you can see, big-data time ūüėČ

As i have two zones, let’s see how active they are – meaning, when was the last packet received at which zone – this helps me out to determine the Per-zone activity and eventual down-times of the zone nodes.

zrzut-ekranu-2016-09-14-o-08-52-49

You can clearly see that both zones received their last packets 2/3 seconds ago – so somebody was hanging around there with a device sending probe requests, good! Let’s move on, let’s find out¬†what is the packet per zone distribution.

zrzut-ekranu-2016-09-14-o-08-54-28

Sensible, the MAIN zone that is outside in the main area of my home is definitely more active, garage makes only 6.7% of all the packets received. Let’s find out what happened in GARAGE in last 24 hours.

zrzut-ekranu-2016-09-14-o-08-55-52

Ahh more graphs, more stats – fantastic. GARAGE Zone received 62 Probe packets in last 24hrs, the peak was around 7 am (where people started to jump into their cars to drive to work). ¬†In total there are 17 UNIQUE devices found, so potentially we can assume there were 17 persons in the zone. Need more info…

zrzut-ekranu-2016-09-14-o-08-57-28

Brilliant, i can see which device was the most ‘popular’ one, meaning was having the biggest ratio of PROBEs sent across the 24hrs time and in total. Beside i can see live packets coming in from the ESP. Let’s look at this B0:79 – looks like the most popular one, a single click on the device and we’re entering a Device tracker sub-page where..

zrzut-ekranu-2016-09-14-o-08-58-54

I can figure out that it is a Motorola device (based on MAC OUI) and

zrzut-ekranu-2016-09-14-o-08-59-04

This Motorola sent 32 packets in last 24hrs, it was seen only in one zone (GARAGE) and it was most active today at 07:00 AM. Great, let’s go deeper…

zrzut-ekranu-2016-09-14-o-09-01-03

Above is a confirmation of the fact that Motorola was seen only in Garage (wasn’t captured by MAIN zone) – live packets on the right hand side. But let’s see the whole history of this device shall we?

zrzut-ekranu-2016-09-14-o-09-02-31

You can clearly look at each of the devices found and track it’s activitiy. What’s even more i can see the RSSI value – that tells me the distance of the device to the zone node.
This Motorola was first found today at 02:52 AM, interesting. The RSSI tells me it was probably on the opposite side of the garage (which is quite big) – meaning far away from the node! Knowing the Garage i can immediately figure out where this person was moving.

Let’s go back and¬†pick up different device.

zrzut-ekranu-2016-09-14-o-09-17-25

zrzut-ekranu-2016-09-14-o-09-17-48

This one is more interesting, somebody was seen in both zones with this device, i wonder at what distance and time.

zrzut-ekranu-2016-09-14-o-09-18-50

First seen at 7:54 today, wen to a garage (7:55) and was pretty close to the NODE at 7:56. Then the signal is getting weaker (jumped into the car probably) as minute later the signal was lost.

As you can see a simple system that costs $3.20 for each node can be turned into a quite sensible real time location service. Knowing that a deployment of 100 of nodes is still low cost – you can imagine how this can help out in business stats, emergency tracking, and advertisement targetting systems (tv screens knowing the profiles ¬†(vendor) of candidates per week/month/season can push targeted ads). There is a big space for project like this to go live and for sale. It’s nothing new – but it’s definitely the most affordable one you can get ($3.20 for a node!).

zrzut-ekranu-2016-09-14-o-09-23-25

zrzut-ekranu-2016-09-14-o-09-23-44

In terms of the project – i keep it live, costs almost nothing so i’ll be adding nodes just for fun ūüėČ

Update. I recalculate RSSI to Meters (distance), it makes more sense now.

zrzut-ekranu-2016-09-14-o-12-05-40

 

WPA Bruteforce test

Just a side note, i’ll be getting to the crackers i wrote later on – but been testing WPA update since couple of days now on WPA Handshakes and the cracker is a success.

ddd

This is a single core task – but have a Distributed Task for GridMan in place already.
The cracker goes through all WPA handshakes that i have sniffed and tries to crack them with a wordlist or a bruteforce generator.

macs

Above a dump from a modified version of a handshake stripper, i take the airmon dump file and strip all WPA Handshakes that were found for a crack-all-at-once approach.

Ok, let’s find out what’s sitting there and let me tell you that i hate Linux support for WiFi – it’s just a mess if you need to do things quick, but as i’m left now with a Raspberry PI on the balcony that has Kali installed, there’s no other option around.

Couple of command line entries and we’re in.

aaaa

 

Now that we know we’re in, let’s checkout the AP.

Zrzut ekranu 2016-08-06 o 22.58.04

Aha! A login form, i can bet the owner didn’t even change admin password.
admin/admin is the default setting for WR340g.

Zrzut ekranu 2016-08-06 o 22.59.05

dodo

Fuzzing continues

Just made a simple fuzzing strategy that shoots random characters on arguments for binary. I am testing this on my MacOSX – and it seems like it works ūüėČ
Zrzut ekranu 2016-08-05 o 23.36.23

The tool itself is dirty coded yet, however fuzzing Rez with argv0 gave

SIGSEGV 11 Invalid memory segment access (ANSI)

 

Splendid.

Blackbox – the big mystery

out3-kopia

This is a screen shot of Hydra running live on 6 Bitcoin markets in parallel. I believe it’s the first time i post it public.¬†The social media call it¬†BlackBox – in other words – a computer running an algorithm (high frequency in this case) to find out nuances on the market and use speed to gain profits before others do. Why do they call it BlackBox? It’s obvious because nobody knows what’s going on inside that generates the profits.

At some point i got interested in financial world – but more from a tech side, not really the financial and business talks. This part of the business is called Fin-Tech and to be precise i got interested in Algorithmic Trading (Blackbox trading) and Direct Market Access.

It wasn’t just because i thought i’ll get rich in one month – i already knew playing the market is a losers approach. The thing that really got me was the Algo-trading wars that are going on behind the scenes. It’s not people who trade the markets today, it’s machines and their algorithms that fight for a penny millions times per second. And this got me, speed and competition in technology that comes with a reward.

Just to be clear, Algo Trading is not¬†Quantitative Trading – these two are coupled together at some point, but Algo Trading is the whole automated/semi automated system that does strategy, risk, portfolio and etc.. Quantitative – is the cool name for all of these that are Masters and PhD’s in maths and physics – these guys make the best algotrading strategies in the world and earn best money around.

I picked¬†BITCOINS as market data is freely available and the fees are still low. Additionally 95% of the Bitcoin markets work with REST api, so there’s no need to deep dive to FIX protocol in the beginning. Sounded like a quick learning curve, but believe me it’s not.

out2
A spike like this can last only 1 second, above – Hydra missed the execution, somebody was faster.

HYDRA
I really like this name – it reflects the way the blackbox works.
The initial idea to start with algotrading was to go with a risk-free strategy called Arbitrage. In few words Рyou find out a spread (difference) between markets and if numbers are fine (after fees, transaction costs) you Buy low (cheapest market) and Sell high (most expensive market). You should always buy the same quantity as you sell for the  arbitrage to work. This is very important, as seen later on Рsometimes (due to slippage/miss) this can be very tricky.

To get things¬†done, HYDRA required to have an access to Market Data and poll it continuously without delays. So a MarketLink’s had to be implemented – these are the classes that talk to Markets via SSL (wolfssl in my case) and HTTPS.


The data required by HYDRA consists mainly of the publicly available OrderBooks for Markets (to get best offers in BID/ASK + Quantities) and my own Portfolio (Wallet) to ensure how much money and BTC i still have at each market.

HYDRA includes a simple deviation value for every market it monitors. This deviation value tells the strength of the change since last time for a given market (ASK/BID) price. Thanks to this HYDRA knows which market moved and by how much, this is useful when a decision : WHERE TO PLACE ORDER FIRST is needed.

Imagine a situation :

13:00:00
MarketA) BID 80 ASK 90
MarketB) BID 80 ASK 90
MarketC) BID 80 ASK 90

13:00:00:133
MarketA) BID 80 ASK 90
MarketB) BID 100 ASK 103
MarketC) BID 80 ASK 90

(Forget about quantities now)
You quickly find out that there is an arbitrage possibility between MarketA and MarketB.
You buy low (ASK 90) at MarketA and Sell high (BID 100) at Market B.

Now after calculating income you find out that executing on this you’ll earn 8$ after fees.
So you start placing order in the naive approach BUY first, SELL last.

You place a BUY order – got executed.

Meantime new data comes in
13:00:00:150
MarketA) BID 80 ASK 90
MarketB) BID 80 ASK 90
MarketC) BID 80 ASK 90

You place a SELL order  on MarketB Рmissed (not executed)

What happened? Well, MarketB “Moved” when you’ve been placing a BUY order on MarketA. This is mostly because placing order takes time.. (ssl, connect, send(), recv()).

This is common, so HYDRA takes that into consideration by calculating strength of the move (deviation) and placing the order first on the market that moved most ( in this situation, HYDRA would place a SELL first and BUY later ). See below, HYDRA missing an order when the Bid (green) spike happened (market manipulation).

out3

So knowing that HYDRA calculates the deviation, how does actually HYDRA choose the order of the action?

It might happen that there are more than two markets at a time vulnerable for Arbitrage.
So HYDRA is calculating what is called OrderProfit by calculating a matrix between all markets and returning best profit found. This is actually pretty easy.

Hydra is calculating buy/sell profit between available markets Рtakes into account BTC quantities both at the current order book and in your wallets on the markets. Then profit matrix is sorted and best BUY/SELL markets are taken for execution.

Zrzut ekranu 2016-08-04 o 22.24.39

At this moment Hydra has all it needs to get the job done – so it fires up the orders.
And here comes the trouble.

 

Approach 1. Buy first (higher deviation), Sell later

What if your first order is missed?
Well nothing happens, you don’t action the second one.
No loss.

What if your first order is slipped
You need to adjust your sell order QTY to ensure you don’t oversell.
Partial loss.

What if your second order is missed?
Well nothing you can do, you bought but didn’t sell.
Loss.

What if your second order is slipped?
Well you didn’t loose but earned a bit less.
Partial loss.

 

Approach 2. Sell first (higher deviation), Buy First

What if your first order is missed?
Nothing, do not action the second one.
No loss.

What if your first order is slipped?
So you sold less than you expected? Ensure that you buy enough to cover the profit.
Partial loss.

What if your second order is missed?
Pity, you sold without a Buy.
Loss.

What if your second order is slipped?
You sold QTY but bought less – unfortunately you’re a bit lost.
Partial Loss.

So as you see, there are couple possible things that can happen and HYDRA takes them all into consideration. Bare in mind you need to include FEES in order to get out from cases like that with a profit (if possible).

Zrzut ekranu 2016-01-11 o 20.59.11

Above Hydra owning the markets.

Additionally thing like socket that hangs on recv() can happen too, so you this needs to be taken into consideration too ūüôā

Ok this was very briefly, i don’t want to make a long post here – let’s treat this as an introduction to HYDRA. I want to describe more details and how did i approached them all in next posts. But by looking into this intro i hope you start to see the challenge and fun in coding a blackbox from scratch.

And if you ask, the GUI – i had to made it myself earlier before. I used it a lot for a Raytracing tool on GridMan.

Hydra was supposed to run on a dedicated (VPS) machine as close as i could get to a markets. So it is using a direct frame buffer operations to render the pixels on the screen – this later on is turned into a PNG file and displayed on the web page for monitoring. However at the beginning i ran it on Raspberry PI at home. This gave an option to click through the GUI and play a bit.

After running it for a couple of days i decided to move to VPS due to network latency issues i had on PI. Secondly Hydra became fully automated – so there was no need for a PANIC button at all :).

It all starts with a single pixel ūüôā

Zrzut ekranu 2016-08-04 o 22.50.55

 

Hope you liked it, stay tuned for more!

Fuzzing

Recently i turned my attention to security, but this time i took a peak into what’s called Fuzzing.

Long story short, fuzzing is mainly brute force vulnerability detection and has a lot of categories. Take a look here. What caught my attention is fuzz testing Web applications and local binaries (mainly black box /grey box fuzzing).

I am total noob in this area, never tried it and never seen a fuzzer doing a good job – however interested in this area i have a low power distributed approach idea somewhere behind.

Just started to read a great book on the topic check it out.

As always, there are some basics that i’ll be focusing on at the start.

  1. Fuzzing arguments passed to suid binary, you just randomly generate a set of characters and put them into binary arguments to discover eventual input handling issues.
  2. Fuzzing WEB applications, this is a bigger topic however here i would like to focus mainly on SQL injection discovery, directory traversal, and POST/GET argument fuzzing.

The whole fuzzing area is wide but i have decided to pick up two pretty major destinations to see if i come up with something interesting. Ideally i would like to be able to write a vulnerable binary, fuzz it and get back with some results. Finally i am thinking of getting this stuff distributed, as it is known fuzzing can take some time Рit would be good to pack it all up with GridMan.

So let’s treat this as an intro, i will be posting in Security/Fuzzing area some updates as i go through the learning curve.

Fingers x.

GridMan – intro

Zrzut ekranu 2016-08-04 o 13.14.13

As everything in my dev life GridMan started because of a need. And yes i did some research on the topic but all projects that i found were not exactly what i was looking for, so i decided to write my own Distributed Computing Library.

So here it begins.
Somewhere around 2014 after diving into the pen testing (again:) i found that working with the WPA and WEP and MD5 cracker on a single host is just a pain in the ass and i waiting weeks or months for some password to popup is just a waste of time.¬†So at that time i was playing around with MD5 hashes and i knew john the ripper could do the job but it was not acceptable to let a 24 core cpu running at home with all his fans operating, cables lying around, power consumption and kids wandering around. I opened my garage and found couple of Raspberry PI’s laying around. Then it started. It was clear decision to start developing¬†with raspberries, why?

Because of costs, power consumtion and lack of noise they generate. If GridMan will work on Raspberries it is obvious it will work on a dedicate Blades/… after recompiling, so the goal was set.

Zrzut ekranu 2016-08-04 o 13.40.28

What i needed at that time is pretty what i am using the GridMan today for.
All i wanted is a network connected group of computers working together on projects that are made of tasks of binaries with payloads.¬†Payload in the sense of passing argv’s to the binary so that each node can start binary with some arguments.

Infrastructure and hardware – I needed to ensure that the whole h/w¬†part is easy¬†– as mentioned, i didn’t want to have cables lying around for weeks, routers¬†and lots of space taken so i had to figure out how to approach it so that the Grid was small and fully customizable and what’s more important – cool looking – so that i could place it somewhere at home and look at it when it’s crunching.

Zrzut ekranu 2016-08-04 o 13.46.56

WiFi – packets flying to/from GridMan are very small, i strip the binaries (tasks) and payloads are just arguments passed so mainly uint_16’s or some chars. WiFi is enough to get this connected and it obviously safes a lot of space around (no cables mess).

Stacking – i used some bolts and screws to stack the boards on top of each other, couple of minutes of work and its done. Even looks good.

Power – a single power unit would be perfect, therefore i use a DLINK 7 port active USB hub to power up 5 PI’s.

Power cycling Рsometimes grid goes off, i used a Belkin WEMO switch to poweron / poweroff the grid from my phone.

Zrzut ekranu 2016-08-04 o 13.52.36

So the hardware part was easy, stack the PI’s however you want, let them connect to your network and play. And btw.. new PI’s have a big advantage over old ones ( 4 cores, Wifi built in ).

The software
Yes, the best part.

GridMan consists of a Dedicated Server binary ( GridMaster ), a Worker (GridNode) and a command line tool for playing around with the projects, tasks, grid itself. In addition to that, there are couple of Grid Tasks that i’ve made that can be kicked off on the grid for crunching. Some of these are : MD5 cruncher, WPA, WEP, Raytraycer, NN learner and something new that i’m working on is Distributed Fuzzer.

GridMaster (Server) – this guy is responsible for work management mainly. You upload a Project (that is just a container of tasks with a name) and tasks you want to run on Nodes.

Zrzut ekranu 2016-08-04 o 14.04.36
Server keeps track of all the tasks and distributes them to idling nodes.
What he really does is finds out the Idling nodes within the grid and sends them the Binary  + Payload, then flags the node as active and keeps getting heartbeats with load average from crunching node. When node finishes, it sends back the new payload (return of the binary) to the server, then the task is set to Complete (or failed) and next task is given to a node / next node meantime.

GridNode (worker) – these guys are like bees. They connect to the GridMaster and wait for tasks to be received. If a task is received, worker saves the binary and payload in the /tmp/. Then it chmods the binary and spawns a child that runs it.
There are two possible options at this moment:

1) the binary exec fails – then a failure is returned to server, you can later on decide what to do (restart for example),

2) the binary starts and keeps working

Zrzut ekranu 2016-08-04 o 14.21.09

After completion, a success code together with payload is returned to server and worker starts waiting for another task.

CommandLine¬†– handy tool for a grid manager ūüėČ
Zrzut ekranu 2016-08-04 o 14.12.04
This command line utility is to drive the whole grid. You can do all project management down to a task operations with a single command.¬†It’s nothing more than exposed api from GridMan lib to control the whole system.

Zrzut ekranu 2016-08-04 o 14.18.09

Above, example showing 2 workers 1 project with 6 tasks. You can also see which node is working on which task.

Zrzut ekranu 2016-08-04 o 14.18.55

Above, example showing 2 tasks running from MD5_TEST project

GridMan IOS Рyeah, the pocket GridMan administrator.
Not everybody likes consoles so i made a fine looking easy to use grid manager. Play with the grid from all over the world ūüėČ A single gif is worth more than thousands of words.

EntityAI

So that’s mainly it for an intro, i will go in to details with next updates on GridMan.

There are couple of GridMan Vids you can watch :YouTube
And a (Polish) article: osworld article