GridMan – intro

Zrzut ekranu 2016-08-04 o 13.14.13

As everything in my dev life GridMan started because of a need. And yes i did some research on the topic but all projects that i found were not exactly what i was looking for, so i decided to write my own Distributed Computing Library.

So here it begins.
Somewhere around 2014 after diving into the pen testing (again:) i found that working with the WPA and WEP and MD5 cracker on a single host is just a pain in the ass and i waiting weeks or months for some password to popup is just a waste of time. So at that time i was playing around with MD5 hashes and i knew john the ripper could do the job but it was not acceptable to let a 24 core cpu running at home with all his fans operating, cables lying around, power consumption and kids wandering around. I opened my garage and found couple of Raspberry PI’s laying around. Then it started. It was clear decision to start developing with raspberries, why?

Because of costs, power consumtion and lack of noise they generate. If GridMan will work on Raspberries it is obvious it will work on a dedicate Blades/… after recompiling, so the goal was set.

Zrzut ekranu 2016-08-04 o 13.40.28

What i needed at that time is pretty what i am using the GridMan today for.
All i wanted is a network connected group of computers working together on projects that are made of tasks of binaries with payloads. Payload in the sense of passing argv’s to the binary so that each node can start binary with some arguments.

Infrastructure and hardware – I needed to ensure that the whole h/w part is easy – as mentioned, i didn’t want to have cables lying around for weeks, routers and lots of space taken so i had to figure out how to approach it so that the Grid was small and fully customizable and what’s more important – cool looking – so that i could place it somewhere at home and look at it when it’s crunching.

Zrzut ekranu 2016-08-04 o 13.46.56

WiFi – packets flying to/from GridMan are very small, i strip the binaries (tasks) and payloads are just arguments passed so mainly uint_16’s or some chars. WiFi is enough to get this connected and it obviously safes a lot of space around (no cables mess).

Stacking – i used some bolts and screws to stack the boards on top of each other, couple of minutes of work and its done. Even looks good.

Power – a single power unit would be perfect, therefore i use a DLINK 7 port active USB hub to power up 5 PI’s.

Power cycling – sometimes grid goes off, i used a Belkin WEMO switch to poweron / poweroff the grid from my phone.

Zrzut ekranu 2016-08-04 o 13.52.36

So the hardware part was easy, stack the PI’s however you want, let them connect to your network and play. And btw.. new PI’s have a big advantage over old ones ( 4 cores, Wifi built in ).

The software
Yes, the best part.

GridMan consists of a Dedicated Server binary ( GridMaster ), a Worker (GridNode) and a command line tool for playing around with the projects, tasks, grid itself. In addition to that, there are couple of Grid Tasks that i’ve made that can be kicked off on the grid for crunching. Some of these are : MD5 cruncher, WPA, WEP, Raytraycer, NN learner and something new that i’m working on is Distributed Fuzzer.

GridMaster (Server) – this guy is responsible for work management mainly. You upload a Project (that is just a container of tasks with a name) and tasks you want to run on Nodes.

Zrzut ekranu 2016-08-04 o 14.04.36
Server keeps track of all the tasks and distributes them to idling nodes.
What he really does is finds out the Idling nodes within the grid and sends them the Binary  + Payload, then flags the node as active and keeps getting heartbeats with load average from crunching node. When node finishes, it sends back the new payload (return of the binary) to the server, then the task is set to Complete (or failed) and next task is given to a node / next node meantime.

GridNode (worker) – these guys are like bees. They connect to the GridMaster and wait for tasks to be received. If a task is received, worker saves the binary and payload in the /tmp/. Then it chmods the binary and spawns a child that runs it.
There are two possible options at this moment:

1) the binary exec fails – then a failure is returned to server, you can later on decide what to do (restart for example),

2) the binary starts and keeps working

Zrzut ekranu 2016-08-04 o 14.21.09

After completion, a success code together with payload is returned to server and worker starts waiting for another task.

CommandLine – handy tool for a grid manager 😉
Zrzut ekranu 2016-08-04 o 14.12.04
This command line utility is to drive the whole grid. You can do all project management down to a task operations with a single command. It’s nothing more than exposed api from GridMan lib to control the whole system.

Zrzut ekranu 2016-08-04 o 14.18.09

Above, example showing 2 workers 1 project with 6 tasks. You can also see which node is working on which task.

Zrzut ekranu 2016-08-04 o 14.18.55

Above, example showing 2 tasks running from MD5_TEST project

GridMan IOS – yeah, the pocket GridMan administrator.
Not everybody likes consoles so i made a fine looking easy to use grid manager. Play with the grid from all over the world 😉 A single gif is worth more than thousands of words.

EntityAI

So that’s mainly it for an intro, i will go in to details with next updates on GridMan.

There are couple of GridMan Vids you can watch :YouTube
And a (Polish) article: osworld article

ESP8266 – WIDS – “Spooky”

Okay this is both #ESP8266 and #Security.

When i first got my hands on the ESP8266 i was curious how much can be done with the chip in terms of security. Knowing that the chip can go ‘monitor’ mode it was clear to me that first thing i’ll code would be the Wireless Intrusion Detection thing running on a 5v low power chip. At that time i ordered something what is called #NODEMCU and the fun started.

IMG_2941

The chip itself is capable of running a callback on ever packet it receives with a function:

static void ICACHE_FLASH_ATTR handle_pkt(uint8_t* buf, uint16_t len)

There is actually an official “Sniffer” PDF document by Espressif, google it for more details.

So the way Spooky works is simple. It captures packets within BSSID that is your local home Access Point. When a packet is received, it scans through 802.11 to find the source and destination. If Source or Destination is not listed on the ‘trusted mac’s’ list then it adds this traffic to a list that is then sent out to your Gmail account every hour.

So long story short, Spooky monitors for a non-trusted traffic (that has to be defined by user) and reports it when found via E-Mail, see below for example.

A Spooky generated e-mail notifying of unknown traffic (this is an unknown device authenticating on my AP)

 

 

And a simple configuration page with trusted devices added

 

static void ICACHE_FLASH_ATTR handle_pkt(uint8_t* buf, uint16_t len)

So how is this all programmed?
Firstly all is coded in C with Arduino IDE – it’s just great we can code chips with C. It was my first adventure with chip programming and i was very happy i didn’t have to lear myself some ASM or other script-kiddie language around.

The code itself is very easy, ESP8266 has a file system (SPIFF) so the configuration is loaded/stored directly from that file. I could easily skip writing/reading from EEPROM thanks to this.

All these Web server things are built in and available as ready to use classes so i will not go into details with that.

The Sniffer Callback function has to be very quickly executed or WatchDog will restart the chip, so there is no room and space and time for a lot of work to do. Knowing that i am only doing quick 802.11 inspection to find out RSSI, BSSID, Source, Destination and packet type in accordance to the below 802.11 header

802dot11_frame

First i find out the packet type by defining them as below:
Zrzut ekranu 2016-08-04 o 10.58.11

Then i filter out all Beacon and Probe request type of frames as they fly around and Spooky will identify them as unknown traffic. So i’m looking only after DATA and MANAGEMENT type frames that are Directed towards user selected Access Point BSSID. This is how i get only the interesting packets out of the chaos around.

Next i read out Source / Destination by looking into ToDS FromDS fields – these are swapped around in regards to the frame type, so be careful how you read them. An example below:

Zrzut ekranu 2016-08-04 o 10.57.11

Then we get the RSSI of the packet, this is easily taken from ESPRESSIF RxControl header within 802.11.

At this moment Spooky knows:
– What is the Packet type
– What BSSID is packet traveling within
– What source is this packet coming from
– What destination is this packet going to
– RSSI

Now having all above and list of trusted MAC addresses, it is easy for Spooky to identify wether a traffic directed within a defined BSSID is trusted or non trusted.
Anything coming from/to a MAC address that is not on the trusted list and having BSSID as defined by user is considered untrusted – and Spooky will email you the details.
All the rest is trusted so Spooky will ignore the traffic.

So the idea of using Spooky is quite simple:
Whenever you get an untrusted traffic notification from Spooky – you either block the MAC address on your Router (if you can’t identify it) or add it to Spooky Trusted MAC’s list (if you know the MAC).

Thanks to this you keep your AP safe and you are notified of unknown events. The word Notified is important because you don’t have to check your AP logs (who does it anyway?)  for intruders and you can respond to events immediately. And yes i know routers can be configured to block untrusted MACs, but i never used that option.

Spooky went into release and some of the users even made a small TicTac version of it.

 

I expect to make some 3d printed cube for spooky in free time. If you are interested in the code i will be posting it shortly.

 

std::sort – let’s arrange things a bit

So, let’s start with arranging bits and pieces on the blog.

Categories:

Bitcoin/Algotrading – as i am interested in what’s called high frequency trading, algotrading and market data – i decided to place a separate category for tools and apps that i wrote for fin tech sector.

ESP8266 – this is something that i picked up this year for the first time, however many apps were written and it’s one of those things that i like to play with from time to time. It’s time consuming (yeah.. cables, soldering) but gives a lot of opportunities with WiFi built in.

Realtime Rendering – my old hobby and full time interest. I am not doing a lot in this are these days but there are quite a few things i want to mention (gamedev included)

GridMan – this is one of my main interests, distributed computing – mainly number crunching, raytracing and other stuff goes here.

Security – well history likes to repeat. In my highschool time i played with this a lot – buffer overflows, remote exploitation and #hacking overall was my daily fun. Today i am trying to do a step forward by playing with security and trying to link i to other mentioned categories above.

I hope i catched it all 😉