Node-Red WSJTX

WSJT-X monitoring with py_wsjtx & Node-RED

An article about Node-RED by G4WNC in a recent Practical Wireless gave me the push to try and use it in my own radio set-up for alerting and monitoring using a spare Raspberry Pi.

The goal is to receive notifications when my own local radio spots new DXCC on HF bands, any WSPR or JT spots on 6m+ and to plot the 2m JT65b beacons I can hear over time amongst other things.

Prior to this I was only monitoring the beacons using a script and forwarding this to openHAB over MQTT to display alongside some house statistics. This wasn’t too flexible and openHAB is a bit of a burden on the Pi which would randomly hang.

For this project I’m wanting to take inputs from different physical radios & SDR with multiple copies of WSJT-X to display, log certain decodes and alert me in multiple ways if interesting things are seen.

The set-up currently has three radio inputs, each of these has a WSJT-X instance with its own configuration:

  1. HF Radio (IC-7300 and/or FT-817)
  2. VHF Radio (FT-847)
  3. GQRX (IF out of FT-847)

Input 1 is set to whatever I’ve left the HF radios monitoring.

Inputs 2 and 3 are usually set to monitor the two JT65B enabled 2m beacons I can hear from this location, GB3VHF and GB3NGI, using the same antenna. I have this graphed on openHAB but it’s not working great so will be using something else and graphing from the database instead at some point.

WSJT-X can output status messages and decodes over the network to a configured address, this is discussed in a previous blog post where we split the output to AlarmeJT and CQRLOG. We will add a third listener on an extra port, py_wsjtx.

Py_wsjtx is a Python network listener that takes the network output from WSJT-X and displays it in a console, either line by line or a curses interface. I have all of the WSJT-X instances sending their data to a single py_wsjtx instance.

py_wsjtx

As can be seen above, this is really handy for monitoring things from a console rather than the GUI and will highlight new DXCC spots and CQ calls. It can also output the decoded messages to an MQTT broker if configure which comes in really useful for what we’re doing here.

Node-RED allows us to easily take these MQTT inputs, process them in whatever way we want and act upon them. The image below shows the current set-up.

Node-Red WSJTX

The purple boxes are MQTT inputs and outputs, each of these points to an MQTT broker (running on the same Raspberry Pi) and listens or sends messages for a particular topic. Py_wsjtx sends MQTT messages in the format py_wsjtx/WSJT-X radioname/messagetype which makes it easy for us to configure Node-RED to process them in the correct manner for instance filtering by radio or by decode type.

Working from the top row of the flow down:

  1. GQRX and FT847 JT65b beacon decodes are converted to JSON, then they are forwarded on in three ways:
    1. All decodes go to openHAB which is graphing things at the moment, this is shown in the image below and I have it copied to my qrz.com page, I’ll be changing this shortly to something more reliable/configurable.Beacon monitoring
    2. All decodes are logged to a MySQL database which I will use for generating graphs when we stop using openHAB.
    3. If the decodes are above set levels, <5 for GB3NGI and <20 for GB3VHF, then send a post to twitter and to me on a local IRC server.
  2. Next we have DXCC alerts from WSJT-X, if it spots a new country then a message is sent to twitter and IRC with the spot, hopefully I will see it and respond. To make it more interesting I had it ring the shack doorbell, I’ve got two ways to do this, using a HackRF to replay the wireless doorbell, which is a bit of a waste of an expensive SDR, or ringing via a second remote unit using the Pi GPIO pins. The ringer got annoying quickly so it’s now turned off, flashing a light may be better!
  3. Next up we have an input for any WSPR spots on any radio. I’m not doing much with this at the moment other than alerting me on local IRC/twitter if there are any spots on 6m/4m/2m, I don’t often have WSPR listeners on these band though but if I think conditions are looking likely I will switch one of the WSJT-X instances to it.
  4. The solar inverter statistics are sent out on 433.9mhz and I use an RTL SDR dongle to receive them and decode with the program rtl_433. These are then rate limited and forwarded to openHAB as well as being written to the database.
  5. The other MQTT inputs are DHT11 temperature and humidity sensors in the house hooked up to various Pi I have. I’ve not got around to doing anything with these in Node-RED yet but they are currently used by openHAB.

Not much more to say other than it works well for me and I plan on playing about with the flow some more to add some more alerting rules and cutting openHAB out of the solution entirely by graphing the outputs from the database in a more accessible way.

SDR / GNU Radio Tutorials

It doesn’t take too long once you get your hands on an SDR to realise that there’s a lot to learn if you want to move beyond being a consumer of programs and applications others produce. This is where I find myself at the moment, and it is a bit like being back at school but fun.

Here’s a few resources I’ve been reading, watching or have bookmarked in no particular order relating to SDR and GNU Radio. As a beginner in this I can’t fully vouch for their quality but they seem okay!

Analysing An Active RFID Card

The next device from the box of bits was a pair of active RFID cards from a top of the range vehicle security system used around 2001-2003. These tags were required to be on your person to start the car in addition to a key. This is just an analysis of the cards RF transmissions and not the system itself as I don’t own one.

Popping open the two cards we can see they have similar layouts but the newer board on the right doesn’t look as clean and does not have any PCB labelling. The keys each have a serial number sticker on the back of the PCB as well as having it printed on the front of the card.

The two RFID tokens.

The two RFID tokens

There are not many components on the card, we can see an SMD EEPROM along with a small SAW device and the PCB antenna. Looking up the SAW device documentation indicated that it will most likely be transmitting in the 433Mhz ISM band on 433.900.

The batteries in the cards had long run out, so we just hooked them up to an external power supply and monitored with the HackRF and GQRX, this would have worked fine with the RTL SDR device too. We see a short transmission on 433.920Mhz roughly every 5 seconds but it does vary a bit. The frequency reading is wrong below as the screen shot was captured off an IQ replay.

Clear signal.

Clear signal.

On looking at this a bit closer with baudline it unsurprisingly looks to be an on off keying AM signal. Importing a saved audio clip in to Audacity shows the signal, the first capture looked a bit odd with dips and other strangeness but it turned out I was a bit off frequency, fixing the frequency gave a clearer capture.

Unclean off frequency capture.

Unclean off frequency capture.

The first thing I wanted to do was level off the signals to allow for an easier visual comparison, the capture below shows the original signal up top and the altered one below. I used a the same script I used with the previous doorbell project and we can count 78 peaks in each tokens transmission.

Full capture with the second channel prettified

Full capture with the second channel prettified

Comparing transmissions showed that the token is beaconing the same pattern over and over so maybe they are easily repayable as with the doorbell. Interestingly when we look at the two tokens with different serial numbers next to each other we can see a quite long period at the start where they roughly match followed by not too dissimilar patterns after. At this stage I’m guessing the serial numbers are likely just being beaconed following the preamble/synchronisation signal.

The signals patch prior to the red line.

The signals match prior to the red line.

The timings were not particularly consistent even in the initial preamble to allow them to be lined up easily and my previous script to guess binary output based on the period of time (NRZ) was not working perfectly due to this.

There appeared to be two spacing times, one around 18 samples and one around 45 samples but these varied by +/- 10 samples which added up to a large error by the end of the 78 state changes. In order to sort this out I added an option to the script we created for the doorbell to make the sample spacing gaps even. So in this instance a sample space of >30 = 40 and <30 = 20 gives us an easy to compare output.

A resized comparison showing some matching digits

A resized comparison showing matching digits

Now that the WAV file has evenly spaced samples, we can compare them easier as can be seen in the image above of the two tokens beside each other. The initial preamble matches perfectly between each transmission up to the previously indicated red line and would seem to have 5 8 bit components going by the peak count. Handily enough, the tokens 6 digit serial numbers share one digit in the same place and we can see where these match between the two tokens, we can also see this digit repeated again in two other places in the second serial number, these are indicated in the image above.

Now on to the actual decoding! The number of peaks, 78, doesn’t really tally with the indication from the preambles/synchronisation that each part is 8 bits. We now have to turn to a different encoding mechanism commonly used for this sort of cheap OOK device, Manchester encoding. This is better explained somewhere like http://www.quickbuilder.co.uk/qb/articles/, but the idea is that instead of counting the peaks and troughs as a high or low as you might when taking a look at it, we use the transition between high and low values in the middle of a set time period to define the 1 or 0.

As we have already altered our file to have a consistent spacing of 20 or 40 samples we can script reading this Manchester encoding easily by starting at the first state change position, measuring forward 39 samples and detecting whether the value increases or decreases in that time period and repeating this process until the end of the file.

This gives us the following binary output which looks very promising, we can see the matching patterns for the shared digits in the serial numbers between the tokens. The first 5 preamble bytes are the same, 6 is different, and 7 is the same matching the digit 0 in each of the tokens at position 2 with token 2 having two further digit 0’s all in bold.

Token 1: 11111111 01010101 10101010 01010101 00110110 00110001 00110000
Token 2: 11111111 01010101 10101010 01010101 00110110 00110100 00110000
Token 1: 00110001 00111001 00110011 00110010 00000000 10111011
Token 2: 00110000 00110111 00110000 00110011 00110000 11100100

Now converting each of these from binary to hexadecimal we get a match for ASCII characters so by adding an extra decode step to the script processing the file containing the resized signal, we can see the serial numbers they are broadcasting!

stats.py -i gqrx_20140905_134055_433900000-tag-101932.resize.wav -m
Manchester encoding, channel: 0 offset: 40 boundary: 40
ASCII: �U�U6101932

stats.py -i gqrx_20140905_134055_433900000-tag-400703.resize.wav -m
Manchester encoding, channel: 0 offset: 40 boundary: 40
ASCII: �U�U64007030�

So we have managed to go from a captured signal to decoding a beacon from an active RFID card in not too many steps. I’m not going to go in to any more details but it doesn’t seem a particularly secure security device on the evidence we have gathered here.

Now this would be a lot nicer if I could do it with GNU Radio as it is happening, but small steps!

HackRF DoorBell Ringer Part 2 – Replay

Following on from capturing the signal in the previous post was to try a simple replay of the signal to see if it would set the doorbell off as expected.

For this we started off using hackrf_transfer, this receives data in to a file then transmits again from the file, perfect for a quick signal replay.

My first attempt at this was to simply set hackrf_transfer to record the signal centred on the frequency of interest 433.780Mhz.

Firstly capturing to a file:

hackrf_transfer -r 433780000.raw -f 433780000

Then transmitting from the file:

hackrf_transfer -t 433780000.raw -f 433780000 -x 20

Initially I hadn’t included a transmit gain option with -x so wasn’t seeing a signal when transmitting, this could have been reduced a bit.

Now the approach taken above did not work as expected. On looking at the signal transmitted by using an RTL-SDR device to see what was going on we can see a problem.

Hackrf_transfer no offset

The image above shows that capturing and transmitting on the same frequency has resulted in in a continuous signal being sent, the red line, through the middle of our real signal. This signal is on the exact same frequency the doorbell receiver is expecting the real signals and unsurprisingly the AM receiver doesn’t like a strong signal interfering so it didn’t ring.

The simple answer here is to offset the record and transmit frequency a bit, 20khz here, so the DC spike that exists in the middle of our recording isn’t where we’re wanting to transmit.

hackrf_transfer -r 433760000.raw -f 433760000
hackrf_transfer -t 433760000.raw -f 433760000 -x 20

As can be seen in the image below, by shifting the capture and transmit frequency 100khz away, the carrier of the spike is no longer on top of our signal and bing bong the doorbell rings!

hackrf_transfer offset

I tried to replay the signal with GNU Radio gnuradio-companion using an offset frequency and it also worked as expected and rang the doorbell first try.

Simple flow to capture to file.

Simple flow to capture to file.

The GNU Radio waterfall from the  capture

The GNU Radio waterfall from the
capture showing the signal (just)

Simple flow to transmit from file.

Simple flow to transmit from file.

The only problem with this is approach is it’s transmitting the full captured sample which could land us transmitting something unintentionally captured. So while it works and is quick, a better approach would include some filtering, I have a look at this in the next post here.

I’d also like to emulate the signal in GNU Radio as retransmitting isn’t that great but that’s a bit of learning away yet.

I uploaded a short video to youtube demonstrating the doorbell ringing.

HackRF DoorBell Ringer Part 1 – Capture

So this is another hello world style project that’s been done a bunch of times by others but not me, take your simple wireless doorbell and try to make it ring with your SDR 🙂

The bottom of my Friedland doorbell receiver unit helpfully showed it was transmitting on 433Mhz, pretty much as expected. A little listening with GQRX identified the signal:

Original CaptureThe signal appeared to comprise of fast on off bursts, On Off Keying. As suggested by other peoples attempts at things like this, I used baudline for the first time to have a closer look at the bursts and we can see things a bit more clearly.

doorbell2 baudlineThis didn’t help too much beyond giving me a count of the number of bursts per button press, they looked similar enough to the eye too. I’m not sure if there would have been a better way to look at this in baudline but will have a look at it again another time.

I recorded the AM signal audio in GQRX to a WAV file, the bursts were quite clear to the ear. On opening this up in audacity we can see groups of pulses making up a single button press. doorbell3 On zooming in to a button press, we can see these button presses are made up of similar looking groups.

doorbell4And closer again we can see the signals are well defined with the first four peaks equidistant which suggests a preamble/sync. Each of the groups within a button press have the same waveform.

doorbell5Now I tried measuring them but there was no easy way to do this by sight or on paper so I wrote a small python script to take the wav file and alter one channel to be either +1 if > 0 or -1 if < 0 to be more clear. I’ve since changed this to be 0.9 and -0.9 as it’s more readable.

./tobin1.py -i doorbellshort.wav -o out.wav -s
Writing to: out.wav

This worked well and gave me something a bit more readable as can be seen in the output from out.wav, the top channel is the original the bottom channel is the altered one:

doorbell6Now the counting was still awkward so I added a sample count to the script to give the distance between each pulse which enabled me to pull off a stream treating it like binary known as non-return-to-zero. This didn’t however give anything that insightful, but I don’t think there’s going to be much point in going any further with this on a doorbell.

This is the output from above script with the leading 0 removed, we can see the 10101010 preamble/sync noted above:

1010101000100000100000100000010000100001000000001001000000100001000001
000001000001000000100000100001000001

The next step will be to try and record and replay the request using hackrf_transfer.

Osmocom Spectrum Browser & Signal Generator

There are a few basic software packages or SDR that can be exceptionally handy and they don’t require much work to install, here’s a couple from gr-osmocom that are likely installed already if you are up and running already.

The osmocom spectrum browser, osmocom_fft, is nice and quick to use and should have been installed already as part of the gr-osmosdr package:

Screenshot - 150814 - 09:51:53

Another handy one that should also be available already is a signal generator, osmocom_siggen, that will allow you to generate some basic signals. Take care to make sure you are transmitting on a frequency you are allowed to, the application starts transmitting straight away so set the frequency on the command line. If you don’t exit it properly it will stick on transmit.

Screenshot - 150814 - 10:47:04

The sweep generated above with the HackRF received on a rtl-sdr dongle:

Sweep

This was my first transmit test, so the HackRF transmits, yay 🙂

Another handy application is osmocom_spectrum_sense. This will give power readings for a frequency or within a range in the console so could be handy for quickly scanning or checking for a strong signal. It can be run over a range or on a single frequency with a 0 range. In the single example below we first get a reading of 15 from a broadcast fm station, the second reading of 2.7 is from 1mhz under the broadcast station where there is just noise.

2014-08-16 20:55:16.523750 center_freq 105650000.0 freq 101900000.0 power_db 15.5508674659 noise_floor_db -84.4425523316

2014-08-16 20:55:37.790722 center_freq 104650000.0 freq 100900000.0 power_db 2.76116236335 noise_floor_db -84.432340649

Toy Delivery – HackRF One

So after a long wait we finally receive the HackRF One plus its telescopic antenna in the post!

HackRF – Long time a coming!

Having played plenty with RTL-SDR dongles already, it was just a case of plug and play to see it working with GQRX.

We started off with gawking at 20Mhz being displayed instead of the 2Mhz of the TV dongle and briefly listening to broadcast FM which is like the “hello world” first step of SDR reception to see it was working as expected.

Opening the HackRF case required using a small screw driver to pop the holding clips in and working the way around after loosening the SMA nuts, it didn’t make too much of a mess thankfully.

HackRF with the lid off.

We also took delivery of the RF shield at the same time, it’s just a frame with a cover that needs to be soldered on to the square on the top right. The shield looks to be easy enough to solder on but I’m going to leave this for after the first 24 hours 🙂

Now to learn SDR..