Node-RED & IRC with ON4KST Chat

Background

Continuing with using Node-RED to handle messaging for radio related things I’ve created some flows for using with the interactive chat service at www.on4kst.com.

This web and Telnet based messaging service is invaluable for VHF+ users for contests, scheduling and band related chat.

The web interface is great for using while at home but when out portable using a mobile device it is a bit fiddly. Additionally, during contest/activity days, the messages can be flying by so quickly a page of text can scroll past in a moment and it’s easy to miss out messages even when directed at you with the /cq prefix. After my last outing portable I returned home to see I had been called but hadn’t noticed.

Prompted by some discussion of this on the ukmicrowaves mailing list and a mention of IRC which I use a lot I thought I should have a go at sorting something out for myself and others if interested.

There are applications that can be used to access KST chat such as the contest logging software tucnak for raw access to the telnet interface and the Windows application kst2me which runs wellin Linux with Wine. Kst2me is great and would be the likely solution for most people with a computer to hand but it’s doesn’t solve my mobile device and alerting requirements.

Using Node-RED & IRC

By pulling the ‘kst chat in to Node-RED my intention was to filter messages to channels on an IRC service based on their content:

  • A channel for all chat, effectively a mirror of whatever chat it’s connected to.
  • A channel for nearby DXCC entries. As I’ll be using this for microwave outings I have some filtering if required during busy periods.
  • A channel for direct messages sent to me with the /cq prefix so I have everything directed to me in one area.

I also want to be able to send messages to KST from my own IRC client via the channels above. A two way set-up is intended for a single user and care needs to be taken to ensure that messages sent to KST are coming from the legitimate logged in user.

I’ve used IRC here because I am comfortable with it and have clients connected to IRC servers 24/7 and can connect to these clients from any device to pick up where I left previously. This means I can leave the Node-RED flow running at home on a contest day and connect to the IRC server as I need and have a full view of everything that has happened in each of the channels. This also means I don’t have to worry about disconnections or leaving it running all the time running batteries down.

Node-RED is however  extremely flexible and allows the messages to be sent to pretty much anything such as an MQTT broker, SMS messages, DMR SMS messages, Web APIs, TCP/UDP servers etc so the use of IRC here should just be taken as an example use.

I have also set the flow up to have CQ messages directed to me on KST to send me a direct message on Twitter to gain my attention on my mobile device when not viewing the IRC channel. I could have used SMS here but the Twitter application with alerts works as well and doesn’t require using a paid for SMS service.

Interfacing With KST Chat – Login

There are two interfaces to the KST chat, the web interface and the Telnet interface, there’s no API that I’m aware of. In this instance I’ve used the Telnet interface to interact with the service as it’s the least complicated for scraping purposes.

In order to log in via Telnet we need our username, password and the band chat we wish to connect to, once these have been provided we are placed in to the selected band chat room. The Telnet outout is shown below connecting to the quiet Warch chat number 11 for test purposes.

Free text at this point will appear in the chat channel and commands need to be prefixed with a forward slash, for instance /quit.

We can log in to the Telnet interface easily with Node-RED using built in nodes and very little customization.

The flow above will initiate the connection to the ON4KST Telnet interface and log in. The nodes do the following:

  1. The first node is set to execute on start-up and contains three values, username, password and the chat number.
  2. The function node takes the user, pass and chat, splits them and sends the username to output 1, the password to output 2 and the chat room number to output 3.
  3. These three outputs are then connected to a TCP Request node, this node takes a hostname and port number, http://www.on4kst.info:23000 in this instance. This node allows both input to and output from the TCP connection.

We also have two delay nodes, as the user, password and chat are entered separately and prompted one after another, we insert a delay after the username and password to allow the next prompt to be displayed. If they are all sent at the same time the ‘kst Telnet interface will not accept them.

If we connect a debug node to the output of the TCP request node we will see the output from the telnet session.

Remember that anything sent to the interface will be immediately sent to the channel so take care not to send your password to an already connected interface. This has caught me out already!

Interfacing With KST Chat – Output

The output from the TCP request node is fed straight in to a function node with four outputs. The function node filters and alters data and directs it to the correct outputs. The four outputs link the function node to three IRC channels and one Twitter output.

The Process Output function node does a number of things. It firstly converts the utf8 input from the TCP node in to a string and strips out newline characters. This allows us to work with it easily.

It then splits the incoming messages up if they are identified as normal messages. The message format is as follows with TOCALLSIGN being the optional CQ message and the NAME being of a variable length:

TIME FROMCALLSIGN FROMNAME > (TOCALLSIGN) MESSAGETEXT

Once we have each of the above split in to variables we start to make decisions as to what to do with the incoming messages based on regex matches.

We have three IRC channels set up for this test, one for all KST chat, one for local KST chat and one for CQ messages directed at us.

If a message matches our callsign, when someone send’s a /CQ message to us, the message is sent to all of the four function outputs resulting in it appearing in all IRC channels and a message to our user on Twitter.

If a message matches a list of prefix’s local to me (2/M/G/EI) they are filtered to the local area IRC channel and also to the all KST IRC channel. This means I have filtered IRC channel containing British Isles chat, which for me in GI matches what I’m able to work, if I was in a position to work other countries it’s easy to add them to the list.

If a message is not a CQ directed to me or in the list of filtered prefixes it is placed in the all KST channel which can be monitored when traffic isn’t heavy.

Interfacing With KST Chat – Input

I have two Input types set up to send text to the KST Telnet interface.

The first was set up for testing purposes in the Node-RED interface and consists of buttons that will send commands when clicked. These buttons send the text to a function node than adds line breaks then forwards them to the TCP request node input.

The second takes input from the IRC channels I’ve configured Node-RED to connect to and carries out some validation of observed messages, processes them then forwards to the add line feed node for sending in to the TCP node.

The Process IRC node checks incoming messages and matches them with some rules. It requires that incoming messages originate from the channels the IRC node is connected to and from a username that matches my IRC clients username.

Once this incoming messages are validated, the node checks the content for commands before actin. For instance it will parse “CQ CALLSIGN Good evening”, prefix the CQ with a forward slash and send it to the telnet service  and it will appear as a valid “/CQ” command on the KSTchat.

Wrap Up

IRC is just a single communication method here, we could do this using any other messaging platform and integrate in to applications easily enough and make this more interesting easily enough. Node-RED allows easy control of messaging and simple coding in JavaScript to manipulate things.

I hope to try the described setup while portable with my microwave setup some time and imagine that the messaging ability to alert me to calls, and the use of a persistent IRC client to interact with people should allow me to keep track of KST chat in a manner I’m more comfortable with.

My code is a horrible mess but I can forward it to anyone that would like to try something like this out. Here’s the whole thing:

Please have a look at some other Node-RED posts for more amateur radio uses if interested in the above:

 

Advertisements

A Node-RED Module for DMR SMS using an openSPOT

I’ve been using Node-RED to send DMR SMS alerts for about 6 months now as described in the earlier post here. This has worked perfectly until now but I would rather have a single configurable node instead of a collection of nodes containing code, so here it is.

Once installed, the node will be available in the social section and can just be dragged in to the flow. The image below shows the previous setup at the top where a number of different nodes were used and the new setup with just the one node and an input.

Once the node is in the flow, the openSPOT server will need to be configured within it by selecting the edit icon next to the openSPOT drop down. The defaults in the server section should suit a basic default install on a local network.

Each SMS node can be configured with all of useful options as described in the HTTP API documentation. The main requirement is that a destination ID is entered, this will be your DMR ID if you’re using this for sending messages to yourself. The rest of the configuration options are as described in the API documentation.

Once configured, any string payload sent to this node should send it to the radio.

The module is available at https://www.npmjs.com/package/node-red-contrib-openspotsms and can be installed in an existing Node-RED instance by changing in to the instance directory and running:

npm install node-red-contrib-openspotsms

It’s also available on github if you’d rather grab it manually. I’m no coder so it’s very messy but it seems to work for now, I’ll get through the todo list in time!

Christmas With RfCat

Having lost the RF remote controlling the power to the Christmas tree lights a couple of times I thought it would be a good time to try RfCat with the YARD stick one that has been sitting neglected since I bought it a while back. Nothing here that hasn’t been written up 101 times by others!

The RF controller has multiple buttons for switching the different mains switches on and off individually or all on/off at the same time. As these are only button presses I’ve just aimed to replay the transmissions not really caring about the content.

The signals are on off keyed and repeated when holding the button, looking at these in audacity was pretty much as described in earlier posts here and here. It’s been a while since I’ve done this sort of thing so was a decent refresher project.

Additionally this time around I’ve used inspectrum with the IQ output from GQRX which along with its cursor option makes getting the timings correct a lot easier.

screenshot-inspectrum-gqrx_20161209_103307_434173979_1800000_fc-raw

Inspectrum with cursors enabled

Using the stats.py script as described in earlier posts with a wav file to make a guess gave the following output (not from the same button as in the screenshot):

./stats.py -i ../allon.wav -c 1 -t 30
 The whole lot: 000000010101111110000001001001010101001010001010010101010010101010100100100101001010010100100100101001000100101001010100101001

After removing the initial zeros this matched up nearly perfectly with the inspectrum/audacity output. I added an additional 1 and 0 to the long on/off periods at the start to have it match better.

Now I needed to send this out using the RfCat with the above input. This was surprisingly easy to get working in Python with RfCat and worked the first time after following some online examples.

Firstly I needed to convert the stream to use with RfCat which gave:

\xaf\xe0\x22\x2a\x8a\x28\xaa\x2a\xa2\x22\x8a\x28\x88\xa2\x22\xaa\x22\x20

Then to send this in RfCat:

d.setMdmModulation(MOD_ASK_OOK)
d.setFreq(434400000)
d.setMaxPower()
d.setMdmSyncMode(0)
d.setMdmDRate((int)(1.0/0.000450))
d.RFxmit("\xaf\xe0\x22\x2a\x8a\x28\xaa\x2a\xa2\x22\x8a\x28\x88\xa2\x22\xaa\x22\x20")

The timing was taken from the inspectrum symbol period as pictured above, modulation and frequency self explanatory.

Comparing the original signal to the one sent by rfcat they are pretty much the same and the switches accept the signals as intended.

screenshot-59

Original signal top, RfCat bottom

I wrapped this all up in a small python script containing all the on/off values and am now using it to turn the lights on and off with a cron job for the month.

Another way to receive the data and skip the iq/wav analysis full stop is just to use RfCat to receive the signal.

I did try this first and didn’t get too far before trying the above instead but it was easier after having done all of the above and having the timings correct and after reading a great post here which is a good guide to this sort of thing: http://andrewmohawk.com/2015/08/31/hacking-fixed-key-remotes-with-only-rfcat/

With the same RfCat settings as above we run d.RFlisten() and get the following:

screenshot-60

RfCat button press output

There’s some obvious repeated data in the screen shot. Taking a few of the repeats out and converting to binary we end up with a string that matches the output from stats.py but for a slightly longer preamble but this doesn’t make a difference to the outcome when we retransmit the above, the switch switches as expected.

It is a lot quicker and less fiddly to just do all of the above with RfCat entirely but had I not worked through it from the method I knew already I’d have struggled to get it working as quickly. Next time with a little better understanding hopefully it will be easier.

Analysing An Active RFID Card

The next device from the box of bits was a pair of active RFID cards from a top of the range vehicle security system used around 2001-2003. These tags were required to be on your person to start the car in addition to a key. This is just an analysis of the cards RF transmissions and not the system itself as I don’t own one.

Popping open the two cards we can see they have similar layouts but the newer board on the right doesn’t look as clean and does not have any PCB labelling. The keys each have a serial number sticker on the back of the PCB as well as having it printed on the front of the card.

The two RFID tokens.

The two RFID tokens

There are not many components on the card, we can see an SMD EEPROM along with a small SAW device and the PCB antenna. Looking up the SAW device documentation indicated that it will most likely be transmitting in the 433Mhz ISM band on 433.900.

The batteries in the cards had long run out, so we just hooked them up to an external power supply and monitored with the HackRF and GQRX, this would have worked fine with the RTL SDR device too. We see a short transmission on 433.920Mhz roughly every 5 seconds but it does vary a bit. The frequency reading is wrong below as the screen shot was captured off an IQ replay.

Clear signal.

Clear signal.

On looking at this a bit closer with baudline it unsurprisingly looks to be an on off keying AM signal. Importing a saved audio clip in to Audacity shows the signal, the first capture looked a bit odd with dips and other strangeness but it turned out I was a bit off frequency, fixing the frequency gave a clearer capture.

Unclean off frequency capture.

Unclean off frequency capture.

The first thing I wanted to do was level off the signals to allow for an easier visual comparison, the capture below shows the original signal up top and the altered one below. I used a the same script I used with the previous doorbell project and we can count 78 peaks in each tokens transmission.

Full capture with the second channel prettified

Full capture with the second channel prettified

Comparing transmissions showed that the token is beaconing the same pattern over and over so maybe they are easily repayable as with the doorbell. Interestingly when we look at the two tokens with different serial numbers next to each other we can see a quite long period at the start where they roughly match followed by not too dissimilar patterns after. At this stage I’m guessing the serial numbers are likely just being beaconed following the preamble/synchronisation signal.

The signals patch prior to the red line.

The signals match prior to the red line.

The timings were not particularly consistent even in the initial preamble to allow them to be lined up easily and my previous script to guess binary output based on the period of time (NRZ) was not working perfectly due to this.

There appeared to be two spacing times, one around 18 samples and one around 45 samples but these varied by +/- 10 samples which added up to a large error by the end of the 78 state changes. In order to sort this out I added an option to the script we created for the doorbell to make the sample spacing gaps even. So in this instance a sample space of >30 = 40 and <30 = 20 gives us an easy to compare output.

A resized comparison showing some matching digits

A resized comparison showing matching digits

Now that the WAV file has evenly spaced samples, we can compare them easier as can be seen in the image above of the two tokens beside each other. The initial preamble matches perfectly between each transmission up to the previously indicated red line and would seem to have 5 8 bit components going by the peak count. Handily enough, the tokens 6 digit serial numbers share one digit in the same place and we can see where these match between the two tokens, we can also see this digit repeated again in two other places in the second serial number, these are indicated in the image above.

Now on to the actual decoding! The number of peaks, 78, doesn’t really tally with the indication from the preambles/synchronisation that each part is 8 bits. We now have to turn to a different encoding mechanism commonly used for this sort of cheap OOK device, Manchester encoding. This is better explained somewhere like http://www.quickbuilder.co.uk/qb/articles/, but the idea is that instead of counting the peaks and troughs as a high or low as you might when taking a look at it, we use the transition between high and low values in the middle of a set time period to define the 1 or 0.

As we have already altered our file to have a consistent spacing of 20 or 40 samples we can script reading this Manchester encoding easily by starting at the first state change position, measuring forward 39 samples and detecting whether the value increases or decreases in that time period and repeating this process until the end of the file.

This gives us the following binary output which looks very promising, we can see the matching patterns for the shared digits in the serial numbers between the tokens. The first 5 preamble bytes are the same, 6 is different, and 7 is the same matching the digit 0 in each of the tokens at position 2 with token 2 having two further digit 0’s all in bold.

Token 1: 11111111 01010101 10101010 01010101 00110110 00110001 00110000
Token 2: 11111111 01010101 10101010 01010101 00110110 00110100 00110000
Token 1: 00110001 00111001 00110011 00110010 00000000 10111011
Token 2: 00110000 00110111 00110000 00110011 00110000 11100100

Now converting each of these from binary to hexadecimal we get a match for ASCII characters so by adding an extra decode step to the script processing the file containing the resized signal, we can see the serial numbers they are broadcasting!

stats.py -i gqrx_20140905_134055_433900000-tag-101932.resize.wav -m
Manchester encoding, channel: 0 offset: 40 boundary: 40
ASCII: �U�U6101932

stats.py -i gqrx_20140905_134055_433900000-tag-400703.resize.wav -m
Manchester encoding, channel: 0 offset: 40 boundary: 40
ASCII: �U�U64007030�

So we have managed to go from a captured signal to decoding a beacon from an active RFID card in not too many steps. I’m not going to go in to any more details but it doesn’t seem a particularly secure security device on the evidence we have gathered here.

Now this would be a lot nicer if I could do it with GNU Radio as it is happening, but small steps!

HackRF DoorBell Ringer Part 2 – Replay

Following on from capturing the signal in the previous post was to try a simple replay of the signal to see if it would set the doorbell off as expected.

For this we started off using hackrf_transfer, this receives data in to a file then transmits again from the file, perfect for a quick signal replay.

My first attempt at this was to simply set hackrf_transfer to record the signal centred on the frequency of interest 433.780Mhz.

Firstly capturing to a file:

hackrf_transfer -r 433780000.raw -f 433780000

Then transmitting from the file:

hackrf_transfer -t 433780000.raw -f 433780000 -x 20

Initially I hadn’t included a transmit gain option with -x so wasn’t seeing a signal when transmitting, this could have been reduced a bit.

Now the approach taken above did not work as expected. On looking at the signal transmitted by using an RTL-SDR device to see what was going on we can see a problem.

Hackrf_transfer no offset

The image above shows that capturing and transmitting on the same frequency has resulted in in a continuous signal being sent, the red line, through the middle of our real signal. This signal is on the exact same frequency the doorbell receiver is expecting the real signals and unsurprisingly the AM receiver doesn’t like a strong signal interfering so it didn’t ring.

The simple answer here is to offset the record and transmit frequency a bit, 20khz here, so the DC spike that exists in the middle of our recording isn’t where we’re wanting to transmit.

hackrf_transfer -r 433760000.raw -f 433760000
hackrf_transfer -t 433760000.raw -f 433760000 -x 20

As can be seen in the image below, by shifting the capture and transmit frequency 100khz away, the carrier of the spike is no longer on top of our signal and bing bong the doorbell rings!

hackrf_transfer offset

I tried to replay the signal with GNU Radio gnuradio-companion using an offset frequency and it also worked as expected and rang the doorbell first try.

Simple flow to capture to file.

Simple flow to capture to file.

The GNU Radio waterfall from the  capture

The GNU Radio waterfall from the
capture showing the signal (just)

Simple flow to transmit from file.

Simple flow to transmit from file.

The only problem with this is approach is it’s transmitting the full captured sample which could land us transmitting something unintentionally captured. So while it works and is quick, a better approach would include some filtering, I have a look at this in the next post here.

I’d also like to emulate the signal in GNU Radio as retransmitting isn’t that great but that’s a bit of learning away yet.

I uploaded a short video to youtube demonstrating the doorbell ringing.

HackRF DoorBell Ringer Part 1 – Capture

So this is another hello world style project that’s been done a bunch of times by others but not me, take your simple wireless doorbell and try to make it ring with your SDR 🙂

The bottom of my Friedland doorbell receiver unit helpfully showed it was transmitting on 433Mhz, pretty much as expected. A little listening with GQRX identified the signal:

Original CaptureThe signal appeared to comprise of fast on off bursts, On Off Keying. As suggested by other peoples attempts at things like this, I used baudline for the first time to have a closer look at the bursts and we can see things a bit more clearly.

doorbell2 baudlineThis didn’t help too much beyond giving me a count of the number of bursts per button press, they looked similar enough to the eye too. I’m not sure if there would have been a better way to look at this in baudline but will have a look at it again another time.

I recorded the AM signal audio in GQRX to a WAV file, the bursts were quite clear to the ear. On opening this up in audacity we can see groups of pulses making up a single button press. doorbell3 On zooming in to a button press, we can see these button presses are made up of similar looking groups.

doorbell4And closer again we can see the signals are well defined with the first four peaks equidistant which suggests a preamble/sync. Each of the groups within a button press have the same waveform.

doorbell5Now I tried measuring them but there was no easy way to do this by sight or on paper so I wrote a small python script to take the wav file and alter one channel to be either +1 if > 0 or -1 if < 0 to be more clear. I’ve since changed this to be 0.9 and -0.9 as it’s more readable.

./tobin1.py -i doorbellshort.wav -o out.wav -s
Writing to: out.wav

This worked well and gave me something a bit more readable as can be seen in the output from out.wav, the top channel is the original the bottom channel is the altered one:

doorbell6Now the counting was still awkward so I added a sample count to the script to give the distance between each pulse which enabled me to pull off a stream treating it like binary known as non-return-to-zero. This didn’t however give anything that insightful, but I don’t think there’s going to be much point in going any further with this on a doorbell.

This is the output from above script with the leading 0 removed, we can see the 10101010 preamble/sync noted above:

1010101000100000100000100000010000100001000000001001000000100001000001
000001000001000000100000100001000001

The next step will be to try and record and replay the request using hackrf_transfer.

Osmocom Spectrum Browser & Signal Generator

There are a few basic software packages or SDR that can be exceptionally handy and they don’t require much work to install, here’s a couple from gr-osmocom that are likely installed already if you are up and running already.

The osmocom spectrum browser, osmocom_fft, is nice and quick to use and should have been installed already as part of the gr-osmosdr package:

Screenshot - 150814 - 09:51:53

Another handy one that should also be available already is a signal generator, osmocom_siggen, that will allow you to generate some basic signals. Take care to make sure you are transmitting on a frequency you are allowed to, the application starts transmitting straight away so set the frequency on the command line. If you don’t exit it properly it will stick on transmit.

Screenshot - 150814 - 10:47:04

The sweep generated above with the HackRF received on a rtl-sdr dongle:

Sweep

This was my first transmit test, so the HackRF transmits, yay 🙂

Another handy application is osmocom_spectrum_sense. This will give power readings for a frequency or within a range in the console so could be handy for quickly scanning or checking for a strong signal. It can be run over a range or on a single frequency with a 0 range. In the single example below we first get a reading of 15 from a broadcast fm station, the second reading of 2.7 is from 1mhz under the broadcast station where there is just noise.

2014-08-16 20:55:16.523750 center_freq 105650000.0 freq 101900000.0 power_db 15.5508674659 noise_floor_db -84.4425523316

2014-08-16 20:55:37.790722 center_freq 104650000.0 freq 100900000.0 power_db 2.76116236335 noise_floor_db -84.432340649