ICOM IC-9700 and IC-7300 coexisting in Linux

I’ve just added an IC-9700 to the desk alongside the IC-7300, they look very nice together and very similar. Unfortunately the Audio and Serial devices also look very similar in Linux!

Both radio sound cards appear as PCM2901 making it hard to differentiate between them, and my previous udev rule to create a symlink for the IC-7300 ttyUSB device on /dev/ic7300 picked up the 9700 instead.

My previous udev serial rule created a symlink to /dev/ttyUSBx from /dev/ic7300 when a device with the 7300’s serial adapter idVendor and idProduct was seen like so:

SUBSYSTEM==”tty”, ATTRS{idVendor}==”10c4″, ATTRS{idProduct}==”ea60″ SYMLINK+=”ic7300″

This worked fine until the 9700 was plugged in as it also has the same idVendor and idProduct number. We can narrow this down easily as another available attribute is “serial” which contains the radios unique serial number along with its name.

We can get the serial numbers using udevadm (or better with lsusb below) against each of the /dev/ttyUSBx devices, the 9700 has two serial devices, the first works fine with hamlib for control but not sure what the second serial device at the moment.

$ udevadm info –attribute-walk –path=/sys/bus/usb-serial/devices/ttyUSB4 | grep IC-
ATTRS{serial}==”IC-9700 13000000 A”

$ udevadm info –attribute-walk –path=/sys/bus/usb-serial/devices/ttyUSB5 | grep IC-
ATTRS{serial}==”IC-9700 13000000 B”

$ udevadm info –attribute-walk –path=/sys/bus/usb-serial/devices/ttyUSB1 | grep IC-
ATTRS{serial}==”IC-7300 03000000″

An easier way to get the serial numbers is by running lsusb as root as follows, thanks to PA3MET for the pointer!

# lsusb -vvvvv | egrep "9700|7300"
iSerial 3 IC-9700 13000000 B
iSerial 3 IC-9700 13000000 A
iSerial 3 IC-7300 03000000

We can take these and create appropriate udev rules for adding symlinks for the radios serial devices including the unique serial numbers. Here is an extract from my /etc/udev/rules.d/99-hamlib.rules file:

SUBSYSTEM==”tty”, ATTRS{idVendor}==”10c4″, ATTRS{idProduct}==”ea60″, ATTRS{serial}==”IC-7300 03000000″, SYMLINK+=”ic7300″

SUBSYSTEM==”tty”, ATTRS{idVendor}==”10c4″, ATTRS{idProduct}==”ea60″, ATTRS{serial}==”IC-9700 13000000 A”, SYMLINK+=”ic9700a”

SUBSYSTEM==”tty”, ATTRS{idVendor}==”10c4″, ATTRS{idProduct}==”ea60″, ATTRS{serial}==”IC-9700 13000000 B”, SYMLINK+=”ic9700b”

Take care if copying the above text as the quotation marks are displaying incorrectly in WordPress. If you copy it as is you will need to replace them all with proper quotation marks.

A reload of udev rules with “udevadm trigger” and we have our serial devices available from convenient symlinks so no chasing about ttyUSB device names:

$ ls -l /dev/ic*
lrwxrwxrwx 1 root root 7 Apr 4 14:27 /dev/ic7300 -> ttyUSB1
lrwxrwxrwx 1 root root 7 Apr 4 14:27 /dev/ic9700a -> ttyUSB4
lrwxrwxrwx 1 root root 7 Apr 4 14:27 /dev/ic9700b -> ttyUSB5

Next up is the sound cards. I use quite a lot of different programs and need to easily switch and adjust my audio inputs/outputs using pavucontrol. The problem here is both audio devices have the same name, PCM2901, meaning I can’t easily tell what sound card belongs to what radio.

There’s no way to differentiate the sound cards like with the serial above as they return the exact same information and attributes. The only way to differentiate them from what I can see is with the physical USB port they are plugged in to. This is fine here as it’s a desktop and they will remain plugged in to the same ports. If you plug the radios in to a different USB socket you will need to update the paths again.

We can list the devices with the following command, this shows both the IC-7300 and IC-9700 sound cards. I’ve removed some other sound cards from the output, as we’re just looking for the “Burr-Brown_from_TI_USB_Audio_CODEC” entries here

$ pacmd list-sources | egrep “name:|sysfs”

name: <alsa_output.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00.analog-stereo.2.monitor>
sysfs.path = “/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2.4/3-2.4:1.0/sound/card4”
name: <alsa_input.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00.analog-stereo.2>
sysfs.path = “/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2.4/3-2.4:1.0/sound/card4″
name: <alsa_output.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00.analog-stereo.3.monitor>
sysfs.path = “/devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3.4/3-3.4:1.0/sound/card5”
name: <alsa_input.usb-Burr-Brown_from_TI_USB_Audio_CODEC-00.analog-stereo.3>
sysfs.path = “/devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3.4/3-3.4:1.0/sound/card5″

We are looking to extract the unique portion from device paths above, here we can see the paths differ at 3-3/3-3.4/3-3.4 and 3-2/3-2.4/3-2.4. We will want to identify which entry belongs to which radio so run it with the USB disconnected then connected to identify which device matches which radio.

We can then run the following to apply a device description of IC9700 to the IC-9700 sound card source and sink which will show against the audio device in pulse applications instead of PCM2901.

The 3-3.4 below is referencing the end of the USB port discovered above.

pacmd update-source-proplist $(pacmd list-sources | egrep “name:.*Burr-Brown*|3-3.4” | grep -B 1 sysfs.path | grep name | sed “s/.*<$.*$>/\1/” | grep -v monitor) device.description=IC9700

pacmd update-sink-proplist $(pacmd list-sinks | egrep “name:.*Burr-Brown*|3-3.4” | grep -B 1 sysfs.path | grep name | sed “s/.*<$.*$>/\1/”) device.description=IC9700

And the same for the IC7300 with 3-2.4:

pacmd update-source-proplist $(pacmd list-sources | egrep “name:.*Burr-Brown*|3-2.4” | grep -B 1 sysfs.path | grep name | sed “s/.*<$.*$>/\1/” | grep -v monitor) device.description=IC7300

pacmd update-sink-proplist $(pacmd list-sinks | egrep “name:.*Burr-Brown*|3-2.4” | grep -B 1 sysfs.path | grep name | sed “s/.*<$.*$>/\1/”) device.description=IC7300

The audio devices should now be available using the name we set above in pavucontrol and other applications. This can be put in a script to run manually or at system startup.

Node-RED & IRC with ON4KST Chat

Background

Continuing with using Node-RED to handle messaging for radio related things I’ve created some flows for using with the interactive chat service at www.on4kst.com.

This web and Telnet based messaging service is invaluable for VHF+ users for contests, scheduling and band related chat.

The web interface is great for using while at home but when out portable using a mobile device it is a bit fiddly. Additionally, during contest/activity days, the messages can be flying by so quickly a page of text can scroll past in a moment and it’s easy to miss out messages even when directed at you with the /cq prefix. After my last outing portable I returned home to see I had been called but hadn’t noticed.

Prompted by some discussion of this on the ukmicrowaves mailing list and a mention of IRC which I use a lot I thought I should have a go at sorting something out for myself and others if interested.

There are applications that can be used to access KST chat such as the contest logging software tucnak for raw access to the telnet interface and the Windows application kst2me which runs wellin Linux with Wine. Kst2me is great and would be the likely solution for most people with a computer to hand but it’s doesn’t solve my mobile device and alerting requirements.

Using Node-RED & IRC

By pulling the ‘kst chat in to Node-RED my intention was to filter messages to channels on an IRC service based on their content:

A channel for all chat, effectively a mirror of whatever chat it’s connected to.
A channel for nearby DXCC entries. As I’ll be using this for microwave outings I have some filtering if required during busy periods.
A channel for direct messages sent to me with the /cq prefix so I have everything directed to me in one area.

I also want to be able to send messages to KST from my own IRC client via the channels above. A two way set-up is intended for a single user and care needs to be taken to ensure that messages sent to KST are coming from the legitimate logged in user.

I’ve used IRC here because I am comfortable with it and have clients connected to IRC servers 24/7 and can connect to these clients from any device to pick up where I left previously. This means I can leave the Node-RED flow running at home on a contest day and connect to the IRC server as I need and have a full view of everything that has happened in each of the channels. This also means I don’t have to worry about disconnections or leaving it running all the time running batteries down.

Node-RED is however extremely flexible and allows the messages to be sent to pretty much anything such as an MQTT broker, SMS messages, DMR SMS messages, Web APIs, TCP/UDP servers etc so the use of IRC here should just be taken as an example use.

I have also set the flow up to have CQ messages directed to me on KST to send me a direct message on Twitter to gain my attention on my mobile device when not viewing the IRC channel. I could have used SMS here but the Twitter application with alerts works as well and doesn’t require using a paid for SMS service.

Interfacing With KST Chat – Login

There are two interfaces to the KST chat, the web interface and the Telnet interface, there’s no API that I’m aware of. In this instance I’ve used the Telnet interface to interact with the service as it’s the least complicated for scraping purposes.

In order to log in via Telnet we need our username, password and the band chat we wish to connect to, once these have been provided we are placed in to the selected band chat room. The Telnet outout is shown below connecting to the quiet Warch chat number 11 for test purposes.

Free text at this point will appear in the chat channel and commands need to be prefixed with a forward slash, for instance /quit.

We can log in to the Telnet interface easily with Node-RED using built in nodes and very little customization.

The flow above will initiate the connection to the ON4KST Telnet interface and log in. The nodes do the following:

The first node is set to execute on start-up and contains three values, username, password and the chat number.
The function node takes the user, pass and chat, splits them and sends the username to output 1, the password to output 2 and the chat room number to output 3.
These three outputs are then connected to a TCP Request node, this node takes a hostname and port number, http://www.on4kst.info:23000 in this instance. This node allows both input to and output from the TCP connection.

We also have two delay nodes, as the user, password and chat are entered separately and prompted one after another, we insert a delay after the username and password to allow the next prompt to be displayed. If they are all sent at the same time the ‘kst Telnet interface will not accept them.

If we connect a debug node to the output of the TCP request node we will see the output from the telnet session.

Remember that anything sent to the interface will be immediately sent to the channel so take care not to send your password to an already connected interface. This has caught me out already!

Interfacing With KST Chat – Output

The output from the TCP request node is fed straight in to a function node with four outputs. The function node filters and alters data and directs it to the correct outputs. The four outputs link the function node to three IRC channels and one Twitter output.

The Process Output function node does a number of things. It firstly converts the utf8 input from the TCP node in to a string and strips out newline characters. This allows us to work with it easily.

It then splits the incoming messages up if they are identified as normal messages. The message format is as follows with TOCALLSIGN being the optional CQ message and the NAME being of a variable length:

TIME FROMCALLSIGN FROMNAME > (TOCALLSIGN) MESSAGETEXT

Once we have each of the above split in to variables we start to make decisions as to what to do with the incoming messages based on regex matches.

We have three IRC channels set up for this test, one for all KST chat, one for local KST chat and one for CQ messages directed at us.

If a message matches our callsign, when someone send’s a /CQ message to us, the message is sent to all of the four function outputs resulting in it appearing in all IRC channels and a message to our user on Twitter.

If a message matches a list of prefix’s local to me (2/M/G/EI) they are filtered to the local area IRC channel and also to the all KST IRC channel. This means I have filtered IRC channel containing British Isles chat, which for me in GI matches what I’m able to work, if I was in a position to work other countries it’s easy to add them to the list.

If a message is not a CQ directed to me or in the list of filtered prefixes it is placed in the all KST channel which can be monitored when traffic isn’t heavy.

Interfacing With KST Chat – Input

I have two Input types set up to send text to the KST Telnet interface.

The first was set up for testing purposes in the Node-RED interface and consists of buttons that will send commands when clicked. These buttons send the text to a function node than adds line breaks then forwards them to the TCP request node input.

The second takes input from the IRC channels I’ve configured Node-RED to connect to and carries out some validation of observed messages, processes them then forwards to the add line feed node for sending in to the TCP node.

The Process IRC node checks incoming messages and matches them with some rules. It requires that incoming messages originate from the channels the IRC node is connected to and from a username that matches my IRC clients username.

Once this incoming messages are validated, the node checks the content for commands before actin. For instance it will parse “CQ CALLSIGN Good evening”, prefix the CQ with a forward slash and send it to the telnet service and it will appear as a valid “/CQ” command on the KSTchat.

Wrap Up

IRC is just a single communication method here, we could do this using any other messaging platform and integrate in to applications easily enough and make this more interesting easily enough. Node-RED allows easy control of messaging and simple coding in JavaScript to manipulate things.

I hope to try the described setup while portable with my microwave setup some time and imagine that the messaging ability to alert me to calls, and the use of a persistent IRC client to interact with people should allow me to keep track of KST chat in a manner I’m more comfortable with. (I did try this in a contest weekend up a hill and it worked just fine!)

My code is a horrible mess but I can forward it to anyone that would like to try something like this out. Here’s the whole thing:

Please have a look at some other Node-RED posts for more amateur radio uses if interested in the above:

A Node-RED Module for DMR SMS using an openSPOT

I’ve been using Node-RED to send DMR SMS alerts for about 6 months now as described in the earlier post here. This has worked perfectly until now but I would rather have a single configurable node instead of a collection of nodes containing code, so here it is.

Once installed, the node will be available in the social section and can just be dragged in to the flow. The image below shows the previous setup at the top where a number of different nodes were used and the new setup with just the one node and an input.

Once the node is in the flow, the openSPOT server will need to be configured within it by selecting the edit icon next to the openSPOT drop down. The defaults in the server section should suit a basic default install on a local network.

Each SMS node can be configured with all of useful options as described in the HTTP API documentation. The main requirement is that a destination ID is entered, this will be your DMR ID if you’re using this for sending messages to yourself. The rest of the configuration options are as described in the API documentation.

Once configured, any string payload sent to this node should send it to the radio.

The module is available at https://www.npmjs.com/package/node-red-contrib-openspotsms and can be installed in an existing Node-RED instance by changing in to the instance directory and running:

npm install node-red-contrib-openspotsms

It’s also available on github if you’d rather grab it manually. I’m no coder so it’s very messy but it seems to work for now, I’ll get through the todo list in time!

Christmas With RfCat

Having lost the RF remote controlling the power to the Christmas tree lights a couple of times I thought it would be a good time to try RfCat with the YARD stick one that has been sitting neglected since I bought it a while back. Nothing here that hasn’t been written up 101 times by others!

The RF controller has multiple buttons for switching the different mains switches on and off individually or all on/off at the same time. As these are only button presses I’ve just aimed to replay the transmissions not really caring about the content.

The signals are on off keyed and repeated when holding the button, looking at these in audacity was pretty much as described in earlier posts here and here. It’s been a while since I’ve done this sort of thing so was a decent refresher project.

Additionally this time around I’ve used inspectrum with the IQ output from GQRX which along with its cursor option makes getting the timings correct a lot easier.

screenshot-inspectrum-gqrx_20161209_103307_434173979_1800000_fc-raw

Inspectrum with cursors enabled

Using the stats.py script as described in earlier posts with a wav file to make a guess gave the following output (not from the same button as in the screenshot):

./stats.py -i ../allon.wav -c 1 -t 30
 The whole lot: 000000010101111110000001001001010101001010001010010101010010101010100100100101001010010100100100101001000100101001010100101001

After removing the initial zeros this matched up nearly perfectly with the inspectrum/audacity output. I added an additional 1 and 0 to the long on/off periods at the start to have it match better.

Now I needed to send this out using the RfCat with the above input. This was surprisingly easy to get working in Python with RfCat and worked the first time after following some online examples.

Firstly I needed to convert the stream to use with RfCat which gave:

\xaf\xe0\x22\x2a\x8a\x28\xaa\x2a\xa2\x22\x8a\x28\x88\xa2\x22\xaa\x22\x20

Then to send this in RfCat:

d.setMdmModulation(MOD_ASK_OOK)
d.setFreq(434400000)
d.setMaxPower()
d.setMdmSyncMode(0)
d.setMdmDRate((int)(1.0/0.000450))
d.RFxmit("\xaf\xe0\x22\x2a\x8a\x28\xaa\x2a\xa2\x22\x8a\x28\x88\xa2\x22\xaa\x22\x20")

The timing was taken from the inspectrum symbol period as pictured above, modulation and frequency self explanatory.

Comparing the original signal to the one sent by rfcat they are pretty much the same and the switches accept the signals as intended.

Original signal top, RfCat bottom

I wrapped this all up in a small python script containing all the on/off values and am now using it to turn the lights on and off with a cron job for the month.

Another way to receive the data and skip the iq/wav analysis full stop is just to use RfCat to receive the signal.

I did try this first and didn’t get too far before trying the above instead but it was easier after having done all of the above and having the timings correct and after reading a great post here which is a good guide to this sort of thing: http://andrewmohawk.com/2015/08/31/hacking-fixed-key-remotes-with-only-rfcat/

With the same RfCat settings as above we run d.RFlisten() and get the following:

RfCat button press output

There’s some obvious repeated data in the screen shot. Taking a few of the repeats out and converting to binary we end up with a string that matches the output from stats.py but for a slightly longer preamble but this doesn’t make a difference to the outcome when we retransmit the above, the switch switches as expected.

It is a lot quicker and less fiddly to just do all of the above with RfCat entirely but had I not worked through it from the method I knew already I’d have struggled to get it working as quickly. Next time with a little better understanding hopefully it will be easier.

WSJT-X alerts to MD-380 with the openSPOT HTTP API

DMR SMS alerts using the SharkRF openSPOT with Node-RED

I recently acquired a SharkRF openSPOT for use as a hotspot to connect to the Brandmeister DMR network with my MD-380 DMR radio as I have no easily accessible repeaters nearby to use for DMR.

I have nothing but good things to say about this device, it works very well, the UI is simple to use, reported bugs are fixed very quickly and new features added with new firmware. The icing on the cake is it is a very accessible device with a HTTP and UDP API to interact with! I’ve only toyed with some features in the HTTP API but happy with what I’ve seen so far.

The first use I came up with for it was receiving DMR SMS messages to my MD-380 from my existing WSJT-X & Node-RED setup. The status-dmrsms API allows us to receive and send SMS messages over the local RF link to our connected DMR radio by specifying its DMR ID. This functionality works exactly as described in the API documentation now and if you follow it you will get a beer.

BEER from the SharkRF openSPOT HTTP API

In order to get this working in to Node-RED a flow was needed to handle authentication. As described in the Login Process, we need to hash our openSPOT password with a provided token to get a digest for use in all communication to the API, this digest is valid for 60 minutes.

The flow below shows the authentication process it as set up at the moment.

Node-RED openSPOT API Login

The inject at the beginning just sends a time stamp, which is unused, to start this flow off on Node-RED start-up and every 30 minutes thereafter. After the login is posted some global variables are set with the login status, the token and the digest if authentication was successful. This should hopefully tick away to ensure we will have a valid digest to hand at all times.

The posting of messages is easy and exactly as documented in the API description. With the digest already in a global variable from the login process above, we take any text input, limit it to 75 characters, convert to UTF16BE HEX and post it in the correct format for our radio. The full flow including the message input from WSJT-X is pictured below.

Full flow for using the openSPOT API

The inject function in the send flow is just there for testing purposes to insert a test message manually and the success function at the end just writes the status to the debug console.

With the above all set up we just wait for the DX to light up our DMR radio with an SMS message, the image below shows this on an earlier version of the same flow.

WSJT-X alerts to MD-380 with the openSPOT HTTP API

Alerts from Node-RED via Twitter or IRC might be easier but at least with the above it is all contained on the RF side and doesn’t need the Internet 🙂

It has been running for a few days now and seems to be working fine. I’ll try and wrap it all up in a more easily deployed function if I get the time but if anyone wants the nasty code before then just drop me a line.

WSJT-X monitoring with py_wsjtx & Node-RED

An article about Node-RED by G4WNC in a recent Practical Wireless gave me the push to try and use it in my own radio set-up for alerting and monitoring using a spare Raspberry Pi.

The goal is to receive notifications when my own local radio spots new DXCC on HF bands, any WSPR or JT spots on 6m+ and to plot the 2m JT65b beacons I can hear over time amongst other things.

Prior to this I was only monitoring the beacons using a script and forwarding this to openHAB over MQTT to display alongside some house statistics. This wasn’t too flexible and openHAB is a bit of a burden on the Pi which would randomly hang.

For this project I’m wanting to take inputs from different physical radios & SDR with multiple copies of WSJT-X to display, log certain decodes and alert me in multiple ways if interesting things are seen.

The set-up currently has three radio inputs, each of these has a WSJT-X instance with its own configuration:

HF Radio (IC-7300 and/or FT-817)
VHF Radio (FT-847)
GQRX (IF out of FT-847)

Input 1 is set to whatever I’ve left the HF radios monitoring.

Inputs 2 and 3 are usually set to monitor the two JT65B enabled 2m beacons I can hear from this location, GB3VHF and GB3NGI, using the same antenna. I have this graphed on openHAB but it’s not working great so will be using something else and graphing from the database instead at some point.

WSJT-X can output status messages and decodes over the network to a configured address, this is discussed in a previous blog post where we split the output to AlarmeJT and CQRLOG. We will add a third listener on an extra port, py_wsjtx.

Py_wsjtx is a Python network listener that takes the network output from WSJT-X and displays it in a console, either line by line or a curses interface. I have all of the WSJT-X instances sending their data to a single py_wsjtx instance.

As can be seen above, this is really handy for monitoring things from a console rather than the GUI and will highlight new DXCC spots and CQ calls. It can also output the decoded messages to an MQTT broker if configure which comes in really useful for what we’re doing here.

Node-RED allows us to easily take these MQTT inputs, process them in whatever way we want and act upon them. The image below shows the current set-up.

The purple boxes are MQTT inputs and outputs, each of these points to an MQTT broker (running on the same Raspberry Pi) and listens or sends messages for a particular topic. Py_wsjtx sends MQTT messages in the format py_wsjtx/WSJT-X radioname/messagetype which makes it easy for us to configure Node-RED to process them in the correct manner for instance filtering by radio or by decode type.

Working from the top row of the flow down:

GQRX and FT847 JT65b beacon decodes are converted to JSON, then they are forwarded on in three ways:
1. All decodes go to openHAB which is graphing things at the moment, this is shown in the image below and I have it copied to my qrz.com page, I’ll be changing this shortly to something more reliable/configurable.
2. All decodes are logged to a MySQL database which I will use for generating graphs when we stop using openHAB.
3. If the decodes are above set levels, <5 for GB3NGI and <20 for GB3VHF, then send a post to twitter and to me on a local IRC server.
Next we have DXCC alerts from WSJT-X, if it spots a new country then a message is sent to twitter and IRC with the spot, hopefully I will see it and respond. To make it more interesting I had it ring the shack doorbell, I’ve got two ways to do this, using a HackRF to replay the wireless doorbell, which is a bit of a waste of an expensive SDR, or ringing via a second remote unit using the Pi GPIO pins. The ringer got annoying quickly so it’s now turned off, flashing a light may be better!
Next up we have an input for any WSPR spots on any radio. I’m not doing much with this at the moment other than alerting me on local IRC/twitter if there are any spots on 6m/4m/2m, I don’t often have WSPR listeners on these band though but if I think conditions are looking likely I will switch one of the WSJT-X instances to it.
The solar inverter statistics are sent out on 433.9mhz and I use an RTL SDR dongle to receive them and decode with the program rtl_433. These are then rate limited and forwarded to openHAB as well as being written to the database.
The other MQTT inputs are DHT11 temperature and humidity sensors in the house hooked up to various Pi I have. I’ve not got around to doing anything with these in Node-RED yet but they are currently used by openHAB.

Not much more to say other than it works well for me and I plan on playing about with the flow some more to add some more alerting rules and cutting openHAB out of the solution entirely by graphing the outputs from the database in a more accessible way.

Solar PV Output in openHAB Using RTL_433

I’ve been playing a little with the fantastic open source home monitoring and automation solution openHAB on a Raspberry PI 2 . Having started off with some small temperature/humidity sensors I was looking for something else to add and the stats from the Solar panels were an obvious want.

I previously had the Inverter monitored using using Auroramon as described here. This was great but it’s stopped working and I’m putting off crawling around the attic to debug the issue for as long as possible.

I can keep an eye on the Inverter output using a standalone wireless monitor, the OWL Micro+, which has a small transmitter sensor clamped to the output from the Inverter giving current generation statistics. There is however no way of hooking this up to a computer to record the stats, so enter rtl_433..

The rtl_433 application uses an RTL SDR dongle to receive and decode a huge selection of wireless sensors transmitting on 433 MHz. I didn’t see the OWL device in the list but on running rtl_433 we see statistics generated every 15 seconds when power is being generated and totals every 60 seconds when there’s no generation.

Energy Sensor CM180 Id 62a0 power: 48W, total: 11242215648W, Total Energy: 3122.837kWh

With the output from this we can pull out the power generated and send it via an MQTT message using mosquitto_pub to a listening MQTT broker we have openHAB set up to use already.

This is ghastly but it works for now, for some reason the unit reads 48w when idle so there’s a bit of fiddling to make it idle at 1:

rtl_433 2> /dev/null | xargs -I {} sh -c “mosquitto_pub -h 192.168.1.1 -t solar -m \$(echo {} | sed -e ‘s/, .*//’ -e ‘s/^.*: $[0-9]\+$.*/\1/’|while read spo; do if [ \$spo -gt 50 ]; then echo \$spo; else echo 1; fi; done)”

I’ve made some changes to my set-up where everything is now being fed in to node-red which processes MQTT messages so the following is a bit more simple and outputs in JSON and lets node-red do the filtering:

rtl_433 -R 12 -F json 2> /dev/null | parallel –tty -k mosquitto_pub -h 192.168.1.1 -t solarout -m {}

At the openHAB side I’ve an item “solar” set up and can see the current generation and day/week charts, it’s been running a few days now and been no problem at all!

Screenshot-16

Future plans with this one will be to capture the output in a less Frankenstein manner or maybe risk broken ribs by crawling around the roof space to fix the monitor on the Inverter.

Using WSJT-X in Linux with CQRLOG and AlarmeJT

WSJT-X offers a handy UDP network service for two way communication with other applications such as logging or monitoring software, but I had some difficulties using it with multiple applications at the same time.

WJST-X is set up to communicate with a listening UDP server in the Settings->Reporting section as the image below, by default it is set up to send packets to the localhost port 2237.

There are a number of software packages in Linux that support this communication from WSJT-X such as CQRLOG and AlarmeJT.

CQRLOG uses it in remote mode to allow automatic logging of QSOs from WSJT-X saving a lot of time and errors.

AlarmeJT, pictured below, is a handy application that takes decodes and displays them alongside information such as whether the DXCC/Locator is required based on logs provided to it and can communicate in response to WSJT-X to tune to the selected call.

The only problem is by default, as far as I understand, we can only use one of these programs at a time with WSJT-X. WSJT-X sends its UDP traffic to one address/port but each of the two consuming applications will try to exclusively bind this port preventing the other application from doing the same.

WSJT-X can be configured to send to multicast addresses to allow multiple applications on the network to consume the same data. However I’m running both applications on the same workstation and one of the applications doesn’t allow the listener IP to be set and the other wouldn’t let me change the settings at all in addition to the binding issues. So the plan was to find some way of duplicating the UDP packets to multiple destinations.

Some Startpage searching before attempting to do this with iptables or Python (a quick test indicated it would be possible) identified an already written user-space tool, samplicator, that does the forwarding we need here. It is easy to compile and use.

The idea is to have WSJT-X configured to send to the samplicator listener on local port 2000 and have it send copies of the received packets to a CQRLOG listener on port 2238 and also to an AlarmeJT listener on the default port 2237.

The command to implement this is as below and I’ve put it in my /etc/rc.local to run at boot.

samplicate -S -p 2000 127.0.0.1/2237 127.0.0.1/2238

Here we are listening on port 2000 and forwarding copies of the UDP packets to 127.0.0.1 port 2237 and 2238. The -S options is especially handy as it spoofs the source addresses to appear exactly as sent which allows the consuming applications to communicate in response such as to allow tuning to a station when clicking on a CQ in AlarmeJT, without this set they can just consume the data.

It seems to work well with the limited traffic we’re using here and lets me parse decodes quickly for their statistics, tune quickly to calls and to log the successful QSO automatically. Now I just need to make a few hundred contacts to make up for the time spent fiddling about with this stuff to save time…

Analysing An Active RFID Card

The next device from the box of bits was a pair of active RFID cards from a top of the range vehicle security system used around 2001-2003. These tags were required to be on your person to start the car in addition to a key. This is just an analysis of the cards RF transmissions and not the system itself as I don’t own one.

Popping open the two cards we can see they have similar layouts but the newer board on the right doesn’t look as clean and does not have any PCB labelling. The keys each have a serial number sticker on the back of the PCB as well as having it printed on the front of the card.

The two RFID tokens

There are not many components on the card, we can see an SMD EEPROM along with a small SAW device and the PCB antenna. Looking up the SAW device documentation indicated that it will most likely be transmitting in the 433Mhz ISM band on 433.900.

The batteries in the cards had long run out, so we just hooked them up to an external power supply and monitored with the HackRF and GQRX, this would have worked fine with the RTL SDR device too. We see a short transmission on 433.920Mhz roughly every 5 seconds but it does vary a bit. The frequency reading is wrong below as the screen shot was captured off an IQ replay.

Clear signal.

On looking at this a bit closer with baudline it unsurprisingly looks to be an on off keying AM signal. Importing a saved audio clip in to Audacity shows the signal, the first capture looked a bit odd with dips and other strangeness but it turned out I was a bit off frequency, fixing the frequency gave a clearer capture.

Unclean off frequency capture.

The first thing I wanted to do was level off the signals to allow for an easier visual comparison, the capture below shows the original signal up top and the altered one below. I used a the same script I used with the previous doorbell project and we can count 78 peaks in each tokens transmission.

Full capture with the second channel prettified

Comparing transmissions showed that the token is beaconing the same pattern over and over so maybe they are easily repayable as with the doorbell. Interestingly when we look at the two tokens with different serial numbers next to each other we can see a quite long period at the start where they roughly match followed by not too dissimilar patterns after. At this stage I’m guessing the serial numbers are likely just being beaconed following the preamble/synchronisation signal.

The signals match prior to the red line.

The timings were not particularly consistent even in the initial preamble to allow them to be lined up easily and my previous script to guess binary output based on the period of time (NRZ) was not working perfectly due to this.

There appeared to be two spacing times, one around 18 samples and one around 45 samples but these varied by +/- 10 samples which added up to a large error by the end of the 78 state changes. In order to sort this out I added an option to the script we created for the doorbell to make the sample spacing gaps even. So in this instance a sample space of >30 = 40 and <30 = 20 gives us an easy to compare output.

A resized comparison showing matching digits

Now that the WAV file has evenly spaced samples, we can compare them easier as can be seen in the image above of the two tokens beside each other. The initial preamble matches perfectly between each transmission up to the previously indicated red line and would seem to have 5 8 bit components going by the peak count. Handily enough, the tokens 6 digit serial numbers share one digit in the same place and we can see where these match between the two tokens, we can also see this digit repeated again in two other places in the second serial number, these are indicated in the image above.

Now on to the actual decoding! The number of peaks, 78, doesn’t really tally with the indication from the preambles/synchronisation that each part is 8 bits. We now have to turn to a different encoding mechanism commonly used for this sort of cheap OOK device, Manchester encoding. This is better explained somewhere like http://www.quickbuilder.co.uk/qb/articles/, but the idea is that instead of counting the peaks and troughs as a high or low as you might when taking a look at it, we use the transition between high and low values in the middle of a set time period to define the 1 or 0.

As we have already altered our file to have a consistent spacing of 20 or 40 samples we can script reading this Manchester encoding easily by starting at the first state change position, measuring forward 39 samples and detecting whether the value increases or decreases in that time period and repeating this process until the end of the file.

This gives us the following binary output which looks very promising, we can see the matching patterns for the shared digits in the serial numbers between the tokens. The first 5 preamble bytes are the same, 6 is different, and 7 is the same matching the digit 0 in each of the tokens at position 2 with token 2 having two further digit 0’s all in bold.

Token 1: 11111111 01010101 10101010 01010101 00110110 00110001 00110000
Token 2: 11111111 01010101 10101010 01010101 00110110 00110100 00110000
Token 1: 00110001 00111001 00110011 00110010 00000000 10111011
Token 2: 00110000 00110111 00110000 00110011 00110000 11100100

Now converting each of these from binary to hexadecimal we get a match for ASCII characters so by adding an extra decode step to the script processing the file containing the resized signal, we can see the serial numbers they are broadcasting!

stats.py -i gqrx_20140905_134055_433900000-tag-101932.resize.wav -m
Manchester encoding, channel: 0 offset: 40 boundary: 40
ASCII: �U�U6101932�

stats.py -i gqrx_20140905_134055_433900000-tag-400703.resize.wav -m
Manchester encoding, channel: 0 offset: 40 boundary: 40
ASCII: �U�U64007030�

So we have managed to go from a captured signal to decoding a beacon from an active RFID card in not too many steps. I’m not going to go in to any more details but it doesn’t seem a particularly secure security device on the evidence we have gathered here.

Now this would be a lot nicer if I could do it with GNU Radio as it is happening, but small steps!

Filtering a Capture with GNU Radio

So after the previous post where I was sending out a rather large sample, I had a look in to filtering things a bit. The first step in the doorbell example should have been to sample as low as possible to limit it to the minimum sample size.

In order to test filtering with GNU Radio, I captured a sample with a short transmission at +100kHz followed by another shortly later at +300kHz using the basic capture flow in the previous post and used this with a waterfall to observe the output. The goal was to filter out the +300kHz signal.

Searching for “filter” turned up a low pass filter which sounded about right, with a bit of trial and error with the “Transition Width” which didn’t work when set to 0 or a low figure we can see the output when the filter is set to 500kHz.

Changing the cut off value to 150kHz shows the following where we can see the +300kHz signal has been filtered out.We can still see the band where it transmits and the signal outside the filter but I assume this is due to the strength of the signal in the original capture.

Trying this out adding the HackRF as a sink and listening with another receiver confirms that only the +100kHz signal is being transmitted with nothing heard on the +300kHz frequency.

I’d still like to figure out what the transition actually is, if anything at all is being transmitted outside the filter frequency and if there’s a way to have it filter in one direction from the centre frequency instead of both although for my purposes the filter above is enough.

Since writing this I’ve messed about with some of the other filters that also work well. I did run in to a problem with some GNU Radio 3.6 blocks being used instead of the 3.7 ones. For instance when trying to use the bandpass filter I received this error:

cannot import name firdes

Removing /usr/local/share/gnuradio/grc/blocks/band_pass_filter.xml did the trick here to get it working and let the 3.7 filter_band_pass_filter.xml be used. I did this for a couple of the blocks before removing all of the older dated ones, I’ve not seen any problems but a total clean up might be in order prior to a reinstall.

x8x.net

Mostly Radio

Author: GI7UGV