A customer of SFL required a rapid proof-of-concept development to support a pitch of a commercial multi-room environmental control system to management. We connected a local Zigbee thermostat and magnet/reed switch to a Zigbee controller, communicated up through Zigbee2MQTT and Eclipse Mosquitto to the Azure Cloud IoT service. How cool! I implemented the custom bits in Zigbee2MQTT (Javascript) for these specific Zigbee devices, and wrote communications to and from the Azure Cloud (Python). The client implemented my code in multiple locations and demonstrated bidirectional communications and control.
Install LTSP in a Private School
Change Home Phone to VoIP
With all the changes in my life 2011-2014, there came a point where I realized that I’d like to carry the same phone number with me. Fortunately, my family’s historical phone number from 1972 had just been given up a few months before by my ex-wife, and since my name was still on the account, I re-registered it. I then ported it to a local VoIP provider, les.net. I used a Grandstream VoIP adapter and a 5-set Vtech Cordless telephone system. It was great having a handset in most areas of my house! That worked well, and I took the VoIP phone (and that number) with me to Phoenix, then back to Winnipeg. In between, I added a Phoenix VoIP number, then a teleconference system, all through Les.net.
In the meantime, I also put a Grandstream VoIP adapter at my mom’s place, and used an old 2-phone Cordless set that she had. It wasn’t intended to take incoming calls, just was for making less expensive long distance calls. She loved it and used it all the time.
The call quality on les.net was never as good as with a “real” landline. True, it was very inexpensive, and I liked supporting local businesses, but there were significant call voice delays that drove me crazy. I put up with it, because I didn’t use it all that much, and it was cheap.
When I moved back to Winnipeg in 2017, I ported my US cell phone number to Les.net, and got a new Manitoba cell phone number. Of course, I kept my original family phone number as well. The rationale was that people would continue to reach me the same way they did before. It worked well!
I really wanted to use SMS on that former US cell phone number. Les.net could do that, but only through a web page, which you had to refresh constantly.
Another service that I found, VoIP.ms, supported SMS through an Android app. I tried it. It was even easier to set up and use, had points of presence all over North America (it can be useful to keep your point-of-presence near the location of the area code your number is in), and ended up having much better call quality!
So now I’ve moved all my VoIP lines over to VoIP.ms… and I use them all the time. Calling France? Yes, use my VoIP line. Getting calls from headhunters in the US? Yes, use my VoIP line. SMS to and from my friends back in Phoenix? You guessed it 🙂
LoRa Proof of Concept for Eye on the Ice In-Ice Sensors
After the disappointment of the Z-Wave implementation of an in-ice sensor for the Eye on the Ice system, Filipe and I did a survey of other RF communications technologies, and settled on LoRa, per this blog entry. After evaluating several modules and then obtaining a few more, we settled on the AcSip S76S system-on-a-chip, which contains an STMicro STM32L073RZ Arm-Cortex M0+ MCU and a Semtech SX1276 LoRa Transceiver.
If used in the more-popular LoRaWAN mode, it is similar to Z-Wave, but has much longer range. However, there is also the less-sophisticated LoRa mode, where there is no inherent mesh networking and a much simpler simplex protocol, which is what we are using for our second proof-of-concept.
We’ve increased the battery size to a CR2450, giving us 600 mA-hr, much more than the approximately 230 mA-hr of the CR2032. However, in order to maintain decent RF antenna operation, we’ve had to split the device into two interconnected boards, complicating the design. We’re currently evaluating the performance and reviewing further options.
In any case, it sure seems like we can get much more consistent operation at up to 250 ft, with the correct dimensional parameters, even in 32 mm diameter size.
Z-Wave Proof of Concept for Eye on the Ice In-Ice Sensors
As outlined in my blog post about Modern RF Communications Modules, around 2008, my team at Norscan developed the Eye on the Ice system for Hans Wuthrich, Master Ice Maker. It is a great system, and still works well. Unfortunately, the Linx FHSS modules have gone obsolete. For the moment, inventory is available to support the relatively small runs of production. But, over the long haul, things will have to change.
In the meantime, Hans wants a small, low cost, perhaps disposable, in-ice sensor that could be used to accurately monitor ice temperature from several points of the rink. He had an engineer doing some experimentation in this area, but since the frequency is 433 MHz, the FHSS module and it antenna are quite large, it was difficult to get it down much smaller than, say, 50 mm x 50 mm with a 150 mm antenna sticking out in one direction or another.
With a former colleague and friend, Filipe Fernandes – analog, RF and EMC developer extraordinaire – we developed a proof of concept that met the requirements. Battery operated from a single CR2032 cell, and only being about 32 mm diameter x 10 mm thick before enclosure, it used Z-Wave to send the temperature to a Raspberry Pi that had both a Z-Wave controller and an FHSS module attached. The Raspberry Pi operated as a gateway, relaying incoming temperature reports into the Eye on the Ice network.
Although the system worked, we had issues with repeatability in manufacture, causing trouble with range. One sensor was able to report from over 80 feet away, another from only 20 feet away. We addressed this issue, but then had trouble with battery life. It seems that the automatic operation of the Z-Wave protocol causes a sensor to continuously retry if it doesn’t get an acknowledgement from the controller. We were sure that there was a way to deal with this, but the lower levels of the protocol and operation of a mostly-sleeping sensor were opaque and difficult to figure out. It would have been time-consuming (and therefore expensive) to resolve these issues, so we abandoned the Z-Wave system and sought an alternative.
Per this blog entry, we found the solution in the form of a LoRa system.
Develop LINUX Test Tool for GE-Alstom COSI-CT Management Processor
The only way to talk to a COSI-CT Management Processor was the COSI-CT Control Panel program in MS-Windows. It knows the structure of the EEPROM parameter storage in the Management Processor, and interprets it accordingly – for instance, if a single byte value is used, it shows it as an integer 0-255, if it’s a string, it shows as a form fill with fixed length, etc. There is a mode that allows it to write to not-defined areas too… but it is clunky. And, to boot, it’s all in decimal, argh. As I’ve said, if it’s a bit mapped value, give it to me in hexadecimal, don’t make me convert it!
Anyways, COSI-CT Control Panel has access to some control flags that indicate whether the parameters of the COSI-CT are under seal – that is, some of the values cannot be changed, probably because they would alter calibration and therefore current values, which dictate revenue to the power company. When the seal is set, Control Panel will not let you even try to modify these settings. That’s great for end users, but for testing, I can’t tell whether the seal is actually working.
For some projects, the seal is more important, than for others. For instance, if you are just doing general disturbance recording or relaying, there’s no reason for anyone to mess with the calibration, so it’s not as important. However, if it’s on a utility-to-utility intertie, or a generator output, then it might be very important.
We had an instance where it was very important, and needed to know that the seal was working properly. Actually, let my rephrase that – we had a corner case that seemed to indicate that the seal was not working completely correctly. So, I had to correct that issue. But, I couldn’t test it!
The serial line protocol to the Management Processor is well documented, but of course, having the source code of the processor itself was very helpful. There are always ambiguities in protocol specifications, unless you happen to give the specification to someone completely in the dark, and have them develop a device to talk that protocol. Well, this was just such a case… but again, I had the source code.
So, I wrote a program in ‘C’ for LINUX to talk to the Management Processor. It wasn’t intended to replace Control Panel, by any stretch, but it ended up expanding and expanding and expanding (as many of my utilities do), so it can run a “write challenge” against every value, to see if it can be modified under seal, can read all values from the device, write individual or all values into the device, and read every real-time value supported. In fact, I expanded it to support every command on the serial link to the Management Processor.
This was quite useful in determining whether the seal had previously been implemented correctly (hint: there were bugs), and to figure out if it was corrected, but also in talking to devices over TCP/IP – like when units were installed at the SRP Perkins substation, and I was talking over a haltingly slow SSH tunnel link. Control Panel would burp, retry, and give up, but, having complete control over the link, I was able to make the connection, lengthen retries, or whatever it took, to make the connection with my command line program, I was able to arrange to get the data back reliably.
In early development, I cross-compiled the code to run from Cygwin in MS-Windows, but the serial port names in MS-Windows tripped it up. I’m pretty sure it would take little to re-port it to MS-Windows.
I advised management of the program, and chatted with Scott Picker (maintainer of COSI-CT Control Panel) that he might want to use it for a test facility, and I gave them the source code, of course, but as far as I’m aware, nothing ever came of it… sigh.
Develop Wireless Remote Access to GE-Alstom Equipment inside SRP Perkins Substation
Salt River Project Power (SRP) had a mystery in its transmission network – some disturbances were unexplained. By installing a COSI-CT and my newly-developed VT, and a GE-Reason recorder at the Perkins Substation, GE would get some valuable operational data from a local substation, and SRP would find out the cause of the problem.
It would be great to have ongoing telemetry from our equipment in the substation, but of course SRP cannot allow another company like GE to have access to its computer network. I was asked to find a wireless way to communicate with the system.
I asked if SRP would allow us to put up an extra antenna – they were already installing a GPS antenna for us – and they agreed. So I undertook to do a quick survey of potential means of communications, and ended up with a cellular phone router – one of the Multi-Tech MTR-H5 family devices.
I was going to get a SIM through GE Corporate, but the required effort & justification was huge. My manager needed this done quickly. So, I went down to my local T-Mobile store and got a “data & SMS only” SIM (intended for tablets etc.). It was only US$10 a month – which, because I was travelling so much, just put on my expense account thereafter.
Then I tried to contact the router – but couldn’t. Apparently incoming connections to T-Mobile’s wireless devices isn’t generally allowed. This is probably a good thing!
We were also installing a GE DAPserver, which would serve as a data concentrator. I dug around a bit and got root access to the DAPserver (no, it wasn’t easy because we were GE – they don’t give that information out!). I then created a Digital Ocean droplet and coded a continuous SSH connection from the DAPserver to the droplet. There were actually two connections – one by IP address, and one by dynamic DNS hostname – just in case one failed. Each one had reverse traffic tunnels brought up, that would link to the various local devices, like each COSI-CT and the recorder.
Well, it turned out that the COSI-CT only provides a serial interface to its administration functions. The process data comes out on an Ethernet link (generally fibre, but could be twisted pair), but the Management Processor interface is RS-232, ugh. So, I had to buy a Lantronix Xpress-DR+ 2-port serial-to-Ethernet gateway… which was fine.
In any case, it all worked well, in the end. I was actually able to log into the recorder and display phasors – although the link was relatively slow, and performance was poor. For text-based communications, like talking to the COSI-CT, it mostly worked fine. The COSI-CT Control Panel didn’t like the link, because turn-around latency was too slow. Well, Control Panel wasn’t designed for Ethernet either, but Scott Picker added this feature for me, but there was some complication in the negotiation of the link (possibly the original serial auto-line-rate detection) that caused it to drop every time. No matter, my little LINUX Test Tool worked just fine, and enabled me to read and write configuration values as needed.
Although it was clunky and slow, it did work – so I considered it a success!
Update Alstom DIT COSI-CT Management Processor
I was only at Alstom Grid DIT in Phoenix a couple of months, when an question came from management, “Does anyone here know PIC assembly language?”
Well, I do! So I took on the modification of the COSI-CT Management Processor firmware. The code was written circa 2003, and was based on earlier efforts. Good designers re-use working code! Anyway, it had been maintained up until about 2010 by Dylan Stewart, but Dylan now had his hands full as Product Manager. So, I took it on.
It started largely as a minor rework. The processor has explicit support for each stored configuration value inside, knowing which one(s) are bytes, which words, which double words, and which are strings. Things had changed in the configuration, but support for these changes was incomplete. So, I addressed that.
A larger issue was that of supported development environment. The old MPLab 8.x was obsolete, with MPLab X taking its place, but MPLab X would not automatically import MPLab 8.x files, especially complex ones. I installed both development systems, and ensured that the outputs were byte-for-byte equivalent – which you can do with assembly language code! It took a lot of tinkering to get the code to assemble properly under MPLab X, even more difficult to get it to assemble under both, but I managed it.
There was some discussion about the apparent corruption of data values on the backplane bus to and from the other internal cards. There was no error checking on this bus. I wrote a proposal to use CRC on the backplane comms. It was carefully tweaked to fit within the protocol, and minimize overhead. I implemented it on the Management Processor, but due to an error in the implementation on the CT card’s FPGA, there was an amiguity in the link, causing protocol failure. I tried to get this error corrected, but it never was, so the effort was abandoned in place. The code is still there, but it is never used.