Implement IRIG-B Decoder in LINUX and on EFM8UB2 Universal Bee
I’ve been advised that one of the biggest challenges in the installation of power utility relays & recorders can be the setup and configuration of the IRIG-B time code distribution.
If there was a small, inexpensive hand-held device that could read the IRIG-B signal, and tell us all its characteristics – modulated/unmodulated, true/inverted, IEEE-1344 extensions active, time zone, offset, and other information – then it would be easier to diagnose the problem and tell the customer what they have to do to get the time sync to work.
I did a search and could not find anything… so after playing with decoding IRIG on my LINUX computer, I bought my own EFM8UB2 Development Kit and proceeded to write a decoder for it, per my blog entry.
The challenge now is to do a user interface for it. I can easily do a LINUX UI (and did!), but what would be really cool is to (say) have an Android app that could connect to the EFM8UB2 using USB OTG – and display the data there.
I actually go the full Google Android SDK and emulator up & running, and was clicking along, learning the development of Android apps, but then along came Eye on the Ice – which took all of my spare non-work hours time. One hobby at a time! I’d like to return to it, and finish, some day soon.
Change Home Phone to VoIP
With all the changes in my life 2011-2014, there came a point where I realized that I’d like to carry the same phone number with me. Fortunately, my family’s historical phone number from 1972 had just been given up a few months before by my ex-wife, and since my name was still on the account, I re-registered it. I then ported it to a local VoIP provider, les.net. I used a Grandstream VoIP adapter and a 5-set Vtech Cordless telephone system. It was great having a handset in most areas of my house! That worked well, and I took the VoIP phone (and that number) with me to Phoenix, then back to Winnipeg. In between, I added a Phoenix VoIP number, then a teleconference system, all through Les.net.
In the meantime, I also put a Grandstream VoIP adapter at my mom’s place, and used an old 2-phone Cordless set that she had. It wasn’t intended to take incoming calls, just was for making less expensive long distance calls. She loved it and used it all the time.
The call quality on les.net was never as good as with a “real” landline. True, it was very inexpensive, and I liked supporting local businesses, but there were significant call voice delays that drove me crazy. I put up with it, because I didn’t use it all that much, and it was cheap.
When I moved back to Winnipeg in 2017, I ported my US cell phone number to Les.net, and got a new Manitoba cell phone number. Of course, I kept my original family phone number as well. The rationale was that people would continue to reach me the same way they did before. It worked well!
I really wanted to use SMS on that former US cell phone number. Les.net could do that, but only through a web page, which you had to refresh constantly.
Another service that I found, VoIP.ms, supported SMS through an Android app. I tried it. It was even easier to set up and use, had points of presence all over North America (it can be useful to keep your point-of-presence near the location of the area code your number is in), and ended up having much better call quality!
So now I’ve moved all my VoIP lines over to VoIP.ms… and I use them all the time. Calling France? Yes, use my VoIP line. Getting calls from headhunters in the US? Yes, use my VoIP line. SMS to and from my friends back in Phoenix? You guessed it 🙂
Contribute Update to NTP Project’s IRIG & WWV Tone Generator
I’ve always wanted to create a WWV(H) decoder. Sigh. In the late 1980s, I played with switched capacitor filters (MF10 springs to mind, but that’s a relatively low order filter, maybe it was the MAX7490 or MAX7413?), and with NE565 PLLs, to try and pull the 100 Hz subcarrier out of the shortwave audio signal WWV(H), then decode it. Unfortunately, in this application, PLLs and high order bandpass filters are basically the same thing, and I didn’t get all the way to building the decoder.
Imagine my delight when I realized that the NTP project already has built-in, a WWV(H) decoder… and as a bonus, also an IRIG-B decoder too (modulated only, no unmodulated… yet). Well, by this time, I didn’t have a shortwave radio handy to pull in WWV with… but, what’s this? A program called tg in the utils directory is a WWV(H) and modulated IRIG-B generator!
I had a look at it, and sure enough, it would generate WWV(H) and modulated IRIG-B signals, but it was pretty limited, didn’t do any of the IEEE-1344 extensions that we used in power utility applications, nor leap seconds etc. That wasn’t enough for me. Hmm, I would rewrite tg to test all these things and generate synthetic IRIG-B. So, I did.
Oops, it was not built for a modern LINUX sound system… so I ported it.
It was good to be able to test an IRIG-B decoder – I could start at an arbitrary time, insert and remove leap seconds, trigger various conditions, etc. So I added these functions.
I found that the computer sound card sample rate wasn’t precise. There were precisely 8,000 samples per second, so after a while, the time emitted by tg would drift. I modified tg to omit or add an extra cycle (in an unused area of the protocol) to correct for this. The program would track the sent time and the “apparent” real time with respect to the computer’s real-time clock, and would correct when needed.
I re-mastered a Knoppix image to include tg2 for demonstration of my work. I thought Elecsys might sell it (or its support) to ERLPhase for cash, but it wasn’t of enough utility to make it worthwhile.
I added unmodulated IRIG-B output, although that was trouble because most sound cards won’t pass frequencies low enough to emit unmodulated IRIG-B. But I unwarped the filter and did something that almost works, sometimes.
I added sophistication to the WWV(H) output as well, with coded pulses lining up more closely to what the actual station does.
Then I submitted it back to Dr. David Mills for inclusion in the NTP sources… he called my program tg2.
Recently, when reviewing my work and making a few tweaks on tg2 and playing with other parts of NTP, I realized that I had violated Dr. Mills’s coding format guidelines pretty badly. I’m surprised he put up with it! So I tried to do better.
LoRa Proof of Concept for Eye on the Ice In-Ice Sensors
After the disappointment of the Z-Wave implementation of an in-ice sensor for the Eye on the Ice system, Filipe and I did a survey of other RF communications technologies, and settled on LoRa, per this blog entry. After evaluating several modules and then obtaining a few more, we settled on the AcSip S76S system-on-a-chip, which contains an STMicro STM32L073RZ Arm-Cortex M0+ MCU and a Semtech SX1276 LoRa Transceiver.
If used in the more-popular LoRaWAN mode, it is similar to Z-Wave, but has much longer range. However, there is also the less-sophisticated LoRa mode, where there is no inherent mesh networking and a much simpler simplex protocol, which is what we are using for our second proof-of-concept.
We’ve increased the battery size to a CR2450, giving us 600 mA-hr, much more than the approximately 230 mA-hr of the CR2032. However, in order to maintain decent RF antenna operation, we’ve had to split the device into two interconnected boards, complicating the design. We’re currently evaluating the performance and reviewing further options.
In any case, it sure seems like we can get much more consistent operation at up to 250 ft, with the correct dimensional parameters, even in 32 mm diameter size.
Z-Wave Proof of Concept for Eye on the Ice In-Ice Sensors
As outlined in my blog post about Modern RF Communications Modules, around 2008, my team at Norscan developed the Eye on the Ice system for Hans Wuthrich, Master Ice Maker. It is a great system, and still works well. Unfortunately, the Linx FHSS modules have gone obsolete. For the moment, inventory is available to support the relatively small runs of production. But, over the long haul, things will have to change.
In the meantime, Hans wants a small, low cost, perhaps disposable, in-ice sensor that could be used to accurately monitor ice temperature from several points of the rink. He had an engineer doing some experimentation in this area, but since the frequency is 433 MHz, the FHSS module and it antenna are quite large, it was difficult to get it down much smaller than, say, 50 mm x 50 mm with a 150 mm antenna sticking out in one direction or another.
With a former colleague and friend, Filipe Fernandes – analog, RF and EMC developer extraordinaire – we developed a proof of concept that met the requirements. Battery operated from a single CR2032 cell, and only being about 32 mm diameter x 10 mm thick before enclosure, it used Z-Wave to send the temperature to a Raspberry Pi that had both a Z-Wave controller and an FHSS module attached. The Raspberry Pi operated as a gateway, relaying incoming temperature reports into the Eye on the Ice network.
Although the system worked, we had issues with repeatability in manufacture, causing trouble with range. One sensor was able to report from over 80 feet away, another from only 20 feet away. We addressed this issue, but then had trouble with battery life. It seems that the automatic operation of the Z-Wave protocol causes a sensor to continuously retry if it doesn’t get an acknowledgement from the controller. We were sure that there was a way to deal with this, but the lower levels of the protocol and operation of a mostly-sleeping sensor were opaque and difficult to figure out. It would have been time-consuming (and therefore expensive) to resolve these issues, so we abandoned the Z-Wave system and sought an alternative.
Per this blog entry, we found the solution in the form of a LoRa system.
Management of Wabtec Embedded Software Team
Wabtec’s Winnipeg plant is actually in Oakbank, about 30 km (18 mi) outside of Winnipeg. Oakbank is a small community of a few thousand people, surrounded by farmland, and, to the northwest, hills with rock quarries.
This was originally a small Winnipeg company called iders (pronounced by most people as I-D-ers), who had been pioneers in the development of PIN pads for Canadian banks and retailers, and later successfully landed the contract to develop the Conviron CMP4000 environmental growth chamber system, among many projects of varying sizes and complexity. In the 2001 to 2008 time frame, they also manufactured the NxtPhase TESLA DC Isolation Module, which Horst Koelzow and I had developed.
One of the industries that iders broke into in the early 2000s was electronics for the railroad industry. Their experience in communications in general, and computer networking in particular, led them over several years, to create what’s now known as the GoLINC ACC (Auxiliary Communications Cage), which has the AAR standard for communications on railroad locomotives.
iders was acquired by GE Transportation in 2016. In 2018, GE Transportation itself was divested from GE, merging with Wabtec. The Oakbank plant was still referred to as “GETW” (GE Transportation Winnipeg) until early 2020, when the migration from GE’s IT systems to Wabtec’s IT systems was mostly complete.
I was hired as the Embedded Software Engineering Manager for GETW in August 2018, with 11 direct reports. A few days after starting, I was advised that I had 3 hires to make – which, for historical reasons, were each assigned to separate hiring managers, and were each managed by separate recruiters. The hiring took some time to implement, as we worked out the projected seniority of the hires and reviewed dozens of applicants… but the big thing is that I was busy in the interim. We made the hires in January 2020.
There was a lot of development going on at GETW. Some was new, some was remedial work, but it was all very intense.
When the COVID-19 pandemic hit, we were quite fortunate to be able to work effectively from our homes, because Wabtec has quite a good internal communications infrastructure. There were issues, mind you – we had trouble accessing the Checkmarx static analysis scanner – but we worked around that using a Raspberry Pi that I put on my desk at the office, until the proper routing could be arranged through the VPN.
The pandemic was not as serious in Manitoba as in many other parts of the world. When things settled down in mid May, we carefully moved back to the office, taking appropriate precautions.
Unfortunately, as with many industries around the world, the locomotive industry was hurt by the economic downturn that accompanied COVID-19. As a result, my employment with Wabtec came to an end in June. Many others were let go at the same time, and apparently there will be a long term strategy to consolidate Wabtec’s electronics manufacturing into other locations, and downsizing the office footprint, saving considerable money in the process.
Preliminary Proof of Concept Work on ERLPhase Fault Location Algorithm
The very cool thing about sigma-delta conversion is that there are trade-offs that can be made with the data after it is captured – sampling rate versus inherent filtering versus resolution versus noise floor. These are all inter-related.
ERLPhase wanted to investigate travelling wave fault location. Technicians captured mock fault data in the lab, which I modelled in Python, performing analysis on the frequency content, and various processing techniques. I developed a proprietary front-end amplifier in LTspice, then implemented the circuit on a proof-of concept PCB. I wrote Python scripts to import the fault data into an Arbitrary Waveform Generator (often called “an ARB”), and played them into the new board, captured the data on the sigma-delta modulator evaluation board, post-processing the captured data in Python (including wavelet analysis), and displayed the results graphically in 2-D, 3-D and colour plots.
I wrote a report on the effort, providing details and outlining the path of product development.
Develop ERLPhase 2nd Generation DC Analog Isolation Module
The original DC isolation module, developed under my direction around 1999, used an unconventional custom off-line power supply and four Analog Devices AD204 isolation amplifiers, to provide isolated voltage and current input from DC to about 1 kHz. This opened up a new market for recording sensors and telemetry signals on the TESLA, as well as some unique DC-level-sensitive AC applications, such as internal levels in static VAR compensators (SVCs) and DC converters.
Modern applications are much more stringent, however, needing better performance than the AD204 can provide (accuracy, noise level, and isolation), and the power supply couldn’t meet today’s restrictive EMC requirements.
So, I was asked to develop a newer high-performance DC Isolation Module. The basic concepts had been established by Steve Peddle, but he was quite busy on other projects.
The original concept was to perform “intelligent” ADC conversion on isolated “islands”, encoding the data in a protocol, and transferring the multiple streams of this protocol using digital isolation devices back to a central location, where it would be converted back by multiple DAC conversion on a circuit not isolated from the TESLA recoreder (as is required by the TESLA analog front-end).
I came across a potential alternative in the form of isolated sigma-delta modulator converters like the Analog Devices AD7401, which could give a highly accurate, flexible solution. Using an evaluation board, I captured data and performed extensive data analysis in Python. I implemented the sinc3 filter in Python, demonstrated the trade-offs, and explored the possibilities of various configurations.
I evaluated the technologies against each other and wrote a report. After discussions with Steve and ERLPhase management, we decided to proceed with the sigma-delta modulator approach.
We obtained the evaluation kit for the Silicon Labs Universal Bee MCU EFM8UB20, and wrote the initial code as proof-of-concept and to bring up the system, handing it over to the firmware development team to implement the core functions.
I developed and documented the isolation plan, which included double 300V isolation gaps. After a design review, I created the schematic, and had two interconnected PCBs developed. I wrote a verification & validation document with guideline for the firmware implementation.
Work with GE-Reason Brazil on Next Generation COSI-CT Architecture
The original design of what’s now the COSI-CT was developed at NxtPhase in about 1999. When I was with NxtPhase Winnipeg from 2001-2003, I was involved in an informal internal evaluation of the system, and recommended significant changes. It wasn’t that the design was bad, but that it could be improved to make it better suited for the power utility market.
A NxtPhase CT redesign was undertaken around 2003. When I joined Alstom Grid in 2013 to work on the COSI-CT, it was practically identical to the 2003 redesign, but had a few extra components grafted on – an optional internal power supply, and a digital card. The design was tired and had developed serious flaws, which we were trying to fix.
Everyone agreed that a new approach, the next generation of COSI-CT (called “Gen4”), was needed. As part of GE-Alstom after the merger, we undertook the exciting new development of Gen4.
I travelled to Florianopolis, Brazil, and met with the team at GE-Reason. They had developed an amazing array of networking, merging unit, and recording equipment before their acquisition by Alstom in 2015. Together, the engineers at Reason and I, set about to architect COSI-CT Gen4.
After my return to Phoenix, we collaborated constantly, often in communication daily. We worked in Atlassian Confluence, developing the architecture. We consulted with other GE experts in Merging Unit design, and our own marketing team.
Unfortunately, just as the initial architecture was completed in late 2017, GE faced a cash crunch and had to lay off many staff, including me. This was a big disappointment, as we were really excited about the potential for the next generation.
Oh well, hopefully Gen4 of the COSI-CT, whenever it emerges, and in whatever form, is wildly successful.