Implement IRIG-B Decoder in LINUX and on EFM8UB2 Universal Bee

I’ve been advised that one of the biggest challenges in the installation of power utility relays & recorders can be the setup and configuration of the IRIG-B time code distribution.

If there was a small, inexpensive hand-held device that could read the IRIG-B signal, and tell us all its characteristics – modulated/unmodulated, true/inverted, IEEE-1344 extensions active, time zone, offset, and other information – then it would be easier to diagnose the problem and tell the customer what they have to do to get the time sync to work.

I did a search and could not find anything… so after playing with decoding IRIG on my LINUX computer, I bought my own EFM8UB2 Development Kit and proceeded to write a decoder for it, per my blog entry.

The challenge now is to do a user interface for it.  I can easily do a LINUX UI (and did!), but what would be really cool is to (say) have an Android app that could connect to the EFM8UB2 using USB OTG – and display the data there.

I actually go the full Google Android SDK and emulator up & running, and was clicking along, learning the development of Android apps, but then along came Eye on the Ice – which took all of my spare non-work hours time.  One hobby at a time!  I’d like to return to it, and finish, some day soon.

LoRa Proof of Concept for Eye on the Ice In-Ice Sensors

After the disappointment of the Z-Wave implementation of an in-ice sensor for the Eye on the Ice system, Filipe and I did a survey of other RF communications technologies, and settled on LoRa, per this blog entry.  After evaluating several modules and then obtaining a few more, we settled on the AcSip S76S system-on-a-chip, which contains an STMicro STM32L073RZ Arm-Cortex M0+ MCU and a Semtech SX1276 LoRa Transceiver.

If used in the more-popular LoRaWAN mode, it is similar to Z-Wave, but has much longer range.  However, there is also the less-sophisticated LoRa mode, where there is no inherent mesh networking and a much simpler simplex protocol, which is what we are using for our second proof-of-concept.

We’ve increased the battery size to a CR2450, giving us 600 mA-hr, much more than the approximately 230 mA-hr of the CR2032.  However, in order to maintain decent RF antenna operation, we’ve had to split the device into two interconnected boards, complicating the design.  We’re currently evaluating the performance and reviewing further options.

In any case, it sure seems like we can get much more consistent operation at up to 250 ft, with the correct dimensional parameters, even in 32 mm diameter size.

Z-Wave Proof of Concept for Eye on the Ice In-Ice Sensors

As outlined in my blog post about Modern RF Communications Modules, around 2008, my team at Norscan developed the Eye on the Ice system for Hans Wuthrich, Master Ice Maker.  It is a great system, and still works well.  Unfortunately, the Linx FHSS modules have gone obsolete.   For the moment, inventory is available to support the relatively small runs of production.  But, over the long haul, things will have to change.

In the meantime, Hans wants a small, low cost, perhaps disposable, in-ice sensor that could be used to accurately monitor ice temperature from several points of the rink.  He had an engineer doing some experimentation in this area, but since the frequency is 433 MHz, the FHSS module and it antenna are quite large, it was difficult to get it down much smaller than, say, 50 mm x 50 mm with a 150 mm antenna sticking out in one direction or another.

With a former colleague and friend, Filipe Fernandes – analog, RF and EMC developer extraordinaire –  we developed a proof of concept that met the requirements.  Battery operated from a single CR2032 cell, and only being about 32 mm diameter x 10 mm thick before enclosure, it used Z-Wave to send the temperature to a Raspberry Pi that had both a Z-Wave controller and an FHSS module attached.  The Raspberry Pi operated as a gateway, relaying incoming temperature reports into the Eye on the Ice network.

Although the system worked, we had issues with repeatability in manufacture, causing trouble with range.  One sensor was able to report from over 80 feet away, another from only 20 feet away.  We addressed this issue, but then had trouble with battery life.  It seems that the automatic operation of the Z-Wave protocol causes a sensor to continuously retry if it doesn’t get an acknowledgement from the controller.  We were sure that there was a way to deal with this, but the lower levels of the protocol and operation of a mostly-sleeping sensor were opaque and difficult to figure out.  It would have been time-consuming (and therefore expensive) to resolve these issues, so we abandoned the Z-Wave system and sought an alternative.

Per this blog entry, we found the solution in the form of a LoRa system.

Develop ERLPhase 2nd Generation DC Analog Isolation Module

The original DC isolation module, developed under my direction around 1999, used an unconventional custom off-line power supply and four Analog Devices AD204 isolation amplifiers, to provide isolated voltage and current input from DC to about 1 kHz.  This opened up a new market for recording sensors and telemetry signals on the TESLA, as well as some unique DC-level-sensitive AC applications, such as internal levels in static VAR compensators (SVCs) and DC converters.

Modern applications are much more stringent, however, needing better performance than the AD204 can provide (accuracy, noise level, and isolation), and the power supply couldn’t meet today’s restrictive EMC requirements.
So, I was asked to develop a newer high-performance DC Isolation Module. The basic concepts had been established by Steve Peddle, but he was quite busy on other projects.

The original concept was to perform “intelligent” ADC conversion on isolated “islands”, encoding the data in a protocol, and transferring the multiple streams of this protocol using digital isolation devices back to a central location, where it would be converted back by multiple DAC conversion on a circuit not isolated from the TESLA recoreder (as is required by the TESLA analog front-end).

I came across a potential alternative in the form of isolated sigma-delta modulator converters like the Analog Devices AD7401, which could give a highly accurate, flexible solution. Using an evaluation board, I captured data and performed extensive data analysis in Python.  I implemented the sinc3 filter in Python, demonstrated the trade-offs, and explored the possibilities of various configurations.

I evaluated the technologies against each other and wrote a report. After discussions with Steve and ERLPhase management, we decided to proceed with the sigma-delta modulator approach.

We obtained the evaluation kit for the Silicon Labs Universal Bee MCU EFM8UB20, and wrote the initial code as proof-of-concept and to bring up the system, handing it over to the firmware development team to implement the core functions.

I developed and documented the isolation plan, which included double 300V isolation gaps. After a design review, I created the schematic, and had two interconnected PCBs developed. I wrote a verification & validation document with guideline for the firmware implementation.

Update Alstom DIT COSI-CT Management Processor

I was only at Alstom Grid DIT in Phoenix a couple of months, when an question came from management, “Does anyone here know PIC assembly language?”

Well, I do!  So I took on the modification of the COSI-CT Management Processor firmware.  The code was written circa 2003, and was based on earlier efforts.  Good designers re-use working code!  Anyway, it had been maintained up until about 2010 by Dylan Stewart, but Dylan now had his hands full as Product Manager.  So, I took it on.

It started largely as a minor rework.  The processor has explicit support for each stored configuration value inside, knowing which one(s) are bytes, which words, which double words, and which are strings.  Things had changed in the configuration, but support for these changes was incomplete.  So, I addressed that.

A larger issue was that of supported development environment.  The old MPLab 8.x was obsolete, with MPLab X taking its place, but MPLab X would not automatically import MPLab 8.x files, especially complex ones.  I installed both development systems, and ensured that the outputs were byte-for-byte equivalent – which you can do with assembly language code!  It took a lot of tinkering to get the code to assemble properly under MPLab X, even more difficult to get it to assemble under both, but I managed it.

There was some discussion about the apparent corruption of data values on the backplane bus to and from the other internal cards.  There was no error checking on this bus.  I wrote a proposal to use CRC on the backplane comms.  It was carefully tweaked to fit within the protocol, and minimize overhead.  I implemented it on the Management Processor, but due to an error in the implementation on the CT card’s FPGA, there was an amiguity in the link, causing protocol failure.  I tried to get this error corrected, but it never was, so the effort was abandoned in place.  The code is still there, but it is never used.

Elecsys starts with a… Last Call?!?

Elecsys started out with what could have been a “bang”, but ended up being a “whimper”.  My “whimpering” from lack of sleep!

Two businessmen from the Riding Mountain area, Terry Ledoux and Rene Roncin, contacted Vansco about development of an remote controlled duck decoy animation system.  Vansco was too busy, so they referred them to Elecsys.

Terry Ledoux was a taxidermist based in McCreary MB, who sold his taxidermy business and went full time into being an outfitter and hunting guide.  He wanted his new product to be called Last Call, a trade name which he owned.

They had started the development, even had nice rotary moulded enclosures made, but they didn’t know how to do the electronics, so Elecsys took them on.

Elecsys was actually operating out of the attic of Jason’s house.  We brought in all our contacts on the job – mechanical development, PCB layout, PIC programming – and ran our credit to the limit to cover all the costs of parts, etc.

The last 48 hours were an absolute nightmare.  We worked non-stop, had all kinds of issues – not the least of which was destroying multiple radio modules – they were not only quite expensive, but we also only had so many to work with.  I so much needed sleep that I went downstairs to lay down on the couch, while one of my colleagues was going to try to commission another unit with our last radio module… when I had a flash of insight!  I had to rush back upstairs and stop them from powering it up!  The radios had been destroyed by inductive kickback from the motor drive circuit, which operated from the same supply.  We hastily reworked the circuit, and with another 14 hours of work, we delivered two functional prototypes to the customer for a business trip to demonstrate them to Cabela’s in the US.

In the meantime, we were all strapped for cash – the customer, Elecsys, Jason and I ourselves… and had to settle for only a portion of the planned amount.  We did cover our costs, but not much more.  Needless to say, the customer didn’t proceed with the next stage of the development, which would have cost a lot more money.  That was unfortunate, because it was a cool product with a cute name, and I think it would have been a winner.

Some time later, we sat around with Ed Van Humbeck and discussed our experience.  He shared with us that something similar happened to him in the early days of Vansco, and counselled us to always get as much as possible up front – to at least cover the bare costs of the project, should the final payment not be forthcoming.  We still had much to learn!

Development of Flex-Header Controller for Honey Bee

Brian Fletcher of Honey Bee had contacted Vansco to do a small electronic assembly for them, but Vansco was too large and busy, so they referred him to Elecsys.  We happily took on the task of creating a controller for their new 42 foot combine flex header.

The challenge is that, with a flexible cutterbar out front, you cannot rely on simple bumpers to keep the cuttterbar from interfering with the reel.  It might flex in the middle, coming up and cutting the plastic tines off of the reel.  Those little plastic tines are expensive and very time consuming to replace.

Honey Bee had created six flex points along the length of the cutterbar, where we placed rugged precision pots to measure the displacement of the cutterbar in that region.  We placed another precision pot on the reel arm.  We were able to calculate interference between the reel and cutterbar, and drive the hydraulics to move the reel up and out of the way.

Life isn’t that simple, though.  We needed to have controls in the cab, to configure the system and monitor its operation.  We created an in-cab controller with small display, rotary dial and pushbuttons, which communicated to the controller on the implement using CAN.  At each end, the processor used was a Microchip PIC18F series MCU.

Over the winter, we performed extensive static testing on a flex header using a portable hydraulic pump, at PAMI in Portage la Prairie.  In the spring, we travelled to Honey Bee in Frontier SK to visit the plant and perform more tests.  Over the summer, Honey Bee did extensive testing all over, and I attended testing in southern Manitoba, North Dakota, Minnesota, and Illinois, riding on the combine, watching the system operation, and tweaking the control programming.

An unexpected challenge arose, that of the hydraulic capacity of different combines.  It turns out that some manufacturers have a huge hydraulic operative capacity, and others quite weak.  It’s difficult to accommodate different manufacturers, even with different programming.

Another issue was the retrofit of the controls to the combine.  The hydraulic controls weren’t meant to be tinkered with.  On some systems, we could just drive the solenoids directly.  On others, we had to diode isolate our controls from the existing controls.  On others, the diode isolation set off alarms indicating open circuit on the solenoids.

It was going to be a long haul to make the system compatible with all the potential combine systems.  In the end, Honey Bee could not invest the capital to make a flexible enough system, and Elecsys could not make this investment either.  A further challenge was that the market for the aftermarket flex header that Honey Bee was making, would not bear the several-hundred-dollar premium for such a controller.

Sadly, the controller project ended with only a handful of (very nice looking) prototypes made.

Developed the DR2/TESLA Main Processor Board (MPB) and Other DR2 & TESLA Boards

I used to say, “those aren’t electrons flowing on the MPB, those are my blood cells!”   In a philosophical sense, it was true – the development of the MPB drained me like no other project effort, before or since.

With multiple processors on board – 2 x PIC 16C MCUs (one for front panel keyboard, one for watchdog processor), a TI TMS320C32 DSP, an Altera Flex 10K FPGA, an Altera MAX 7000 CPLD, 3 x National Semiconductor LM12458 microsequenced ADCs, and a Motorola MC68HC11 MCU for IRIG processing, 3 x NS16550 FIFO UARTs, and a 16 bit PC-104 bus connection, it was quite busy.  An X86-based PC-104 card was mounted on top of the MPB – lower powered on the DR2, but much more powerful model on the TESLA.

The concept was that sifting, sorting, compression, and decision making was done on the DSP, where memory was expensive but calculations were easy.  The resultant data was streamed through a high-speed DSP-to-X86 FIFO, where the X86 DMA’d it into main memory and stored it to disk.  The X86 was used because its memory was plentiful and inexpensive, and disk I/O was fast and easy.

The other boards in the system weren’t as complex, but were just as important.  The DR2 AIB, mounted on the bottom of the chassis, had up to 18 miniature PCB-mount instrumentation transformers on it – CTs or PTs, depending on the application – and analog processing to transfer to the ADCs on the MPB.  The TESLA AIB was mounted on the rear of the chassis, had more channels but was less sophisticated – non isolated +/- 2.5V (nominal) input.  The DR2 and TESLA each had their own interpretation of digital I/O – DR2 with high current relay outputs, and dedicated ultra reliable digital inputs, and the TESLA with fewer and lower level outputs and simpler digital inputs.

The front panel of the DR2 had a 2 line x 24 character FIP display, a small keyboard, and several indicator LEDs, including a large bright red “Target” LED, as is expected on a relay.  The TESLA front panel had the smaller LEDs only.

I established the bus structure and pin-outs for all the I/O.  The front panel and digital I/O were intentionally designed to be interchangeable, with the intention of future mixed mode configuration, or expansion.  Although some of the buses had extra connections (front panel serial port, for instance), this was kept to the ends of the connectors, so combining and splitting ribbon cables would be relatively straightforward.  This was very well documented in extensive spreadsheets, which clearly showed connections and levels.

The memory and I/O maps of the processors was also documented in spreadsheets, with well defined bit states and clearly defined defaults.

Later, I wrote an exhaustive 70+ page programming guide to the MPB, which outlined all its features, how each processor interacted, the resources used, assumptions in programming, and how to compose images for execution.