I’m a bit of a time-nut…  well, not quite as adamant as some of those on that list  ( Allen deviations?   Seriously?   Who would talk about Allen that way?!? ), but I am crazy about keeping time, accurate time (well to within the millisecond or so, anyway), and distributing time.

IRIG-B Decoder Box

Since my first encounter with IRIG-B in, oh, was that 1988 (?), doing an IRIG-B decoder box at Vansco, on contract for Manitoba Hydro, I’ve been fascinated by the IRIG-B time code.  Here’s the official specification, and an unofficial site by someone trying to sell you something.  Truth be told, the original MC68xx (might have been MC68701 because it had both accumulators and more than one TCAP pin) IRIG-B decoder firmware was written by Filipe Fernandes…   then I took it from there, as Filipe got gobbled up by all the other stuff going on at Vansco 🙂  That decoder box took in modulated IRIG-B only, put out a contact for when it was valid or invalid, and had an RS-232 interface for another device to tell it the IRIG-B time.  What fun!

Extensions

Then came the IEEE 1344 extensions, which defined a use for the extra IRIG-B undefined bits, giving more information that originally intended.  The original IRIG-B decoder box did not support these extensions, but our next product did.

APT Power Technologies had IRIG-B

Anyways, while tangled up in all the other matters during development of the APT relay, like running the APT division, hiring staff, arranging trade shows, and designing the entire data acquisition system, I designed the hardware and ported & upgraded the firmware for the on-board MC68711 IRIG-B decoder.  Oh yes, and designed the time coordination system between the multiple processors in the system, which one had the master time and when, etc.  Very interesting.  During that time, we added unmodulated IRIG-B input as well.  The unit cycles between searching for valid modulated IRIG-B and unmodulated IRIG-B.

Getting IRIG-B for Testing

The only way that we had to generate IRIG-B was with an Arbiter 1084 satellite clock.  It would put out a modulated IRIG-B stream, and an unmodulated IRIG-B stream (TTL level), and a 1 pulse-per-second (generally referred to as 1 PPS or just “PPS”) (also TTL level).  There was also a serial interface – reminiscent of what we did in the original IRIG-B decoder box – but we rarely used it.  It could change settings on the IRIG-B interface signal, but we generally had models with the front panel buttons and display, so we changed parameters using that.

Satellite clocks were expensive back then.  Well, industrial grade ones still are – but we needed IRIG-B signals for testing – in product development, but also in manufacturing test.  So, we had one central satellite clock, and distributed IRIG-B through the building using CAT5 twisted pair, with specially marked RJ45 drops at developer’s desks, in production, in customer support, etc.

Experimenting with various formats was a pain.  We had to get access to the satellite clock (it was in the locked server room), change the settings, then get other users’ acceptance of the change…  oops, did I get that out of sequence?  Ha ha, I did get that out of sequence several times.  It turned out that most locations only had a single drop, so if I changed from unmodulated to modulated when users weren’t expecting it, ugh it caused us some heartache.  To the point where production got two separate direct wire drops.  Then, we had loading issues, so production got its own clock.  It didn’t have to be accurate, just functional, so I’m not sure that it was even always satellite locked.

Rudimentary Test Recordings

Sometimes, we recorded IRIG-B signals and just played them back.  After all, modulated IRIG-B was developed in the 50s (or was it 60s?) and was intended to be recorded on multi-track audio recorders and strip chart recorders for the telemetry of various military tests (think nuclear weapons testing).  I experimented with cassette tape recorderings in the early days, then with MP3 players like my little Samsung YP-U3 that we used on our trip to England & Scotland in 2005.  It had mixed results.  Modulated worked quite well, but unmodulated was unreliable – since it’s not really audio, and requires DC offset to be maintained, or restored, or something.

Enter tg

Around 2005, I found out that the NTP project had a program called tg in its “utils” subdirectory.   tg stands for “Timecode Generator”, and it was designed to put out modulated simulated IRIG-B or WWV through the audio output port of a Sun workstation computer (I suspect that this was the original target for the NTP executable).  I could be wrong, maybe it was a different *NIX system, but anyway, it didn’t work on the LINUX that I was running at the time, probably Mandrake LINUX.  So, I hacked it to use OSS to make it run on my computer.  What fun!

tg isn’t good enough for really accurate time sync!  It’s just good enough to test the decoder, test edge cases, etc.

Deano went a-hacking

Of course, I couldn’t leave well enough alone, so I Weiten-ified it, as Michael Miller used to say.  I added a slew of options and tweaks.  I added optional unmodulated IRIG-B output, tweaked its time code generation, added IEEE 1344 extension support, added the ability to start at an arbitrary specified time & date, added the ability to insert and remove leap seconds on demand, and much more.  I found that the audio card output sample rate wasn’t quite good enough over long term testing to keep accurate time – the output sample rate of 8,000 samples per second and the sinewaves were precisely, you know, 8 samples or 80 samples (IRIG or WWV) per cycle, so if the output sample rate were off by 0.1%, the frequency was off by 0.1%, and this would accumulate to eventually cause time drift.

So, I arranged to insert or remove a single cycle (1 mSec or 10 mSec) at what I decided was an innocuous part of the time code – at least for my decoders, heh heh.  The program compared the time to the LINUX real-time clock (which often is sync’d to global standard using NTP, how cool is that), and if the amount became detectable, it would slip in an extra cycle, or remove an extra cycle per second, to get back in sync.

I also added options to create file output instead of audio, so could be played back later (through audio port, as before, or through an Arbitrary Waveform Generator).

The Fork Less Travelled

So anyways, when I was done, I was quite proud of my work, so I sent it back to the NTP project – specifically to Dr. David Mills, chief NTP guy , and author of the book on network time sync – and I’m sure he was absolutely horrified at what I had done to the program.  In retrospect, I didn’t read nor respect the code formatting guidelines, I hacked and slashed the original audio driver code, yikes.  Dr. Mills was very polite, but wouldn’t accept my changes to tg on top of tg itself, but instead called it tg2.  If you download NTP today, you will see util/tg2 in there, waiting to be hacked some more 🙂

Oh, and I have hacked it some more.    Source code in tarball with RCS tracking available online.  I am trying to be more respectful of code formatting guidelines, and maintaining backward compatibility these days.

tg2‘s Greatest Hits?

Around 2006, I re-mastered a Knoppix CD with tg2 built in, so I could demo all its features on a live CD.  At the time, Jason Fuith and I were operating our own business called Elecsys Solutions (and starving, at least financially, while doing it), and we wanted to maybe sell it to folks like Krish Narendra, who was now in charge of Product Development at ERLPhase, and still faced all the problems of satellite clocks, IRIG-B distribution etc.

The idea that we had wasn’t necessarily to sell the code, but to sell support of it – maybe install it on custom embedded hardware, or put in specific customization, etc.

tg2 + Knoppix = NAN

Although interesting, it didn’t sweep Krish off his feet, and even at a modest cost, he could not justify the expenditure.  Well, I gave him the CD anyway, and I don’t think it went any further.

After Krish left ERLPhase, I worked out of his office for a while in 2018, and I stumbled across that CD, heh heh.  I gave it to Mark Poole, as he is now ERLPhase’s IRIG-B guy – and he was quite interested.  Mark and I had a long chat, and my next “hobby” project was born!  More on that later.

Meanwhile, Back doing Real Work

I tinkered with tg2 from time to time, per above, but didn’t do much with it.  In the meantime, Elecsys was purchased by Norscan Instruments and I went to work there as Product Development Manager for 3-1/2 years.  They didn’t have any specific interest in IRIG-B, NTP, or tg2 – they needed to make some money to pay our wages, imagine that!

Then back to ERLPhase for a bit, a great year at GE Multilin in Markham, back in Winnipeg for a short time, and off to Phoenix to work for Alstom Grid Digital Instrument Transformers, or Alstom Grid DIT (now GE Grid Solutions DIT).

Optical IRIG

Now, at DIT, they used satellite clocks all right – Arbiter 1084s, in fact, among others (like Reason, now GE, ones).  There was one important difference though – DIT used the now-standard orange 62.5/125 multimode fibre with 820 nm pulsed light (using the Avago/Broadcom HFPL1414TZ transmitter).  On the Arbiter 1084, this option is called “option 20”, and the single ST output can be configured to output IRIG-B unmodulated (of course 🙂 ), IRIG-B modified manchester coding, or 1 PPS.  DIT used the 1 PPS, because its primary use was to synchronize sampling / output of its product – what’s now known in the industry as a “primary converter merging unit” – so that it can be synchronized in time to other merging units in the system.

We did switch our satellite clocks back and forth between 1 PPS and IRIG-B unmodulated, when we were synchronizing with Reason (now GE) merging units.  I was intrigued.

Dean Scores!  ( an Arbiter 1084C satellite clock )

I always wanted a satellite clock, but, as I said before, they are expensive.  I started trolling for one in late 2015.  They are still expensive – generally selling for US700 used, and up to US2100 if new in box.  I just happened to find one selling at BMI Surplus for something like US250, talked them down to US236 inc shipping and tax, and it was mine, yay!  As a bonus, it was the Arbiter 1084C model, which has the big seven segment LED display on the front, way cool 🙂

It didn’t come with option 20, so I (carefully!) disassembled one of DIT’s 1084 clocks, made a parts list and took pictures, ordered what I needed, and installed them in my clock.  Now I have option 20, heh heh heh.

This came in handy for testing of the Reason merging unit with RogoFlex at PowerTech in Vancouver in August 2017, when I took my own 1084 clock to sync the systems together.

What to Do with a Satellite Clock?

My satellite clock didn’t come with a GPS antenna, so I got one through Amazon, of course, put that up with the many other antennas above my house, and got the clock working.

I had a lot of fun with the Arbiter 1084C and my oscilloscope – marvelling at the time code, watching the different signals, cross referencing the different formats (yes, I am that weird).

The clock was affected by a rollover bug.  I ordered new EPROMs for US$60 from Arbiter.  Easy peasy, replaced those, just like in my early days.

GPS Module Troubles

At one point, the clock stopped working properly.  Sad day!  The GPS module had died.  It seems that it’s writing something to the flash on the GPS module constantly, and it just wore out.

The module, an old Motorola OnCore unit, is of course obsolete.  There are non-drop-in replacements, but they aren’t binary compatible.  Arbiter will upgrade your unit – as I recall, it’s about US$300 – more than I paid for the satellite clock in the first place!  Now that was a very sad day.

I bought a SMT soldering/desoldering station, flash programmer and some spare flash parts, pulled the old flash off the board and rolled it out to disk… programmed the new part, but could not get it installed back on the board properly.  Seems as though I damaged the board removing the parts, so sad.

Fortunately, I found a (less) used GPS module of the exact same model, at some place in the far east.  I received it, plugged it in, and was back in business.

Oh, and the backup battery had of course died – being nickel-cadmium and probably 25 years old!  But, it had also corroded its connections badly.  I repaired the board and replaced the battery, and all is good again.   The battery is actually optional, probably should have just left it off.

Now What to do with a Satellite Clock?

The satellite clock sat running on my shelf for a year or so, before I got to thinking that I should use the IRIG-B signal output.  At first, I wanted to time sync a local computer and create my own (effectively) stratum 1 NTP time reference.  I don’t recall, I might have even had it working briefly, but what do I need a stratum 1 NTP time reference for?  That’s kind of a dead-end project, at least for me.

If You Can’t Generate ‘Em, Decode ‘Em?

Back to that conversation with Mark Poole, he mused that it would be nice to have a tool that could display the details of IRIG-B signals, so Customer Service, Applications and Product Development personnel, could trouble shoot issues, including various clocks that apparently give wrong or conflicting information, get the parity wrong, use reversed time zone-to-UTC offset etc.

So I started playing with NTP’s refclock_irig, not to sync my computer’s clock (in fact, I neutered it, so it would not), but to fully decode and display the signal.  Along the way, I added support for unmodulated IRIG-B as well – that was a challenge, since an audio input port is used to read the signal, and it does not maintain DC levels.  In fact, I did a lot of work on DC restoration à la NTSC TV signal recovery, back in the day.  That was fraught with peril, eventually causing a numerical overflow or a drifting DC baseline, so I eventually settled on a much simpler “look for fast shifts” solution, and that worked well.  I made the printout of the time code optional, using “fudge” tweak configuration bits already in the code, to dump output to the existing log, then tail -f in a separate console to display them.  I also generated some (huge) CSVs from time to time, both raw analog data (do you see what I see?) and various decode internals (do you decode what I decode?).  A lot of work to keep putting debug in, take it out, etc.  There are only 4 fudge tweak bits, and 3 of them are already in use… I pulled one back, so I could use 2, but still, too much hassle to tweak it inside the NTP framework.

Stand-Alone Decoder Program

Eventually, I undertook an effort to break my refclock_irig code away from NTP, creating read_irig.  This program stands alone, and supports too many options:

Read and decode modulated/unmodulated IRIG-B/modulated IRIG-E from audio signal, v0.43, 2019-05-08 dmw

RCS Info:
  Header: /home/dmw/src/ntp/refclock_irig/RCS/read_irig.c,v 1.57 2019/05/08 06:07:07 dmw Exp

Usage: read_irig [option]*

Options: -b                             Disable automatic input gain control
         -c <csv_output_filename>       Put out CSV file of bunch of internal values for portion of time
         -e <lines_between_headers>     Repeat full header every this many lines (default 20)
         -f <format_type>               Force only format:
                                             0 = Any (default)
                                             1 = Normal Modulated
                                             2 = Normal Unmodulated
                                             3 = Inverted Unmodulated
                                             4 = Normal or Inverted Unmodulated
         -g <initial_gain>              Initial gain setting (default 50), from 10 to 100
         -i <input_file>                Input file instead of audio input
         -j <csv_output_delay_samples>  Delay in samples for "-c" CSV output after startup (default 8000)
         -m                             Enable IRIG-B modulated bandpass filter
         -n                             Non-inverting input stream (normally input is inverted)
         -o <raw_signal_csv_out>        Put out CSV file of just raw input for as long as program runs
         -r <bitmap_of_outputs>         Bitmap of outputs:
                                             bit 0 = decoded data
                                             bit 1 = raw data
                                             bit 2 = do not repeat full header while in low verbosity
                                             bit 3 = 8 lines x 16 chars/line flat panel display output (disables all others)
                                             (default value shows raw and decoded data, and repeats full header periodically)
         -s <seconds_to_run>            Run this many seconds then exit (default forever), must be greater than 7
         -t <timeout_seconds>           Change timeout on successful decode, must be greater than 4.0 sec
         -v                             Increase verbosity of output
         -x                             Use 2nd stream (right? left?) instead of 1st
         -y                             Invert raw_signal_csv_out from "-o" option
         -z <threshold_amount>          Unmodulated edge detect threshold amount (default +/-0.20) in +/- amount of max

Notes: 1. Starting with "  Normal Unmodulated" format, default time out after about 64,000 samples
          (about 8.0 sec) without successful decode, then will cycle through "  Normal   Modulated",
          "  Normal Unmodulated", and "Inverted Unmodulated", with about 8.0 sec timeout on each format.
       2. Setting timeout on successful decode to less than about 8 sec may produce intermittent lock or
          failure to lock on modulated IRIG carrier.
       3. It can take a few seconds to lock onto a signal, so output may not be <seconds_to_run> frames.
       4. Modulated bandpass filter for IRIG-B tends to "shmoosh" the input signal and make it difficult
          to decode, and most IRIG-B signals are very clean, so it's not generally necessary.
       5. Default outputs include both raw and decoded data.  If both are turned off using "-r" option,
          decoded data is turned on alone.

This software licenced under the GPL, derived from refclock_irig, changes made 2018, 2019 by Dean Weiten
Contact: Dean Weiten, Winnipeg, MB, Canada, ph (204)-888-1334, E-mail dmw@weiten.com

The output, I think, is way cool:

#
#--------------------------------------------------------------------------------------------------||------------------------------------------------------------|
#         Inverted Unmodulated IRIG-B Raw (inverted audio) (Gain 100 giving -12.9 dB)              || Inverted Unmodulated IRIG-B (inv   , gain 100 -> -12.9 dB) |
# StrtBinSecs (SBS) |    Control Bits   |   Year  |    Day of Year    |  Hours  | Minutes |Seconds ||Day|   Date and Time    |SBS hex|Leap| DST |Offset|Qual|Par |
# ........ .........|......... .........|.... ....|       .. .... ....|  .. ....| ... ....|........||...|.... .. .. .. .. .. | . ....|....|.....|......|  ..|....|
#                   |                   |         |                   |         |         |        ||   |                    |       |    |     |      |    |    |
.001010000.000011001.000100000.010111000.000101001.000000001.001100000.000100001.001000011.00000101. 130 2019-05-10 11:23:05   0_A019 !lsp  DST  UTC-05   00 1 ok 
.001010000.000011010.000100000.010111000.000101001.000000001.001100000.000100001.001000011.00000110. 130 2019-05-10 11:23:06   0_A01A !lsp  DST  UTC-05   00 1 ok 
.001010000.000011011.000000000.010111000.000101001.000000001.001100000.000100001.001000011.00000111. 130 2019-05-10 11:23:07   0_A01B !lsp  DST  UTC-05   00 0 ok 
.001010000.000011100.000000000.010111000.000101001.000000001.001100000.000100001.001000011.00001000. 130 2019-05-10 11:23:08   0_A01C !lsp  DST  UTC-05   00 0 ok 
...

The raw and inverted decode sections can be enabled independently, and the program can auto-detect (slowly, mind you) modulated IRIG, unmodulated IRIG, and inverted unmodulated IRIG.

I’m still a sad case… I can sit and watch the codes go by, constantly.  It turns in a separate TTY on my computer, in the background, all the time.  Heh heh heh.

Source code in tarball with RCS tracking available online.

What Now?

In keeping with How Could They Possibly Improve Upon That?, you have to wonder where I could go from there.  Stay tuned! 🙂