Product Development – Page 4

Developed the PDR (Power Disturbance Recorder) into the DSR (Dynamic Swing Recorder)

Vansco was an important partner of the IAMC for the development of a new product for Manitoba Hydro – the Power Disturbance Recorder (PDR). This was an innovative system that had a Central Station and multiple Remote Stations, connected by modems. In practice, we had Remote Stations be as far as 1500 km (2400 mi) away, way up north on the Nelson River power system. The Remote Station would detect power disturbances and contact the Central Station to alert it of the occurrence. The Central Station would then connect all other Remote Stations and instruct them to capture a time-synchronized snapshot around the event.

The key concepts here were time synchronization and snapshot retrieval. The Remote Station processors had enough storage to keep very long buffers of data – so even if the disturbance disrupted communications for a time, events would still be captured, network-wide. The communications protocol between the Master and Remote Stations always had an initial multiple-pass time-synchronization exchange to resolve the clock offsets between the stations, without external synchronizing source. Of course, now time sync over wide area is easy, due to ubiquitous satellite navigation & timing systems, but at this time, the PDR implementation was innovative and worked well.

After IAMC closed down, ongoing PDR development was transferred to Vansco. Later, former IAMC developer Michael Miller joined our team and was instrumental in its ongoing development.

The PDR was renamed the Dynamic Swing Recorder (DSR) by our Manitoba Hydro team. That team, especially Dave Fedirchuk, felt that more utilities could use this technology. We tried to market it more widely, but later decided that a newer approach was required – creating a more generalized fault recorder, similar to the leaders at the time, the Rochester Instruments or Mehta Tech recorders. This is where the idea of the TESLA 2000 started to take hold.

Developed the DR2/TESLA Main Processor Board (MPB) and Other DR2 & TESLA Boards

I used to say, “those aren’t electrons flowing on the MPB, those are my blood cells!” In a philosophical sense, it was true – the development of the MPB drained me like no other project effort, before or since.

With multiple processors on board – 2 x PIC 16C MCUs (one for front panel keyboard, one for watchdog processor), a TI TMS320C32 DSP, an Altera Flex 10K FPGA, an Altera MAX 7000 CPLD, 3 x National Semiconductor LM12458 microsequenced ADCs, and a Motorola MC68HC11 MCU for IRIG processing, 3 x NS16550 FIFO UARTs, and a 16 bit PC-104 bus connection, it was quite busy. An X86-based PC-104 card was mounted on top of the MPB – lower powered on the DR2, but much more powerful model on the TESLA.

The concept was that sifting, sorting, compression, and decision making was done on the DSP, where memory was expensive but calculations were easy. The resultant data was streamed through a high-speed DSP-to-X86 FIFO, where the X86 DMA’d it into main memory and stored it to disk. The X86 was used because its memory was plentiful and inexpensive, and disk I/O was fast and easy.

The other boards in the system weren’t as complex, but were just as important. The DR2 AIB, mounted on the bottom of the chassis, had up to 18 miniature PCB-mount instrumentation transformers on it – CTs or PTs, depending on the application – and analog processing to transfer to the ADCs on the MPB. The TESLA AIB was mounted on the rear of the chassis, had more channels but was less sophisticated – non isolated +/- 2.5V (nominal) input. The DR2 and TESLA each had their own interpretation of digital I/O – DR2 with high current relay outputs, and dedicated ultra reliable digital inputs, and the TESLA with fewer and lower level outputs and simpler digital inputs.

The front panel of the DR2 had a 2 line x 24 character FIP display, a small keyboard, and several indicator LEDs, including a large bright red “Target” LED, as is expected on a relay. The TESLA front panel had the smaller LEDs only.

I established the bus structure and pin-outs for all the I/O. The front panel and digital I/O were intentionally designed to be interchangeable, with the intention of future mixed mode configuration, or expansion. Although some of the buses had extra connections (front panel serial port, for instance), this was kept to the ends of the connectors, so combining and splitting ribbon cables would be relatively straightforward. This was very well documented in extensive spreadsheets, which clearly showed connections and levels.

The memory and I/O maps of the processors was also documented in spreadsheets, with well defined bit states and clearly defined defaults.

Later, I wrote an exhaustive 70+ page programming guide to the MPB, which outlined all its features, how each processor interacted, the resources used, assumptions in programming, and how to compose images for execution.

TESLA 2000 Released

The TESLA 2000 was the most sophisticated disturbance recorder of its time. It used two MPBs, connected together by a bidirectional DSP-to-DSP synchronous serial link. This gave the TESLA 36 analog inputs.

However, it was the software which made the TESLA so outstanding. Although the serial port connection could be a pain to set up on MS-Windows, once it was set up, it worked like a dream. The RecordBase / RecordGraph system was amazing and intuitive. It was an instant hit in the Power Utility Community.

L-PRO and T-PRO Released

The delay was stressful, but the results were worth it! Manitoba’s very own 6 input (x 3 phase, always) protective relay became a reality!

The Analog Input Board (AIB) was configured to manage any mix of CTs and PTs, up to 18 discrete inputs.

The L-PRO came out first, then the T-PRO. Their primary hardware difference was the mix of CTs & PTs, and the T-PRO had a separate input for oil temperature transducer.

Challenged More and More: DR2 and TESLA on the Same Platform

We did focus group studies, discussed with customers, went to trade shows and conferences, and finally, settled on the development of two platforms for the power utility market: a TMS320C32-based platform for the relay, which we would call the DR2 (DR1 would have been the units developed using TMS320C30 at the UofM), and a TMS320C44-based platform for the recorder.

The ‘C3x family is fast and inexpensive. The ‘C44 family is well connected and easily networked. No comparable family at the time gave us both.

Management deemed this too expensive and risky, so we were told to go back to the drawing board and do it with only one platform. We gasped… sighed… and started over.

I can’t say that it was wrong. The development was successful… the product met market demand… and, with updates and changes, still in production today.

The core Main Processor Board developed as part of that effort, was in active production and use until being finally retired in 2018. Not bad!

System Undervoltage Controller on DR2 Platform

Manitoba Hydro required a special controller at the Dorsey Substation. We developed the System Undervoltage Controller (SUVC) to implement their algorithm on our new APT Relay platform, which we called DR2.

Developed an Alternator Limit Controller for a New Flyer Bus

New Flyer had a problem on their hands. They had shipped 26 buses to a property in southern California where they had omitted the little vendor-supplied 24V alternator from the Thermo-King air conditioning system, instead opting to use the main 24V bus to operate the fans. Unfortunately, at idle, the big, monster Delco 50DN alternator did not produce enough current to maintain the battery voltage, and batteries were going dead. When New Flyer offered to retrofit the vendor-supplied 24V alternator from Thermo-King, as was originally intended, the customer refused and wanted the little Motorola alternator to charge the main battery.

Unfortunately, parallel alternators do not share well – one will cook itself while the other one loafs. In this case, it’s even worse – the Motorola alternator was puny beside the 50DN, so you know there was going to be trouble.

At first, the New Flyer engineer counselled me to just draw a little current out of the base of the regulator’s drive transistor to keep the Motorola alternator from going over its rating. I protested that this wasn’t likely to work over component variation or temperature, but he insisted. We tried… and it didn’t work reliably. But, now he insisted that I was in the game and had to complete the job!

They brought a 40 foot bus over and parked it in front of Vansco, for me to work on. What fun that was, heh heh. I didn’t realize that buses often have pneumatic start – like an impact wrench – very loud and scary if you happen to be standing by the engine!

Well, I designed a module that used a Hall Effect sensor through which you would run the output wire of the Motorola alternator. We would open up the alternator and tap the base drive of the integrated regulator’s pass transistor. When the output current approached the alternator’s output limit, the circuit would draw away the base drive for the pass transistor, limiting the output current.

The assembly was a bit of a nightmare. It was a potted assembly with a hole in it for the alternator output wire to pass through. The potting would leak into the hole if the seal wasn’t solid, rendering the assembly useless. We had trouble with every batch. Fortunately, there were only 2 or 3 batches ever made!

Developed the Conviron Temperature Limit Controller (TLC)

Relatively simple in concept, this was a 3 channel comparator system based on one or two remote LM35 temperature sensors placed in the growth chamber, and two remote precision linear potentiometers mounted on the front panel . For simple chambers – those without the CMP3000 control system – it would function as a simple heat and/or cool controller.

For larger systems, it replaced the electro-mechanical failsafe alarm/shutdown system, providing an accurate, repeatable, easily settable minimum and/or maximum limit for chamber temperature.

It was a fairly simple circuit, but had to accomplish its task with only a single supply, single ended comparator chips, and one circuit board for multiple configurations. The system was designed to be fail-safe in the event of a sensor malfunction or open pot. There were configurations where its range was -30C to +30C, and others where it was +10C to +70C.

This product taught me straight analog circuitry, summing notes, differential nodes, analog hysterisis, fail-safe operation, and chatter reduction. Pretty cool for a fairly simple circuit!

Developing Specifications and Architecture for the Conviron CMP4000

The CMP3000 was starting to show its age in the early 1990s. It was agreed all around that we should start working on the next generation, the CMP4000.

I spent over 6 months pretty much full time working with Chuck Leibert, R&D Manager of Conviron, in the definition of the CMP4000 – specifying its feature set, minimum requirements, protocols, and expected technologies to be used.

During this time, I travelled to Indianapolis IN, where, after attending the T&D World Expo, I met up with Brian McCuskee, and we visited the Monsanto installation, discussing requirements with their staff, and getting good intelligence on where the product should go next.

Unfortunately, once the specification was completed, while waiting for the next stage of the development process to begin, Conviron was required to seek alternate bids for the development. A promised lower price, faster time to market, and pressure from the Government of Canada’s IRAP on who would get a grant for its development, and the work went instead to iders, another firm in Winnipeg. Sad day.

Worked on an IRIG-B Decoder

Manitoba Hydro wanted an IRIG-B decoder unit. It would listen to modulated IRIG-B, decode the time, and provide details of the time on a serial port. Filipe Fernandes and I did the system architecture. Filipe designed the board, based on a Motorola MC68HC11 development kit, and did the programming in assembly language.

The project was a success, but he was moved off to another project just as it was finishing, so I completed the programming. The design was a success.

This was my first exposure to IRIG-B, and I was fascinated.