Messaging on Computer

Of course, using Facebook Messenger on the computer is just as easy as on your phone…  or is your phone just as easy as computer?  No matter, it’s easy.  For other messengers, you have to get more creative.

WhatsApp

This one isn’t too difficult.  Go to the WhatsApp web site, then try to log in.  It will give you a QR code.  Go to your cell phone, get onto WhatsApp, go to the upper right menu, choose “WhatsApp Web” and point it to the QR code…  Boom!  You have your WhatsApp on your computer and on your phone.

You can’t have WhatsApp web on two computers at once.  If you sign on at work (say), then your home computer is logged out.  Not too bad though, since it’s easy to sign back in again.

This only works when the phone is on-line and has data through WiFi or its own data connection.

This is even easier.  If you are logged onto your Google account in your browser, just go to the Hangouts page, and there it is.  You can be logged in on as many computers as you want for this.

Hangouts works whether or not the phone is on, or connected.

SMS (Text Messaging)

There are two approaches here.  Both requires the phone to be on-line and have data (WiFi or its own data) and SMS access.

PushBullet

This is really cool, and was my go-to solution before I had Google Messages.  Put the PushBullet application on your phone, then go to the PushBullet web page and sign in on both.  You can send and receive your phone’s text messages.

There are two drawbacks to PushBullet.  One is that it costs money to sign up and keep it running (sorry, I forget how much).  The other is that, reading a text message on PushBullet does not mark it as read on your phone, so the notification stays up until you do read it on your phone (or at least cancel the notification).

Kik Messenger

I think here, you are out of luck.  For security reasons, you don’t want to use other company’s programs to dig into the Kik Messenger messages.

I find it cool that I still have the same telephone number that I had in high school.  Yes, if you knew what my phone number was in high school, you can still reach me there now.

Once upon a Land Line

My family moved to Headingley, just west of Winnipeg, in October 1971.  At that time, Headingley was an independent rural municipality, separate from the city of Winnipeg.  We got a rural telephone number (204)-864-xxxx, typical of St. Francois Xavier and Lido Plage, to the west and north, further out of town.  This meant that calling into Winnipeg was long distance, even though it was only about 9 km (5.5 miles) away!  Very annoying.

There was quite a fuss going on in Headingley at the time.  Manitoba Telephone System (MTS), the government owned monopoly on phone service, claimed that if we wanted to have local calls to Winnipeg, then we had to be part of the city.  That was crap.  Communities to the northeast of the city, like Lockport, had local calling to the city, and were not part of the city.  The city of Winnipeg was actually several independent cities at that time, each with its own council and mayor/reeve.

Nonetheless, around 1972 the province of Manitoba amalgamated all the independent cities in an effort called “Unicity”, and in so doing, subsumed Headingley into the City of Winnipeg.  Now, I think that the Unicity concept was a good thing, especially in the area of control of development – although on the other hand, Winnipeg hasn’t done a very good job of development control…  more on that, another time.

So, around 1972, my family got a new Winnipeg phone number, (204)-xxx-1334.   All of my friends knew this number, of course.  And I knew all my friends’ numbers.  It was what we did in those days!

Dean gets his Own Phone – in His Basement Lab

Around 1980 or so, while living at home, I was frustrated by not being able to use the phone when I wanted to, and I had decent income, so I got my own phone installed into my lab in the basement, (204)-xxx-5620.  I was so thrilled!  When I tried to call my best friend Dave, his number was disconnected… oh no!  It turns out that when I was at Polo Park getting my number assigned, Dave’s mom was at another MTS store getting a new number as well, and she got the one just before mine, (204)-xxx-5619.  My personal number is long gone now, but she still has hers, and that’s how I remember it – one less than mine, ha ha ha.

Grabs Control of Old Family Telephone Number

Later, when Dayna and I bought the house and moved out to Headingley with Eric, I called MTS and told them that they had my first initial wrong (my mom’s initial is “M”), so effectively transferred the phone to me, heh heh.  Later, we “ported” the number to Shaw, our cable TV and Internet provider.

When I moved away in 2011/2012, Dayna let the phone lapse.  I called Shaw to get my own phone, they told me it had lapsed, so I picked it up again.

Moving to VoIP… and to Arizona!

In 2013, when I moved to Phoenix, I ported it to VoIP provider Les.net.  I had a Grandstream HandyTone 286 adapter, with a 4-phone Uniden cordless telephone system.  It was very nice in Phoenix, as I could have several phones about the house.

Way cool, I had my long time Manitoba telephone number, and it rang in Phoenix.  Caused pollsters and telephone solicitors no end of confusion, hehehe.

While I was in Phoenix, I actually signed up for a couple of other Les.net numbers, including a conference call number in Winnipeg, conference call number in Phoenix, a toll-free (800) number, just going crazy 🙂

Toll Free Line

The toll free number was to encourage my mom, my son and my brother to call more often, if they had no long distance.

Conference Call

I used the conference call number for governing board teleconference meetings, when I was chair of SAE Arizona & Nevada Section.  It was a bit expensive, as it cost me 1.5 cents per minute per caller…   and, as I seem to recall, it cost me big one time that someone didn’t hang up before I did!

Features

All the telephone numbers rang to the same phone.  It worked well.  An additional feature was an embedded answering machine – a message left on any number would be converted to a WAV file and E-mailed to me.  I could get them anywhere, how convenient.

Caller ID Foibles

The one challenge that I’ve had using VoIP.ms is the caller ID information on outgoing calls.

I don’t know if it’s my VoIP desk phone or the VoIP.ms system, but it sometimes gives the Arizona number for calls made from line 1 (the Manitoba line) and the Manitoba number for calls made from line 2 (the Arizona line).

Eric also reported that his phone claimed I was calling from Egypt (country code +20), so his phone was misinterpreting the caller ID information.  The start of the “204” Manitoba area code was getting interpreted as the country code.  Contrary to what VoIP.ms tells you, when it asks you to give the 10 digit caller ID number, give them 11 digits, prepending the “1”, and that fixed the problem.

Incoming Calls: the One Challenge

For some reason, incoming calls have been a challenge.  The VoIP server must be able to route back to your VoIP phone, in order to ring it and set up the connection.  Years ago, this seemed easy, and it all worked.  Of course, I didn’t make any notes about how to set it up (argh).  But recently, I found out that neither of my numbers would ring through to my desk phone.  Curses.

My solution was to forward a bunch of ports through my firewall to my VoIP phone.  UDP ports 5060 (1st line), 5062 (2nd line), and 10000-10200, to be precise.  Then I had to deal with my dynamic IP resolution.  Well, I have a subscription to dyndns.org, so I can readily resolve (say) something.dyndns.org (or whatever, they have lots of options for TLDs) to my local firewall’s outside address.  The challenge is getting the dyndns.org  (oops, now that it’s owned by Oracle, they call it dyn.com) server updated.  In the old days, I used an old WRT-54G firewall/router flashed with DD-WRT.  That worked well back in the day, but I’ve abandoned it recently because the WRT-54G got a bit sluggish and lacked features (like 5 GHz band support).  If you go that route, be careful – there are a lot of WRT-54G variants, and not all of them can be flashed to DD-WRT… and some that can, are restricted in functionality once flashed.

Anyway, without my handy DD-WRT router, I set up a Raspberry Pi that runs inside my network, with ddclient connected to dyndns (see directions), which keeps the IP address on dyndns pointed to my firewall/router.

IRIG2LCD()

Well, I did it – what I’ve been trying to accomplish for some time now.  I have a “live” decode of IRIG-B to my little 5×7-character-based LCD display.

• d: show decoded data
• r: show raw data 3 lines at a time (60 bits at a time), and if raw data already showing, reverse the bit sequence
• t: show raw data 1 line at a time (20 bits at a time) with header, and if already showing, reverse the bit sequence
• u/l: move between windows of present format up/down (e.g. show next or previous 20 bits of raw data)
• h: freeze the display at the present time

The second thread takes a full structure of present time data and displays it on the screen, in accordance with the present format and situation.

Displayed Output

Reversed Raw Display

Reversed raw display is convenient because the bits are arranged from most significant (MSB) to least significant (LSB) from left to right, which is convenient for most people to read.

What’s Next?

I was thinking that I might introduce a third display mode, that of raw data with headers, more like the console output of read_irig:

#
#--------------------------------------------------------------------------------------------------||------------------------------------------------------------|
#           Normal Unmodulated IRIG-B Raw  (noninv audio)  (Gain  93 giving  +1.0 dB)              ||   Normal Unmodulated IRIG-B (noninv, gain  93 ->  +1.0 dB) |
# StrtBinSecs (SBS) |    Control Bits   |   Year  |    Day of Year    |  Hours  | Minutes |Seconds ||Day|   Date and Time    |SBS hex|Leap| DST |Offset|Qual|Par |
# ........ .........|......... .........|.... ....|       .. .... ....|  .. ....| ... ....|........||...|.... .. .. .. .. .. | . ....|....|.....|......|  ..|....|
#                   |                   |         |                   |         |         |        ||   |                    |       |    |     |      |    |    |
.001010101.010111101.000100000.010111000.000101001.000000001.010000000.000100010.000001000.01001001. 140 2019-05-20 12:08:29   0_AABD !lsp  DST  UTC-05   00 1 ok
.001010101.010111110.000000000.010111000.000101001.000000001.010000000.000100010.000001000.01100000. 140 2019-05-20 12:08:30   0_AABE !lsp  DST  UTC-05   00 0 ok

And Then, and Then!?!

You think that would be enough for me?  Well, apparently not!  read_irig takes very little computing power, so it could easily run on a smaller processor…  say, an ARM… on a Raspberry Pi, or a Beagleboard – then it could be made portable.  I have two Beagleboard Blacks and like them.  The problem here is that none of these boards have easy audio input ports, like my laptop.  Some do have ADC inputs, but they need to be protected, amplified, provided with a jack, etc.  Hmm, have to think about this.

Almost every device these days has an I2C port, so there shouldn’t be much trouble there.

Talking to an I2C Character Based Display

Shortly after I got the SPIDriver, I obtained the similar I2CDriver through CrowdSupply.  It’s made by the same folks, and has a very similar interface.  The software provided is different though.  Anyways…  I managed to figure out the ‘C’ programming language interface, added a missing “i2c_stop” prototype to the header file, and get it working in test.

I purchased a TC2004A LCD display from Amazon, here is the datasheet:  TC2004A-01 .

The I2C Interface

I2C means Inter-IC-Communication, with the “2” originally meant to be superscripted, as in “I squared C”.  It was developed by Phillips Semiconductor in the 1980s.  It’s a relatively slow, but pin-efficient interface… not as efficient as Dallas Semiconductor’s “1-wire” interface, but more ubiquitous – available everywhere, and adapted for many uses.

The I2C Display

Anyways, modern LCD displays all seem to have a built-in I2C interface. It’s slow, but it works.  More on the speed, and tricks to get around this, later.

I2C Conversion Chip

The old parallel interface appears to still be there, but there’s almost always a daughterboard attached containing a PCF8574 I2C-to-parallel conversion chip.  Here’s the datasheet for the conversion chip : PCF8574 .

Display Controller

The original Hitachi HD44780 controller chip is still there, here is datasheet: HD44780 .  There appears to be a second source now, the Sitronix ST7066U, here is datasheet: ST7066U_v2.4 .

The HD44780 and friends, always did have a 4/8 bit bus, based on the Motorola MC6800 bus interface – probably because Hitachi second-sourced a bunch of Motorola MC68xx parts in the 1980s.  But I digress 🙂

Anyway, this bus had two control lines:

• RW – read/write.  This is actually “read/not write” but many source code languages don’t like the embedded use of slashes in symbol names, and  it’s difficult to do an above bar in most simple word processors.  Anyways, this is high for read and low for write.  Its state is latched on the rising edge of E and must be stable through the end of E high.
• E – enable.  This is an active-high strobe.

The HD44780 has one address pin called RS for register select.  Like RW, this is latched on the rising edge of E and must be stable through the end of E high.  When RS is low, the command/status register is accessed.  When RS is high, data memory is accessed.

There are eight data lines, D0 through D7.  Generally, outgoing data must be valid before the rising edge through the falling edge of E.  Most devices latch the data in on the falling edge of E, although this is not always the case.

Connections from I2C Conversion Chip to Display Controller

With only 8 bits total output from the I2C conversion chip, the display only implements the 4 bit interface.  The 4 bit interface is made up of only the top 4 data lines, D4 through D7.

 HD44780 Input PCF8574 Output RS P0 RW P1 E P2 D4 P4 D5 P5 D6 P6 D7 P7

To control the HD44780, the bus interface has to be emulated by bit-banging.  It takes multiple I2C transfers to do a single transaction.

More than Two Lines

The original HD44780 could control one or two line character displays with lines that were up to 64 characters in width.  Today, displays often have more than two lines, but generally not nearly as wide as 64 characters – most often, not even half this…  so they split the lines.

The original internal display ram was 128 bytes of addressable memory.  The top line at addresses 0x00 to 0x00+LINE_LENGTH-1, the lower line at addresses 0x40 to 0x40+LINE_LENGTH-1.  The rest of the memory was present and could be used as scratchpad memory, with no effect on the display.

What modern displays appear to do, is split the display into multiple 2-line parts.  The top two lines work just like the original, using addresses 0x00 to 0x00+LINE_LENGTH-1, and 0x40 to 0x40+LINE_LENGTH-1, as before.  The third line uses addresses 0x00+LINE_LENGTH to 0x00+(2*LINE_LENGTH)-1, the fourth line uses 0x40+LINE_LENGTH to 0x40+(2*LINE_LENGTH)-1.

So, in my case, for a 4 line by 20 character display, the lines start at 0x00, 0x40, 0x14 (decimal 20) and 0x54 (0x80+decimal 20), respectively.

There are also devices that have larger displays – more rows, or longer lines that don’t allow them to play the “split original 2 lines” trick.  For this, the common approach is to use two HD44780 controllers (or equivalent), and have 2 “E” signals.

Accessing the Display

As with the SPIDriver, there are an assortment of drivers and sample programs provided to play with.  I built the little i2ccl ‘C’ program that was provided, and that showed me the basics of access.  I reviewed the lcd1602.py script.  That gave me everything I needed.  I adapted a very old LCD access library that I wrote many years ago, do use the i2cdriver functions.  I wrote a ‘C’ program called TestLcd which exercised these libraries, and found all kinds of errors (of course).

Embrace…   CG Characters

One of the cool features of the HD44780 type displays is their ability to display compose characters, or CG characters.  You can compose up to 8 arbitrary different 5×7 patterns, store the patterns in RAM, and access them just like regular characters.  This is helpful, because, well, the character set is a little weird – having some kind of foreign symbol where backslash should be, value 0x5C – see below.

It can also be cool for creating industry-specific compose characters too.  For instance, my first exposure to the HD44780 was in the late ’80s, where we designed it into the Kodiak Scoretec scoreboard system (I developed the electronics for the remote keyboard assembly, eventually became responsible for support of the entire system).  It had little numbers for periods, number of fouls, a little football for possession, special characters for bonus etc.

Extend…   CG Palette

The LCD library has the concept of a CG palette.  There are far more than 8 compose characters defined, but only 8 can be used at any given time.  When this library tries to display a CG character, it checks to see if it’s already in the palette.  If it is, it uses it.  If it is not, it finds an available CG location, loads it with the desired CG character, and uses it.  If there are no CG locations available, it displays a question mark ‘?’.

I added a function to this, so that if no CG locations are available, every present CG character is cross checked to every character in the display RAM, to see if that CG character is presently in use.  The first CG location that’s not in use, is removed to make room for the new CG character.  At the end, if there just aren’t enough CG locations, again a question mark ‘?’ is displayed.

Extinguish…   Backlight and Display Clear

I changed the clear() function to use the direct clear command, instead of writing spaces out to the screen.  Much faster!

I also added a function to control the backlight, seeing as it can be turned on and off using I2C.

Speed, Speed…   Always Need More Speed!

It turns out that accessing the display through I2C is quite slow.  Now, I found a few easy things to speed things up… and there may be places where I can save time, but are more complex.  However, those may be for a later time.

I modified the library to check whether a character is already in the display, before writing it out.  If it is already there, it skips the write, saving precious I/O time.

Compared to the I/O time, the processor time is nothing, so this speeds things up considerably, especially if very little changes from second to second…  like maybe, in an IRIG-B decoder display, hmmmmmm?

Let the (Graphical) Fun Begin!

I bought a cool device called SPIDriver through CrowdSupply.  Actually, I bought two – and got them each with a neat little SPI based 132 x 162 colour LCD display.

It’s a very neat device, appearing as a USB serial port to your system.  On my LINUX system, it appears as /dev/ttyUSBn.  There are several sample programs and libraries available to drive SPIDriver, including for the display.

The display operates with a Sitronix ST7735 controller – here are a couple of different version datasheets: ST7735 ST7735S_v1.1 .

Anyway, I hacked the included Python program st7735s.py to do my own bidding.   irig_to_st7735.py takes STDIN text, uses the PIL library to rasterize it into graphics, and paints it into the little display.

Unfortunately, it takes about 1-1/2 seconds to get it transferred onto the display – it is a full graphics display, and it is a SPI interface, after all.  I added an option to read_irig to put out 8 lines of text every 2 seconds, which I then piped into the Python program.  It worked fine.  Not terribly useful, but fine 🙂

#!/usr/bin/env python3
# coding=utf-8

#***************
# Import functions.
#---------------
import array
import getopt
import struct
import sys
import time

from PIL import Image, ImageDraw, ImageFont
from spidriver import SPIDriver

#***************
# Constants.
#---------------
Version			= 0
Issue			= 3
IssueDate		= "2019-05-06"
DefaultDevice	= "/dev/ttyUSB0"

LightBlue	= (102, 255, 255)
PaleBlue	= (102, 204, 255)
Red			= (255,   0,   0)
Yellow		= (255, 255,   0)
Lime		= (  0, 255,   0)
White		= (255, 255, 255)

NOP = 0x00
SWRESET = 0x01
RDDID = 0x04
RDDST = 0x09

SLPIN = 0x10
SLPOUT = 0x11
PTLON = 0x12
NORON = 0x13

INVOFF = 0x20
INVON = 0x21
DISPOFF = 0x28
DISPON = 0x29
CASET = 0x2A
RASET = 0x2B
RAMWR = 0x2C
RAMRD = 0x2E

PTLAR = 0x30
COLMOD = 0x3A

FRMCTR1 = 0xB1
FRMCTR2 = 0xB2
FRMCTR3 = 0xB3
INVCTR = 0xB4
DISSET5 = 0xB6

PWCTR1 = 0xC0
PWCTR2 = 0xC1
PWCTR3 = 0xC2
PWCTR4 = 0xC3
PWCTR5 = 0xC4
VMCTR1 = 0xC5

RDID1 = 0xDA
RDID2 = 0xDB
RDID3 = 0xDC
RDID4 = 0xDD

PWCTR6 = 0xFC

GMCTRP1 = 0xE0
GMCTRN1 = 0xE1

DELAY = 0x80

#***************
# Pure Python rgb to 565 encoder for portablity
#---------------
def as565(ProcessedImage):
#print ("ProcessedImage:", ProcessedImage)

OriginalRed, OriginalGreen, OriginalBlue = [list(c.getdata()) for c in ProcessedImage.convert("RGB").split()]

def MultiplyAndShift(ColourValue, ShiftBy):
return ColourValue * (2 ** ShiftBy - 1) // 255

d565 = [(MultiplyAndShift(BlueValue, 5) << 11) | (MultiplyAndShift(GreenValue, 6) << 5) | MultiplyAndShift(RedValue, 5) for (RedValue, GreenValue, BlueValue) in zip(OriginalRed, OriginalGreen, OriginalBlue)]

d565h = array.array('H', d565)
#print ("d565h:", d565h)
d565h.byteswap()
d565s = d565h.tostring()
#print ("d565s:", d565s);
return array.array('B', d565s)

def debug565(OriginalColour):
print ("OriginalColour:", OriginalColour)

OriginalRed, OriginalGreen, OriginalBlue = OriginalColour

def MultiplyAndShift(ColourValue, ShiftBy):
return ColourValue * (2 ** ShiftBy - 1) // 255

d565 = [(MultiplyAndShift(OriginalBlue, 5) << 11) | (MultiplyAndShift(OriginalGreen, 6) << 5) | MultiplyAndShift(OriginalRed, 5)]

d565h = array.array('H', d565)
print ("d565h:", d565h)
d565h.byteswap()
d565s = d565h.tostring()
print ("d565s:", d565s);
return array.array('B', d565s)

#***************
# Class for wrangling with the ST7735 160 x 128 dot display
#---------------
class ST7735:
def __init__(self, sd):
self.sd = sd
self.sd.unsel()

def write(self, a, c):
self.sd.seta(a)
self.sd.sel()
self.sd.write(c)
self.sd.unsel()

def writeCommand(self, cc):
self.write(0, struct.pack("B", cc))

def writeData(self, c):
self.write(1, c)

def writeData1(self, cc):
self.writeData(struct.pack("B", cc))

def cmd(self, cc, args=()):
self.writeCommand(cc)
n = len(args)
if n != 0:
self.writeData(struct.pack(str(n) + "B", *args))

def setAddrWindow(self, x0, y0, x1, y1):
self.writeData(struct.pack(">HH", x0, x1))
self.writeData(struct.pack(">HH", y0, y1))
self.writeCommand(RAMWR)  # write to RAM

def rect(self, x, y, w, h, color):
self.setAddrWindow(x, y, x + w - 1, y + h - 1)
self.writeData(w * h * struct.pack(">H", color))

def start(self):
self.sd.setb(0)
time.sleep(.001)
self.sd.setb(1)
time.sleep(.001)

self.cmd(SWRESET)   # Software reset, 0 args, w/delay
time.sleep(.180)
self.cmd(SLPOUT)    # Out of sleep mode, 0 args, w/delay
time.sleep(.180)

commands = [
(FRMCTR1, (     # Frame rate ctrl - normal mode
0x01, 0x2C, 0x2D)),  # Rate = fosc/(1x2+40) * (LINE+2C+2D)
(FRMCTR2, (     # Frame rate control - idle mode
0x01, 0x2C, 0x2D)),  # Rate = fosc/(1x2+40) * (LINE+2C+2D)
(FRMCTR3, (     # Frame rate ctrl - partial mode
0x01, 0x2C, 0x2D,  # Dot inversion mode
0x01, 0x2C, 0x2D)),  # Line inversion mode
(PWCTR1, (      # Power control
0xA2,
0x02,       # -4.6V
0x84)),     # AUTO mode
(PWCTR2, (      # Power control
0xC5,)),    # VGH25 = 2.4C VGSEL = -10 VGH = 3 * AVDD
(PWCTR3, (      # Power control
0x0A,       # Opamp current small
0x00)),     # Boost frequency
(PWCTR4, (      # Power control
0x8A,       # BCLK/2, Opamp current small & Medium low
0x2A)),
(PWCTR5, (      # Power control
0x8A, 0xEE)),
(VMCTR1, (      # VCOM control
0x0E,)),
(MADCTL, (      # Memory access control (directions)
(COLMOD, (      # set color mode
0x05,)),    # 16-bit color
(GMCTRP1, (     # Gamma + polarity Correction Characterstics
0x02, 0x1c, 0x07, 0x12,
0x37, 0x32, 0x29, 0x2d,
0x29, 0x25, 0x2B, 0x39,
0x00, 0x01, 0x03, 0x10)),
(GMCTRN1, (     # Gamma - polarity Correction Characterstics
0x03, 0x1d, 0x07, 0x06,
0x2E, 0x2C, 0x29, 0x2D,
0x2E, 0x2E, 0x37, 0x3F,
0x00, 0x00, 0x02, 0x10)),
(NORON, ()),    # Normal display on
(DISPON, ()),   # Main screen turn on
]
for c, args in commands:
self.cmd(c, args)

def clear(self):
self.rect(0, 0, 128, 160, 0x0000)

def writestrings(self, ListOfLineStrings):
#print ("On entry into writestrings(), ListOfLineStrings: ", ListOfLineStrings)

#print ("Starting writestrings")
BaseWidth  = 160
BaseHeight = 128

# make a blank image for the text, initialized to transparent text color
TextImage = Image.new('RGB', (BaseWidth, BaseHeight), ( 16,  16,  16))

TextSize = int(BaseHeight/ 8)
LineColours = (White, Lime, Lime, PaleBlue, LightBlue, LightBlue, Yellow, Red)
#print ("Length of list of LineColours: " + "{0:d}".format(len(LineColours)))
StartColumnOffset = 0

# get a font
#FontToWrite = ImageFont.truetype('Pillow/Tests/fonts/FreeMono.ttf', int(TextSize))
FontToWrite = ImageFont.truetype('Courier_New.ttf', int(TextSize))
# get a drawing context
DrawContext = ImageDraw.Draw(TextImage)

# draw text, full opacity
LineNumber = 1
for LineString in ListOfLineStrings:
#print ("LineString: " + LineString)
if  (LineNumber > len(LineColours)):
print ("Past end of colour list, LineNumber = " + "{0:d}".format(LineNumber) + " and length of LineColours list = " + "{0:d}".format(len(LineColours)))
print ("Length of ListOfLineStrings = " + "{0:d}".format(len(ListOfLineStrings)))
print ("ListOfLineStrings: ", ListOfLineStrings )
print ()
LocalColour = White
else:
LocalColour = LineColours[LineNumber-1]

DrawContext.text((StartColumnOffset, (LineNumber-1)*TextSize), LineString, font=FontToWrite, fill=LocalColour)
LineNumber += 1

FinalImage = TextImage

# debug by saving images to disk
#print ("Output 01txt.jpg")
#SaveImage = TextImage.convert('RGB')
#SaveImage.save("01text.jpg")

#print ("Ouput 02base.jpg")
#SaveImage = BaseImage.convert('RGB')
#SaveImage.save("02base.jpg")

#print ("Ouput 03final.jpg")
#SaveImage = FinalImage.convert('RGB')
#SaveImage.save("03final.jpg")

#print ("FinalImage.size")
#print (FinalImage.size)

if FinalImage.size[0] > FinalImage.size[1]:
#print ("Rotating 90 degrees")
FinalImage = FinalImage.transpose(Image.ROTATE_90)
#w = 160 * FinalImage.size[0] // FinalImage.size[1]
#FinalImage = FinalImage.resize((w, 160), Image.ANTIALIAS)
#(w, h) = FinalImage.size
#if w > 128:
#FinalImage = FinalImage.crop((w // 2 - 64, 0, w // 2 + 64, 160))
#elif w < 128:
#c = Image.new("RGB", (128, 160))
#c.paste(FinalImage, (64 - w // 2, 0))
#FinalImage = c
#st.writeData(as565(FinalImage.convert("RGB")))
st.writeData(as565(FinalImage))

#***************
# Function to provide usage help.
#---------------
def usage():
print ("\nTake in read_irig LCD output and display to ST7735 display attached to SPIDriver, v"+"%1d" % (Version)+"."+"%1d" % (Issue)+" "+IssueDate+" dmw")
print ("\nTypical usage: "+sys.argv[0]+" [option]* ")
print ("\n      Options: ")
print   ("               -o <device>           Output device for SPIDriver (default " + DefaultDevice + ")")
print   ("               -v                    More verbose")

print ("\n      RCS Info:")
print   ("               Header: /home/dmw/src/ntp/refclock_irig/RCS/irig_to_st7735.py,v 1.4 2019/05/17 03:37:27 dmw Exp")

print ("\n")

#***************
# Main function.
#---------------
if __name__ == '__main__':
#***************
# Get command line options.  Error results in help text dump and exit.
#---------------
try:
opts, args = getopt.getopt(sys.argv[1:], "o:v")
except getopt.GetoptError as Error:
# print help information and exit:
print ("\n")
print (Error) # will print something like "option -a not recognized"
print ("\n------------------------------")
usage()
sys.exit(2)

#***************
# Set defaults.
#---------------
UseDevice	= DefaultDevice
Verbose		= False

#print ("Checking options now")

#***************
# Parse values from command line options.  Error results in message and exit.
#---------------
for Option, Argument in opts:
#print ("Checking option: " + Option + ", with argument: " + Argument)
if   Option in ("-o"):              # Output file name.
UseDevice = Argument
#print ("\nUsing device: " + UseDevice)
elif Option in ("-v"):              # Turn on verbosity.
Verbose = True
else:
print ("\nUnknown option \"" + Option + " " + Argument + "\", aborting...")
print ("\n------------------------------")
usage()
sys.exit(2)

if  (Verbose):
print("\nLightBlue")
debug565(LightBlue)

print("\nPaleBlue")
debug565(PaleBlue)

print("\nRed")
debug565(Red)

print("\nYellow")
debug565(Yellow)

print("\nLime")
debug565(Lime)

print ("\nWhite")
debug565(White)

print ()
print ("\nUsing device: " + UseDevice)

print ()

#exit(0)

#***************
# Open ST7735 display through SPIDriver, initialize and clear.
#---------------
st = ST7735(SPIDriver(UseDevice))
st.start()
st.clear()

#***************
# Initial message.
#---------------
LineList = [
#         1111111
#1234567890123456
"  IRIG Decoder",
"v"+"%1d" % (Version)+"."+"%1d" % (Issue)+" "+IssueDate,
"    (c) 2019",
"  Dean Weiten",
"  Winnipeg, MB",
" (204)-888-1334",
" dmw@weiten.com"]
st.writestrings( LineList )

#***************
# Loop on input.
# Expecting up to 8 lines per frame,
# up to 16 characters per line.
# Colours are fixed sequence.
# An exception (often a ctrl-C break)
# results in clearing and exit message display.
#---------------
try:
LineIndex = 0
LineList = []
for line in sys.stdin:
FindFF = line.find('\f')
if  (FindFF>=0):
if  (Verbose):
print ("Got FF at " + "{0:d}".format(FindFF))
print( "The line up to the FF is \"" + line[:FindFF] + "\"." )

LineToAppend = line[:FindFF].rstrip("\r\n\f")
if  (len(LineToAppend) > 0):
LineList.append(LineToAppend)
st.writestrings( LineList )
LineList = []
LineToAppend = line[FindFF+1:].rstrip("\r\n\f")
if  (len(LineToAppend) > 0):
#print ("First line \"" + LineToAppend + "\" has length " + "{0:d}".format(len(LineToAppend)) + ", so will be appended, list is at present: \"", LineList, "\".")
LineList.append(LineToAppend)
else:
LineToAppend = line.rstrip("\r\n\f")
if  (len(LineToAppend) > 0):
#print ("Line \"" + LineToAppend + "\" has length " + "{0:d}".format(len(LineToAppend)) + ", so will be appended.")
LineList.append(LineToAppend)
#print ("Length of LineList after append is " + "{0:d}".format(len(LineList)) + ".")

if  (Verbose):
print( "The line list is:" )
print(LineList)
#st.writestrings(LoopNumber)
#LoopNumber += 1

finally:
st.clear()

#time.sleep(3)

LineList = [
#         1111111
#1234567890123456
"  IRIG Decoder",
"v"+"%1d" % (Version)+"."+"%1d" % (Issue)+" "+IssueDate,
"",
"    Exiting"]
st.writestrings( LineList )

time.sleep(4)

st.clear()


That’s *Mister* IRIG to you, sir…

I’m a bit of a time-nut…  well, not quite as adamant as some of those on that list  ( Allen deviations?   Seriously?   Who would talk about Allen that way?!? ), but I am crazy about keeping time, accurate time (well to within the millisecond or so, anyway), and distributing time.

IRIG-B Decoder Box

Since my first encounter with IRIG-B in, oh, was that 1988 (?), doing an IRIG-B decoder box at Vansco, on contract for Manitoba Hydro, I’ve been fascinated by the IRIG-B time code.  Here’s the official specification, and an unofficial site by someone trying to sell you something.  Truth be told, the original MC68xx (might have been MC68701 because it had both accumulators and more than one TCAP pin) IRIG-B decoder firmware was written by Filipe Fernandes…   then I took it from there, as Filipe got gobbled up by all the other stuff going on at Vansco 🙂  That decoder box took in modulated IRIG-B only, put out a contact for when it was valid or invalid, and had an RS-232 interface for another device to tell it the IRIG-B time.  What fun!

Extensions

Then came the IEEE 1344 extensions, which defined a use for the extra IRIG-B undefined bits, giving more information that originally intended.  The original IRIG-B decoder box did not support these extensions, but our next product did.

Anyways, while tangled up in all the other matters during development of the APT relay, like running the APT division, hiring staff, arranging trade shows, and designing the entire data acquisition system, I designed the hardware and ported & upgraded the firmware for the on-board MC68711 IRIG-B decoder.  Oh yes, and designed the time coordination system between the multiple processors in the system, which one had the master time and when, etc.  Very interesting.  During that time, we added unmodulated IRIG-B input as well.  The unit cycles between searching for valid modulated IRIG-B and unmodulated IRIG-B.

Getting IRIG-B for Testing

The only way that we had to generate IRIG-B was with an Arbiter 1084 satellite clock.  It would put out a modulated IRIG-B stream, and an unmodulated IRIG-B stream (TTL level), and a 1 pulse-per-second (generally referred to as 1 PPS or just “PPS”) (also TTL level).  There was also a serial interface – reminiscent of what we did in the original IRIG-B decoder box – but we rarely used it.  It could change settings on the IRIG-B interface signal, but we generally had models with the front panel buttons and display, so we changed parameters using that.

Satellite clocks were expensive back then.  Well, industrial grade ones still are – but we needed IRIG-B signals for testing – in product development, but also in manufacturing test.  So, we had one central satellite clock, and distributed IRIG-B through the building using CAT5 twisted pair, with specially marked RJ45 drops at developer’s desks, in production, in customer support, etc.

Experimenting with various formats was a pain.  We had to get access to the satellite clock (it was in the locked server room), change the settings, then get other users’ acceptance of the change…  oops, did I get that out of sequence?  Ha ha, I did get that out of sequence several times.  It turned out that most locations only had a single drop, so if I changed from unmodulated to modulated when users weren’t expecting it, ugh it caused us some heartache.  To the point where production got two separate direct wire drops.  Then, we had loading issues, so production got its own clock.  It didn’t have to be accurate, just functional, so I’m not sure that it was even always satellite locked.

Rudimentary Test Recordings

Sometimes, we recorded IRIG-B signals and just played them back.  After all, modulated IRIG-B was developed in the 50s (or was it 60s?) and was intended to be recorded on multi-track audio recorders and strip chart recorders for the telemetry of various military tests (think nuclear weapons testing).  I experimented with cassette tape recorderings in the early days, then with MP3 players like my little Samsung YP-U3 that we used on our trip to England & Scotland in 2005.  It had mixed results.  Modulated worked quite well, but unmodulated was unreliable – since it’s not really audio, and requires DC offset to be maintained, or restored, or something.

Enter tg

Around 2005, I found out that the NTP project had a program called tg in its “utils” subdirectory.   tg stands for “Timecode Generator”, and it was designed to put out modulated simulated IRIG-B or WWV through the audio output port of a Sun workstation computer (I suspect that this was the original target for the NTP executable).  I could be wrong, maybe it was a different *NIX system, but anyway, it didn’t work on the LINUX that I was running at the time, probably Mandrake LINUX.  So, I hacked it to use OSS to make it run on my computer.  What fun!

tg isn’t good enough for really accurate time sync!  It’s just good enough to test the decoder, test edge cases, etc.

Deano went a-hacking

Of course, I couldn’t leave well enough alone, so I Weiten-ified it, as Michael Miller used to say.  I added a slew of options and tweaks.  I added optional unmodulated IRIG-B output, tweaked its time code generation, added IEEE 1344 extension support, added the ability to start at an arbitrary specified time & date, added the ability to insert and remove leap seconds on demand, and much more.  I found that the audio card output sample rate wasn’t quite good enough over long term testing to keep accurate time – the output sample rate of 8,000 samples per second and the sinewaves were precisely, you know, 8 samples or 80 samples (IRIG or WWV) per cycle, so if the output sample rate were off by 0.1%, the frequency was off by 0.1%, and this would accumulate to eventually cause time drift.

So, I arranged to insert or remove a single cycle (1 mSec or 10 mSec) at what I decided was an innocuous part of the time code – at least for my decoders, heh heh.  The program compared the time to the LINUX real-time clock (which often is sync’d to global standard using NTP, how cool is that), and if the amount became detectable, it would slip in an extra cycle, or remove an extra cycle per second, to get back in sync.

I also added options to create file output instead of audio, so could be played back later (through audio port, as before, or through an Arbitrary Waveform Generator).

The Fork Less Travelled

So anyways, when I was done, I was quite proud of my work, so I sent it back to the NTP project – specifically to Dr. David Mills, chief NTP guy , and author of the book on network time sync – and I’m sure he was absolutely horrified at what I had done to the program.  In retrospect, I didn’t read nor respect the code formatting guidelines, I hacked and slashed the original audio driver code, yikes.  Dr. Mills was very polite, but wouldn’t accept my changes to tg on top of tg itself, but instead called it tg2.  If you download NTP today, you will see util/tg2 in there, waiting to be hacked some more 🙂

Oh, and I have hacked it some more.    Source code in tarball with RCS tracking available online.  I am trying to be more respectful of code formatting guidelines, and maintaining backward compatibility these days.

tg2‘s Greatest Hits?

Around 2006, I re-mastered a Knoppix CD with tg2 built in, so I could demo all its features on a live CD.  At the time, Jason Fuith and I were operating our own business called Elecsys Solutions (and starving, at least financially, while doing it), and we wanted to maybe sell it to folks like Krish Narendra, who was now in charge of Product Development at ERLPhase, and still faced all the problems of satellite clocks, IRIG-B distribution etc.

The idea that we had wasn’t necessarily to sell the code, but to sell support of it – maybe install it on custom embedded hardware, or put in specific customization, etc.

tg2 + Knoppix = NAN

Although interesting, it didn’t sweep Krish off his feet, and even at a modest cost, he could not justify the expenditure.  Well, I gave him the CD anyway, and I don’t think it went any further.

After Krish left ERLPhase, I worked out of his office for a while in 2018, and I stumbled across that CD, heh heh.  I gave it to Mark Poole, as he is now ERLPhase’s IRIG-B guy – and he was quite interested.  Mark and I had a long chat, and my next “hobby” project was born!  More on that later.

Meanwhile, Back doing Real Work

I tinkered with tg2 from time to time, per above, but didn’t do much with it.  In the meantime, Elecsys was purchased by Norscan Instruments and I went to work there as Product Development Manager for 3-1/2 years.  They didn’t have any specific interest in IRIG-B, NTP, or tg2 – they needed to make some money to pay our wages, imagine that!

Then back to ERLPhase for a bit, a great year at GE Multilin in Markham, back in Winnipeg for a short time, and off to Phoenix to work for Alstom Grid Digital Instrument Transformers, or Alstom Grid DIT (now GE Grid Solutions DIT).

Optical IRIG

Now, at DIT, they used satellite clocks all right – Arbiter 1084s, in fact, among others (like Reason, now GE, ones).  There was one important difference though – DIT used the now-standard orange 62.5/125 multimode fibre with 820 nm pulsed light (using the Avago/Broadcom HFPL1414TZ transmitter).  On the Arbiter 1084, this option is called “option 20”, and the single ST output can be configured to output IRIG-B unmodulated (of course 🙂 ), IRIG-B modified manchester coding, or 1 PPS.  DIT used the 1 PPS, because its primary use was to synchronize sampling / output of its product – what’s now known in the industry as a “primary converter merging unit” – so that it can be synchronized in time to other merging units in the system.

We did switch our satellite clocks back and forth between 1 PPS and IRIG-B unmodulated, when we were synchronizing with Reason (now GE) merging units.  I was intrigued.

Dean Scores!  ( an Arbiter 1084C satellite clock )

I always wanted a satellite clock, but, as I said before, they are expensive.  I started trolling for one in late 2015.  They are still expensive – generally selling for US700 used, and up to US2100 if new in box.  I just happened to find one selling at BMI Surplus for something like US250, talked them down to US236 inc shipping and tax, and it was mine, yay!  As a bonus, it was the Arbiter 1084C model, which has the big seven segment LED display on the front, way cool 🙂

It didn’t come with option 20, so I (carefully!) disassembled one of DIT’s 1084 clocks, made a parts list and took pictures, ordered what I needed, and installed them in my clock.  Now I have option 20, heh heh heh.

This came in handy for testing of the Reason merging unit with RogoFlex at PowerTech in Vancouver in August 2017, when I took my own 1084 clock to sync the systems together.

What to Do with a Satellite Clock?

My satellite clock didn’t come with a GPS antenna, so I got one through Amazon, of course, put that up with the many other antennas above my house, and got the clock working.

I had a lot of fun with the Arbiter 1084C and my oscilloscope – marvelling at the time code, watching the different signals, cross referencing the different formats (yes, I am that weird).

The clock was affected by a rollover bug.  I ordered new EPROMs for US$60 from Arbiter. Easy peasy, replaced those, just like in my early days. GPS Module Troubles At one point, the clock stopped working properly. Sad day! The GPS module had died. It seems that it’s writing something to the flash on the GPS module constantly, and it just wore out. The module, an old Motorola OnCore unit, is of course obsolete. There are non-drop-in replacements, but they aren’t binary compatible. Arbiter will upgrade your unit – as I recall, it’s about US$300 – more than I paid for the satellite clock in the first place!  Now that was a very sad day.

I bought a SMT soldering/desoldering station, flash programmer and some spare flash parts, pulled the old flash off the board and rolled it out to disk… programmed the new part, but could not get it installed back on the board properly.  Seems as though I damaged the board removing the parts, so sad.

Fortunately, I found a (less) used GPS module of the exact same model, at some place in the far east.  I received it, plugged it in, and was back in business.

Oh, and the backup battery had of course died – being nickel-cadmium and probably 25 years old!  But, it had also corroded its connections badly.  I repaired the board and replaced the battery, and all is good again.   The battery is actually optional, probably should have just left it off.

Now What to do with a Satellite Clock?

The satellite clock sat running on my shelf for a year or so, before I got to thinking that I should use the IRIG-B signal output.  At first, I wanted to time sync a local computer and create my own (effectively) stratum 1 NTP time reference.  I don’t recall, I might have even had it working briefly, but what do I need a stratum 1 NTP time reference for?  That’s kind of a dead-end project, at least for me.

If You Can’t Generate ‘Em, Decode ‘Em?

Back to that conversation with Mark Poole, he mused that it would be nice to have a tool that could display the details of IRIG-B signals, so Customer Service, Applications and Product Development personnel, could trouble shoot issues, including various clocks that apparently give wrong or conflicting information, get the parity wrong, use reversed time zone-to-UTC offset etc.

So I started playing with NTP’s refclock_irig, not to sync my computer’s clock (in fact, I neutered it, so it would not), but to fully decode and display the signal.  Along the way, I added support for unmodulated IRIG-B as well – that was a challenge, since an audio input port is used to read the signal, and it does not maintain DC levels.  In fact, I did a lot of work on DC restoration à la NTSC TV signal recovery, back in the day.  That was fraught with peril, eventually causing a numerical overflow or a drifting DC baseline, so I eventually settled on a much simpler “look for fast shifts” solution, and that worked well.  I made the printout of the time code optional, using “fudge” tweak configuration bits already in the code, to dump output to the existing log, then tail -f in a separate console to display them.  I also generated some (huge) CSVs from time to time, both raw analog data (do you see what I see?) and various decode internals (do you decode what I decode?).  A lot of work to keep putting debug in, take it out, etc.  There are only 4 fudge tweak bits, and 3 of them are already in use… I pulled one back, so I could use 2, but still, too much hassle to tweak it inside the NTP framework.

Stand-Alone Decoder Program

Eventually, I undertook an effort to break my refclock_irig code away from NTP, creating read_irig.  This program stands alone, and supports too many options:

Read and decode modulated/unmodulated IRIG-B/modulated IRIG-E from audio signal, v0.43, 2019-05-08 dmw

RCS Info:

Options: -b                             Disable automatic input gain control
-c <csv_output_filename>       Put out CSV file of bunch of internal values for portion of time
-f <format_type>               Force only format:
0 = Any (default)
1 = Normal Modulated
2 = Normal Unmodulated
3 = Inverted Unmodulated
4 = Normal or Inverted Unmodulated
-g <initial_gain>              Initial gain setting (default 50), from 10 to 100
-i <input_file>                Input file instead of audio input
-j <csv_output_delay_samples>  Delay in samples for "-c" CSV output after startup (default 8000)
-m                             Enable IRIG-B modulated bandpass filter
-n                             Non-inverting input stream (normally input is inverted)
-o <raw_signal_csv_out>        Put out CSV file of just raw input for as long as program runs
-r <bitmap_of_outputs>         Bitmap of outputs:
bit 0 = decoded data
bit 1 = raw data
bit 2 = do not repeat full header while in low verbosity
bit 3 = 8 lines x 16 chars/line flat panel display output (disables all others)
(default value shows raw and decoded data, and repeats full header periodically)
-s <seconds_to_run>            Run this many seconds then exit (default forever), must be greater than 7
-t <timeout_seconds>           Change timeout on successful decode, must be greater than 4.0 sec
-v                             Increase verbosity of output
-x                             Use 2nd stream (right? left?) instead of 1st
-y                             Invert raw_signal_csv_out from "-o" option
-z <threshold_amount>          Unmodulated edge detect threshold amount (default +/-0.20) in +/- amount of max

Notes: 1. Starting with "  Normal Unmodulated" format, default time out after about 64,000 samples
(about 8.0 sec) without successful decode, then will cycle through "  Normal   Modulated",
"  Normal Unmodulated", and "Inverted Unmodulated", with about 8.0 sec timeout on each format.
2. Setting timeout on successful decode to less than about 8 sec may produce intermittent lock or
failure to lock on modulated IRIG carrier.
3. It can take a few seconds to lock onto a signal, so output may not be <seconds_to_run> frames.
4. Modulated bandpass filter for IRIG-B tends to "shmoosh" the input signal and make it difficult
to decode, and most IRIG-B signals are very clean, so it's not generally necessary.
5. Default outputs include both raw and decoded data.  If both are turned off using "-r" option,
decoded data is turned on alone.

This software licenced under the GPL, derived from refclock_irig, changes made 2018, 2019 by Dean Weiten
Contact: Dean Weiten, Winnipeg, MB, Canada, ph (204)-888-1334, E-mail dmw@weiten.com

The output, I think, is way cool:

#
#--------------------------------------------------------------------------------------------------||------------------------------------------------------------|
#         Inverted Unmodulated IRIG-B Raw (inverted audio) (Gain 100 giving -12.9 dB)              || Inverted Unmodulated IRIG-B (inv   , gain 100 -> -12.9 dB) |
# StrtBinSecs (SBS) |    Control Bits   |   Year  |    Day of Year    |  Hours  | Minutes |Seconds ||Day|   Date and Time    |SBS hex|Leap| DST |Offset|Qual|Par |
# ........ .........|......... .........|.... ....|       .. .... ....|  .. ....| ... ....|........||...|.... .. .. .. .. .. | . ....|....|.....|......|  ..|....|
#                   |                   |         |                   |         |         |        ||   |                    |       |    |     |      |    |    |
.001010000.000011001.000100000.010111000.000101001.000000001.001100000.000100001.001000011.00000101. 130 2019-05-10 11:23:05   0_A019 !lsp  DST  UTC-05   00 1 ok
.001010000.000011010.000100000.010111000.000101001.000000001.001100000.000100001.001000011.00000110. 130 2019-05-10 11:23:06   0_A01A !lsp  DST  UTC-05   00 1 ok
.001010000.000011011.000000000.010111000.000101001.000000001.001100000.000100001.001000011.00000111. 130 2019-05-10 11:23:07   0_A01B !lsp  DST  UTC-05   00 0 ok
.001010000.000011100.000000000.010111000.000101001.000000001.001100000.000100001.001000011.00001000. 130 2019-05-10 11:23:08   0_A01C !lsp  DST  UTC-05   00 0 ok
...


The raw and inverted decode sections can be enabled independently, and the program can auto-detect (slowly, mind you) modulated IRIG, unmodulated IRIG, and inverted unmodulated IRIG.

I’m still a sad case… I can sit and watch the codes go by, constantly.  It turns in a separate TTY on my computer, in the background, all the time.  Heh heh heh.

What Now?

In keeping with How Could They Possibly Improve Upon That?, you have to wonder where I could go from there.  Stay tuned! 🙂

Encrypting a Disk Partition with LUKS

I need to encrypt the contents of my Drobo – at least the Drobo partitions that are going to contain my personal data. The Drobo is too old to natively support encryption.

Article about hard disk partition encryption using LINUX Unified Key Setup (LUKS). Herein, I will copy portions of this article, and adjust it for my own purposes. Thanks to nixCraft for a great article, their source data is gratefully acknowledged.

Prerequisite: cryptsetup utility

To do LUKS, you need cryptsetup package. Use apt-get or apt command.

Configure LUKS partition

WARNING! The following command will remove all data on the partition that you are encrypting. You WILL lose all your information! So make sure you backup your data to an external source such as NAS or hard disk before typing any one of the following command.

In this example, I’m going to encrpt /dev/sdj1. Type the following command:
# cryptsetup -y -v luksFormat /dev/sdj1

Sample outputs:

 WARNING! ======== This will overwrite data on /dev/sdj1 irrevocably. Are you sure? (Type uppercase yes): YES Enter LUKS passphrase: Verify passphrase: Command successful. 

This command initializes the volume, and sets an initial key or passphrase. Please note that the passphrase is not recoverable so do not forget it.

Switching to UUID Instead of Block Device Name

At this point, you can switch to using UUID reference, so the mapping won’t change if, say, your block devices show up in a different sequence.  To find the UUID for a given partition, do the following:

# lsblk -f

Sample output:

 NAME FSTYPE LABEL UUID MOUNTPOINT ... sdj └─sdj1 crypto_LUKS 85af2419-bde3-49e7-939a-2f231532a8b2 ... 

Now, wherever you would see “/dev/sdj1”, you put UUID=<the UUID number copied from above>.

Open the Crypto Block Device

Type the following command create a mapping:
# cryptsetup luksOpen /dev/sdj1 crypt_drobo2-5

Or, using UUID:
# cryptsetup luksOpen UUID=85af2419-bde3-49e7-939a-2f231532a8b2 crypt_drobo2-5

Sample outputs:

Enter passphrase for /dev/sdj1:

You can see a mapping name /dev/mapper/crypt_drobo2-5 after successful verification of the supplied key material which was created with luksFormat command extension:
# ls -l /dev/mapper/crypt_drobo2-5

Sample outputs:

lrwxrwxrwx 1 root root 7 Oct 19 19:37 /dev/mapper/crypt_drobo2-5 -> ../dm-0

You can use the following command to see the status for the mapping:
# cryptsetup -v status crypt_drobo2-5

Sample outputs:

/dev/mapper/crypt_drobo2-5 is active.
type:    LUKS1
cipher:  aes-cbc-essiv:sha256
keysize: 256 bits
device:  /dev/sdj1
offset:  4096 sectors
size:    419426304 sectors
Command successful.


You can dump LUKS headers using the following command:
# cryptsetup luksDump /dev/sdj1

Or, using UUID:
# cryptsetup luksDump UUID=85af2419-bde3-49e7-939a-2f231532a8b2

Sample outputs:

 LUKS header information for /dev/sdj1 Version: 1 Cipher name: aes Cipher mode: xts-plain64 Hash spec: sha256 Payload offset: 4096 MK bits: 256 MK digest: 21 07 68 54 77 96 11 34 f2 ec 17 e9 85 8a 12 c3 1f 3e cf 5f MK salt: 8c a6 3d 8c e9 de 16 fb 07 fd 8e d3 72 d7 db 94 7e e0 75 f9 e0 23 24 df 50 26 fb 92 f8 b5 dd 70 MK iterations: 222000 UUID: 4dd563a9-5bff-4fea-b51d-b4124f7185d1 Key Slot 0: ENABLED Iterations: 2245613 Salt: 05 a8 b4 a2 54 f7 c6 ee 52 db 60 b6 12 7f 2f 53 3f 5d 2d 62 fb 5a b1 c3 52 da d5 5f 7b 2d 38 32 Key material offset: 8 AF stripes: 4000 Key Slot 1: DISABLED Key Slot 2: DISABLED Key Slot 3: DISABLED Key Slot 4: DISABLED Key Slot 5: DISABLED Key Slot 6: DISABLED Key Slot 7: DISABLED 

Optional: Wipe LUKS partition

First, you need to write zeros to /dev/mapper/crypt_drobo2-5 encrypted device. This will allocate block data with zeros. This ensures that outside world will see this as random data i.e. it protect against disclosure of usage patterns:
# dd if=/dev/zero of=/dev/mapper/crypt_drobo2-5

The dd command may take many hours to complete. I suggest that you use pv command to monitor the progress:
# pv -tpreb /dev/zero | dd of=/dev/mapper/crypt_drobo2-5 bs=128M

Sample outputs:

dd: error writing '/dev/mapper/crypt_drobo2-5': No space left on device                                                                                                            ]
200GiB 0:16:47 [ 203MiB/s] [                      <=>                                                                                                                      ]
1600+1 records in
1599+1 records out
214746267648 bytes (215 GB, 200 GiB) copied, 1008.19 s, 213 MB/s

You can also pass the status=progress option to the dd command:
# dd if=/dev/zero of=/dev/mapper/crypt_drobo2-5 status=progress

Sample outputs:

2133934592 bytes (2.1 GB, 2.0 GiB) copied, 142 s, 15.0 MB/s

Create Filesystem on LUKS partition

Next, create a filesystem i.e. format filesystem, enter:
# mkfs.ext4 /dev/mapper/crypt_drobo2-5

Or, using UUID:
# mkfs.ext4 UUID=85af2419-bde3-49e7-939a-2f231532a8b2

Sample outputs:

mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 52428288 4k blocks and 13107200 inodes
Filesystem UUID: 1c71b0f4-f95d-46d6-93e0-cbd19cb95edb
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Mounting Partition

To mount the new filesystem at /backup2, enter:
# mkdir /mnt/drobo2-5 # mount /dev/mapper/crypt_drobo2-5 /mnt/drobo2-5 # df -H # cd /mnt/drobo2-5 # ls -l

Closing the Paritition

Type the following commands:
# umount /mnt/drobo2-5 # cryptsetup luksClose backup2

Re-Opening the Paritition

Type the following command:
# cryptsetup luksOpen /dev/sdj1 backup2 # mount /dev/mapper/crypt_drobo2-5 /mnt/drobo2-5 # df -H # mount
Sample outputs:

See shell script wrapper that opens LUKS partition and sets up a mapping for nas devices.

Changing Passphrase (password) on the Partition

Type the following command
### see key slots, max -8 i.e. max 8 passwords can be setup for each device #### # cryptsetup luksDump /dev/sdj1 # cryptsetup luksAddKey /dev/sdj1

Enter any passphrase:
Enter new passphrase for key slot:
Verify passphrase:


Remove or delete the old password:
# cryptsetup luksRemoveKey /dev/sdj1
Please note that you need to enter the old password / passphrase.

Using the Encrypted Partition

Check out the man page for cypttab. It’s a bit cryptic, heh heh.

exFAT? Who are you calling exFAT?

Yeah, in the machinations around my phones and breaking them, I wanted to put a few files onto the microSD card from the phone. I put it into an adapter, put the adapter into my computer, and… filesystem exFAT not recognized?!? Hmm.

I guess when I loaded up Ubuntu 18.04 onto this computer, I didn’t put the exFAT utilities onto it.

It turns out that it’s as simple as:
sudo su - apt-get update apt-get install exfat-utils

Of course, you need to enter the root password when you “sudo su”.

Centre Button BullSomething on LINUX

It has been a minor annoyance for as long as I can remember. When using LINUX, or more specifically X-Windows on LINUX, clicking the mouse centre button would perform a paste wherever the cursor is right now. When you are using the scroll-wheel to scroll through a document, often the centre button has a hair-trigger and will perform a click (and therefore a paste) without even realizing it – and, boom! You have random (well, arbitrary) text in the middle of your document. Fortunately, if it’s source code, the compile or execution will almost always fail… so you can fix it. But, when it’s a word processing document… ugh!
Continue reading “Centre Button BullSomething on LINUX”

S5 Flip Flap Flop

When we last left our hero (me? ha ha), he had installed the latest LineageOS installed on my second T-Mobile Samsung S5 (actually my original from T-Mobile). I like it! It’s not quite as pretty as the stock O/S (OK OK, stock ROM), but it works well, more quickly, and, most importantly (related to the quickness), uses much, much less flash!

I had to (re-re-) install all of my programs, of course, and that was a pain. Worse, several of my “go to” programs are now “abandonware”. Sigh. Notably, “Profiles”, which gave me one-touch change of settings – I used this to turn off BlueTooth, dim display, and, in earlier phones, change volumes etc. However, in the S5, even with the T-Mobile ROM, I had to use “do not disturb” to prevent overnight E-mail & text notifications.

Anyway, so now the phone works… mostly. Still adding programs…

My use case is to leave the phone plugged in, face down, on my dresser overnight. Often my tablet is sitting there as well, plugged in, because I like to read the news, devotions, etc., before going to sleep. Well, what do you know, the tablet is large and thin, so I put it on the dresser first, then the phone on top, face down.

The past two mornings, the phone is working – I can see notifications in my web-based “WhatsApp” and text messages on “PushBullet” – but it seems to be locked up! The clock is present on a blue background in the top half of the screen, but the bottom half of the screen is black 🙁 It’s doing the haptic feedback on button presses (including the “soft” buttons), but won’t change screens for anything. If you plug it into the charger, the blue background changes to yellow, whoo hoo. My only recourse was to open it up, remove the battery, and reboot the phone that way.

Well, it turns out that this is a known problem with the Galaxy S5 – well, actually, a feature! Ugh. Apparently, there are some cases out there (or “covers” as they are sometimes called) that have a magnet in them, so that the S5 knows when the face is closed. It’s called a “flip flap”. Well, when the face is closed for a period of time, it automatically goes into “flipflap” mode, which shows the clock on the solid background, per above, through a lightly tinted upper window in the cover.

There were posts on this issue from a year ago – see https://www.reddit.com/r/LineageOS/comments/6yzpn1/how_to_disable_flip_flap_klte/ . Apparently there was a setting to disable this so-called “feature” (does my disdain show here?), but it would keep re-enabling itself “automagically”, as one post put it. You have to use “debloat” under “Magisk”. Now, here is a journey down a rabbit hole.

Installing Terminal Emulator
Go to the Google Play store and install Terminal Emulator from Jack Palevich. It will eventually need root privileges, but that should be automatic. Just remember that.

Installing Magisk

Optionally Check for Root
The next part requires root access, so you can check for root with Root Checker from joeykrim. This may fail the first time, because Magisk has to wake up and ask to give it root privileges, which, if you want it to detect root privileges, you will say “yes” to. Ha ha.

Installing Debloater
Unfortunately, you are just getting started. You may have to reboot to get Magisk Manager fully loaded. Open Magisk Manager, click the upper left menu, go to Modules. It will be empty. OK then, click the upper left menu, go to Downloads. Install Bash for Android. Do not install Debloater from here, unless you are on Magisk 19 or later – this is a bug in my opinion. Instead, that thread leads to a scary filesharing site to get old version. Now go back to Modules, click the bottom middle plus, navigate and select the downloaded older debloat ZIP archive, and it will install.

Now, reboot.

Doing the Deed in Terminal Emulator
Once I got Terminal Emulator running, I just:

• su – up to superuser mode
• debloat – if it says “module not found” then you have too new of debloat, go to this thread which leads to a scary filesharing site to get old version
• 2 System Priv-Apps – which will list available applications to be removed / masked by debloat
• 25 – FlipFlap program!
• exit – to get out of “su” mode
• exit – to exit the terminal emulator
• – now, reboot the phone to apply changes
• Load Terminal Emulator again, do “su”, “debloat”, “2 System Priv-Apps” again, and “FlipFlap” is no longer in the list, yay!

Blank Terminal Emulator Screen
I goofed and uninstalled Bash for Android instead of the too-new version of Debloater. ( hint: there’s no double check, it just does it – so be careful! )

After I reinstalled Bash for Android, Terminal Emulator just gave a blank screen! I’m not sure if it was related. Maybe I hit the “X” to close a window, rather than typing “exit”.

Anyway, it did come in after some fiddling. Not sure what I did. I waited a long time, I went to “windows” and tried to add one (killed the program?), and I tried to add a window using the big plus on the screen. I uninstalled and reinstalled Terminal Emulator.

If this does happen, be patient. Click things, try things, it will eventually work.

Better, be careful when clicking things in Magisk.

Shall see if this fixes the problem. Maybe will find out tomorrow morning 🙂