Filling the gap in the plain text accounting world felt like an apt use case to kick the tires on claude code. A few sessions and 257155 opus-4.5 tokens later, claude had written a working ledger-cli LSP implementation.

Key observations:

• The hardest part of the entire process was halting development everytime I hit the session limit on the Claude Pro ($20/mo) plan
• Claude developed this over a few days and it would have taken me weeks/months by hand (prohibitively expensive for the value I would get out of the LSP)
• I don't know Rust. I did not learn Rust during this project and I can't evaluate the code quality. But it works and it's performant

My prompts were straightforward - 1) come up with a plan 2) what features are we missing 3) review the code. This pattern was repeated across seven sessions (usually running up against the Pro session limit) until I reached enough features for my use case.

Hey claude. You are an expert in rust and writing Language Server Protocol (LSP) servers. What we want to do is create a no code plan on how to implement a LSP for the "ledger-cli" domain specific language. The documentation for the ledger-cli language is given in ledger.html. Please come up with a plan on how we should extend the tower-lsp language server to support the ledger-cli syntax. Ask as many clarifying questions as necessary

Claude you are very familiar with ledger-cli and Rust. This directory contains a language service (LSP) for the ledger-cli language. Evalute the existing features and come up with some other features we should add or extend into this LSP that are specifically useful for double entry accounting. We do not want to make claims about account balances since the ledger software is quite complex. Instead we want to make sure the editing experience is good.

Claude you are an expert in rust and well versed in language servers. Please review this implementation of ledger-lsp, a ledger language server in rust.

The initial design decision I made before prompting claude was that the LSP would be in rust since tower-lsp appeared fairly mature. During the implementation another key decision was to steer claude away from writing an error-prone regex based syntax parser to using tree-sitter - a surprisingly difficult ask. At every turn, claude wanted to revert to regex. Similarily, claude staunchly believed this LSP should be keeping track of balances across files, i.e. move all of of the ledger logic into the LSP. Nearly every session required steering the LLM towards supporting the editing experience and not trying to replace ledger. Despite the required steering, claude came up with some novel ideas I would not have thought of. For example, a warning if entries in a file were out of chronological order.

Overall, a few nights of coaching an LLM resulted in a useful tool in a language I'm unfamiliar with and my workflow for editing ledger files has noticeably improved. Several months ago I would think very carefully about whether I needed a tool before investing my time in to developing it. Today, I can cut a check to Anthropic and spend a few hours guiding an LLM - xkcd 1319 may no longer be relevant.

Zigbee Home Automation (ZHA) is a great piece of software.

SmartIR is a great piece of software.

So why is it so difficult to get them to control a Mitsubish Mini-Split HVAC system? Newish hardware, that's why.

Several Zigbee IR blasters have entered the Home Automation scene in recent years. Some are battery powered (Moes UFO-R11), some are powered via USB (ZS06), but all of them use the Tuya TS1201 (Zigbee) platform under the hood. Until the Zigbee blasters appeared, Broadlink was the main player in the automation space, but it only supports Wi-Fi networking. None of the upstream software packages have caught up with the proliferation of Zigbee IR blasters.

Note: This post will hopefully not age gracefully as these patches get absorbed into upstream projects

ZHA

Getting Home Assistant (via ZHA) to recognize the TS1201 chipset requires installing a custom quirk.

# Make a directory for quirks if none exists
$ mkdir /config/quirks
# Inform Home Assistant about it
$ cat /config/configuration.yaml
...
zha:
  custom_quirks_path: quirks/
...
# Download the quirk
$ cd /config/quirks
$ wget https://raw.githubusercontent.com/ferehcarb/zha-device-handlers/dev/zhaquirks/tuya/ts1201.py
# Restart Home Assistant

After performing the above steps, ZHA should recognize the device:

Click "Zigbee Info" in this panel and record the IEEE address for the device.

Verifying the device is functioning is performed by clicking the kebab menu and selecting "Manage Device". From there, set the Zigbee Cluster to ZosungIRControl and the Command to IRLearn.

Issue the command and record a signal from an IR emitter. Click Attributes and read the last_learned_ir_code attribute. Copy and paste this string back into the IRSend command and verify that the device is receiving and transmitting values.

Note: On my system, I had to fully reboot the Home Assistant server and power cycle the Zigbee IR blaster to actually send IR data.

For simpler IR devices the ride ends here - record values and play them back via Home Assistant scriipts. For more complicated devices, like Mitsubishi HVAC units with a complex payload, SmartIR is the better path forward.

SmartIR

Once the device is communicating with ZHA properly, install a forked version of SmartIR with ZHA support.

Clone davidrpfarinha's fork of SmartIR and extract the contents into the custom_components directory of the Home Assistant configuration.

$ git clone https://github.com/davidrpfarinha/SmartIR
$ mkdir /config/custom_components
$ cp -r SmartIR/custom_components /config/custom_components

Next, modify the Home Assistant configuration.yaml for this device:

...
zha:
  custom_quirks_path: quriks/

smartir:

climate:
  - platform: smartir
    name: Office AC
    unique_id: office_ac
    device_code: <YOUR_DEVICE_CODE>
    controller_data: <YO:UR:DE:VI:CE:MA:CA:DR>
    temperature_sensor: sensor.office_ac_temperature
    humidity_sensor: sensor.office_ac_humidity
    power_sensor: binary_sensor.office_ac_power

Set controller_data equial to the IEEE address reported in the ZHA Device Info panel above.

Find the "device code" number for the HVAC system you are trying to control in the SmartIR database.

Note that nearly all of the devices only have support for "Broadlink" blasters. This is problematic. Broadlink and the TS1201 both store data in a different format. Thankfully, the TS1201 protocol has been mostly reverse engineered and scripts exist to convert from the Broadlink protocol to the TS1201 protocol.

$ wget https://gist.githubusercontent.com/svyatogor/7839d00303998a9fa37eb48494dd680f/raw/66cba20e653f84aab0b9a31ea5b9ca497d038d8a/broadlink_to_tuya.py
$ python3 broadlink_to_tuya.py your_device_code.json > /config/custom_components/smartir/codes/climate/your_device_code.json

The device codes in the new JSON file should be different from those in the original file:

Broadlink: "31": "JgBMAmw5DisOKg0PDg4NEA8pDg4ODw4qDioNDw0rDhANDg..."

Tuya: "31": "ENkMyAarAR4FqwEABYwByQGrIAECjAH..."

There is still more work to do! These converted files are still marked as being for the Broadlink. Set the supportedController field in the file to "ZHA" and restart Home Assistant.

With some luck the climate panel should have automatically loaded and allow control of the device.

Did this save you hours of debugging? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

Currently, this is an issue with Radioddity's GA-510 10W radio. Radioddity updated the device's chipset in mid-2023 without changing the product ID and the new chipset does not have support in CHIRP. Radioddity only provides a Windows executable for programming the radio.

On Linux there are a few options for intercepting serial data that will not work directly on Windows like slsnif and socat in addition to the methods described below. None of the methods described here are dependent on Windows and should work on most platforms.

So how do we free the bits from a Windows binary blob?

Option 1: Wireshark + USBPcap

Wireshark is primarily a network protocol analyzer but with USBPcap installed it also supports tracking and analyzing USB streams.

Once USBPcap is installed it will show up as a capture device in Wireshark. Clicking the gear icon allows selecting a specific USB device to capture.

Once the capture has started the USB frames are displayed in the GUI.

It is completely possible to click through line by line and copy the relevant data out of each packet. Fortunately, there are many tools to work with pcap files that can simplify the process.  usbrply is a small utility to convert a USB capture into python or C code that replays the USB frames in the pcap file. But for reversing the GA-510 it is more useful to inspect the data than replay it.

tshark is a command line based analyzer that is installed alongside Wireshark. tshark allows display filters to be applied to previously captured data, so the USB capture data can be stripped from the Wireshark pcap.

> tshark.exe -r capture.pcapng -Y usb.capdata -T fields -e usb.src -e usb.capdata
host    50534541524348
2.1.2   06444d5231373032
host    50415353535441
2.1.2   500000
host    535953494e464f
2.1.2   06
host    5600000a0d
2.1.2   560d0a0a0d6900000000000000

From the output we can see that the host sends \x50\x53\45\41\52\43\48 (PSEARCH) and the device replies back with \x06\x44\x4d\x52\x31\x37\x30\x32 ("\x06DMR1702").

Option 2: Logic Analyzer

If it is not possible to install Wireshark on the Windows machine an external logic analyzer (e.g. Saleae Logic, Digilent Analog Discovery) may work for sniffing the serial bus. If GND/TX/RX can be accessed on the device via pads or test points then the logic analyzer can be attached directly to the PCB. If the device has a non-USB connector at the end of the programming cable (e.g. a Baofeng programming cable uses a 2.5mm and 3.5mm jack) then an interception harness can be used to read the serial data directly off the wire. However, if the harness or device does not expose GND/TX/RX then a logic analyzer is not an option.

Sniffing the serial bus with a logic analyzer is a time consuming option but provides definitive data on what is being sent to the device.

Option 3: Windows VM + qemu + socat

Sometimes a Windows VM is available and a physical machine is not (or the physical machine is across the room). In that case, qemu can be used to virtualize Windows and socat can be used to MITM the serial device.

In the notional setup, the physical device is connected to a Linux host and enumerates as /dev/ttyUSB0. A Windows VM is running within qemu on the Linux host. A virtual Serial Device is attached to the VM (either via qemu XML or virt-manager) which enumerates as a COM device in Windows. On the Linux host the virtual Serial Device appears as a pts.

virt-manager will generate a new source path for the serial device on every boot:

socat can be used to link the physical /dev/ttyUSB0 to the virtual pts device:

# socat -d -d -x 2> log.txt /dev/ttyUSB0,b57600,raw,echo=0 /dev/pts/9,raw,echo=0

Now, the pts in qemu is linking /dev/ttyUSB0 to COM1 in Windows. The data that is flowing is recorded in log.txt:

< 2024/01/24 19:16:44.000948516  length=1 from=14 to=14
 50
< 2024/01/24 19:16:44.000948586  length=2 from=15 to=16
 53 45
< 2024/01/24 19:16:44.000948614  length=1 from=17 to=17
 41
< 2024/01/24 19:16:44.000948645  length=1 from=18 to=18
 52
< 2024/01/24 19:16:44.000948708  length=1 from=19 to=19
 43
< 2024/01/24 19:16:44.000948805  length=1 from=20 to=20
 48
> 2024/01/24 19:16:44.000954053  length=8 from=7 to=14
 06 44 4d 52 31 37 30 32

Once again, the hex bytes of "PSEARCH" are present in the logs, albeit in a painful format. A little python can convert this into a reasonable format:

def read_log(log):
    with open(log, 'r') as radio:
        transactions = []
        direction = ''
        transaction = ''
        for line in radio.readlines():
            if (line.startswith('<') and direction != '<') or (line.startswith('>') and direction != '>'):
                # We've changed direction
                #print(f"Direction change!\n {direction}\t{transaction}")
                transactions.append(transaction)
                direction = line[0]
                transaction = direction
            elif line.startswith(' '):
                transaction += line.strip('\n')
                #print(transaction)
            else:
                pass
                #print(line)
    return transactions
for t in read_log('log.txt'):
    print(t)

< 50 53 45 41 52 43 48
> 06 44 4d 52 31 37 30 32

Linux Bonus: If sniffing a serial device on Linux, socat can create the virtual device directly. In this case passing /dev/pts/5 to a program using a serial device instead of /dev/ttyUSB0 will allow the data to be streamed.

# socat -d -d -x /dev/ttyUSB0,b57600,raw,echo=0 pty,raw,echo=0
socat[22063] N opening character device "/dev/ttyUSB0" for reading and writing
socat[22063] N PTY is /dev/pts/5
socat[22063] N starting data transfer loop with FDs [5,5] and [6,6]
> 2024/02/20 20:06:48.000165060  length=5 from=0 to=4

Option 4: Reverse Engineering the Binary

Sniffing the serial bus, either physically or virtually, is by far the easiest path to understanding what is being sent to the device. The "nuclear option" is to reverse engineer the Windows executable to determine what it is writing to the device. Determining what the binary is doing is a tedious process but can be done; below is the "PSEARCH" string in the binary using Ghidra.

Despite the challenges with reverse engineering, on occasion there are functions with hidden functionality that aren't exposed in the GUI or are behind a password. For example, the GA-510 program allows users to access a secret menu to set the font and boot logo if they enter BAOFENGH5:

Was this an interesting read? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

Table of Contents

1. Cookbook of the Year
2. American Cooking / U.S. Foodways
3. Baking and Dessert
4. Beverage
5. Bread
6. Restaurant / Professional
7. General Cooking
8. Health
9. Food Issues and Advocacy
10. International
11. Photography / Visuals
12. Reference, History, and Scholarship
13. Single Subject
14. Vegetable Focused and Vegetarian
15. Writing and Literature

Cookbook of the Year

American Cooking / U.S. Foodways

Books with recipes focused on the cooking or foodways of regions or communities in the United States.

Baking and Dessert

Books with recipes focused on the art and craft of baking pastries and desserts, both sweet and savory items, including ingredients, techniques, equipment, and traditions.

Beverage

Combined with and without recipes

Bread

Books with recipes focused on the art and craft of making bread, including ingredients, techniques, equipment, and traditions.

Restaurant / Professional

Books written by a culinary professional or restaurant chef with recipes that may include advanced cooking techniques, or the use of specialty ingredients and professional equipment, including culinary arts textbooks.

Food Issues and Advocacy

Books that include investigative journalism, food policy, deep dives, and critical analysis of the changing social landscape.

General Cooking

Books with recipes that address a broad scope of cooking, not just a single topic, technique, or region.

Health

International

Books with recipes focused on food or cooking traditions of countries, regions, or communities outside of the United States.

Photography / Visuals

Books on food or beverage with exceptional graphic design, art, or photography.

Reference, History, and Scholarship

Includes manuals, guides, encyclopedias, and books that present research related to food or foodways.

Single Subject

Books with recipes focused on a single ingredient, dish, or method of cooking.

Vegetable Focused and Vegetarian

Books on vegetable cookery with recipes that are meatless, vegetarian, or vegan.

Writing and Literature

Narrative nonfiction books, including memoirs, culinary tourism, investigative journalism, food advocacy, and critical analysis of food and foodways for a general audience. Includes memoir, biography, personal essays, culinary travel, and culture.

‌Was this an interesting read? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

Background

The Freestyle Libre 3 is a Continuous Glucose Monitor (CGM) produced by Abbott. The sensor records glucose readings every minute from interstitial fluid for 14-days before being replaced with a new one.

The actual sensor is a small channel filled with glucose oxidase which releases electrons when it interacts with glucose. The probe of the sensor has a few small wells to collect interstitial fluid which is then routed over the glucose oxidase pad.

The first iteration of the sensor had a small community of enthusiasts who thoroughly reversed engineered the device to use it as a wireless ADC after the 14-day period. Unfortunately, the 3rd generation of the sensor doesn't seem to have a teardown outside of the blurry images in the FCC filing.

An encryption key is generated and shared with the Libre 3 via NFC to "initialize" the sensor. From then on, glucose readings are communicated via BLE to Android and iOS apps. The glucose readings are stored in an encrypted RealmDB which is then synced to the cloud. The apps display the glucose readings on a graph.  A CSV of glucose data can be downloaded from the LibreView web interface by agreeing to the Terms & Conditions of a LibreView account and living in a geo-fenced region.

However, the graph is not zoomable and the timeseries data cannot be exported without using the cloud sync. The motivation behind these terrible product decisions is unknown. It is extremely useful to compare blood glucose readings across various confounding variables (heart rate, temperature, food intake, etc.).

Let's get those blood glucose readings out of this horrendous UI!

Prior Work

xDrip+ is an Android app to collect health data from a variety of sources but does not interface directly with the Libre 3. A variety of methods exist to migrate data between the Libre 3 app and xDrip but many of them require a rooted device or an active internet connection. Juggulco is an "offline" non-root method but requires a LibreView account to be associated with the sensors. The majority of the methods do not disclose the actual mechanism behind the data bridge and some require sideloading apks of unknown origin.

The BLE data is encrypted but the DiaBLE and Juggluco projects seems to have made some progress on decrypting the readings. However, Abbott will swing the DMCA hammer if too much is published despite the reverse engineering being done to "achieve interoperability" of the CGM user's own glucose data.

To date, there is nothing published for accessing the data for iOS users.

iOS

Since a feature of the app is to see non-interactive plots of historical glucose data, it seemed reasonable that there must be a local database of glucose readings stored on the device filesystem. The idevicebackup2 utility from libimobiledevice is a straightforward way to access the iOS filesystem on a non-jailbroken device, albeit a space consuming one since the entire phone is backed up.

$ mkdir iPhone-backup 
$ idevicebackup2 backup iPhone-backup

The iOS device backup is a collection of files that are enumerated in the SQLite Manifest.db  located inside iPhone-backup.

$ sqlitebrowser Manifest.db

As expected, there is a database associated with the Libre 3 app on the filesystem! Realm is a "fast, scalable alternative to SQLite with mobile to cloud data sync that makes building real-time, reactive mobile apps easy." Additionally there are "elog" files (which are likely "encrypted" logs).

Unfortunately, trying to open trident.realm with Realm Studio results in a very disappointing message: "The Realm might be encrypted".

However, another interesting file exists in the backup: com.abbott.libre3.us.plist. It is by default a binary plist but can be converted to XML quickly:

plistutil -i com.abbott.libre3.us.plist > libre.xml.plist

The file contains some interesting but not terribly useful information like the current sensor UUID, the last time the sensor was read, etc. But right at the end of the file there is a juicy field!

        <key>RealmEncryptionKey</key>
        <array>
                <integer>96</integer>
                <integer>10</integer>
				...
                <integer>157</integer>
        </array>

Could this actually be the RealmEncryptionKey? It turns out this array is 92-bytes while a real Realm key is 64-bytes (or 128-characters when hex encoded). The first and last 64 byte windows of the key did not unlock the Realm, so the odds were good the key was wrapped. In theory the wrap/unwrap functions should be obvious from disassembling and decompiling the iOS app.

Searching for the RealmEncryptionKey in ghidra after loading the Libre 3 ipa confirmed that it was indeed a key used for decrypting the Realm database. The key is generated via a call to SecRandomCopyBytes and is unique for each installation. But, the candidate function for unwrapping the key (arguments were the 92-byte RealmEncryptionKey data and 0x5c (92 in decimal)) was full of obtuse references due to Swift/Objective-C calling conventions. Additionally, Ghidra did not fully disassemble the entire function. Hopper teased apart many of the Objective-C and Swift calling conventions but also did not successfully disassemble the function - there is a decent chance obfuscation methods were used for this section of code.

Without a jailbroken iOS device, performing dynamic analysis to understand this function was out of the question.

But, with some luck, the Android app may behave in the same way as the iOS app.

Android

Static / jadx

Acquiring the Android apk is straightforward from the usual apk mirrors and decompiling it with jadx is just as easy.

After searching for "Realm" in the jadx decompilation it looks like the Android app behaves in a very similar manner -  AppDetailsForRealmDatabase is loaded from the App's Shared Preferences.

Chasing down the unWrapDBEncryptionKey turns up an interesting development: the Libre 3 developers hid the interesting functionality of the libreSKBCryptoLib class behind a JNI/NDK interface.

At first this seemed pretty promising - the apk contains a liblibre3extension.so for four different CPU architectures. However, ghidra failed to disassemble the unwrap functions for each of the architectures.

With a failed static analysis across two platforms (and five architectures) it was time to look into dynamic analysis of the key wrapping.

Dynamic / Frida

Getting a rooted Android device running on Linux is very straightforward with waydroid (it also runs on X11).

# waydroid init -f
# waydroid container start
$ waydroid show-full-ui
$ adb devices
List of devices attached
192.168.17.132:5555    device

After acquiring an apk, it can be installed into the waydroid session and launched via adb:

$ adb install libre3.apk
$ adb shell am start -n com.freestylelibre3.app.us/com.adc.trident.app.startup.SplashActivity

Everything looks good until the app refuses to move past the Splash Screen with "Please install this App from Google Play." Somehow the app is detecting the sideload and refusing to run. Fortunately, it loads enough!

Also, since the app has been launched once, the trident.realm has been created within waydroid:

$ adb shell ls /data/user/0/com.freestylelibre3.app.us/files
... trident.realm ...

And lo-and-behold a key for the database stored in XML!. The Android app stores the wrapped key as base64 encoded text while the iOS app does not; the base64 encoding is clear from the jadx decompilation. Note: this key is different than the key used to encrypt the iOS realm database - the keys are unique per installation.

$ adb shell cat /data/user/0/com.freestylelibre3.app.us/shared_prefs/AppDetailsForRealmDatabase.xml
<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
    <string name="DBIdentifier">CoPojFQfHHI+Ls5nwRChIkqZ+bq0aYUL5S7RPSt3jLWfu0e8yCTaqFT7HY+JdE6Qoy+5bWgutGtk&#10;dyqZJFsGtJyhqxVJAqw8jvLQC5xEgltQ75iN+5vRObBLykc=&#10;    </string>
</map>

Using Frida it is possible to call the native library functions with our own arguments. Installation is well-documented.

With the Libre 3 app launched and in the "Please install this App from Google Play." state, we can launch the frida-server running inside waydroid and connect to it:

$ adb root
$ adb shell "/data/local/tmp/frida-server &"
$ frida -U "Libre 3"

Once frida has "hooked" into the Libre 3 app it is possible to call the native functions hidden in liblibre3extension.so.

Frida-> Java.perform(function(){}); // Seems necessary to use Java.use
Frida-> var crypto_lib_def = Java.use("com.adc.trident.app.frameworks.mobileservices.libre3.security.Libre3SKBCryptoLib");
Frida-> var crypto_lib = crypto_lib_def.$new()
Frida-> unwrapped = crypto_lib.unWrapDBEncryptionKey([96, 10, ..., -99])
[
    93,
    -81,
    ...
    -122,
    26
]
Frida -> unwrapped.length
64

Success! A 64-byte Realm encryption key. But first, some information on massaging the data from the plist for the call to succeed.

The unWrapDBEncryptionKey function requires an array of Int8 while the iOS plist has an array of UInt8. The following snippet of python converts a UInt8 iOS wrapped key stored as one array entry per-line in realm.key to Int8 with numpy:

with open('realm.key', 'r') as f:
	data = f.readlines()
u = np.array(data, dtype=np.uint8)
i = u.astype(np.int8)
for b in i:
	print(str(b)+', ', end='')

[96, 10, ..., -99]

The printed array can then be copy/pasted into the Frida call to unWrapDBEncryptionKey.

Similarly, we can use python to convert the Int8 results of unWrapDBEncryptionKey into the 128-character key required by Realm.

import struct
with open ('unwrapped.txt', 'r') as f:
	data = f.readlines()
key = ''.join([struct.pack('b', int(i)).hex() for i in data])
print(key)
5daf8d031405bfeafffc64659bcffb692f186164e3d1899d628ed16de32eaaccb0c66662360d772a92749eeda7eb53fe29096fc4cb34fa2b04eb14f3a7af861a

Note: Since it is 2023, it is worth noting all Python code in this post was generated directly via perplexity.ai from human-language prompts.

And finally, after jumping through many hoops, the Realm database from the iOS installation is revealed!

Several pieces of the puzzle aren't solved despite having the glucose readings:

• What do unWrapDB and wrapDB actually do? Are they RFC 3394?
• How are the *.elog files encrypted and what do they contain?

Both answers lie somewhere in (the likely obfuscated) liblibre3extension.so.

Summary

At a high level, the steps to reproduce this are:

1. Backup iOS device and isolate trident.realm and com.abbott.libre3.us.plist files
2. Extract the RealmEncryptionKey from the plist file and convert it to a Int8 array
3. Install waydroid, sideload Libre 3 apk, and install frida server
4. Launch Libre 3 app and hook Frida
5. Use Frida to unwrap the Int8 array from Step 2
6. Use the unwrapped key to unlock trident.realm using Realm Studio

It is important to note that the iOS backup and trident.realm isolation steps need to be performed every time a copy of the data is desired.

Bonus: The FDA 510(k) for the Libre 3 contains a snippet on Cybersecurity:

ADC has provided cybersecurity risk management documentation for the System that includes analysis of confidentiality, integrity, and availability for data, information and software related to the System accordance with FDA Draft Guidance “Content of Premarket Submissions for Management of Cybersecurity in Medical Devices.” For each identified threat and vulnerability risk event scenario, risk assessment of impact to confidentiality integrity, and availability was performed and documented within the cybersecurity risk management documentation. Appropriate risk mitigation controls have been implemented and tested.

It's a shame that the risk assessment was not published. It does not seem like "The RealmDB gets decrypted" was in the assessment considering it contains API keys to third party services.

Was this an interesting read? Send your thanks to Sundar Pichai for refocusing Everyday Robots and enabling a summer sabbatical! If you're interested in discussing employment opportunities reach out to <author>@gmail.com.

The Arduino 2.x IDE is a step in the right direction compared to the 1.x IDE, but is still too simplistic for intermediate and advanced projects. The editor is developed on Eclipse Theia, which is based on Visual Studio Code, but does not directly support VS Code plugins.

The new IDE  is in many ways a frontend to the new arduino-cli which is itself a front-end to the arduino daemon. The arduino daemon makes one connection to the microcontroller and brokers serial data read/write to "pluggable monitors". The two monitors included out of the box are the serial plotter and serial monitor which receive serial data as JSON over websockets. Once again, these two tools are nice for introductory work but do not cut it for advanced use cases. Unfortunately, the pluggable monitor specification is not documented enough to create an arbitrary monitor from scratch. Another neat feature of the arduino-cli is the ability to compile and flash sketches directly from the command line. This is a very nice solution for using alternative editors if the Arduino Sketch directory format and dependency resolution is suitable.

However, sometimes a project needs an alternative directory layout and demanding plotting needs. This post outlines using a Makefile to compile and flash firmware to Arduino development kits as well as using a serial<->UDP bridge to allow using PlotJuggler as a serial visualizer.

System Configuration

A non-root user cannot access the Arduino microcontroller by default on most Linux installations. A udev rule can be used to grant permission.

A sample udev rule for an Arduino Nano saved as /etc/udev/rules.d/99-arduino.rules:

SUBSYSTEMS=="usb",ATTRS{idVendor}=="1a86",ATTRS{idProduct}=="7523",GROUP="plugdev",MODE="0666"

This rule adds the device with VID:PID 1a86:7523 to the plugdev group. The VID:PID for a device can be determined with lsusb.

$ lsusb
...
Bus 001 Device 010: ID 1a86:7523 QinHeng Electronics CH340 serial converter
...

The current user must be in the plugdev group to access the Arduino.

# usermod -a -G plugdev <username>
$ groups
... plugdev ...

If the user is in the plugdev group and the udev rules has the proper VID:PID, then loading and triggering the new udev rule should result in rootless access to the Arduino interface:

# udevadm control --reload
# udevadm trigger

Toolchain

The easiest method to get avr-gcc (the compiler) and avrdude (firmware uploader/downloader) is to install either the Arudino 2.x IDE or the Arduino CLI. The installers should populate a $HOME/.arduinoXX directory with the necessary binaries. The Arduino CLI may require installing the avr core via the CLI.

sudar/Arduino-Makefile is a great starting point for using a Makefile to compile Arduino sketches. Create a directory third_party and either git clone or git submodule the Arduino-Makefile repository.

my_project
├── src
└── third_party
    └── Arduino-Makefile

The Arduino Makefile project takes care of many aspects of compiling and uploading Arduino code once configured properly. To do so, create a Makefile in src with the paths for the installed version of avr-gcc, avrdude, and the Arduino MCU.

ARDMK_VENDOR = .
ARDUINO_VERSION = 186

# Set paths to avr-gcc and avrdude
ARDUINO_DIR = ${HOME}/.arduino15/packages/arduino
AVR_TOOLS_DIR = ${ARDUINO_DIR}/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7
AVRDUDE_DIR = ${ARDUINO_DIR}/tools/avrdude/6.3.0-arduino17
AVRDUDE = ${AVRDUDE_DIR}/bin/avrdude
AVRDUDE_CONF = ${AVRDUDE_DIR}/etc/avrdude.conf

Similar to the Arduino IDE, the Arduino board needs specified:

# Arduino Board Configurations
AVRDUDE_ARD_BAUDRATE = 115200
AVRDUDE_ARD_PROGRAMMER = arduino
MCU = atmega328
AVRDUDE_MCU = m328p

ARCHITECTURE	= avr/1.8.6
BOARD_TAG	= nano

Unlike the Arduino IDE, the libraries that are being used must be specified. The below lines say that user-added libraries are in third_party , of which we expect to use the Adafruit_NeoPixel library. Additionally, the system libraries Wire and EEPROM are used. Any additional libraries are required to be added to ARDUINO_LIBS

USER_LIB_PATH += $(realpath ../third_party)

ARDUINO_LIBS += Wire EEPROM 
ARDUINO_LIBS += Adafruit_NeoPixel

Finally, the build output directory and Arduino Makefile locations are specified:

OBJDIR = $(realpath ../build-$(BOARD_TAG))

include $(realpath ../third_party/Arduino-Makefile/Arduino.mk)

A directory layout for the above Makefile, including the Arduino-Makefile project and Adafruit NeoPixel library looks like:

my_project
├── src
│   ├── main.ino
│   └── Makefile
└── third_party
    ├── Adafruit_NeoPixel
    │   ├── Adafruit_NeoPixel.cpp
    │   └── Adafruit_NeoPixel.h
    └── Arduino-Makefile
 

Running make will compile main.ino and make upload will flash main.ino to the Arduino.

Autocomplete, Lint, Formatting, etc

Any IDE can be used with the Makefile based toolchain outlined above. Despite the overkill that is using Electron for an IDE, codium (a Microsoft-free build of VS Code) provides a simple way to use clangd as a Language Server.

The clangd extension can be installed from the Extensions sidebar. In order to use code completion and other features, clang needs a compile_commands.json file to explain the build process. bear provides an easy way to generate the compilation database:

$ make clean
$ apt install bear
$ bear -- make

Next, configure clangd to use clang-tidy by modifying the clangd extension settings to launch clangd with the --clang-tidy argument.

clang-tidy is configured by a .clang-format file, a good example being:

BasedOnStyle: Google
SeparateDefinitionBlocks: Always
EmptyLineBeforeAccessModifier: LogicalBlock
ColumnLimit: 100

The compile_commands.json and .clang-format files should be placed at the root of the project directory:

my_project
├── .clang_format
├── compile_commands.json
├── src
│   ├── main.ino
│   └── Makefile
└── third_party
    ├── Adafruit_NeoPixel
    │   ├── Adafruit_NeoPixel.cpp
    │   └── Adafruit_NeoPixel.h
    └── Arduino-Makefile

Adding a .gitignore to the project root prevents git from slurping up some of the one-off parts of the configuration. Depending on user preferences it may also be advisable to add third_party/ to .gitignore.

build-*/
compile_commands.json
.cache

Plotting

The Arduino 2.x IDE includes a Serial Plotter that plots data in a CSV-like format of key1:value1, key2:value2. For example, voltage:3.1,current:6.28\n results in two separate traces for voltage and current data. There are many usability flaws with the built-in Serial Plotter - axis limits cannot be set, X-Y plots are missing, only ten traces can be viewed at a time, the time window is fixed, etc. Fortunately there are many other plotting solutions that can be used.

Ideally, any plotting solution is backwards compatible with the built-in serial plotter. Kst is a great plotting tool but does not offer support for the key:value CSV-like format. serialplot supports the key:value format, interfaces directly with serial devices, and includes the ability to write data to the serial port, but lacks multiple plots. PlotJuggler offers a nice interactive user-interface but does not communicate directly with serial ports and does not provide a method for writing serial data; however, PlotJuggler's UX makes it worthwhile to overcome these limitations.

Despite what a LLM may say it is not this simple to get serial data into PlotJuggler:

However, PlotJuggler does provide a method to plot JSON packets sent over UDP and this one-liner pipes serial key:value data directly to PlotJuggler:

socat stdio /dev/ttyUSB0,raw,b115200 | sed -un 's/^\$//p' | jq -R --unbuffered 'split(",") | map(split(":")) | map({(.[0]): .[1] | tonumber}) | add' 2> /dev/null | socat - udp-datagram:127.0.0.1:9870

This one-liner uses socat to stream serial data to sed which filters for only lines that start with $, which are then passed to jq to convert the key:value pairs into valid JSON, which is then passed back to socat to stream the JSON as UDP packets to the PlotJuggler UDP server. Using the $ sigil at the start of a line allows other libraries to print debug information to the serial terminal without being plotted. PlotJuggler will use the "/" delimiter to group signals in the Timeseries list, in the following example both pitch and pitch_dot are grouped under r because they were printed as ...r/pitch:34.31,r/pitch_dot:0... .

While the above solution allows quick plotting of serial data, it does not allow for writing data to the serial port, or viewing the lines that do not start with $. Additionally, it is not very Windows friendly.

Fortunately, Python has nice libraries for interacting with both serial ports and UDP packets. A simple program that replicates the bash one-liner above is:

import json
import serial
import socket
import sys

arduino = serial.Serial(port="/dev/ttyUSB0", baudrate=115200)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)


def line_to_dict(line):
    """
    Convert a key:value CSV line in the Arduino Serial Plotter
    format to a dict.

    For example, "time:10,b:2,c:3" becomes {"time":10, "b":2.0, "c":3.0}
    """

    tree = {}
    for pair in line.strip().split(','):
        t = tree
        key, value = pair.split(':')
        parts = key.split('/')
        for part in parts[:-1]:
            t = t.setdefault(part, {})
        if parts[-1] == "time":
            t[parts[-1]] = int(value)
        else:
            t[parts[-1]] = float(value)
    return tree


while True:
    if arduino.in_waiting > 0:
        data = arduino.readline().decode()
        if data.startswith('$'):
            json_data = json.dumps(line_to_dict(data))
            sock.sendto(json_data.encode(), ('127.0.0.1', 9870))

A full-featured script to handle forwarding CSV-like serial data to UDP, MQTT, or websocket servers and writing serial data is available at bsilvereagle/serial_forward.

Live Reset

One of the nicest features of the Arduino IDE is the orchestration of stopping the serial plotter and serial monitor to upload new programs. Normally, a serial monitor like screen or minicom would retain full control of the hardware interface to the serial port, preventing avrdude from flashing new firmware. The Arduino IDE uses a daemon to broadcast the serial data to the serial monitor and plotter via websockets. When "upload" is clicked the daemon release control of the serial device and reclaims it once avrdude finishes uploading.

This behavior can be replicated with a little bit of signal handling. The Arduino Makefile has a RESET_CMD argument that can be overridden to send a HUP signal to any process using the Arduino device.

# HUP for Live Reset
RESET_CMD = fuser -k -HUP $(DEVICE_PATH) || /bin/true

The python program can handle the SIGHUP using the signal library.

import functools
import signal
...

def sighup_handler(signum, frame, aios):
    logger.info("Received HUP signal")
    aios.close()
    logger.info("Serial closed")
    time.sleep(10)
    logger.info("Reopening serial")
    aios.open()
    logger.info("Serial Opened")
    
handler = functools.partial(sighup_handler, aios=aios)
signal.signal(signal.SIGHUP, handler) 

Was this an interesting read? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

I found myself in this situation as part of an ongoing project to write custom firmware for the Elegoo Tumbller self-balancing robot.

The Tumbller uses an Arduino Nano clone with a MPU6050 IMU. The MPU6050 features a neat on-board Digital Motion Processor (DMP) which takes care of fusing the gyroscope and accelerometer readings into a usable pose. However, the Elegoo developers opted to hand-tune a Kalman filter with raw gyroscope and accelerometer readings; I wanted to use the DMP.

Fortunately, there exists an easy-to-use open-source driver to read from the DMP.  

Unfortunately, there were periodic spikes in the DMP output which threw a wrench in tuning a controller. The following figure shows pitch (derived from the DMP quaternion output) for a stationary robot:

Every so often the DMP output would erroneously jump before settling back to steady-state within a few hundred milliseconds:

So what to do? Swapping for a new MPU6050 (twice) and a new Arduino did nothing for eliminating the spikes, hinting that there was a bug lurking in the custom firmware.

To get a handle on the issue, I collected 500 samples of the DMP spikes and measured the time between consecutive occurrences. The hypothesis was that the time deltas between events would be a rough Exponential distribution, which models the time between events in a Poisson process. A histogram of the time delta measurements is below:

The histogram looks remarkably similar to an Exponential distribution! Fitting a curve was straightforward with scipy.stats.expon.fit.

With an Exponential distribution model in hand, it is possible to determine how long we need to sample without observing the DMP output drift to say with confidence that the bug is fixed. The Exponential distribution models how much time elapses in between events but we need to know how long to wait for an event to occur. The Poisson distribution models how many events occur in a given time window. Fortunately, there is a simple relationship between the two distributions that allows calculating the probability that an event will occur within a certain time period.

$$P(T > t) = \exp^{-\lambda t}$$ gives the probability for no occurences to happen in time \(t\) milliseconds. In this case, the \(\lambda\) parameter for the Exponential distribution is \(4.65\text{e-}5\) (1/scale from scipy.stats.expon.fit). For example, after starting the system the probability of observing a DMP spike within 15 seconds (15000 ms) is ~50%. Running for 64 seconds gives a probability of ~95%, and coincidentially running for 99 seconds gives a probability of ~99%.

In other words, if a fix is made to the firmware and the system runs for ~99 seconds without observing the DMP spike there is a 99% chance that it is fixed.

Armed with the knowledge of how long to test a change, it is possible to begin shotgun debugging. Fortunately, 50% of the time a change only needs to run for ~15 seconds to determine that it doesn't fix the issue so the debug-iteration loop is at an okay rate.

In this case, the issue was caused by resetting the FIFO buffer, waiting for a packet, then reading from the reset FIFO buffer.

  ...
  mpu.resetFIFO();
  if(!mpu.dmpPacketAvailable()){
    return -1;
  }

  mpu.getFIFOBytes(fifoBuffer, 42);
  mpu.dmpGetQuaternion(&orientation, fifoBuffer);
  ...

Invensense's documentation is very sparse, so it is difficult to piece together why clearing the DMP's FIFO buffer causes intermittent output drift, but removing the FIFO reset eliminated the spikes:

  ...
  //mpu.resetFIFO();
  if(!mpu.dmpPacketAvailable()){
    return -1;
  }

  mpu.getFIFOBytes(fifoBuffer, 42);
  mpu.dmpGetQuaternion(&orientation, fifoBuffer);
  ...

With no apparent link between the fix and the bug symptoms, the "proof" of the bug resolving is in the statistics.

Was this an interesting read? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

Unfortunately, Qucs requires Qt4, which has been deprecated. Debian completely removed Qt4 support from Debian 11 ("bullseye").

Fortunately, podman can be used to run a containerized version of qucs without issue. The Dockerfile below starts from a Debian 10 base image, installs the necessary dependencies for Qucs, downloads the version 0.0.20 source code, and compiles it.

FROM debian:buster
RUN apt update && apt install -y build-essential libqt4-dev libqt4-qt3support automake libtool gperf flex bison texlive-font-utils wget pkg-config libxml-libxml-perl && mkdir -p /opt/qucs && cd /opt/qucs
RUN wget -c https://github.com/Qucs/ADMS/releases/download/release-2.3.7/adms-2.3.7.tar.gz -O - | tar -xz && cd adms-2.3.7 && ./configure --prefix=/usr && make install
RUN wget -c https://sourceforge.net/projects/qucs/files/qucs/0.0.20/qucs-0.0.20-rc2.tar.gz/download -O - | tar -xz
RUN cd qucs-0.0.20 && ./bootstrap && ./configure --disable-doc && make -j 8 && make install

The container can be ran from the command line:

podman run -it \\
           -v .:/data \\
           -e DISPLAY=:0 \\
           -v /tmp/.X11-unix:/tmp/.X11-unix \\
           -v /home/user/.Xauthority:/tmp/.XAuthority \\
           --env=QT_X11_NO_MITSHM=1 \\
           6fa55ff299d \\
           qucs

The launch command is fairly straightforward with some nuances for using an X11 application in the container.

-e DISPLAY=:0 sets the current X11 server within the docker container. The volume mounts of .X11-unix and .Xauthority map the X11 server sockets and authentication into the container. --env=QT_X11_NO_MITSHM=1 prevents Qt from using MIT-SHM extensions which require access to /dev/shm.

This method should extend to any program that requires Qt4, at least as long as Debian continues to provide apt support for buster.

Was this an interesting read? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

Many, many, forms that accept e-mail addresses do not comply with the RFC 3696 specification for e-mail addresses:

Without quotes, local-parts may consist of any combination of alphabetic characters, digits, or any of the special characters

 ! # $ % & ' * + - / = ?  ^ _ ` . { | } ~

period (".") may also appear, but may not be used to start or end the local part, nor may two or more consecutive periods appear. Stated differently, any ASCII graphic (printing) character other than the at-sign ("@"), backslash, double quote, comma, or square brackets may appear without quoting

This commonly manifests itself when trying to take advantage of Gmail's task specific e-mail addresses using the '+' sign; the example below show's bayareafastrak.org's non-compliance.  

Using '+' unlocks virtually unlimited Gmail addresses so each service can have an individual address to track down leaks, etc. Fortunately, Gmail offers another way to create a finite set of e-mail addresses - dots don't matter in Gmail addresses.

Note the RFC verbiage on using dots/periods - the address cannot start, end, or contain repeated dots. So, a single period can go between any two characters meaning that if the "local" (username) part of a Gmail address has 12 characters, there are 2¹¹ or 2048 unique e-mail addresses that can be created by leveraging how Gmail handles periods.

The issue becomes handling these finite e-mail addresses. How do we make sure every e-mail address is unique? One option is to start sticking periods in the first available slot and checking a password manager to make sure that the addresses are unique.

Manually inserting periods works just fine, but requires a password manager to map from a given e-mail address to a service. The '+' method makes the mapping very easy - bsilvereagle+fastrak is clearly used for Fastrak but b.silve.ereagle has no human readable element. Fortunately, using some steganography we can embed human understandable elements into the e-mail.

The plan is to limit the information to lowercase letters and attempt to embed meaningful 2-3 character representations into the e-mail address. For example, "Hacker News" may be "hn", "Reddit" may be "r", etc.

It takes 5 bits to represent the 26 lowercase letters (log₂(26) ≈ 4.7) which can be done by mapping them to their respective digits on a 0-index basis, i.e. 'a' is '0', 'b' is '1', etc.

Next, we can convert the digits to a binary representation and then stuff them into the e-mail address. "hn" becomes "00111 01101" which can be represented in period form with "--...-..-.".

After that, the period/binary string can be slotted into the e-mail address, as shown below:

Note that the amount of encodable information is limited by the length of the original e-mail address. "bsilvereagle" can only fit 11 bits of period-encoded information and is limited to two characters, or 26² (276) unique combinations. Many of those combinations will not be meaningful ("zj", "xq", "xv", etc) so the useful combination range is significantly lower. An e-mail address that has 15 slots (at least 16 characters) opens up to 26³ (17576) combinations.

Fortunately, the script at the end of this post can automate the encoding and decoding of the period e-mails.

This steganographic method does not really improve the usability of a password manager handling the mapping of e-mails to services and is largely impractical.

Was this an interesting read? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

from itertools import chain, zip_longest


def alternate(list1, list2):
    # Merges two lists in an alternate fashion
    # ['A','B','C'], ['a','b'] -> ['A','a','B','b','C']
    return [x for x in chain.from_iterable(zip_longest(list1, list2)) if x]


def int2binlist(integer):
    # Converts an integer 5 into a list of binary representation padded to 5 spaces [0, 0, 1, 0, 1]
    if (integer >= 31) or (integer < 0):
        return [1, 1, 1, 1, 1]

    binary = [int(x) for x in list('{0:0b}'.format(integer))]
    return [0]*(5 - len(binary)) + binary


def encode(email, identifier, width=5):
    payload = "abcdefghijklmnopqrstuvwxyz"
    local, domain = email.split('@')
    # Determine the number of identfiers that can be encoded using one character per 5 available "slots"
    identifier_count = (len(local) - 1) // width
    # Look up identifier in payload, get binary representation, then convert to "." binary encoding
    encoding = ['.' if x else '' for x in chain(*map(int2binlist, map(payload.index, identifier[:identifier_count])))]
    return ''.join(alternate(local, encoding)) + '@' + domain


def decode(email, width=5):
    payload = "abcdefghijklmnopqrstuvwxyz"
    local, domain = email.split('@')
    encoded = []
    local_iter = iter(local[1:-1])  # Skip first and last character
    for c in local_iter:
        if c == '.':
            encoded.append(1)
            next(local_iter)  # We've recorded something for this slot, move to the next one
        else:
            encoded.append(0)
    identifier_count = len(encoded) // width
    result = ''
    for i in range(identifier_count):
        result += payload[int(''.join([str(i) for i in encoded[i*width:(i+1)*width]]), 2)]

    return result

print(encode("aaaaaaaaaaa@gmail.com", "secretkey"))
a.aaa.aaaa.aaa@gmail.com

Numerous requests have flooded in requesting an enumeration of the home media releases of Scooby-Doo in a similar fashion to the breakdown of Bugs Bunny, so in Halloween Spirit, here it is.

The following lists are only for Scooby-Doo works produced before Hanna-Barbera was bought by TBS in 1991 and then folded into Cartoon Network and Warner Bros. Episodes where Scooby-Doo was included in a large cast of Hanna-Barbera properties are also excluded.

Many, many, Scooby-Doo compendiums have been released that are samplings of episodes from different seasons. These are best avoided. Fortunately, Hanna-Barbera has made a concerted effort to also release complete seasons despite being a subsidiary of Warner Bros.

Complete Television Series Releases

The following six series are available in their entirety, providing 137 episodes of Scooby and the Gang:

Partial Television Series Releases

The Richie Rich/Scooby-Do Show and Laff-A-Lympics have both been partially released. The first seven episodes of The Richie Rich/Scooby-Do Show have been released. The second season of Laff-A-Lympics has not been released, but the whole first season has over two separate disks.

Separated Television Series

The New Scooby and Scrappy-Doo Show aired two seasons in 1983 and 1984. The second season was renamed New Scooby Doo Mysteries. Neither season has been released to disk in their entirety, but both are viewable on Amazon Prime. The first season is streamed as Season 5 of the "Scooby-Doo and Scrappy Doo Show". The second season is artificially split into two more seasons and watchable as "New Scooby Doo Mysteries".

17 of 44 individual episodes from this show have been trickled across various Scooby-Doo compilation disks.

Did this post help you enjoy Scooby-Doo? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

It is not obvious how to turn these features off, and even when you do, some features are left behind cluttering up the HTML.

In the Settings -> Membership, change Subscription access to "Nobody".

Despite not allowing subscriptions, pesky "Subscribe" buttons and forms are left behind in the default Casper theme. These can be hidden by going to "Settings" -> "Code injection" and applying the following CSS in the Site Header:

<style type='text/css'>
    /* Hide the Membership aka Subscribe actions etc. */
    #ghost-portal-root,
    a[href^="https://ghost.org"],
    a[href^="#/portal/signup"],
    .footer-cta { display: none; }
</style>

Did this make your blog look cleaner? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

An interesting no-brand air quality monitor showed up in an Amazon search recently. The sensor is two bare PCBs with an Asair AM2120 temperature & humidity sensor, a SenseAir S8 CO2 sensor, a Plantower PMS5003 PM2.5 sensor, and a 2.8" LED display all sandwiched between two pieces of acrylic via plastic standoffs.

The monitor is only ~$25 more than the qty 1 price of the sensors, which seems like a reasonable markup to not have to write any code. An identical monitor can also be found on banggood that is priced much closer to BOM. Since the sensors are not soldered to the PCBs, if the monitor dies they can be easily removed and moved to another project.

The main downside to the monitor only applies to Fahrenheit users - the temperature display is only in Celsius!

Let's patch this thing & prove the seller wrong!

Finding the Debug Header

There are several unpopulated 0.1" headers around the PCB. From images on banggood, other variants of the monitor have additional sensors populating them. In order to dump the firmware and patch it, one of the headers needs to carry either JTAG or SWD signals.

Finding the debug header was fairly straightforward, using a very similar method to the one demonstrated by Eric Schlaepfer.  The monitor was disassembled and well-lit pictures of both sides of the PCB were overlaid with reduced opacity so traces on the top and bottom layers could be seen together.

Note: Mirroring the side of the PCB containing the MCU makes it very difficult to match pins to the datasheet!

The STM32F103C8 was given away by the silkscreen and the datasheet highlighted the likely SWD pins.

Fortunately, PA13 & PA14 traced out to the same header with NRST - the SWD debug header was found! The remaining two pins in the header were continuity checked against the 3.3V and GND from the AM2120 to determine polarity.

Dumping the Firmware

With the debug header discovered and mapped, it is straightforward to wire up a JTAG programmer (STLink, J-Link EDU, etc).

Dumping the firmware can be accomplished with openocd, a sample config file for this target with a J-Link is:

source [find interface/jlink.cfg]

transport select swd

source [find target/stm32f1x.cfg]

Fortunately, the flash is not read-protected and can be extracted with flash read_bank 0 bank0.bin 0x0 0x1FFF via the telnet interface to openocd.

$ openocd -f openocd.cfg
Open On-Chip Debugger 0.10.0 
Licensed under GNU GPL v2 
For bug reports, read 
	http://openocd.org/doc/doxygen/bugs.html
adapter speed: 1000 kHz
adapter_nsrst_delay: 100
none separate
cortex_m reset_config sysresetreq
Info : No device selected, using first device.
Info : J-Link V10 compiled Mar  7 2019 15:19:19
Info : Hardware version: 10.10
Info : VTarget = 3.328 V
Info : clock speed 1000 kHz
Info : SWD DPIDR 0x1ba01477
Info : stm32f1x.cpu: hardware has 6 breakpoints, 4 watchpoints

$ telnet 0 4444 
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
Open On-Chip Debugger 
> flash list
{name stm32f1x base 134217728 size 0 bus_width 0 chip_width 0}
> flash read_bank 0 bank0.bin 0x0 0x1FFFF
device id = 0x20036410
flash size = 64kbytes
wrote 131071 bytes to file bank0.bin from flash bank 0 at offset 0x00000000 in 1.897898s (67.443 KiB/s)

Finding the Temperature Drawing Function

With the firmware stored in bank0.bin it's time to fire up ghidra and determine where the temperature is being displayed.

PM DETECTOR appears on the monitor's display which makes it a good candidate to find the screen drawing function(s). Fortunately, PM DETECTOR appears as a "Defined String" in the binary. The function defined at 00003bbc uses the string.

‌ Taking a deeper look at this function reveals that there are 10 calls to a function with 4 arguments; coincidentally there are 10 lines on the display. The arguments to this function also appear to be in an x1,y1,x2,y2 format. Additionally, _DAT_0003fb8 is set to 0x9bdd - a shade of blue in the RGB565 space. So, it seems that FUN_00003bbc is responsible for drawing the main screen of the monitor.

FUN_000012d2 appears to be drawing strings to the screen, and the call on L30 has the address for the "Temp" string. Ghidra does not detect strings less than 5 characters, so "Temp" was not formatted in a similar fashion to "PM DETECTOR".

‌The "Function Call Tree" feature of ghidra highlights which functions call a particular function. Renaming FUN_00003bbcto Screen1Static() makes it clearer what is going on in the parent function, FUN_000047a6.

FUN_000047a6 calls Screen1Static() once before a infinite loop, and then again in the infinite loop which has different behavior depending on the value of iVar4. Taking a look inside of FUN_00003142, it is clear it is returning the state of a digital input; when the side button of the monitor is pressed, the display will cycle through three screens, suggesting iVar4 maps to the side button and bVar1/ pbVar2 maps to the number of times the button has been pressed.

A similar state change based on iVar4 /bVar1 occurs later on in FUN_000047a6 (presumably the "main" function), suggesting that FUN_00004634 may be the function performing the dynamic update of the sensor readings.

FUN_00004634 is a wrapper function that only calls four other functions, the second being FUN_000042f4. FUN_000042f4 contains two function calls which divide an argument by 10; the AM2120 datasheet specifies that humidity and temperature readings are 10 times larger than the actual value. It looks very promising that this is the function displaying the temperature and humidity to the screen.

Furthermore, L31 - L33 print a number divided by 10, a decimal point, and then the modulus of the number with respect to 10. In other words, these three lines print "716" as "71.6". The only value printed to the monitor's screen which has a decimal is the temperature reading - the temperature drawing function has been found!

Patching in Fahrenheit

Because the temperature value returned by the AM2120 is 10x the actual temperature in Celsius, the usual Fahrenheit to Celsius formula needs adjusted:

$$F = \frac{9}{5}C + 32$$
$$F^* = 10F$$
$$C^* = 10C$$
$$\frac{F^* }{10} = \frac{9 C^* }{50} + 32$$
$$F^* = \frac{9 C^* }{5} + 320 = \frac{9 C^* + 1600}{5}$$

Transcribing the formula to ARM assembly is fairly straightforward. Unfortunately, many ARM assembly simulators do not support sdiv so the first time the code was tested was live on the processor!

ldr        r0,[Temperature]		# Load the temperature address
ldrh       r0,[r0,#0x0]			# Load the temperature value
mov        r1,#0x9			# Move 9 into r1
mul        r0,r0,r1			# Multiply temperature by 9
mov.w      r1,#0x640			# Move 1600 to r1
add        r0,r0,r1			# Add 9*C to 1600
mov        r1,#0x5			# Move 5 to r1
sdiv       r0,r0,r1			# Divide 9*C+1600 by 5
ldr        r1,[Temperature]		# Load temperature address
strh       r0,[r1,#0x0]			# Store the temperature

With the assembly in hand, the next step was to find an appropriate place to insert the ten instructions. Right above the call to FUN_0000114C (draw a number to the screen) the color for the number draw is selected.

The color  (DAT_00004474) is based on the value of the temperature reading (10x the temperature in ℃). The color scheme is RGB565 which fits in 2 bytes instead of the three bytes required for RGB888. The values can be trivially converted to RGB888 color space using the snippet of Python below.

def rgb565_to_rgb888(rgb565):
    red = (rgb565 & 0xf800) >> 8
    green = (rgb565 & 0x7e0) >> 3
    blue = (rgb565 & 0x1f) << 3
    return red, green, blue
    
rgb565_to_rgb888(0x1c9f)
(24, 144, 248)

Displaying a different color for values above 50℃ does not necessarily provide a lot of value, so that functionality is a prime candidate for replacing with the Fahrenheit conversion function.

The block of assembly at  00004376 sets the Color to 0x1f (Blue) and all of the jumps to that label occur in the block 00004358, right above it, so it looks like overriding this functionality will work cleanly.

Starting at 0000437a and working up, the 10 Fahrenheit instructions fit in neatly with one dangling instruction at 00004360 which can be converted to either a nop or a direct jump to 00004362. Since all branches of the color selection tree should calculate Fahrenheit, all of the jumps to 0000437c (the ℃ read) need updated to  00004362 (the ℉ read). Note that at first blush it looks like the instructions at 00004378:437a are redundant since the Temperature value is stored and then immediately read. Storing the Temperature is necessary for the tenth of a degree draw call after the decimal point is drawn.

In theory, the temperature reading should be converted to Fahrenheit and printed to the screen!

Fixing the ℃ Glyph

Calculating the temperature in Fahrenheit is only half the battle - the display has a  ℃ glyph which needs to be updated to ℉. In the screen setup function there are two similar calls right after the "Temp" and "RH" (Relative Humidity) strings are drawn to the screen. Both the "Temp" and "RH" cells in the also have static units drawn in them,℃ and %, respectively.

Diving into the suspicious function, it appears to have three nested loops, the outer running 16 times, the middle running 8 times, and the inner running 2 times for a total of 256 (or 16x16) iterations. The ℃ and the % glyphs on the display both look like 16x16 bitmaps, further suggesting that this function is indeed blitting a 16x16 image to the display. Both the ℃ and % appear to be in the same x column on the display, suggesting that the first argument to FUN_00000dac is the x-coordinate, the second argument being the y-coordinate,  with the third paramter potentially being an index into a sprite array.

Assuming that third parameter is indeed a sprite index, it is possible to calculate the memory address for each glyph by multiplying the index by 256 and adding that to the address located at DAT_000000e50 (Line 15 in the disassembly). However, for reasons that will become apparent, this binary data did not seem to match the expected pattern for the ℃ glyph.

Fortunately, with a bit of imagemagick black magic we can visually hunt for these glyphs! The following snippet reads in the binary file as a 16x16384 image (one long column), breaks that column into 16x16 sections, and then stitches them back together, effectively y-axis overflow wrapping the 16x16384 column.

montage <(convert -depth 1 -size 16x16384 gray:bank0.bin -crop 16x16 miff:-) -tile 1x32 -geometry 16x16+0+0 miff:- | montage - -geometry +0+0 -tile x1 binary.png

With a little bit of  searching it's possible to find the ℃ which just so happens to be at the exact memory location it was supposed to be ( 0x47c6 + 0x18*256)! However, the image is rotated and has an 8 bit offset.

With the format known, it was trivial to create a similar ℉ glyph and patch the binary with the hex representation.

Flashing the modified firmware

Diffing the two firmware images shows the three distinct regions that were patched; a fixed typo at 00004060, the Fahrenheit formula at 00004320, and the °F glyph update at 00004ac0:

diff -u <(xxd bank0.bin) <(xxd bank0.bin.mod)

Reflashing the modified binary to the monitor is straightforward using  openocd:

halt
stm32f1x unlock 0
reset halt
stm32f1x mass_erase 0
flash write_bank 0 bank0.bin.mod 0x0

And now the temperature displays in Fahrenheit with a proper unit designation!

Edit: Many thanks to dmitrygr for playing code golf on the core of the asm function!

movw r1, #0xe677
muls r1, r0
lsrs r1, #15
add r1, #320

Did this inspire you to go out and hack something? Say thanks and help keep this site ad & analytic free by using my Amazon Affilliate URL. I'll receive a small portion of any purchases made within 24 hours.

Bitwarden provides docker images to self-host an open-source version of the Bitwarden server. After recent updates to the Bitwarden client which broke compatiblity with rubywarden the official image is a viable option despite requiring mssql.

However, to unlock Premium features the self-hosted version must have a "verified" e-mail address. Unfortunately, this requires setting up an SMTP relay or taking the self-hosted server onto the internet.

Fortunately, the "verification" is just a bit in the MSSQL tables, which can be trivially set without the need to click a verification e-mail link!

1. Find the unique password for the bitwarden mssql server

$ cat bwdata/env/mssql.override.env 

2. Launch a shell in the mssql server docker container

sudo docker exec -it bitwarden-mssql /bin/bash

3. Launch the mssql client

root@5223f3f519cc:/opt/mssql-tools/bin# ./sqlcmd -S 127.0.0.1 -U sa -P <key>

4. Find the user who needs verified

USE master
GO
SELECT Name, Email, EmailVerified FROM [User]
GO

5. Update the EmailVerified field

UPDATE [User] SET EmailVerified = 1 WHERE [User].Name="<Name>"

6. Wait a few minutes and then refresh the web interface

Did your security posture improve because of this post? Consider saying thanks by using my Amazon Affilliate URL and help to keep this site ad & analytics free.