python-dectest — Improved unittest.TestCase

I have released the first version of dectest for Python (Github), an improved version of unittest.TestCase. It is a drop-in replacement with two improvements:

  • decorators for tests, setup, and teardown methods. This is not only more explicit than the “magic” names of unittest, it also allows multiple setup and teardown methods per class.
  • patch() helper method that calls unittest.mock.patch(), but stops the patch during teardown.

Python asserts and json-get with type hints

I released new versions of python-asserts  (0.8.2) and python-json-get (1.1.0) that include type hints, so that typecheckers like mypy will now be able to type check calls to those libraries. json-get uses asserts for its tests and it’s now possible to type check it without --ignore-missing-imports.

In addition, json-get now has a json_get_default() function that returns a default value instead of raising an exception if a path is not found.

Friendly Printers

At the offices of one of my customers, the printers are extra-friendly. Whenever I arrive or leave, they all greet me with a small concert.

I suspect that connecting or disconnecting USB devices from my laptop (such as the mouse dongle), CUPS is sending a broadcast over the network, querying printers of their status. This seems to wake up all Epson printers and force them to do a small self-check. This certainly needs more research.

dbupgrade – A Database Migration Tool – published

Yesterday I published dbupgrade, a simple database migration tool, written in Python. It allows you to put database migration files (simple SQL files with a special header) into a directory and upgrade your database schema. While this is a new project, it was extracted from an internal library I’ve been using for years and should work fairly reliably for MySQL/MariaDB and Postgres databases.

dbupgrade is available at github and at pypi or via pip: pip install dbupgrade.

Chipcard Woes

One of my customers, a large dental practice, had a problem from time to time when reading Krankenversicherungskarten (German health insurance card, KVK). The insurance number got all garbled up, which led to problems when reading the patient’s insurance data into the health care application. After their sysadmin and I analysed the situation it seemed that the application reading the data from the card and writing it to a file in a custom format was to blame. This program (KVKTool) was included with the Cherry card reader and had been modified by another consultant to write the read data to a file rather than displaying it on the screen.

This C++ program was a huge stinking pile. All card reading was done in a method called CKVKToolDlg::Onreadcard. The name CKVKToolDlg hints at it: This is the class responsible for showing the dialog. (Yes, it had been left in the program by the previous consultant. Only the dlg.DoModal() call had been commented out.) The method was a gigantic 837 lines long and features all kinds of goodies such as lots of copied-and-pasted code that had been modified in some instance to fix bugs, but not in others.

After a long refactoring session (which turned into a rewrite), I was able to spot the problem: a wrong workaround to bugs in a sizable amount of KVKs. Normally, the data on KVKs is structured into fields in an arbitrary order. These fields consist of a one-byte tag to identify the field, a one-byte length marker, and then n bytes of data, where nis the length. All KVK fields are identified by tags between 0x80 and 0x92. But all KVK fields are wrapped by a super-field called the “Versichten-Template” (insurant template, VT) with tag 0x60. Thus, data on a typical (non-buggy) KVK card would start like this:

0x60 0x89 0x85 0x09 0x53 0x65 0x62 0x61 …
VT tag VT len First name tag First name length Seba …

Now, buggy cards seem to insert a stray 0x81 byte after the 0x60 tag:

0x60 0x81 0x89 0x85 0x09 0x53 0x65 0x62 0x61 …
VT tag ??? VT len First name tag First name length Seba …

This code in the KVKTool was supposed to handle this problem:

// skip tag 60 and length
int i=2;

// some cards seem to have a damaged tag 0x89 after 0x81
if (responsearray[1] == 0x81)
    i++;

(Snipped the huge indentation.)

responsearray is a buffer that contains the KVK data and i is an index into that buffer. This code works fine in most circumstances. It works around cards with the stray byte. But there is one situation where it utterly fails: On correct cards where the Versichten-Template is exactly 129 bytes long. In this case, the first tag is skipped entirely, which usually is the insurant’s number. Since it was not, junk (random memory content) was written to the file in place of the insurant’s number. Not good, but hopefully fixed now.

VirtualBox on i386 with amd64 Kernel

I have recently started to use an amd64 kernel on my i386 Debian unstable system. Unfortunately, VirtualBox OSE does not work with that setup. When I try to start a virtual machine, it fails with an oblique error message:

RTR3Init failed with rc=-1912 (rc=-1912)

The VirtualBox kernel modules do not fit to this version of VirtualBox. The installation of VirtualBox was apparently not successful. Executing

‘/etc/init.d/vboxdrv setup’

should fix that problem. Make sure that you don’t mix the OSE version and the PUEL version of VirtualBox.

Debian bug #456391 explains the problem. In that report Michael Meskes alludes to running VirtualBox in an amd64 chroot jail, so I tried this myself. It works flawlessly, once I got it setup. Here is what I did (as root):

robinson:~# mkdir /srv/amd64
robinson:~# cdebootstrap --arch amd64 sid /srv/amd64 http://ftp.debian.org/debian/
[...]
robinson:~# chroot /srv/amd64
robinson:/# apt-get update
[...]
robinson:/# apt-get upgrade
[...]
robinson:/# apt-get install virtualbox-ose # add more packages here if needed
[...]
robinson:/# adduser --uid 1000 --no-create-home --disabled-password --disabled-login srittau
[...]
robinson:/# 

These commands install the base system and create a user account. Now I created a script called /usr/local/bin/amd64.sh:

#!/bin/sh

CHROOT=/srv/amd64

if test ! -e $CHROOT/dev/.udev; then
    mount -t none /dev $CHROOT/dev/ -o bind
fi
if test `ls $CHROOT/proc | wc -l` = "0"; then
    mount -t proc none $CHROOT/proc
fi
if test `ls $CHROOT/sys | wc -l` = "0"; then
    mount -t sysfs none $CHROOT/sys
fi
if test `ls $CHROOT/home | wc -l` = "0"; then
    mount --bind /home $CHROOT/home
fi
chroot $CHROOT sh -c "su - srittau"

Running sudo amd64.sh will now enter the chroot environment as user srittau where I can start virtualbox normally.

Helmar Fanselau dies aged 56

I just learned that one of my former teacher, Helmar Fanselau, died on January 13th, aged 56. He was a strange man. Many students liked him because of his antics and strange sense for humor. Others did not like him for his harsh and unforgiving marks in his subjects, Latin and History. I think he was always fair, though, even if that worked to my disadvantage, because he actually liked me, but I got bad marks nevertheless. I still fondly remember him balancing rulers on his nose and for his ability to speak several different German dialects. His “weather show” featured them all in a stand-up comedy-like routine.

He always was a controversial person, though. I just found an article I wrote back in ’96 or ’97 about a school mess held by him during which the school parson actually put out the candles in protest.

There were always various rumours about Mr. Fanselau floating around in the school. For example, he was well known for eating from student’s lunch boxes. So, the story goes, a student prepared a special sandwich with tooth paste and put it on the teacher’s desk. Mr. Fanselau entered, saw the sandwich and ate it. When I questioned him about the story, Mr. Fanselau confirmed it, but was quick to add: “Well, I didn’t eat the whole sandwich.”

Finally, the way I found out about Mr. Fanselau’s death is as bizarre as was his personality: A rogue edit on Wikipedia. Goodbye, Helmar!

Upper Deck Entertainment vs. Konami

This is my take on the conflict between Konami and Upper Deck Entertainment (UDE) that is currently upsetting the Yu-Gi-Oh! Trading Card Game community. While I don’t play Yu-Gi-Oh! myself, as a Magic: The Gathering judge I have a certain interest in what happens in the industry overall. Oh, and I like soap operas. All information presented is scrounged from various sources around the net, including press releases by the companies, statements from employees, and court documents (or copies thereof provided in forums). Mixed in is my own opinion and assumptions, so don’t take anything in this article for granted.

This is what I think happened so far:

Konami is the producer of the Yu-Gi-Oh! Trading Card Game and owner of the rights to it. Upper Deck Entertainment is the distributor of the game in Europe and the US. UDE also handled Organized Play (OP) for the game in these territories. The distribution contract ended at the end of 2007, but there was a mutual agreement (either verbal or in the form of a Letter of Intent) between both companies that the contract was to be renewed for 2008 through 2010. There was no formal contract, though, so the companies just kept on going. (And I thought those were serious businesses …)

UDE was unhappy with the way Konami handled the game and how they were supported by Konami. So they took more and more liberties with how they ran the game. This reached a climax when in the summer of 2008 UDE manufactured copies (or “counterfeits” as Konami calls them) of special Yu-Gi-Oh! promotional cards and provided them to Vintage Sports Cards, who packaged them with three boosters and sold those packs in Toys-R-Us stores. Konami found out about the counterfeits and sued Vintage Sports Cards. When they found out that UDE had provided the cards, they allegedly asked UDE for clarification. When were giving insufficient information and threatened Konami with a breach of contract suit, Konami amended their suit at the end of 2008 to include UDE as a defendant. But before they could do so, UDE counter-sued on December, 12th 2008.

This led to the well-known exchange of press releases. (December, 11th 2008, Konami: “We’re taking over the distribution of Yu-Gi-Oh!“; December, 16th 2008, UDE: “Not true!“; December, 17th 2008, Konami: Yes, true!) At December 22th UDE wins an injunction against Konami in Europe that voids the cancellation of the contract. And a few days later the same happens in the US.

So, this year both companies are flexing their muscles and the background of the story became known. The main lawsuits are still pending, so it will stay interesting. In the long term it’s obvious that UDE will lose any distribution rights to the Yu-Gi-Oh! TCG, so all their bets are now on various World of Warcraft products. I am not sure what that means for Upperdeck’s OP department. I know that I will keep a watchful eye on further developments.

Coincidences

I turned 30 yesterday. I’m still not sure whether that’s a reason to celebrate or a reason to mourn. Anyway, there were two coincidences. Late on Sunday evening I was watching the remake of Shaft and started to wikipedia (doesn’t have the same sound as “google”, right?) for among other things Blaxploitation. Funnily, later that evening I watched Live and Let Die, the blaxploitation Bond movie.

Still reading Wikipedia I started to read about Tarantino movies and came across the article of Zoë Bell, the stunt double of Uma Thurman in Kill Bill and playing herself in Death Proof. It seems she has birthday on the exact same day as I. So, happy birthday to you, Zoë!