Tag: software development

Categories
Software Development

Chipcard Woes

One of my customers, a large dental practice, had a problem from time to time when reading Krankenversicherungskarten (German health insurance card, KVK). The insurance number got all garbled up, which led to problems when reading the patient’s insurance data into the health care application. After their sysadmin and I analysed the situation it seemed that the application reading the data from the card and writing it to a file in a custom format was to blame. This program (KVKTool) was included with the Cherry card reader and had been modified by another consultant to write the read data to a file rather than displaying it on the screen.

This C++ program was a huge stinking pile. All card reading was done in a method called CKVKToolDlg::Onreadcard. The name CKVKToolDlg hints at it: This is the class responsible for showing the dialog. (Yes, it had been left in the program by the previous consultant. Only the dlg.DoModal() call had been commented out.) The method was a gigantic 837 lines long and features all kinds of goodies such as lots of copied-and-pasted code that had been modified in some instance to fix bugs, but not in others.

After a long refactoring session (which turned into a rewrite), I was able to spot the problem: a wrong workaround to bugs in a sizable amount of KVKs. Normally, the data on KVKs is structured into fields in an arbitrary order. These fields consist of a one-byte tag to identify the field, a one-byte length marker, and then n bytes of data, where nis the length. All KVK fields are identified by tags between 0x80 and 0x92. But all KVK fields are wrapped by a super-field called the “Versichten-Template” (insurant template, VT) with tag 0x60. Thus, data on a typical (non-buggy) KVK card would start like this:

0x60 0x89 0x85 0x09 0x53 0x65 0x62 0x61 …
VT tag VT len First name tag First name length Seba …

Now, buggy cards seem to insert a stray 0x81 byte after the 0x60 tag:

0x60 0x81 0x89 0x85 0x09 0x53 0x65 0x62 0x61 …
VT tag ??? VT len First name tag First name length Seba …

This code in the KVKTool was supposed to handle this problem:

// skip tag 60 and length
int i=2;

// some cards seem to have a damaged tag 0x89 after 0x81
if (responsearray[1] == 0x81)
    i++;

(Snipped the huge indentation.)

responsearray is a buffer that contains the KVK data and i is an index into that buffer. This code works fine in most circumstances. It works around cards with the stray byte. But there is one situation where it utterly fails: On correct cards where the Versichten-Template is exactly 129 bytes long. In this case, the first tag is skipped entirely, which usually is the insurant’s number. Since it was not, junk (random memory content) was written to the file in place of the insurant’s number. Not good, but hopefully fixed now.

Categories
Software Development

Programming Warts Wiki

There is a new Programming Warts Wiki. I think this is a great idea, since it encourages discussion about the various advantages and disadvantages of different programming languages and may help to remove some kinks in those languages.

Categories
Python

setuptools breaks PYTHONPATH

setuptools and egg files are a great way to distribute Python packages and programs. But today I stumbled over a really braindead design decision: setuptools overrides Python’s standard module search path algorithm in a very inconvenient way. Normally, when Python looks for a module or package, it first looks in the current directory, then in any path specified in the PYTHONPATH environment variable, and finally in the default search path. setuptools changes the lookup order to the following:

  1. the current directory
  2. egg files specified in PYTHONPATH
  3. egg files in the default search path
  4. regular modules in PYTHONPATH
  5. regular modules in the default search path

This means that an egg file in the default search path overrides a non-egg module in PYTHONPATH. I wonder what the rationale for that decision is (and it seems to be a deliberate decision). PYTHONPATH is a means to override the default search path and is perfectly suited for testing, developing, and forcing to use a particular (e.g. a bundled) version of a package. Not anymore! Now, the only way to override an egg is to create your own egg. And this is non-trivial and often means an extra step during development.

Here is where it bites me: I have created a library of custom classes, that I use in several projects. I break API frequently, since it is developed in a just-in-time fashion: whenever I need some functionality, I implement it. Therefore I bundle the library (or parts of it) with most projects. This works well for projects that go into a separate directory, like web projects. Nevertheless I use the library for some other projects that are installed system-globally. Otherwise I would need some way to namespace it for the different projects. Not my idea of fun. Normally, this is not a problem, since I can just set PYTHONPATH for all projects that have a local copy, at least when developing. Due to the strange path handling of setuptools, this method breaks and I have to work around it. Great …

Categories
GNOME Python

gnome-keyring with Python

The documentation on gnome-keyring I discovered helped me to access it successfully with Python. I’ve written a small module that fetches and stores a username and a password for some server.

Some notes:

  • The attributes are freeform, but there a some common attributes for network hosts. These are: “user”, “domain”, “server”, “object”, “protocol”, “authtype”, and “port”. Actually there is a convenience API for network hosts.
  • libgnome-keyring requires that the application is correctly set using g_set_application_name. Unfortunately the Python module gnomekeyring does not do that for us and pygtk does not provide a function to set the application name. Therefore you need to include the module gtk to make it work. (Filed as bug 460515.)Update: g_set_application_name was added to pygtk in the 2.13 development version.
  • Python’s find_network_password_sync function does not allow None to be passed as arguments. The C API allows NULL to be passed here. This means that this function is basically unusable in Python, since you never want to provide all values. (Filed as bug 460518.)
Categories
GNOME

Documentation for libgnome-keyring

I was trying to find documentation for libgnome-keyring for little project I am writing, which accesses a password-protected web service. Unfortunately there is no real documentation for it. No API documentation (well, there are a total of two functions documented), no tutorial. Finally I found this document in GNOME’s Subversion repository. Better then nothing, but why isn’t this part of the API documentation?

Filed as bug 460475.

Update: It seems that libgnome-keyring has gained documentation a few weeks ago, so far only found in Subversion. Kudos to the maintainers!

Categories
GNOME

Selection in GtkTextBuffer

I’ve recently played around with GtkTextBuffer. It’s a rather nice text editing widget (or rather widget part). Unfortunately it misses one functionality, which is also missing from GtkEditable derived widgets: A signal for selection changes. There are two workarounds:

  1. You can setup a notification on the “has-selection” property like this: 
    buffer.connect("notify", on_buffer_notify)

def on_buffer_notify(buffer, prop):
    if prop.name == "has-selection":
        if buffer.get_has_selection():
            ...
        else:
            ...

    This method only works if you are interested in whether there is a selection or not, not if you are interested in any selection changes. Also, this method works only for GtkTextBuffers, not for GtkEditables.

  2. Tomboy uses a timeout to check the selection status: 
    namespace Tomboy
{
	public class NoteWindow : ForcedPresentWindow 
	{
                ...
		public NoteWindow (Note note) : 
			base (note.Title) 
		{
                        ...
			// Sensitize the Link toolbar button on text selection
			mark_set_timeout = new InterruptableTimeout();
			mark_set_timeout.Timeout  = UpdateLinkButtonSensitivity;
                        ...
                }

                ...

		void UpdateLinkButtonSensitivity (object sender, EventArgs args)
		{
			link_button.Sensitive = (note.Buffer.Selection != null);
		}

                ...
        }
}

    This method works, but seems a bit complicated, when a simple signal should do the trick.

Filed as bug #448261.

Categories
Java

Java File API

I don’t like Java’s File API. It’s main problem is that is mixes several responsibilities into one class. One responsibility is handling of “abstract” file and path names, i.e. operating system independent file names. The other responsibilities are all file system related: Querying file meta information (access rights, access times, file type), creating files, and listing directory contents. In my opinion these should be clearly separated.

I am also missing a constructor or construction method that creates a File instance from a list of path components like this:

  File f = new File("/", "path", "to", "file");

Maybe I should start a Java warts page, similar to my Python warts page …

Categories
Java

Java: Iterators are not Iterable

This is something I stumbled across multiple times now in Java: Iterators are not Iterable. Java 1.5 introduced the foreach statement, which allows easy iteration over collections like this:

for (MyClass item: myCollection) {
    doStuffWithItem(item);
}

For this to work class MyClass must implement the Iterable interface. This interface defines just one method that returns an Iterator:

    Iterator<T> iterator();

This works fine in many cases. But now suppose we have a class that can be iterated in multiple ways. For example a Tree class that can be iterated breadth first or depth first. I would define two methods and use them like this:

class Tree {
    Iterator<Node> depthFirstIterator();
    Iterator<Node> breadthFirstIterator();
}

// ...
    for (Node node: myTree.depthFirstIterator()) {
    }

This won’t work in Java. Iterators do not extend the Iterable interface, like they in other languages. For example in Python an iterator will return itself when its iterator() equivalent is called. Instead in Java I have to return an object that itself implements the Iterable interface like this:

class IterableIterator<T> implements Iterable<T> {
    private Iterator<T> iter;

    public IterableIterator(Iterator<T> iter) {
        this.iter = iter;
    }

    public Iterator<T> iterator() {
        return iter;
    }
}

class Tree {
    IterableIterator<Node> depthFirstIterator();
    IterableIterator<Node> breadthFirstIterator();
}

Not only is this boilerplate code for no gain, a generic class like the IterableIterator above is not included in Java’s standard library.

Categories
Javascript

JavaScript Includes Revisited

I’ve given in in the JavaScript and Modules matter. I’m doing now what all the cool kids are doing: Using XMLHttpRequest synchronously to load external JavaScript files. This has the advantage that it works synchronously, a very important thing for includes. Therefore I don’t need callback and moduleLoadedhackery anymore. Also I have sensible error checking, i.e. I can notice when loading a JavaScript module failed. This enables me to have multiple search paths, for example, and sensible error handling.

But it comes at a cost, too. XMLHttpRequest is only supported by “modern browsers”. But since moderns browsers means any browser of the last five years I don’t care too much. When XMLHttpRequest is not supported, I suppose I can’t rely on other useful stuff neither. The synchronous requests also mean that there might be a slight performance hit. But the advantages of synchronous includes outweigh this by far. Finally, XMLHttpRequest doesn’t provide the means to recognize when a synchronous request is stalled. This is not of too much concern, since a stalled request usually means that an include files can’t be downloaded and so the JavaScript application is not usable anyways. It might be a concern for sensibly degrading, though.

Update: Here is the code to the updated module. Using it is now as simple as this:

<script src="include-http.js"></script>
<script>
    include.include("my.module");
    myfunc(...);
</script>
Categories
Javascript

Porting JavaScript to Internet Explorer

While porting a small AJAX application I wrote to Internet Explorer, I encountered a few problems:

  1. IE doesn’t like <script> tags without a matching closing tag. This is especially a problem if you use the src attribute and try to use an XHTML-like closing tag like this: 
    <script src="..."/>

    In this case IE doesn’t draw anything at all, since it keeps looking for the end of the script and doesn’t find it. This is especially a problem with templating toolskits like Kid. If Kid encounters a construct like <foo></foo> it will correctly shrink this to <foo/> in the XML output. The solution is rather easy: Insert a newline character between openening and closing tag in the template. Since Kid has now way to determine whether this whitespace is significant it will keep it in the output.

  2. It seems that IE doesn’t like trailing commas in JavaScript object literals: 
    var o = {
    x: 'Foo',
    y: 1,
};

    I like to insert trailing commas in cases like this. It’s easy to forget to add one if extending the property list. Firefox parses the trailing comma, while IE doesn’t. (IE follows the ECMAScript standard here, which doesn’t allow the trailing comma. Indeed in array literals a trailing comma has a special meaning, so it’s probably a good idea to disallow the trailing comma in object literals.)

  3. My AJAX application queries a web service for a list of items and displays that list in a table. The table is generated in HTML code like this: 
    <table id="mytable">
    <tr>...</tr>
</table>

    I then used JavaScript code like the following to populate that table:

    var table = document.getElementById("mytable");
var trTag = createRow();
table.appendChild(trTag);

    This worked fine in Firefox, but no matter what I did, IE refused to render the table rows. After much hair tearing I remembered that Firefox’s DOM Inspector showed a TBODY element just beneath the TABLE element and TR elements are attached to the former. The DOM Inspector basically showed the following structure:

    • TABLE id=mytable
      • TBODY
        • TR (my header row from the HTML)
      • TR (a dynamically generated row)
      • …

    So I changed my code to append the newly generated rows to the TBODY instead:

    var table = document.getElementById("mytable").
    getElementsByTagName("tbody")[0];
var trTag = createRow();
table.appendChild(trTag);

    This works in Firefox and IE.