Reversing Quixant Linux drivers

I work with a company that develops casual games for the Internet but a recent project is moving us into foreign ground. We want to build physical slot machines using hardware from TommyBear and Quixant.

TommyBear assembled the machinery and provides the casing while the internals were delivered by Quixant. Quixant also delivered drivers and API's for working with the hardware but we only got binaries.

 

Now, I have a passion for programming and for computer security and two of my goals in life is to code for the Linux kernel and do some reverse engineering, so this could look like a perfect oportunity to try out both and the following indicates that I will not have any legal problems in doing so:

$ modinfo qxtio.ko qxtmem.ko|grep license
license:        GPL
license:        GPL

The qxtio driver drives the coin collector, coin hopper, buttons and similar hardware while qxtmem drives a two megabyte SRAM chip.

Goal

Now, if the drivers are licensed under the GPL why do I not simply ask for the source ?

Well, my primary objective is the actual reversing and recoding, so getting the code from Quixant will not be satisfactory.

I do have secondary goals too, and that is to be able to build the drivers for different kernels myself, and to be able to build a virtual driver that simply emulates the hardware. That way I can develop using the Quixant API's and testing on any machine without having the actual hardware. But these are secondary goals.

Tools

For this task I will be using the following set of tools:

  • IDA Pro for the disassembling
  • VirtualBox for running Windows and a Linux box for testing.
  • gcc for building the module
  • An actual TommyBear and Quixant platform for tracing the real drivers and to see if they match my own.

Findings

The calling convention in the Linux kernel is similar to the Borland fastcall. First three arguments to functions get passed through EAX, EDX and ECX respectively and any extra arguments are put on the stack (leftmost on the top), the return value is put in EAX and the caller cleans up.

Also, ESI, EBX and EDI should maintain their values through function calls (maybe other registers as well).

I found an interesting issue. I had a function which took three arguments, but the assembly referenced the last one from the stack. It turned out that the second argument had type loff_t which is 64 bits. So apparently it is split between two registers.

The same function returned type loff_t which is returned in EAX and EDX. The least significant dword is in EAX and most significant dword is in EDX.