A brief status update

Got a new device last week!

The device and peripherals.
Closeup of the device.

The device has an i.MX283 processor and a touchscreen attached to the other side. It booted from the image that came installed on the SD card:

It even has Doom installed on it! Which, I found, is actually easy to install with Buildroot. Who knew video games on embedded devices were so popular?

Since I’m going to be working on a kernel module (mxs-lradc) for i.MX28, I have to put a new kernel on the device. The device came with a manual that described a way to do that, which is really nice, since I have no experience with embedded devices.

The SD card the device boots from is formatted with three partitions. The first has some initial i.MX23/28 boot code on it (“bootlets”), the second has the bootloader (they use Barebox), and the third has the root filesystem, which also contains the kernel image (and device tree binaries) in the /boot directory. From what I understand, the bootlets run first (after the firmware) and start the bootloader, which in turn starts the kernel.

Updating all of this should be pretty easy using Buildroot. They provide a Github repository with a Buildroot configuration file for the device. By downloading it and just running make crystalfontz_cfa10036_defconfig and make, Buildroot will download and build all the necessary parts for the system. This includes things like the cross-compilation toolchain, the bootlets, Barebox and its configuration, and their version of the kernel. The locations of all of these are specified in the configuration file.

The result is two images and a compressed root filesystem, which can be placed on the three partitions. The manufacturer even provided a handy Python script that formats the SD card and puts the images on it.

Trouble is, after I did all this, the device wouldn’t boot. It just hung on a blank screen. Good thing I backed up the original image (with dd).

My initial reaction was to try to get some output from the system, using the debug UART on the processor (on the image above, the four golden headers (holes) to the right of the red plate). I spent over two days on it, so far unsuccessfully (more on this below).

At some point, however, I noticed that the working system was running kernel version 3.12.17, while the kernel being built was v3.15-rc5, and the configuration file used was generated for v3.9 (presumably the new options were set to the defaults). So I thought I’d build v3.12.17 instead. I got the configuration for it from /proc/config.gz on the working system, and modified Buildroot to build it.

I once again placed the images and rootfs on the SD card, inserted it in the device, and… it didn’t boot. Nothing on the screen again.

Eventually I came up with the idea of just taking the working image, mounting the rootfs on it on my computer, and simply replacing the zImage file in the /boot directory with the one I built. And that worked! The device booted again with no problems.

Next I tried the same with v3.13. It booted, but the touchscreen didn’t work. Well, okay. Then I tried 3.14. Blank screen again, didn’t boot. sigh

Haven’t tried to find the reason yet, but it’s a start. At least now I have a way to boot my own kernel.

But I’d also like to figure out what’s wrong with the bootlets and/or Barebox images I put on the card, and why they don’t boot. I can see from the bootlet code that it prints debug information out to the debug UART on the board. So it would be great if I could read from the UART.

Furthermore, the kernel also boots with the command line parameter console=/dev/ttyAMA0, which is the debug UART. That means system messages are printed to it, and reading it could also help me find out what’s wrong with v3.14. (Though I could probably also override the console parameter by finding where the command line is parsed in the kernel.)

I tried to read from the UART using the serial to USB converter that came with the device. It connects the UART to my computer’s USB port. I should be able to read from it using the ftdi-sio driver, by issuing “normal” serial commands to /dev/USB0. I tried it with both Minicom and a small program using termios (example code), but so far no luck. I’m just getting garbage:

Minicom output.

I got the serial communication parameters from sysfs and the tcgetattr() system call on the device (with a working kernel). Not sure if that’s the right way to do it. And I can see the AMBA PL011 driver running on the device (or at least it says so in the system log during bootup). And yet nothing I write to /dev/console on the device is arriving on my computer, or I’m not handling it right. I must be doing something wrong. There are also weird SysRq “help” messages in the system log on the device, which implies that the UART is getting something from my end or the converter.

I’ve spent several days messing with this thing, and while I’d really like to get it to work, I’ll probably leave it alone for the time being. It’s not really what I’m supposed to be working on (the mxs-lradc driver is), and I’d like to get something done by the end of the internship, so I’ll probably do some other things next, like delete some drivers, and do some IIO subsystem driver cleanup that doesn’t require any hardware.

Which reminds me, this isn’t all I’ve worked on during the past couple of weeks. I also removed one more driver, and talked to some driver “owners” about their drivers, either on the mailing lists or some of them emailed me personally. I never got a response about 7 drivers, and was going to ask the maintainers (or original authors) about them again, but Greg thought I should just delete them, so I’ll do that instead.

In addition, I spent a lot of time learning about the IIO subsystem, by reading the documentation and the code of the IIO “core”, a “dummy” driver, and userspace examples.

I also subscribed to a bunch more kernel mailing lists related to the device and the mxs-lradc driver (input, pm (power management), ARM, and devicetree). It’s interesting since I get to see what kind of work goes on in the kernel. Even though 99% of the time I don’t understand what they’re talking about… And the volume is pretty high, so I can’t read everything anyway, but I at least try to scan the subject lines, and read the email only if it seems interesting.

It’s also nice to see people making mistakes :) Makes me worry less about messing up myself, since I see it’s usually not a big deal. Oh, and I also found that the LKML archives go back to 1996, which is pretty cool (and humbling).

I’ve also found myself in various “detours” reading code from parts of the kernel not strictly related to what I’m working on, like the module infrastructure, device trees, how devices and drivers are initialized, or how the kernel starts running. I should probably avoid going off track like this.

To finish up, I’d say it’s been a fun couple of weeks. Despite a few frustrating days with the UART port, I’m still very happy to have an actual device to work with and test my changes on, and I feel like I’ve learned a lot about Linux and embedded devices, which is nice.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s