Install AOSP build to PandaBoard


This post continues the post on building AOSP for PandaBoard. Read that post, if you haven’t already. This post covers installing the AOSP images to PandaBoard. It is much more adventuresome that I initially imagined it would be. The procedure described here, with some variations, is based on instructions in the device/ti/panda/README file.

usbboot

PandaBoard’s usbboot boot loader runs when no SD card is provided. In this mode, the board can receive the next stage boot loader over the USB bus, using the usbboot utility located under device/ti/panda. Unfortunately, VirtualBox Ubuntu VM is able to use the PandaBoard only after Windows host has loaded a valid driver for it. I modified Google’s USB driver to work with PandaBoard’s usbboot mode.

I am using Windows 8 as the host OS. It will not install unsigned drivers by default. I test signed the driver using Windows Driver Kit. Once the driver is installed, you’ll need to stop the VM, and edit its settings to use device “PandaBoard UsbBoot Interface”. You’ll know VirtualBox is using the USB device when lsusb lists it, and it does not appear in Windows Device Manager on the host. The same driver also works with Android’s fastboot mode. Here’s what lsusb -v dumps when PandaBoard is in usbboot mode.

Bus 001 Device 012: ID 0451:d00f Texas Instruments, Inc.
Couldn't open device, some information will be missing
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.10
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  idVendor           0x0451 Texas Instruments, Inc.
  idProduct          0xd00f
  bcdDevice            0.00
  iManufacturer          33
  iProduct               37
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          1
    bmAttributes         0xc0
      Self Powered
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    255 Vendor Specific Subclass
      bInterfaceProtocol    255 Vendor Specific Protocol
      iInterface              2
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0

Finally, here’s the command you need to run so that usbboot puts PandaBoard in fastboot mode.

sudo ./device/ti/panda/usbboot device/ti/panda/bootloader.bin

fastboot

Android’s fastboot command line utility can flash Android to a device when it is in fastboot mode. Build of fastboot from source under out/host/linux-x86/bin didn’t work, until I installed the Oracle VM VirtualBox Extension Pack and enabled USB 2.0 support. If you want to avoid installing that extension pack, Android SDK for Windows installs fastboot in the platform-tools folder. You can use that instead of using the one built from source.

You’ll need to copy all image files built by make under out/target/product/panda to the Windows host and run the following commands from the command prompt. Change ANDROID_SDK_PATH appropriately.

set ANDROID_SDK_PATH=C:\Users\user\AppData\Local\Android\android-sdk
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe oem format
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe flash xloader xloader.bin
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe flash bootloader bootloader.bin
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe erase cache
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe -p panda flash userdata userdata.img
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe -p panda flash boot boot.img
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe -p panda flash system system.img
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe -p panda flash recovery recovery.img
%ANDROID_SDK_PATH%\platform-tools\fastboot.exe -p panda flash cache cache.img

The more convenient flashall option of fastboot fails as follows. Hence, the need to flash image files individually.

%ANDROID_SDK_PATH%\platform-tools\fastboot.exe -p panda flashall
error: could not load android-info.txt: No error

At this point you can reset the PandaBoard so that it loads Android!

To boot into fastboot mode ever again, hold the GPIO_121 button (furthest button from SD card) and press and release the PWRON_RESET button. Release the GPIO_121 button after a while.

Proprietary binaries for PowerVR

Android failed to load on PandaBoard after my first build. The serial port console printed the following message.

[  297.078613] PVR_K:(Error): BridgedDispatchKM: Driver initialisation not completed yet. [4836, drivers/gpu/pvr/bridged_pvr_bridge.c]

I discovered the version of the PowerVR binary required by my Android branch is as follows.

shell@android:/ $ cat /proc/pvr/version
Version CustomerGoogle_Android_ogles1_ogles2_GPL sgxddk 18 1.8@785978 (release) omap4430_android
System Version String: None

shell@android:/ $ ls -l system/vendor/lib/egl
-rw-r--r-- root     root         4812 2013-05-24 17:25 libEGL_POWERVR_SGX540_120.so
-rw-r--r-- root     root       445892 2013-05-24 17:25 libGLESv1_CM_POWERVR_SGX540_120.so
-rw-r--r-- root     root       371532 2013-05-24 17:25 libGLESv2_POWERVR_SGX540_120.so

The binary I had provided pre build was a different version.

$ strings vendor/imgtec/panda/proprietary/libGLESv2_POWERVR_SGX540_120.so | grep build
OpenGL ES 2.0 build 1.8@905891
OpenGL ES GLSL ES 1.00 build 1.8@905891

The solution was to clobber and build again with the correct version of the proprietary driver.

Does it work?

This Android build is painfully slow. adb shell dmesg shows repeating omapfb message below.

%ANDROID_SDK_PATH%\platform-tools\adb.exe shell dmesg
...
[   10.060485] omapfb omapfb: Unknown ioctl 0x40044f40

The CPU itself is relatively idle as reported by monitor and shown in the figure below. Changing display size using adb am display-size or display density using adb am display-density, has no noticeable impact.

monitor

logcat shows the following messages repeating very often.

01-02 15:22:48.375: I/PowerManagerService(346): Going to sleep due to screen timeout...
01-02 15:22:48.382: E/SurfaceFlinger(99): eventControl(0, 1) failed Invalid argument
01-02 15:22:49.390: W/SurfaceFlinger(99): Timed out waiting for hw vsync; faking it
01-02 15:22:53.375: I/PowerManagerService(346): Waking up from sleep...

All that leads me to believe graphics are hardware accelerated but buggy. If and when I figure this out I’ll post an update.

Here’s the Android screen captured using monitor on Windows.

Android 4.2.2

Enjoy!

Free with subscription bought over the internet model


Apps that naturally depend on the Internet can use a model I term as “free with subscription” or freesub. In this model, the app itself is free to download with some features available from the outset. Other features are enabled based on subscriptions or credits purchased over the Internet. An example that comes to mind is the Skype app. The landline calling feature is only available when you are a subscriber, or purchase additional credit using the desktop app or the Internet browser.

This may be one means of circumventing the App/Play/Windows Store cut. I wonder if the stores have specific terms that prohibit this. I am aware that the apps themselves cannot accept payments directly.

Thoughts?

IP fowarding


Normally, commercial operating systems do not act as routers, although they posses the ability to.

It is fairly easy to enable it and there are instructions all over the Internet to do so for Windows and Linux.

You’ll also need to add appropriate route settings to the routing table, so that the packets destined to a network are properly routed through a network interface on that network. This can be done using the route add command on Windows and Linux.

I have had problems routing multicast packets, from IP addresses 224.0.0.0 through 239.255.255.255 (formerly referred as Class D addresses). Windows does not route multicast traffic by default. Using EnableMulticastForwarding does not automatically enable it either.

Dealing with segmented data in a Wireshark dissector written in Lua


Protocols based on stream-oriented transport protocols like TCP may get segmented i.e. the PDU boundaries may not be preserved. If you are not familiar with writing dissectors for Wireshark in Lua, read Create a Wireshark dissector in Lua.

Protocols based on TCP

You can simply let the TCP dissector reassemble segments by telling it how much data the dissector function still needs.

Adding the following logic to the dissector function does the trick. Reading the PDU length field is protocol specific, change it appropriately.

    local i = 0
    repeat
        if buf:len() - i >= 2 then    -- change length field size appropriately
            -- we have length field
            i = i + buf(i,2):uint() -- change appropriately
            if i > buf:len() then
                -- we don't have all the data we need yet
                pkt.desegment_len = i - buf:len() 
                return
            end
        else
            -- we don't have all of length field yet
            pkt.desegment_len = DESEGMENT_ONE_MORE_SEGMENT 
            return
        end
    until i >= buf:len()
    
    -- rest of the dissector function remains the same,
    -- but use a repeat / until loop to read all PDUs

The TCP dissector calls the dissector function with progressively larger chunks of data taken by aggregating data from the following TCP segments. Once data has been aggregated, your dissector function will proceed as usual.

Protocols not based on TCP

If your protocol is not based on TCP, you will have to handle segmentation on your own. It bears to keep in mind that Wireshark will call the dissector function for each packet, from first to the last, when you first open a capture file. This knowledge can be used to preprocess each packet, and save some state information such as whether it is segmented or not. If it is segmented, its raw data can be appended to a global byte array. This process will repeat until the whole message is ascertained to have been read. A new Tvb can then be created and used for dissection. The final byte array is saved in state of the last packet in sequence.

    local state = pktState[pkt.number]

    if state ~= nil then
        -- we've already processed this packet
        if state.complete == true then
            pkt.info = "Command [complete]"
            buf = ByteArray.tvb(state.buffer, "Complete Command")
        else
            pkt.info = "Command [incomplete]"
            return -- nothing to do
        end
    else
        -- first time here, capture file has just been opened?
        state = {}
        if partialBuffer == nil then
            partialBuffer = buf(0):bytes()
        else
            partialBuffer:append(buf(0):bytes())
            buf = ByteArray.tvb(partialBuffer, "Command") -- create new tvb for packet
        end
        local i = 0
        repeat
            if buf:len() - i >= 2 then -- change length field size appropriately
                -- we have length field
                i = i + buf(i,2):uint() -- change appropriately
                if i > buf:len() then
                    -- we don't have all the data we need yet
                    state.complete = false 
                    pktState[pkt.number] = state
                    return
                end
            else
                -- we don't have all of length field yet
                state.complete = false 
                pktState[pkt.number] = state
                return
            end
        until i >= buf:len()
        state.complete = true
        state.buffer = partialBuffer
        pktState[pkt.number] = state
        partialBuffer = nil
    end

    -- perform dissection of buf

Remember to initialize pktState to an empty table i.e. {} in the init function. partialBuffer is a file local variable used within the dissector function.

Building Android Open Source Project for the PandaBoard


If you ever decide to build AOSP on a Virtual Machine you’ll need tonnes of patience. This post describes how to build the AOSP on a Ubuntu 13.04 VM, for the PandaBoard.

Preparing the VM

I created a VirtualBox (4.2.12) VM with a 100 GB virtual drive on a laptop that has an Intel Core i5 dual core CPU with four logical processors. I configured the VM to use all four logical processors. The amount of physical memory available to the guest OS was configured to four GiB, but you may be able to do with less if you use make with less parallel jobs. The host laptop has 8 GB of DDR3 SDRAM. I then installed Ubuntu 13.04 64-bit version from an ISO file.

Prepare Ubuntu and download AOSP source code

This is a lengthy and bandwidth-intensive procedure. I don’t want to describe it all since it’s well documented elsewhere. I chose to build the android-4.2.2_r1 branch. You may want to add the -j 20 option to repo sync if you have lots of available bandwidth. You’ll need a big virtual drive. du -h shows that I have used up 13 GB after downloading the source! You’ll obviously need much more than that once you are done building, the same folder swelled up to about 35 GB in my case.

Building AOSP

Again, this is well documented by the official team. For my build, I chose the target as full_panda-eng, since I want to run Android on a PandaBoard. If you restart your VM, you’ll need to repeat the entire procedure described in the documentation, right upto the make command. If you see the build being killed for no apparent reason, it could be because you have allocated too little memory to the VM. You can also try invoking make with less parallel jobs by tweaking the -j option e.g. -j2.

You’ll need to download and extract some proprietary binaries for PandaBoard before starting the build, basically just the graphics driver. Further instructions can be found in file device/ti/panda/README under the source tree. If you add the binaries after the build, remember to call make clobber to clean up existing output, and then start the build again. The build takes approximately four hours with four simultaneous jobs.

Stay posted for a follow-up post on flashing Android to the PandaBoard.

Incremental upgrade of an embedded relational database


Several years ago I worked on a project that required incrementally upgrading an application and its embedded relational database. What follows is an overview of some best practices we used. Several open source projects, books and articles use or cite similar practices.

A full create script and incremental upgrade scripts

It is vital that you have a full create script for fresh installs, and an incremental upgrade script per application version. The create script has CREATE SQL statements mostly. The upgrade script mostly has ALTER statements, for the changes required from a version of the application to the next.

Maintain your scripts under version control

This ensures that you always know the changes that have been made, can rollback changes, and easily identify changes to create incremental update scripts.

Traceability between database scripts and application code

You should use the same label to tag the database scripts and the application code in the version control system, so you know which script versions match an application release.

Installation

The installer should upgrade the database from an older version of the application to the newest. You’ll need to have a procedure in place, a batch script for instance, that knows all application versions and can apply corresponding upgrade scripts in sequence.

Tips for Portuguese speakers writing English


I have seen Portuguese speakers repeat the same mistakes very often when writing English. This post tries to document these mistakes with links to worthwhile references.

Different of / Different from / Different than

This mistake usually happens when you literally translate diferente deDifferent of is wrong, use different from or different than.

Once / Since

This mistake usually happens when you literally translate uma vez in a phrase. If you mean because, use since. If you mean after, use once.

Must to / Have to

This mistake usually happens when you literally translate tem que in a phrase. Must to is incorrect, use have to or just must.