Arduino USB Serial firmware from scratch


I have need to change the USB descriptors of the Arduino Uno R3, for a project that will have it acting as a peripheral to another device.

This post documents how you can make from scratch the USB Serial firmware for ATmega16U2 on the Arduino Uno R3, using Atmel Studio 7 and its LUFA Library extension. It can be flashed quite easily using the built-in DFU firmware and Atmel’s FLIP tool. Source code is available at GitHub.

Install LUFA Library extension if not already installed (Tools -> Extensions and Updates…).

lufa-install

Source files that perform the actual USB/serial proxying are reused from USB to Serial Converter example project for at90usb1287 (File -> New -> Example Project…). Go ahead and create that project and grab the files. Some additions to the USB/serial code such as allowing a sketch to be uploaded from the Arduino IDE, have been taken from the official source code at GitHub.

usb-serial-lufa.png

Create a new project for the ATmega16U2 (File -> New -> Project…).

usb-serial-project.png

usb-serial-project-board.png

Use ASF Wizard (ASF -> ASF Wizard) to add LUFA modules and remove the Generic board support (driver) module.

usb-serial-asf-wizard.png

Configure GCC symbols (Project -> Properties). Change BOARD to BOARD_UNO, and add F_CPU=16000000UL, and F_USB=16000000UL.

usb-serial-symbols.png

Resolve build errors by mostly eliminating code. Source code that builds without errors is available for comparison at GitHub. Program (Tools -> Device Programming) the firmware using the FLIP tool.

usb-serial-flip

Reboot the Arduino.

Arduino USB DFU firmware from scratch


The Device Firmware Upgrade (DFU) firmware for the ATmega16U2 on Arduino UNO R3 is used to flash the USB Serial firmware, among others, using Atmel’s FLIP tool.

This post documents how you can build from scratch DFU firmware using Atmel Studio 7 and its LUFA Library extension. You’ll need an ISP/ICSP programmer to program the ATmega16U2.

You can install the LUFA extension from the Extensions and Updates dialog (Tools -> Extensions and Updates).

lufa-install.png

Create a new project based on the DFU Bootloader example (File -> New -> Example Project…).

dfu-lufa-example.png

Change the compiler optimization setting to -Os in toolchain properties (Project -> Properties).

dfu-optimize-for-size.png

Build the solution and flash (Tools -> Device Programming) using an ISP/ICSP programmer connected to the ICSP2 header.

dfu-jtagice3.png

 

Integer division and timer resolution


I have this very specific need to send a list of pre-timed messages from an embedded system. Each message has a specific time when it needs to be sent out. The time is specified in milliseconds (ms) with respect to the previous message in the list. The first message being at time zero.

The algorithm to send messages out is very simple. I send out the first message, and arm a timer with the time of the second message. As soon as the timer expires, I send out the second message, and rearm the timer to send the third message, and so on. Thus, it is pretty clear that timer resolution is very important. Since the smallest integer time interval I require is 1 ms, a timer resolution of 1 ms or less would be ideal.

The embedded system I am dealing with, has a timer resolution of 2.5 ms i.e. 1 tick of the timer is 2.5 ms long. The timer routine does not accept a time of 0, 1 tick is the smallest integer value the routine expects. To convert time in milliseconds to ticks, I need a routine that can convert milliseconds to ticks. The conversion routine, putting it simply, would receive time in milliseconds, divide it by 2.5, and round the result to an integer value. I don’t have access to floating point math though, so the routine I’ve developed looks like this

typedef unsigned int uint32_t;
const unsigned short MULTIPLY_BY = 2;
const unsigned short DIVIDE_BY = 5; // never ever set to zero

uint32_t MillisToTicks(uint32_t timeInMillis, int *remainder)
{
    uint32_t timesTwo;
    uint32_t result;

    timesTwo = timeInMillis * MULTIPLY_BY;
    result =  timesTwo / DIVIDE_BY;
    *remainder += timesTwo % DIVIDE_BY;
    if (*remainder >= DIVIDE_BY)
    {
        result += 1;
        *remainder -= DIVIDE_BY;
    }
    else if(*remainder <= -1 * DIVIDE_BY && result > 1)
    {
        result -= 1;
        *remainder += DIVIDE_BY;
    }
    else if (result == 0)
    {
        result = 1;
        *remainder -= DIVIDE_BY - result;
    }
    return result;
}

MillisToTicks never returns zero. Dividing by 2.5 is the same as multiplying by 2 and dividing by 5. 2 is read from a constant called MULTIPLY_BY and 5 from another constant called DIVIDE_BY. Since the integer division will produce a remainder, MillisToTicks requires a pointer to integer where it can store the remainder of the integer division. That remainder is updated and checked during each call. If it exceeds DIVIDE_BY, the result is incremented by one, and remainder decremented by DIVIDE_BY.

If the result of integer division is zero, I force it to a value of 1, and decrement remainder by DIVIDE_BY. Next time MillisToTicks gets called, if remainder is less than or equal to -DIVIDE_BY, I decrement result by 1, and increment remainder by DIVIDE_BY. Putting it simply, there will be moments when I’ll stray away from timing the messages perfectly, but given enough messages, I’ll stray back on track. To test that, I have implemented the following code

int remainderMillisToTicks = 0;

uint32_t times1[] = {6, 6, 6, 2, 6, 6, 2, 2, 6, 6, 7, 8, 1, 1, 2, 3, 6, 20, 30, 9, 30, 100, 3000, 1, 1, 8000, 10000, 23, 1, 1, 19, 6, 5, 26, 201, 503, 901};

uint32_t times2[] = {6, 6, 6, 2, 6, 6, 2, 2, 6, 6, 7, 8, 503, 901, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 6, 20, 30, 9, 30, 100, 3000, 1, 1, 8000, 10000, 23, 1, 1, 19, 6, 5, 26, 201, 503, 901};

uint32_t times3[] = {6, 6, 6, 2, 6, 6, 2, 2, 6, 6, 7, 8, 1, 1, 2, 3, 6, 20, 30, 9, 30, 100, 3000, 8000, 10000, 23, 19, 6, 5, 26, 201, 503, 901, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1};

void test(uint32_t *times, unsigned short size)
{
    int i;
    uint32_t totalMillis = 0;
    uint32_t ticks = 0;
    double ticksToMillis = 0;
    double totalTicksToMillis = 0;

    for (i = 0; i < size; i++)
    {
        totalMillis += times[i];
        ticks = MillisToTicks(times[i], &remainderMillisToTicks);
        ticksToMillis = ((double)ticks*DIVIDE_BY) / MULTIPLY_BY;
        totalTicksToMillis += ticksToMillis;
        //printf("%d\t%f\t%d\t%d\t%f\n", times[i], ticksToMillis, remainderMillisToTicks, totalMillis, totalTicksToMillis);
    }
    printf("Ideal total time required: %d, time achieved: %f\n", totalMillis, totalTicksToMillis);
}

int main()
{
    test(times1, sizeof(times1) / sizeof(uint32_t));
    remainderMillisToTicks = 0;
    test(times2, sizeof(times2) / sizeof(uint32_t));
    remainderMillisToTicks = 0;
    test(times3, sizeof(times3) / sizeof(uint32_t));
}

Here’s how the output of that test code looks like

Ideal total time required: 22953, time achieved: 22952.500000
Ideal total time required: 24363, time achieved: 24362.500000
Ideal total time required: 22965, time achieved: 22985.000000

Enable the commented printf statement in test and you’ll see a call by call log. At times, when the time value requested is too small, you’ll see the messages stray away from perfect timing, and stray back to a normal cadence later. The overall time is thus unaffected or affected only slightly. The last call to test represents a failure condition that can only be avoided by using a higher resolution timer.

21st Century C by Ben Klemens; O’Reilly Media


21st Century C

21st Century C is a useful read for the budding Linux hacker, especially due to its focus on the GNU toolchain. Among its highlights is the constant reminder of improvements to the C language since the original ANSI C standard (C89). It covers a wide range of topics, you may want to skim particular topics and return to them when you see the need.

The book is divided into two parts. Part 1 starts with mundane things like installing a C compiler and compiling your C program. It then moves on to useful topics such as debugging using GDB, checking for errors using Valgrind, unit testing, building documentation using Doxygen and CWEB, and packaging your project using Autotools. If you have executed “./configure” and then “make install”, but never looked under the hood, now is the time to do that. The chapter on version control using Git is probably the best coverage of Git I have seen in any book.

Part 2 focuses on the C language itself. This part starts with coverage of pointers, followed by a chapter on C syntax that programmers should avoid. The following chapters discuss usage of macros to write succinct code, text handling, and better structs and typedefs. The chapter on object oriented programming delves into using anonymous structs for extending objects, function pointers, and building and freeing objects. The last chapter wraps up by introducing several popular open source libraries such as GLib, POSIX, GSL, SQLite, libxml, and cURL.

You can cobble together the knowledge dispensed in this book by reading manuals and articles over the internet, but it is convenient to have it all together in one place to be read at your leisure.

Operator precedence


Programmers should use parentheses more often. The following C language snippets are mind-blowing examples of when operator precedence is not obvious and can bite you.

x &amp; 0x01 == 0&lt;/code&gt;

== is evaluated first and then &.

a + b ^ c + d
  • is evaluated first and then ^.

MISRA C and CERT Secure Coding site actually have rules against depending on operator precedence, for instance EXP00-C.

Access USB device on Mac OS X using I/O Kit


This post shows how you can access a USB device from user space on Mac OS X. I used Xcode 4.3.2 on Mac OS X Lion (10.7.3) for testing the code.

Apple provides the USB Device Interface Guide that shows how USB devices can be accessed from user space. The book OS X and iOS Kernel Programming is also very helpful, see Chapter 15 – User-Space USB Drivers.

Create an Xcode console application. Add the following libraries to the default target.

  • IOKit.framework
  • CoreFoundation.framework

Xcode will have created main.c, add the following headers.

#include <stdio.h>
#include <CoreFoundation/CoreFoundation.h>
#include <IOKit/IOKitLib.h>
#include <IOKit/IOCFPlugIn.h>
#include <IOKit/usb/IOUSBLib.h>
#include <IOKit/usb/USBSpec.h>

Let’s now implement the main function. Add the following variable declarations.

    CFMutableDictionaryRef matchingDictionary = NULL;
    SInt32 idVendor = 0x0000; // set vendor id
    SInt32 idProduct = 0x0000; // set product id
    io_iterator_t iterator = 0;
    io_service_t usbRef;
    SInt32 score;
    IOCFPlugInInterface** plugin;
    IOUSBDeviceInterface300** usbDevice = NULL;
    IOReturn ret;
    IOUSBConfigurationDescriptorPtr config;
    IOUSBFindInterfaceRequest interfaceRequest;
    IOUSBInterfaceInterface300** usbInterface;
    char out[] = { 0x00, 0x00 }; // set data to send
    char* in;
    UInt32 numBytes;

We now try to find the USB device using the vendor and product id.

    matchingDictionary = IOServiceMatching(kIOUSBDeviceClassName);
    CFDictionaryAddValue(matchingDictionary,
                         CFSTR(kUSBVendorID),
                         CFNumberCreate(kCFAllocatorDefault,
                                        kCFNumberSInt32Type, &idVendor));
    CFDictionaryAddValue(matchingDictionary,
                         CFSTR(kUSBProductID),
                         CFNumberCreate(kCFAllocatorDefault,
                                        kCFNumberSInt32Type, &idProduct));
    IOServiceGetMatchingServices(kIOMasterPortDefault,
                                 matchingDictionary, &iterator);
    usbRef = IOIteratorNext(iterator);
    if (usbRef == 0)
    {
        printf("Device not found\n");
        return -1;
    }
    IOObjectRelease(iterator);
    IOCreatePlugInInterfaceForService(usbRef, kIOUSBDeviceUserClientTypeID,
                                      kIOCFPlugInInterfaceID, &plugin, &score);
    IOObjectRelease(usbRef);
    (*plugin)->QueryInterface(plugin,
                              CFUUIDGetUUIDBytes(kIOUSBDeviceInterfaceID300),
                              (LPVOID)&usbDevice);
    (*plugin)->Release(plugin);

Now that we have found the device, we open the device and set the first configuration as active.

    ret = (*usbDevice)->USBDeviceOpen(usbDevice);
    if (ret == kIOReturnSuccess)
    {
        // set first configuration as active
        ret = (*usbDevice)->GetConfigurationDescriptorPtr(usbDevice, 0, &config);
        if (ret != kIOReturnSuccess)
        {
            printf("Could not set active configuration (error: %x)\n", ret);
            return -1;
        }
        (*usbDevice)->SetConfiguration(usbDevice, config->bConfigurationValue);
    }
    else if (ret == kIOReturnExclusiveAccess)
    {
        // this is not a problem as we can still do some things
    }
    else
    {
        printf("Could not open device (error: %x)\n", ret);
        return -1;
    }

Having done that, we need to find the interface we want to use for sending and receiving data. Our device is a USB RNDIS device that has two bulk data endpoints on the second interface (#1).

    interfaceRequest.bInterfaceClass = kIOUSBFindInterfaceDontCare;
    interfaceRequest.bInterfaceSubClass = kIOUSBFindInterfaceDontCare;
    interfaceRequest.bInterfaceProtocol = kIOUSBFindInterfaceDontCare;
    interfaceRequest.bAlternateSetting = kIOUSBFindInterfaceDontCare;
    (*usbDevice)->CreateInterfaceIterator(usbDevice,
                                          &interfaceRequest, &iterator);
    IOIteratorNext(iterator); // skip interface #0
    usbRef = IOIteratorNext(iterator);
    IOObjectRelease(iterator);
    IOCreatePlugInInterfaceForService(usbRef,
                                      kIOUSBInterfaceUserClientTypeID,
                                      kIOCFPlugInInterfaceID, &plugin, &score);
    IOObjectRelease(usbRef);
    (*plugin)->QueryInterface(plugin,
                              CFUUIDGetUUIDBytes(kIOUSBInterfaceInterfaceID300),
                              (LPVOID)&usbInterface);
    (*plugin)->Release(plugin);

Now that we have found the interface, let us open it.

    ret = (*usbInterface)->USBInterfaceOpen(usbInterface);
    if (ret != kIOReturnSuccess)
    {
        printf("Could not open interface (error: %x)\n", ret);
        return -1;
    }

Finally, we can send and receive some data. The pipe references are in the same order as the end points that appear in the Bus Probe tab of USB Prober. In our case, pipe 0 is the default control endpoint (does not have an endpoint descriptor and will not appear in USB Prober), pipe 1 is the bulk data output endpoint, and pipe 2 is the bulk data input endpoint.

For some reason, if we read less than 64 bytes I/O Kit returns a kIOReturnOverrun error. Coincidentally, the max packet size of the bulk input pipe of the device is also 64 bytes.

    // Send data through pipe 1
    (*usbInterface)->WritePipe(usbInterface, 1, out, sizeof(out));

    // Read data through pipe 2
    numBytes = 64;
    in = malloc(numBytes);
    ret = (*usbInterface)->ReadPipe(usbInterface, 2, in, &numBytes);
    if (ret == kIOReturnSuccess)
    {
        printf("Read %d bytes\n", numBytes);
    }
    else
    {
        printf("Read failed (error: %x)\n", ret);
    }

To wrap it all, we close the interface and device, and return from main.

    (*usbInterface)->USBInterfaceClose(usbInterface);
    (*usbDevice)->USBDeviceClose(usbDevice);

    return 0;

Found an issue? Leave a comment below.

C language style guides


I don’t have a particular style that I like, I do have a style that I use, frequently without being aware of it. Often, the style of the surrounding code determines my style. I tend to use as references links returned by Google for “c style guide”. I have found the following useful:

I don’t like using hungarian notation for primitive types, I prefer descriptive names and grep, or a good IDE.