Arduino USB Serial firmware from scratch


I have need to change the USB descriptors of the Arduino Uno R3, for a project that will have it acting as a peripheral to another device.

This post documents how you can make from scratch the USB Serial firmware for ATmega16U2 on the Arduino Uno R3, using Atmel Studio 7 and its LUFA Library extension. It can be flashed quite easily using the built-in DFU firmware and Atmel’s FLIP tool. Source code is available at GitHub.

Install LUFA Library extension if not already installed (Tools -> Extensions and Updates…).

lufa-install

Source files that perform the actual USB/serial proxying are reused from USB to Serial Converter example project for at90usb1287 (File -> New -> Example Project…). Go ahead and create that project and grab the files. Some additions to the USB/serial code such as allowing a sketch to be uploaded from the Arduino IDE, have been taken from the official source code at GitHub.

usb-serial-lufa.png

Create a new project for the ATmega16U2 (File -> New -> Project…).

usb-serial-project.png

usb-serial-project-board.png

Use ASF Wizard (ASF -> ASF Wizard) to add LUFA modules and remove the Generic board support (driver) module.

usb-serial-asf-wizard.png

Configure GCC symbols (Project -> Properties). Change BOARD to BOARD_UNO, and add F_CPU=16000000UL, and F_USB=16000000UL.

usb-serial-symbols.png

Resolve build errors by mostly eliminating code. Source code that builds without errors is available for comparison at GitHub. Program (Tools -> Device Programming) the firmware using the FLIP tool.

usb-serial-flip

Reboot the Arduino.

Arduino USB DFU firmware from scratch


The Device Firmware Upgrade (DFU) firmware for the ATmega16U2 on Arduino UNO R3 is used to flash the USB Serial firmware, among others, using Atmel’s FLIP tool.

This post documents how you can build from scratch DFU firmware using Atmel Studio 7 and its LUFA Library extension. You’ll need an ISP/ICSP programmer to program the ATmega16U2.

You can install the LUFA extension from the Extensions and Updates dialog (Tools -> Extensions and Updates).

lufa-install.png

Create a new project based on the DFU Bootloader example (File -> New -> Example Project…).

dfu-lufa-example.png

Change the compiler optimization setting to -Os in toolchain properties (Project -> Properties).

dfu-optimize-for-size.png

Build the solution and flash (Tools -> Device Programming) using an ISP/ICSP programmer connected to the ICSP2 header.

dfu-jtagice3.png

 

Getting Started with the ESP-03


The ESP-03 is a very affordable Wi-Fi module built around the ESP8266EX chip by Espressif. TheESP8266 has become very popular among makers who want to affordably add wireless smarts to things at home and work.

The ESP-03 has two useful modes of operation that can be initiated by controlling its GPIO pins – normal mode and flash mode.

Normal mode wiring is show in the figure below. ESP-03 shown here is powered using SparkFun’s FTDI Basic Breakout – 3.3V – USB to serial module.

Normal Mode

In this mode, ESP-03 executes firmware programmed to the SPI Flash. SPI Flash is an external NOR Flash chip where program instructions are stored, and retrieved during execution.

The ESP-03 has a 4 Mbit 25Q40BT part which allows for 512 KB of program space. Of that, about 423 KB is available for your own programs.

Serial Flash Chip

In flash mode, new program instructions can be flashed to SPI Flash, using tools such as the ESP Flash Download Tool. The wiring is similar to that for normal mode, with the addition of GPIO0 connected to GND.

Flash Mode

ESP8266 can be programmed using an SDK distributed by Espressif. Popular embedded development platforms such as the Arduino IDE, MicroPython, and Lua can also be used.

Adafruit provides instructions for configuring the Arduino IDE for ESP8266 development. Here’re the settings I use with the Arduino IDE – under Tools menu.

Arduino Settings

Try the example project under File -> Examples -> ESP8266WiFi. With it, you’ll be controlling a GPIO pin on the ESP-03, over your Wi-Fi network, in no time at all.

wifi-web-server.gif
Wi-Fi Web Server

USB bulk data transfer


USB bulk data transfers are the more commonly used method of communication between a USB host and a peripheral. In this post, I’ll show how you can analyze bulk data transfers captured using a Total Phase Beagle USB 480.

The figure below, shows USB bulk data transfers captured using Total Phase’s Data Center software and a Beagle USB 480. According to the USB specification, all bits in a byte are written to the bus in little-endian fashion, and multi-byte values are also written in little-endian order. While the latter is valid with the Beagle USB 480, single byte values appear in a more readable big-endian order.

bulk transfer

Each USB bulk data transfer is broken into smaller data transfers. Each of the smaller data transfers is initiated by a token packet, followed by data packets, and an ACK packet.

Incoming transfers are initiated by the host using a token packet that has a packet identifier (PID) of IN (byte 0x69). The PID is contained in the last 4 bits (0x9), the first 4 bits are a one’s complement of that value (0x6). The second and third bytes, form a single word in little-endian byte order, containing a 5-bit CRC, followed by a 4-bit endpoint ID, followed by a 7-bit device ID. Thus ASCII hex sequence 81 58 in the figure above, should be interpreted as 0x5881, which results in a CRC of 0x0B, an endpoint ID of 0x1, and a device ID of 0x01.

The device responds to the token packet with data packets, or NAK packet if it is unable to handle the request. The PID of the data packets may be DATA0 (0xC3) or DATA1 (0x4B) for full-speed, and also DATA2 (0x87) for high-speed bulk transfers. The data packet has a maximum payload size determined by the USB specification, and is described by wMaxPacketSize attribute in the endpoint descriptor. It may be 8, 16, 32, or 64 bytes for full-speed bulk endpoints (USB 1.1), and also 512 bytes for high-speed (USB 2.0) bulk endpoints. It is 64 bytes in the capture shown above. The payload is followed by a 2-byte CRC.

The end of a bulk data transfer is signaled by a data packet that has less than wMaxPacketSize bytes. If the last, or only, data packet has wMaxPacketSize bytes, the end is signaled by sending a data packet with a payload size of 0 bytes. The device acknowledges a successful transfer using an ACK packet.

Outgoing transfers work in the same manner. The host initiates data transfer using token packet with PID of OUT (0xE1), followed by data packets. The device acknowledges a successful transfer using an ACK packet. If it is unable to receive data, it sends a NAK packet.

Outgoing bulk data transfers can appear jumbled up with incoming bulk data transfers. Applications that leverage Beagle API need to take this into consideration when parsing data captured on the bus.

Physical serial port redirection


I have previously posted about Virtual serial port redirection on Windows, to develop and test applications that use serial ports. I’ve found the available drivers increasingly buggy on Windows 8.1. This post shows how you can wire two USB-Serial cables to achieve the same objective.

Serial Port Redirection

You’ll need two USB-Serial cables such as USB to TTL Serial Cable for Raspberry Pi from Adafruit or several similar ones from Amazon. Wire the cables so that the ground wires are connected, and receive wire of one cable is connected to the transmit of another and vice-versa. Plug the cables in and install the appropriate drivers. Windows usually works with FTDI chipset based cables without requiring installation of drivers.

Now, you should see two serial ports, which redirect data to each other.

USB Serial


Serial port access can be very useful during embedded systems development.

I do most of my development on Mac OS X, or Ubuntu and Windows virtual machines. I use a USB to serial cable/breakout to connect a serial port to the Mac, which is then redirected by Parallels Desktop to the guest OS.

If you have a cable that uses the Prolific USB Serial chipset, getting up and running is well documented by Plugable. The driver they provide works for me. Most other cables use a chipset from FTDI, but Mac OS X already provides a driver for that.

To interact with a terminal on the embedded system you need some kind of terminal emulator.

I use miniterm.py and screen on Ubuntu

miniterm.py -b 115200 /dev/ttyUSB0

Ensure that you have access to the device

sudo chmod 777 /dev/ttyUSB0

On Mac OS X I usually use screen

screen /dev/cu.usbserial 115200

USB CDC ACM driver for Windows


The LUFA library provides a template INF file that can be easily extended to create a serial port driver for your USB CDC ACM device. Here’s the template with CatalogFile added by me

;     Windows LUFA CDC ACM Setup File
; Copyright (c) 2000 Microsoft Corporation

[DefaultInstall]
CopyINF="LUFA CDC-ACM.inf"

[Version]
Signature="$Windows NT$"
Class=Ports
ClassGuid={4D36E978-E325-11CE-BFC1-08002BE10318}
Provider=%MFGNAME%
CatalogFile="LUFA CDC-ACM.cat"
DriverVer=07/01/2012,10.0.0.0

[Manufacturer]
%MFGNAME%=DeviceList, NTx86, NTamd64, NTia64

[SourceDisksNames]

[SourceDisksFiles]

[DestinationDirs]
DefaultDestDir=12

[DriverInstall]
Include=mdmcpq.inf
CopyFiles=FakeModemCopyFileSection
AddReg=DriverInstall.AddReg

[DriverInstall.Services]
Include=mdmcpq.inf
AddService=usbser, 0x00000002, LowerFilter_Service_Inst

[DriverInstall.AddReg]
HKR,,EnumPropPages32,,"msports.dll,SerialPortPropPageProvider"

;------------------------------------------------------------------------------
;  Vendor and Product ID Definitions
;------------------------------------------------------------------------------
; When developing your USB device, the VID and PID used in the PC side
; application program and the firmware on the microcontroller must match.
; Modify the below line to use your VID and PID.  Use the format as shown below.
; Note: One INF file can be used for multiple devices with different VID and PIDs.
; For each supported device, append ",USB\VID_xxxx&PID_yyyy" to the end of the line.
;------------------------------------------------------------------------------
[DeviceList]
%DESCRIPTION%=DriverInstall, USB\VID_03EB&PID_2044

[DeviceList.NTx86]
%DESCRIPTION%=DriverInstall, USB\VID_03EB&PID_2044

[DeviceList.NTamd64]
%DESCRIPTION%=DriverInstall, USB\VID_03EB&PID_2044

[DeviceList.NTia64]
%DESCRIPTION%=DriverInstall, USB\VID_03EB&PID_2044

;------------------------------------------------------------------------------
;  String Definitions
;------------------------------------------------------------------------------
;Modify these strings to customize your device
;------------------------------------------------------------------------------
[Strings]
MFGNAME="http://www.lufa-lib.org"
DESCRIPTION="LUFA CDC-ACM Virtual Serial Port"

Replace VID_03EB&PID_2044 with something appropriate for your device.

To use this on Windows 8.1 without disabling signed driver check, you’ll need to self-sign the driver. Install Windows Driver Kit (WDK) 8.1 and execute the following from a privileged command prompt

makecert.exe -r -pe -ss PrivateCertStore -n CN=Test.org(Test) Test.cer

Inf2cat.exe /driver:. /os:7_x64,7_X86 

Signtool.exe sign /v /s PrivateCertStore /n Test.org(Test) /t http://timestamp.verisign.com/scripts/timstamp.dll "LUFA CDC-ACM.cat"

certmgr.exe /add Test.cer /s /r localMachine root

Note that in Windows Driver Kit 7.1 Signtool.exe is located under bin\selfsign, and in Windows Driver Kit 8.1 it is located under bin\x86

Now, you should be able to manually install the driver by pointing Windows 8.1 to the folder containing the INF and cat files.