Getting Started with the ESP-03


The ESP-03 is a very affordable Wi-Fi module containing the ESP8266EX chip by Espressif. The latter has become very popular among makers who want to affordably add wirelessly controlled smarts to things at home and work.

The ESP-03 has two useful modes of operation that can be initiated by controlling its GPIO pins. The normal mode connections are show in the following figure. The ESP-03 shown here is powered using SparkFun’s FTDI Basic Breakout – 3.3V – USB to serial module.

Normal Mode

In normal mode, the ESP-03 runs firmware programmed in the SPI Flash. SPI Flash is an external NOR Flash chip where program instructions can be stored, and retrieved later for execution. The ESP-03 has a 4 Mbit 25Q40BT part which allows for 512 KB of program space. Of that, about 423 KB is available for your own programs.

Serial Flash Chip

The second mode is the flash mode, in which new program instructions can be flashed to the SPI Flash, using tools such as the ESP Flash Download Tool. The connections are similar to those for normal mode, with the addition of GPIO0 connected to GND.

Flash Mode

The ESP8266 can be programmed using an SDK distributed by Espressif. Popular embedded development platforms such as the Arduino IDE and MicroPython, can also be used to develop firmware for the ESP-03.

Adafruit provides instructions for configuring the Arduino IDE for ESP8266 development. Here’re the settings I use with the ESP-03.

Arduino Settings

Try out the WiFiWebServer example project under File -> Examples -> ESP8266WiFi. With it, you’ll be controlling a GPIO pin on the ESP-03, over your Wi-Fi network, in no time at all.

Run homebridge as a service upon reboot


This post shows how to run homebridge automatically upon reboot using upstart.

Install upstart

sudo apt-get install upstart

Create configuration file /etc/init/homebridge.conf with

start on stopped rc
stop on shutdown

setuid pi

script
    export HOME="/home/pi"
    export NODE_PATH=$HOME/node_modules/
    gpio -g mode 27 out
    gpio -g mode 27 down
    gpio export 27 out
    exec /usr/local/bin/homebridge
end script

start on stopped rc ensures that avahi-daemon has been started by its SysV script under /etc/init.d before homebridge is started.

Test the job by running it thus

sudo start homebridge

Use the following command to check the output of the job

sudo tail -f /var/log/upstart/homebridge.log

The following command can be used to verify that homebridge job has been started

sudo initctl list | grep homebridge

To stop the above job

sudo stop homebridge

To run job as a service that will run automatically at boot

sudo service homebridge start

To stop the service forever

sudo service homebridge stop

Toggle GPIO on Raspberry Pi using HomeKit


In this post, I take my HomeKit Raspberry Pi integration a step further, by turning on/off a LED using the homebridge-gpio-wpi plugin. With the ability to control the GPIO pins, I should be able to turn on/off much bigger things using solid state relays and such.

Install homebridge-gpio-wpi

The installation should be pretty straightforward. Assuming you are at the command line in the home folder, run

npm install homebridge-gpio-wpi

That should install all node modules under ~/node_modules/.

Configure homebridge by editing ~/.homebridge/config.json. Here’s mine

{
    "bridge": {
        "name": "Homebridge",
        "username": "CC:22:3D:E3:CE:32",
        "port": 51826,
        "pin": "031-45-155"
    },
    
    "description": "This has some fake accessories",

    "accessories": [
        {
            "accessory":      "FakeBulb",
            "name":           "Test lamp",
            "bulb_name":      "Lamp 1"
        },
        {
            "accessory": "GPIO",
            "name": "GPIO2",
            "pin": 27
        }
    ],

    "platforms": []
}

Configure GPIO2 using the gpio utility, and start homebridge

gpio -g mode 27 out
gpio -g mode 27 down
gpio export 27 out
export NODE_PATH=$HOME/node_modules/
homebridge

See also how to run homebridge as a service upon reboot.

Test with HomeKit

If you’ve configured the Homebridge peripheral in an iOS app such as Hesperus, it should now show you a new device called GPIO2, and allow you to switch it on/off.

LED

Hesperus allows you to create a schedule to turn on and off devices.

Schedule

Make things smart with HomeKit and Raspberry Pi


As an avid iOS user I have been keen on using HomeKit. That’s when I read about a new – and currently free – HomeKit app in the iOS App Store called Hesperus. I don’t have a HomeKit compatible thing at home, but a quick internet search revealed that I could run a HomeKit compatible service called homebridge on a Raspberry Pi. This post only goes so far as configuring a fictitious light bulb plugin that can be controlled remotely.

Setup Raspberry Pi Image

I decided to download a headless (console-only) version of Raspbian called RASPBIAN JESSIE LITE. Instructions for setting up an SD card appropriately can be found here. I tend to use the Windows based Win32 Disk Imager, I trust it and have used it – for as long as I can remember – to write Ubuntu ARM images.

Powering the Raspberry Pi

I didn’t want to use an HDMI display with the Raspberry Pi, and wanted to power it using my laptop. I have used a USB to serial adapter to do that in the past. This time, I went with Adafruit’s USB Serial TTL cable as described in this lesson. That done, I was able to power up and login to the Raspberry Pi using a serial terminal. I tend to use screen on Linux or Mac OS X

screen /dev/ttyUSB0

Configuring Wi-Fi

RASPBIAN JESSIE LITE lacks a full-fledged user interface, making Wi-Fi configuration slightly painful. I am using a Wi-Fi stick and had some issues getting the driver to work. Hopefully, you’ve got a Raspberry Pi 3, or a compatible Wi-Fi stick that does not require too much tinkering. You can also use Ethernet. The following can be used to check whether your network interface can be listed

ifconfig

Look for an interface called wlan0 if using Wi-Fi, or eth0 if using ethernet.

This is how you can create a configuration file for your Wi-Fi access point

wpa_passphrase your_SSID your_passphrase > your_SSID.conf

Copy the contents of your_SSID.conf and paste them into /etc/wpa_supplicant/wpa_supplicant.conf using any text editor. I used vi thus

sudo vi /etc/wpa_supplicant/wpa_supplicant.conf

Having done that, Wi-Fi was up and running. I had internet access, and could access the Raspberry Pi on the local network via ssh.

Installing packages

A few additional Linux packages and configuration steps are required before homebridge may be installed. Packages can be installed thus

sudo apt-get update
sudo apt install nodejs npm git libavahi-compat-libdnssd-dev

Updating Node.js

The version of Node.js installed by apt-get is rather dated, and will not work with homebridge. To update node, use the following commands

sudo npm install -g n
sudo n stable

Install homebridge

homebridge can be installed using npm thus

sudo npm install -g homebridge

Find and install plugins

To do anything interesting with homebridge you’ll require a plugin, and have it configured in ~/.homebridge/config.json. One simple plugin called homebridge-fakebulb can be installed thus

sudo npm install -g homebridge-fakebulb

Its sample configuration file can be used to create the config.json file mentioned above. This is what my config.json looks like

{
    "bridge": {
        "name": "Homebridge",
        "username": "CC:22:3D:E3:CE:32",
        "port": 51826,
        "pin": "031-45-155"
    },
    
    "description": "This has some fake accessories",

    "accessories": [
        {
            "accessory":      "FakeBulb",
            "name":           "Test lamp",
            "bulb_name":      "Lamp 1"
        }
    ],

    "platforms": []
}

Use with HomeKit

Apple’s HomeKit has been app-less since launch. Siri is the only way you were able to control HomeKit devices. HomeKit has a rich API and it didn’t take long for paid apps to appear in the App Store. Hesperus is a new free app that I opted to use to control homebridge.

Here’s Hesperus with the Homebridge perihperal paired and working, showing the Test lamp device’s status. I can control the Test lamp (turn it on/off) anywhere I have an internet connection because I have an Apple TV (generation 4 – but 3 should also work) at home. Apple TV needs to be signed into the same iCould account as the iOS device paired with the Homebridge peripheral.

IMG_1979

Create and use NuGet packages


To follow the (brief) instructions in this post, you’ll need to download NuGet.

To create NuGet package spec (nuspec) from a project file, execute the following in the folder where the project file is located

nuget spec

Edit the file appropriately.

To create a NuGet package (a glorified zip file) containing project output and all dependencies, run

nuget pack ProjectName.csproj -IncludeReferencedProjects -Prop Configuration=Release -Prop Platform=AnyCPU

Packages can be distributed by creating account at, and uploading nupkg to, nuget.org.

Package references can be managed using NuGet Package Manager extension in Visual Studio (see option under Tools menu). Referenced packages are listed in the packages.config located in a project’s folder; recommend adding it to source control. VS will download all packages specified there before the project is built. VS caches downloaded packages in the packages folder located in the solution’s root folder; exclude it from source control.

nupkg files can also be distributed by other means, and added manually into project using the Package Management Console

Install-Package SomePackage -Source Path

SomePackage is a fully qualified package name, Path is an absolute or relative path containing the nupkg file.

Install-Package may fail with the following message, when packages need to be restored from custom source

Install-Package : Some NuGet packages are missing from the solution. The packages need to be restored in order to build the dependency graph. Restore the packages before performing any operations.

To restore packages from custom source, use nuget command, thus

nuget restore -Source Path

Note that Path has to be an absolute path; multiple paths can be separated by semicolon e.g. C:\packages;https://www.myget.org/F/nuget.

HTTP/S capture using mitmproxy


This post shows how to install mitmproxy on Mac OS X (El Capitan) to capture HTTP/S traffic, especially useful when debugging applications.

I’ve been using Telerik Fiddler on Windows for sniffing HTTP/S traffic. It supports WebSockets. It isn’t very reliable on Mac or Linux. mitmproxy fills the lacuna well, although it does not yet support WebSockets.

Install

Use pip to install mitmproxy thus

pip install mitmproxy

I encountered several compilation issues while installing through pip. I’ll go through them one by one. The first error results from failure to compile cryptography

    building '_openssl' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/_openssl.c -o build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7/_openssl.o
    build/temp.macosx-10.10-x86_64-2.7/_openssl.c:431:10: fatal error: 'openssl/aes.h' file not found
    #include <openssl/aes.h>
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1

That can be resolved by executing pip to install cryptography thus

env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install mitmproxy

With that dependency resolved, mitmproxy install fails with the following error

    building 'lxml.etree' extension
    clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -I/usr/include/libxml2 -Isrc/lxml/includes -I/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.10-x86_64-2.7/src/lxml/lxml.etree.o -w -flat_namespace
    In file included from src/lxml/lxml.etree.c:323:
    src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found
    #include "libxml/xmlversion.h"
             ^
    1 error generated.
    Compile failed: command 'clang' failed with exit status 1
    cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.c -o var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.o
    /var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.c:1:10: fatal error: 'libxml/xpath.h' file not found
    #include "libxml/xpath.h"
             ^
    1 error generated.
    *********************************************************************************
    Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
    Perhaps try: xcode-select --install
    *********************************************************************************

Luckily, that error also shows the solution, run

xcode-select --install

Now, mitmproxy should install successfully.

Run

To capture HTTP/S traffic using mitmproxy traffic, run

mitmproxy

mitmproxy should show which port it is listening at; 8080 is the default. Use http://localhost:8080 as the HTTP proxy setting in browsers and applications.

Android emulator

This is how you can execute Android emulator to use mitmproxy as an HTTP proxy

export DYLD_FALLBACK_LIBRARY_PATH=~/Library/Android/sdk/tools/lib64
~/Library/Android/sdk/tools/emulator64-x86 -avd Nexus_S_API_21_x86 -http-proxy http://localhost:8080

The first line is needed so that the emulator can find the necessary libraries such as OpenGLES emulation library.

Pinned Certificates

If you try to access any site in the Android browser, or run any application that uses HTTP/S, mitmproxy will capture all traffic. To capture SSL traffic mitmproxy presents its own certificate to the applications. The root certificate that mitmproxy uses will need to be added to the certificate store, to avoid failures in certificate chain validation. This can be done by navigating to the special mitm.it URL in the browser, and picking your platform from the resulting page.

If you use certificate pinning in an application you develop, you’ll need to add ~/.mitmproxy/mitmproxy-ca-cert.cer to the list of certificates.

WebSocket traffic

You can setup mitmproxy to ignore (avoid intercepting) traffic to a certain host:port. This can be useful when you want it to ignore WebSocket traffic.

mitmproxy --ignore 192\.168\.1\.10:888[1-9]