Make things smart with HomeKit and Raspberry Pi

As an avid iOS user I have been keen on using HomeKit. That’s when I read about a new – and currently free – HomeKit app in the iOS App Store called Hesperus. I don’t have a HomeKit compatible thing at home, but a quick internet search revealed that I could run a HomeKit compatible service called homebridge on a Raspberry Pi. This post only goes so far as configuring a fictitious light bulb plugin that can be controlled remotely.

Setup Raspberry Pi Image

I decided to download a headless (console-only) version of Raspbian called RASPBIAN JESSIE LITE. Instructions for setting up an SD card appropriately can be found here. I tend to use the Windows based Win32 Disk Imager, I trust it and have used it – for as long as I can remember – to write Ubuntu ARM images.

Powering the Raspberry Pi

I didn’t want to use an HDMI display with the Raspberry Pi, and wanted to power it using my laptop. I have used a USB to serial adapter to do that in the past. This time, I went with Adafruit’s USB Serial TTL cable as described in this lesson. That done, I was able to power up and login to the Raspberry Pi using a serial terminal. I tend to use screen on Linux or Mac OS X

screen /dev/ttyUSB0

Configuring Wi-Fi

RASPBIAN JESSIE LITE lacks a full-fledged user interface, making Wi-Fi configuration slightly painful. I am using a Wi-Fi stick and had some issues getting the driver to work. Hopefully, you’ve got a Raspberry Pi 3, or a compatible Wi-Fi stick that does not require too much tinkering. You can also use Ethernet. The following can be used to check whether your network interface can be listed

ifconfig

Look for an interface called wlan0 if using Wi-Fi, or eth0 if using ethernet.

This is how you can create a configuration file for your Wi-Fi access point

wpa_passphrase your_SSID your_passphrase > your_SSID.conf

Copy the contents of your_SSID.conf and paste them into /etc/wpa_supplicant/wpa_supplicant.conf using any text editor. I used vi thus

sudo vi /etc/wpa_supplicant/wpa_supplicant.conf

Having done that, Wi-Fi was up and running. I had internet access, and could access the Raspberry Pi on the local network via ssh.

Installing packages

A few additional Linux packages and configuration steps are required before homebridge may be installed. Packages can be installed thus

sudo apt-get update
sudo apt install nodejs npm git libavahi-compat-libdnssd-dev

Updating Node.js

The version of Node.js installed by apt-get is rather dated, and will not work with homebridge. To update node, use the following commands

sudo npm install -g n
sudo n stable

Install homebridge

homebridge can be installed using npm thus

sudo npm install -g homebridge

Find and install plugins

To do anything interesting with homebridge you’ll require a plugin, and have it configured in ~/.homebridge/config.json. One simple plugin called homebridge-fakebulb can be installed thus

sudo npm install -g homebridge-fakebulb

Its sample configuration file can be used to create the config.json file mentioned above. This is what my config.json looks like

{
    "bridge": {
        "name": "Homebridge",
        "username": "CC:22:3D:E3:CE:32",
        "port": 51826,
        "pin": "031-45-155"
    },
    
    "description": "This has some fake accessories",

    "accessories": [
        {
            "accessory":      "FakeBulb",
            "name":           "Test lamp",
            "bulb_name":      "Lamp 1"
        }
    ],

    "platforms": []
}

Use with HomeKit

Apple’s HomeKit has been app-less since launch. Siri is the only way you were able to control HomeKit devices. HomeKit has a rich API and it didn’t take long for paid apps to appear in the App Store. Hesperus is a new free app that I opted to use to control homebridge.

Here’s Hesperus with the Homebridge perihperal paired and working, showing the Test lamp device’s status. I can control the Test lamp (turn it on/off) anywhere I have an internet connection because I have an Apple TV (generation 4 – but 3 should also work) at home. Apple TV needs to be signed into the same iCould account as the iOS device paired with the Homebridge peripheral.

IMG_1979

Create and use NuGet packages

To follow the (brief) instructions in this post, you’ll need to download NuGet.

To create NuGet package spec (nuspec) from a project file, execute the following in the folder where the project file is located

nuget spec

Edit the file appropriately.

To create a NuGet package (a glorified zip file) containing project output and all dependencies, run

nuget pack ProjectName.csproj -IncludeReferencedProjects -Prop Configuration=Release -Prop Platform=AnyCPU

Packages can be distributed by creating account at, and uploading nupkg to, nuget.org.

Package references can be managed using NuGet Package Manager extension in Visual Studio (see option under Tools menu). Referenced packages are listed in the packages.config located in a project’s folder; recommend adding it to source control. VS will download all packages specified there before the project is built. VS caches downloaded packages in the packages folder located in the solution’s root folder; exclude it from source control.

nupkg files can also be distributed by other means, and added manually into project using the Package Management Console

Install-Package SomePackage -Source Path

SomePackage is a fully qualified package name, Path is an absolute or relative path containing the nupkg file.

Install-Package may fail with the following message, when packages need to be restored from custom source

Install-Package : Some NuGet packages are missing from the solution. The packages need to be restored in order to build the dependency graph. Restore the packages before performing any operations.

To restore packages from custom source, use nuget command, thus

nuget restore -Source Path

Note that Path has to be an absolute path; multiple paths can be separated by semicolon e.g. C:\packages;https://www.myget.org/F/nuget.

HTTP/S capture using mitmproxy

This post shows how to install mitmproxy on Mac OS X (El Capitan) to capture HTTP/S traffic, especially useful when debugging applications.

I’ve been using Telerik Fiddler on Windows for sniffing HTTP/S traffic. It supports WebSockets. It isn’t very reliable on Mac or Linux. mitmproxy fills the lacuna well, although it does not yet support WebSockets.

Install

Use pip to install mitmproxy thus

pip install mitmproxy

I encountered several compilation issues while installing through pip. I’ll go through them one by one. The first error results from failure to compile cryptography

    building '_openssl' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/_openssl.c -o build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7/_openssl.o
    build/temp.macosx-10.10-x86_64-2.7/_openssl.c:431:10: fatal error: 'openssl/aes.h' file not found
    #include <openssl/aes.h>
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1

That can be resolved by executing pip to install cryptography thus

env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install mitmproxy

With that dependency resolved, mitmproxy install fails with the following error

    building 'lxml.etree' extension
    clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -I/usr/include/libxml2 -Isrc/lxml/includes -I/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.10-x86_64-2.7/src/lxml/lxml.etree.o -w -flat_namespace
    In file included from src/lxml/lxml.etree.c:323:
    src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found
    #include "libxml/xmlversion.h"
             ^
    1 error generated.
    Compile failed: command 'clang' failed with exit status 1
    cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.c -o var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.o
    /var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.c:1:10: fatal error: 'libxml/xpath.h' file not found
    #include "libxml/xpath.h"
             ^
    1 error generated.
    *********************************************************************************
    Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
    Perhaps try: xcode-select --install
    *********************************************************************************

Luckily, that error also shows the solution, run

xcode-select --install

Now, mitmproxy should install successfully.

Run

To capture HTTP/S traffic using mitmproxy traffic, run

mitmproxy

mitmproxy should show which port it is listening at; 8080 is the default. Use http://localhost:8080 as the HTTP proxy setting in browsers and applications.

Android emulator

This is how you can execute Android emulator to use mitmproxy as an HTTP proxy

export DYLD_FALLBACK_LIBRARY_PATH=~/Library/Android/sdk/tools/lib64
~/Library/Android/sdk/tools/emulator64-x86 -avd Nexus_S_API_21_x86 -http-proxy http://localhost:8080

The first line is needed so that the emulator can find the necessary libraries such as OpenGLES emulation library.

Pinned Certificates

If you try to access any site in the Android browser, or run any application that uses HTTP/S, mitmproxy will capture all traffic. To capture SSL traffic mitmproxy presents its own certificate to the applications. The root certificate that mitmproxy uses will need to be added to the certificate store, to avoid failures in certificate chain validation. This can be done by navigating to the special mitm.it URL in the browser, and picking your platform from the resulting page.

If you use certificate pinning in an application you develop, you’ll need to add ~/.mitmproxy/mitmproxy-ca-cert.cer to the list of certificates.

WebSocket traffic

You can setup mitmproxy to ignore (avoid intercepting) traffic to a certain host:port. This can be useful when you want it to ignore WebSocket traffic.

mitmproxy --ignore 192\.168\.1\.10:888[1-9]

Handle custom CAs in iOS app

This post discusses how to handle custom CAs in a Swift 2 iOS app that uses Alamofire.

Certificates need to be in the DER encoded binary format. If they are in PEM (BASE64) encoded text format, you need to convert them to DER encoded format. This can be achieved with openssl, thus

openssl x509 -inform PEM -in infile.cer -outform DER -out outfile.cer

Then, just drag the certificate files into your Xcode project so that they are embedded as resources.

Next, we need to create a custom Alamofire manager with a custom ServerTrustPolicyManager

        let serverTrustPolicies: [String: ServerTrustPolicy] = [
            "example.com": .PinCertificates(
                certificates: ServerTrustPolicy.certificatesInBundle(),
                validateCertificateChain: true,
                validateHost: true
            )
        ]
        
        manager = Manager(
            serverTrustPolicyManager: ServerTrustPolicyManager(policies: serverTrustPolicies)
        )

All REST requests should then be created using manager.request instead of Alamofire.request.

In iOS 9, additional transport security exceptions are required to be set in info.plist.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
...
    <key>NSAppTransportSecurity</key>
    <dict>
        <key>NSExceptionDomains</key>
        <dict>
            <key>example.com</key>
            <dict>
                <key>NSExceptionAllowsInsecureHTTPLoads</key>
                <true/>
                <key>NSExceptionRequiresForwardSecrecy</key>
                <false/>
                <key>NSIncludesSubdomains</key>
                <true/>
                <!-- Optional: Specify minimum TLS version -->
                <key>NSTemporaryExceptionMinimumTLSVersion</key>
                <string>TLSv1.2</string>
            </dict>
        </dict>
    </dict>
...
</dict>
</plist>

Tackling OAuth 2.0 in iOS 9 with Swift 2

In an earlier post I discussed how to tackle OAuth 2.0 in an Android app. This post discusses how to do that in iOS 9 with Swift 2.

Step 1, in the post referenced above, is performed in a UIWebView. The view containing the UIWebView implements UIWebViewDelegate, intercepts all accessed URLs in webView:shouldStartLoadWithRequest:navigationType: function, and acquires the authorization code using regular expressions.

class ViewController: UIViewController, UIWebViewDelegate {
    @IBOutlet weak var webView: UIWebView!

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view, typically from a nib.
        webView.delegate = self;
        webView.loadRequest(loginURL) // initialize loginURL
    }

    func webView(webView: UIWebView, shouldStartLoadWithRequest: NSURLRequest, navigationType: UIWebViewNavigationType) -> Bool {
        var continueToUrl = true
        do {
            let regex = try NSRegularExpression(pattern: "code=(.+?(?=&|$))", options: NSRegularExpressionOptions())
            let url: String? = shouldStartLoadWithRequest.URL?.absoluteString;
            let result: NSTextCheckingResult? = regex.firstMatchInString(url!, options: NSMatchingOptions(), range: NSMakeRange(0, url!.characters.count))
            let codeNSRange = result?.rangeAtIndex(1)
            if (codeNSRange != nil) {
                let startIndex = (url?.startIndex)!.advancedBy((codeNSRange?.location)!)
                let endIndex = (url?.startIndex)!.advancedBy((codeNSRange?.location)! + (codeNSRange?.length)!)
                let code = url?.substringWithRange(Range<String.Index>(start: startIndex, end: endIndex))
                debugPrint(code)
                acquireAccessToken(code!);
                continueToUrl = false
            }

        } catch {
        }
        return continueToUrl;
    }

I found it challenging to use the interfaces for regular expression matching, and extracting substrings. They are quite different from those found in other popular languages, but should be familiar to Objective C developers.

Step 2, to acquire the access token, can be performed thus

    func acquireAccessToken(code: String) {
        manager!.request(.POST, tokenURL, parameters: ["code": code], encoding: .URLEncodedInURL)
            .responseJSON { (response) -> Void in
                switch response.result {
                case .Success:
                    if let value = response.result.value {
                        let json = JSON(value)
                        self.accessToken = json["access_token"].stringValue
                        debugPrint("Access Token: \(self.accessToken)")
                    }
                    
                case .Failure(let error):
                    debugPrint(error)
                }
        }
    }

The code above utilizes Alamofire for performing REST calls, and SwiftyJSON to handle JSON. Obtain those two frameworks using Carthage. Create a Cartfile with

github "Alamofire/Alamofire" ~> 3.0
github "SwiftyJSON/SwiftyJSON"

Then, invoke carthage to build the frameworks

carthage update

Drag the .framework files under Carthage/Build/ into your Xcode project. The files are added by reference. Adjust project target settings to ensure that the frameworks are embedded, otherwise you’ll get an error such as

dyld: Library not loaded: @rpath/SwiftyJSON.framework/SwiftyJSON
  Referenced from: ...

If you use git for version control, remember to adjust .gitignore to exclude Carthage/ folder.

Dealing with .NET’s messy WebBrowser control

I’ve been trying hard to coax .NET’s WebBrowser control to log in using PingFederate federation server. I particularly don’t want to mess with the registry to change Internet Explorer’s browser emulation settings due to a single application. Something that with Android is amazingly simple, requires a lot of extra effort with .NET for the Desktop.

The WebBrowser control defaults to IE7 emulation as seen by the following User-Agent header, discovered using Fiddler.

User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729)

There’s a bug in the browser control that sends a trailing null character in POST data.

pf.username=xxx&pf.pass=xxx&pf.ok=clicked&pf.cancel=<NULL>

I had to extend the WebBrowser control, gain access to its internal ActiveX control, and use events of that control to modify the behavior just enough to be able to log in using PingFederate. The code that does that is reproduced below.

    public class ExtendedWebBrowser : WebBrowser
    {
        bool renavigating = false;

        public string UserAgent { get; set; }

        public delegate void BeforeNavigateDelegate(string url, ref bool cancel);

        public event BeforeNavigateDelegate HandleBeforeNavigate;

        public delegate void NavigateErrorDelegate(string url, ref bool cancel);

        public event NavigateErrorDelegate HandleNavigateError;

        public ExtendedWebBrowser()
        {
            DocumentCompleted += SetupBrowser;

            //this will cause SetupBrowser to run (we need a document object)
            Navigate("about:blank");
        }

        void SetupBrowser(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
            DocumentCompleted -= SetupBrowser;
            SHDocVw.WebBrowser xBrowser = (SHDocVw.WebBrowser)ActiveXInstance;
            xBrowser.BeforeNavigate2 += BeforeNavigate;
            xBrowser.NavigateError += NavigateError;
        }

        private void NavigateError(object pDisp, ref object URL, ref object Frame, ref object StatusCode, ref bool Cancel)
        {
            if (HandleNavigateError != null)
                HandleNavigateError.Invoke((string)URL, ref Cancel);
        }

        void BeforeNavigate(object pDisp, ref object url, ref object flags, ref object targetFrameName,
            ref object postData, ref object headers, ref bool cancel)
        {
            if (renavigating)
            {
                renavigating = false;
                if (HandleBeforeNavigate != null)
                {
                    HandleBeforeNavigate.Invoke((string)url, ref cancel);
                }
            }
            else
            {
                byte[] pSrc = (byte[])postData;
                byte[] p = pSrc;

                if (pSrc != null && pSrc[pSrc.Length - 1] == 0)
                {
                    // remove trailing null from POST data
                    p = new byte[((byte[])postData).Length - 1];
                    Array.Copy(((byte[])postData), p, p.Length);
                    renavigating = true;
                }

                if (!string.IsNullOrEmpty(UserAgent))
                {
                    headers += string.Format("User-Agent: {0}\r\n", UserAgent);
                    renavigating = true;
                }

                if (renavigating)
                {
                    Navigate((string)url, (string)targetFrameName, p, (string)headers);
                    cancel = true;
                }
            }
        }
    }

The authorization code returned by PingFederate can be obtained by registering for HandleNavigateError event. Using the HandleBeforeNavigate event handler does not work, because it is not invoked when the browser control is redirected after a 302 Not Found response.

            extendedWebBrowser1.HandleNavigateError += delegate (string url,
                ref bool cancel)
            {
                cancel = ExtractAuthorizationCode(url);
            };