Randomly shuffle lines in a file


shuf -o output.txt input.txt

Install coreutils on Mac OS X using Homebrew, if not already installed. Call gshuf instead of shuf.

Advertisements

Run homebridge as a service upon reboot


This post shows how to run homebridge automatically upon reboot using upstart. If you’re using systemd, the default initialization system these days, see Running Homebridge on Bootup (systemd).

Install upstart

sudo apt-get install upstart

Create configuration file /etc/init/homebridge.conf with

start on stopped rc
stop on shutdown

setuid pi

script
    export HOME="/home/pi"
    export NODE_PATH=$HOME/node_modules/
    gpio -g mode 27 out
    gpio -g mode 27 down
    gpio export 27 out
    exec /usr/local/bin/homebridge
end script

start on stopped rc ensures that avahi-daemon has been started by its SysV init script under /etc/init.d before homebridge is started.

Test the job by running it thus

sudo start homebridge

Use the following command to check the output of the job

sudo tail -f /var/log/upstart/homebridge.log

The following command can be used to verify that homebridge job has been started

sudo initctl list | grep homebridge

To stop the above job

sudo stop homebridge

To run job as a service that will run automatically at boot

sudo service homebridge start

To stop the service forever

sudo service homebridge stop

Toggle GPIO on Raspberry Pi using HomeKit


In this post, I take my HomeKit Raspberry Pi integration a step further, by turning on/off a LED using the homebridge-gpio-wpi plugin. With the ability to control the GPIO pins, I should be able to turn on/off much bigger things using solid state relays and such.

Install homebridge-gpio-wpi

The installation should be pretty straightforward. Assuming you are at the command line in the home folder, run

npm install homebridge-gpio-wpi

That should install all node modules under ~/node_modules/.

Configure homebridge by editing ~/.homebridge/config.json. Here’s mine

{
    "bridge": {
        "name": "Homebridge",
        "username": "CC:22:3D:E3:CE:32",
        "port": 51826,
        "pin": "031-45-155"
    },
    
    "description": "This has some fake accessories",

    "accessories": [
        {
            "accessory":      "FakeBulb",
            "name":           "Test lamp",
            "bulb_name":      "Lamp 1"
        },
        {
            "accessory": "GPIO",
            "name": "GPIO2",
            "pin": 27
        }
    ],

    "platforms": []
}

Configure GPIO2 using the gpio utility, and start homebridge

gpio -g mode 27 out
gpio -g mode 27 down
gpio export 27 out
export NODE_PATH=$HOME/node_modules/
homebridge

See also how to run homebridge as a service upon reboot.

Test with HomeKit

If you’ve configured the Homebridge peripheral in an iOS app such as Hesperus, it should now show you a new device called GPIO2, and allow you to switch it on/off.

LED

Hesperus allows you to create a schedule to turn on and off devices.

Schedule

HTTP/S capture using mitmproxy


This post shows how to install mitmproxy on Mac OS X (El Capitan) to capture HTTP/S traffic, especially useful when debugging applications.

I’ve been using Telerik Fiddler on Windows for sniffing HTTP/S and WebSocket traffic, but it isn’t very reliable on Mac or Linux. mitmproxy fills the lacuna well, but it does not yet support WebSocket traffic.

Install

Use pip to install mitmproxy thus

pip install mitmproxy

I encountered several compilation issues while installing through pip. I’ll go through them one by one. The first error results from failure to compile cryptography

    building '_openssl' extension
    clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/_openssl.c -o build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7/_openssl.o
    build/temp.macosx-10.10-x86_64-2.7/_openssl.c:431:10: fatal error: 'openssl/aes.h' file not found
    #include <openssl/aes.h>
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1

That can be resolved by executing pip to install cryptography thus

env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install mitmproxy

With that dependency resolved, mitmproxy install fails with the following error

    building 'lxml.etree' extension
    clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -I/usr/include/libxml2 -Isrc/lxml/includes -I/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.10-x86_64-2.7/src/lxml/lxml.etree.o -w -flat_namespace
    In file included from src/lxml/lxml.etree.c:323:
    src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found
    #include "libxml/xmlversion.h"
             ^
    1 error generated.
    Compile failed: command 'clang' failed with exit status 1
    cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.c -o var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.o
    /var/folders/3v/zgzrr9h96_34db7lt9_fx1wr0000gn/T/xmlXPathInitdIvQjA.c:1:10: fatal error: 'libxml/xpath.h' file not found
    #include "libxml/xpath.h"
             ^
    1 error generated.
    *********************************************************************************
    Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
    Perhaps try: xcode-select --install
    *********************************************************************************

Luckily, that error also shows the solution, run

xcode-select --install

Now, mitmproxy should install successfully.

Run

To capture HTTP/S traffic using mitmproxy traffic, run

mitmproxy

mitmproxy should show which port it is listening at; 8080 is the default. Use http://localhost:8080 as the HTTP proxy setting in browsers and applications.

Android emulator

This is how you can execute Android emulator to use mitmproxy as an HTTP proxy

export DYLD_FALLBACK_LIBRARY_PATH=~/Library/Android/sdk/tools/lib64
~/Library/Android/sdk/tools/emulator64-x86 -avd Nexus_S_API_21_x86 -http-proxy http://localhost:8080

The first line is needed so that the emulator can find the necessary libraries such as OpenGLES emulation library.

Pinned Certificates

If you try to access any site in the Android browser, or run any application that uses HTTP/S, mitmproxy will capture all traffic. To capture SSL traffic mitmproxy presents its own certificate to the applications. The root certificate that mitmproxy uses will need to be added to the certificate store, to avoid failures in certificate chain validation. This can be done by navigating to the special mitm.it URL in the browser, and picking your platform from the resulting page.

If you use certificate pinning in your applications, you can add ~/.mitmproxy/mitmproxy-ca-cert.cer to the list of certificates.

WebSocket traffic

mitmproxy does not support WebSocket traffic so connection establishment will fail. You can however setup mitmproxy to ignore traffic to a certain host:port. This can be leveraged to ask it to ignore WebSocket traffic.

mitmproxy --ignore 192\.168\.1\.10:888[1-9]

Folders consuming most disk space using du


du is available natively on almost all Linux distributions, and on Mac OS X. If you are in need of reclaiming disk space, and want to quickly find which folders to focus your attention on, run the following command

du -h -d 1

That will quickly list all the folders under the current folder and their disk space usage. Use the following command to check space left on each disk

df -h

Virtual serial port redirection using socat


Here’s how socat can be used to redirect one serial port to another on Ubuntu

sudo socat -d -d pty,link=/dev/ttyS0,raw,echo=0 pty,link=/dev/ttyS1,raw,echo=0

Assuming both serial devices above are not assigned to a real device. If the above command is successful, you can send data from on serial port to another using a terminal emulator such as screen

screen /dev/ttyS0 115200

Install screen using sudo apt-get screen if not already installed. Repeat the above command on another prompt for the other serial device, and you’re good to go. Quit screen using Ctrl-A .

socat can also be used to perform serial to TCP redirection

sudo socat -d -d pty,link=/dev/ttyS0,raw,echo=0 tcp-listen:8000

Now, you should be able to use screen to send and receive data on the serial port, while you use something like netcat to receive and send data to the serial port

netcat 127.0.0.1 8000

Enable Ubuntu Unity


I have been having problems using Ubuntu Unity on my Parallels VM, that I was finally able to resolve with help from the Ubuntu forums. It turns out Ubuntu Unity Plugins was disabled for some reason. Use the following sequence of commands to run CompizConfig Settings Manager.

sudo apt-get install compizconfig-settings-manager
export DISPLAY=:0
ccsm

Select Ubuntu Unity Plugin. Check Enable Ubuntu Unity Plugin if it is disabled. Enable other required plugins if you are prompted to do so.