Link dynamic pads of demuxer


Demuxers do not have any pads till they receive the buffers to parse. As data is available to parse, pads are dynamically added based on the streams available.

The pad-added signal

The pad-added signal can be used to attach new elements to the pipeline when a new pad gets added. Use the g_signal_connect function to listen for pad-added. In the callback function, you can add new elements to the pipeline and link them to the demuxer based on the name of the pad. If the pad name starts with audio, for instance, you can link the element for audio playback. The state of these new elements needs to set to GST_STATE_PLAYING.

Here’s how you can register a callback for pad-added

g_signal_connect (demux, "pad-added", (GCallback)demux_pad_added, NULL);

Here’s a sample callback function for the matroskademux element

void
demux_pad_added (GstElement* demux, GstPad* pad, gpointer user_data)
{
  char* name;
  GstPad *sinkpad;
  GstElement *tee, *sink;

  name = gst_pad_get_name(pad);

  if (strncmp(name, "audio", 5) == 0)
  {
    // link audio sink pad of demuxer to src pad of audio tee
    tee = gst_element_factory_make ("tee", "audiotee");
    sink = gst_element_factory_make ("fakesink", "audiosink");
    gst_bin_add_many (GST_BIN (pipeline), tee, sink, NULL);
    gst_element_link (tee, sink);
    sinkpad = gst_element_get_static_pad(tee, "sink");
    gst_pad_link(pad, sinkpad);
    gst_object_unref (sinkpad);
    gst_element_set_state(tee, GST_STATE_PLAYING);
    gst_element_set_state(sink, GST_STATE_PLAYING);
    g_print ("Linked pad %s of demuxer\n", name);
  }
  else if (strncmp(name, "video", 5) == 0)
  {
    // link src pad of demuxer to sink pad of video tee
    tee = gst_element_factory_make ("tee", "videotee");
    sink = gst_element_factory_make ("fakesink", "videosink");
    gst_bin_add_many (GST_BIN (pipeline), tee, sink, NULL);
    gst_element_link (tee, sink);
    sinkpad = gst_element_get_static_pad(tee, "sink");
    gst_pad_link(pad, sinkpad);
    gst_object_unref (sinkpad);
    gst_element_set_state(tee, GST_STATE_PLAYING);
    gst_element_set_state (sink, GST_STATE_PLAYING);
    g_print ("Linked pad %s of demuxer\n", name);
  }

  g_free (name);
}

The no-more-pads signal

Another signal that can be used is no-more-pads. You can check for its existence with your version of GStreamer by using gst-inspect e.g. gst-inspect avidemux. In the callback of that signal you can link new elements to the demuxer using gst_element_link_filtered. Call the function once for each caps. The caps parameter required by the function can be created using gst_caps_new_simple e.g. gst_caps_new_simple ("video/x-vp8", NULL). Again, the state of these new elements needs to be set to GST_STATE_PLAYING.

Here’s how you can register a callback for no-more-pads

  g_signal_connect (demux, "no-more-pads", (GCallback)demux_no_more_pads, NULL);

Here’s a sample callback function.

void
demux_no_more_pads (GstElement* demux, gpointer user_data)
{
  GstCaps *caps;
  GstElement *tee, *sink;

  tee = gst_element_factory_make ("tee", "videotee");
  sink = gst_element_factory_make ("fakesink", "videosink");
  gst_bin_add_many (GST_BIN (pipeline), tee, sink, NULL);
  gst_element_link (tee, sink);
  caps = gst_caps_new_simple ("video/x-vp8", NULL);
  gst_element_link_filtered (demux, tee, caps);
  gst_element_set_state(tee, GST_STATE_PLAYING);
  gst_element_set_state(sink, GST_STATE_PLAYING);

  tee = gst_element_factory_make ("tee", "audiotee");
  sink = gst_element_factory_make ("fakesink", "audiosink");
  gst_bin_add_many (GST_BIN (pipeline), tee, sink, NULL);
  gst_element_link (tee, sink);
  caps = gst_caps_new_simple ("audio/x-vorbis", NULL);
  gst_element_link_filtered (demux, tee, caps);
  gst_element_set_state(tee, GST_STATE_PLAYING);
  gst_element_set_state(sink, GST_STATE_PLAYING);
}

Debugging

As usual, if you have any issues you need to troubleshoot with your pipeline, you can try setting the environment variable GST_DEBUG to 5. GStreamer and its elements will print copious amounts of information as they execute.

export GST_DEBUG=5

Port 443 (HTTPS)


I was testing the TCP relay mechanism using a VM on the Amazon EC2 and came across an interesting realization regarding port 443. My network at work blocks all ports except 80 (HTTP) and 443 (HTTPS). I tried relaying using port 80 and realized that they also have a squid instance that caches data, which basically restricts any access through port 80 to purely HTTP requests. I then decided to relay over port 443 and it worked! So, if you ever want to use a port to get out of even a tightly secured network, try port 443.

For the curious who may be asking themselves as to how I could access Amazon EC2 from my work network, I used the excellent VPNUK service. It only works from my work network over TCP and port 443.

A TCP relay mechanism with Node.js


These TCP relay scripts can be used to expose any TCP/IP service running behind a NAT. This includes services that use HTTP, SSH, and so on. The latest source code is available at GitHub. Use npm to install.

The relay server script is meant to be executed on the server visible on the internet as follows

tcprelays --relayPort port --servicePort port

relayPort is the port where the relay server will listen for incoming connections from the relay client. servicePort is the port where external clients can connect to the service exposed through the relay.

The relay client script is meant to be executed on a machine behind a NAT as follows

tcprelayc --host host --port port --relayHost host --relayPort port [--numConn count]

host is any server visible to the machine behind the NAT, it can also be localhost. port is the port of the service you want to expose through the relay. relayServer is the host or IP address of the server visible on the internet, and already executing the relay server script. relayPort is the port where this script will connect with the relay server. numConn is the number of unused sockets relay client maintains with the relay server. As soon as it detects data activity on a socket, relay client establishes another connection.

If you’re using HTTP/S, use a reverse proxy such as http-proxy, between the relay client and the local service e.g.

var httpProxy = require('http-proxy');
httpProxy.createProxyServer({target:'http://host:port'}).listen(port);

One external module is used in these scripts to parse commands line options. It can be installed using npm as follows

sudo npm -g install optimist@latest

Programming Interface

Create and start a relay client thus

var relayClient = require("node-tcp-relay")
var newRelayClient = relayClient.createRelayClient("hostname", 8080, "relayserver", 10080, 1);

End relay client

newRelayClient.end();

Create and start a relay server thus

var relayServer = require("node-tcp-relay");
var newRelayServer = relayServer.createRelayServer(10080, 10081);

End relay server

newRelayServer.end();    

Desktop remoting for Ubuntu using x11vnc


x11vnc is a neat way to remote an existing X11 session to another machine. You can use a regular VNC client on the remote machine to interact with your Ubuntu Desktop.

Installation

Execute the following command to install x11vnc

sudo apt-get install x11vnc

Then, execute x11vnc thus

x11vnc

VNC Client

There are several VNC clients you can choose from but I prefer the TightVNC Viewer. Once installed, point the viewer to host_name:display_number or host_ip:display_number e.g. 192.168.0.2:0. The display number can be obtained from a terminal window on Ubuntu thus

echo $DISPLAY

Some neat features to try out:

  • Send keyboard commands like Alt+Tab or Ctrl+Alt+Del
  • Copy to or paste from the remote machine

WebKit/GTK+ on Ubuntu


Ubuntu has fairly good independent browsers, FireFox and Chromium come to mind. If you want a cutting-edge WebKit port that is deeply integrated with the Ubuntu Desktop, WebKit/Gtk+ is worth taking a look at. Its UI is based on Gtk and the multimedia playback is based on GStreamer.

Obtain source code

Source can be downloaded from webkitgtk.org. There are archives of the stable and development trees there.

Build

You’ll need to install the following packages (for the development tree at version 1.5.1) and any dependencies. Other packages may need to be installed for newer versions if ./configure fails.

  • bison
  • flex
  • gperf
  • libjpeg8-dev
  • libicu-dev
  • libgail-dev
  • libxt-dev
  • libsoup2.4-dev
  • libsqlite3-dev
  • libxslt1-dev

Then, run the following commands

tar xvzf webkit-1.5.1.tar.gz
cd webkit-1.5.1
./configure
make

Test

Run the browser using a simple helper program

./Programs/GtkLauncher

Open http://www.youtube.com/html5. Opt in to the html5 trial. Watch any video. WebM videos should play nicely on Ubuntu 11.04.

Quick Hack – play video without start up lag

I am experimenting with HTTP streaming for real time communication (RTC). For that to work with no lag, I need the video tag to start playing the video immediately. I commented all lines of code that instructs the pipeline to enter the state GST_STATE_PAUSED in file ./Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp. For instance, this line in method MediaPlayerPrivateGStreamer::pause()

    if (changePipelineState(GST_STATE_PAUSED))
        LOG_VERBOSE(Media, "Pause");

Quick Hack – GtkLauncher in fullscreen with no address and status bars

Edit the source file ./Tools/GtkLauncher/main.c. In function createWindow, comment out the two lines shown within comments below

    /*gtk_box_pack_start(GTK_BOX(vbox), createToolbar(uriEntry, webView), FALSE, FALSE, 0);*/
    gtk_box_pack_start(GTK_BOX(vbox), createBrowser(window, uriEntry, statusbar, webView), TRUE, TRUE, 0);
    /*gtk_box_pack_start(GTK_BOX(vbox), statusbar, FALSE, FALSE, 0);*/

Then, in function main, add the following line before the call to gtk_main

    gtk_window_fullscreen((GtkWindow*)main_window);

Ubuntu package management from the command line


These are the commands that I mostly use to manage packages from the CLI. See man command for more detailed help.

apt-cache

apt-cache search text to search for packages containing the specified text. Listed packages can be installed using apt-get. To see details about a package use apt-cache showpkg packages.

apt-add-repository

apt-add-repository url to add a new repository source

apt-add-repository -r url to remove

apt-get

sudo apt-get install package_name to install or upgrade a package

sudo apt-get update to update package cache from active respository sources, usually after a apt-add-repository

dpkg

dpkg -l to list all packages installed and their versions. pipe to grep to filter.

sudo dpkg -i deb_package to install from a package archive

dpkg -S file to discover which package a file belongs to
sudo dpkg -r --force-depends to forcefully remove a broken package e.g. sudo dpkg -r --force-depends libssl1.0.0

Moving to Blogger


I plan moving this blog to Blogger at tewarid.blogspot.com, but the conversion utility messes up code within pre tags. Giving up on the transition for now.