GStreamer pipeline with Tee


The tee element is useful to branch a data flow so that it can be fed to multiple elements. In this post we’ll use the tee element to split live test video and audio sources after encoding, mux the output as live WebM, and stream the result using the tcpclientsink element. This procedure can be repeated several times to stream to multiple clients, the only limit being the CPU and bandwidth. By encoding only once we avoid taxing the CPU, encoding being the most intensive operation it must perform. The code presented below has been tested with GStreamer 0.10.32.

Creating a pipeline with Tee

Example C code that creates a dynamic GStreamer pipeline using tee follows

  GstElement *pipeline, *videosrc, *colorspace, *videoenc,
    *videotee, *audiosrc, *conv, *audioenc, *audiotee;

  // Create elements
  pipeline = gst_pipeline_new ("tcp-streamer");
  videosrc = gst_element_factory_make ("videotestsrc", "videosrc");
  colorspace = gst_element_factory_make ("ffmpegcolorspace", "colorspace");
  videoenc = gst_element_factory_make ("vp8enc", "videoenc");
  videotee = gst_element_factory_make ("tee", "videotee");
  audiosrc = gst_element_factory_make ("autoaudiosrc", "audiosrc");
  conv = gst_element_factory_make ("audioconvert", "converter");
  audioenc = gst_element_factory_make ("vorbisenc", "audioenc");
  audiotee = gst_element_factory_make ("tee", "audiotee");

  if (!pipeline || !videosrc || !colorspace || !videoenc
    || !videotee || !audiosrc || !conv || !audioenc || !audiotee) {
    g_printerr ("One element could not be created.\n");
    return NULL;
  }

  // set the properties of elements
  g_object_set (G_OBJECT (videosrc), "horizontal-speed", 1, NULL);
  g_object_set (G_OBJECT (videosrc), "is-live", 1, NULL);
  g_object_set (G_OBJECT (videoenc), "speed", 2, NULL);

  // add all elements to the pipeline
  gst_bin_add_many (GST_BIN (pipeline),
    videosrc, colorspace, videoenc, videotee, audiosrc, conv,
    audioenc, audiotee, NULL);

  // link the elements together
  gst_element_link_many (videosrc, colorspace, videoenc,
    videotee, NULL);
  gst_element_link_many (audiosrc, conv, audioenc,
    audiotee, NULL);

Branching from a Tee on a running Pipeline

We create a sub-pipeline using a bin. Creating a new branch from the tee, on a running pipeline, can be achieved thus

  GstElement *bin, *videoq, *audioq, *muxer, *sink,
    *videotee, *audiotee;

  GstPad *sinkpadvideo, *srcpadvideo, *sinkpadaudio, *srcpadaudio;

  bin = gst_bin_new (NULL);
  videoq = gst_element_factory_make ("queue2", NULL);
  audioq = gst_element_factory_make ("queue2", NULL);
  muxer = gst_element_factory_make ("webmmux", NULL);
  sink = gst_element_factory_make ("tcpclientsink", NULL);

  if (!bin || !videoq || !audioq || !muxer || !sink) {
    g_printerr ("One element could not be created.\n");
    return FALSE;
  }

  g_object_set (G_OBJECT (muxer), "streamable", 1, NULL);

  g_object_set (G_OBJECT (sink), "port", port,
    "host", "localhost", NULL);

  gst_bin_add_many (GST_BIN (bin), videoq, audioq,
    muxer, sink, NULL);

  // link src pad of video queue to sink pad of muxer
  srcpadvideo = gst_element_get_static_pad(videoq, "src");
  sinkpadvideo = gst_element_get_request_pad(muxer, "video_%d");
  gst_pad_link(srcpadvideo, sinkpadvideo);

  // link src pad of audio queue to sink pad of muxer
  srcpadaudio = gst_element_get_static_pad(audioq, "src");
  sinkpadaudio = gst_element_get_request_pad(muxer, "audio_%d");
  gst_pad_link(srcpadaudio, sinkpadaudio);

  gst_element_link(muxer, sink);

  // Create ghost pads on the bin and link to queues
  sinkpadvideo = gst_element_get_static_pad(videoq, "sink");
  gst_element_add_pad(bin, gst_ghost_pad_new("videosink", sinkpadvideo));
  gst_object_unref(GST_OBJECT(sinkpadvideo));
  sinkpadaudio = gst_element_get_static_pad(audioq, "sink");
  gst_element_add_pad(bin, gst_ghost_pad_new("audiosink", sinkpadaudio));
  gst_object_unref(GST_OBJECT(sinkpadaudio));

  // set the new bin to PAUSE to preroll
  gst_element_set_state(bin, GST_STATE_PAUSED);

  // Request source pads from tee and sink pads from bin
  videotee = gst_bin_get_by_name (GST_BIN(pipeline), "videotee");
  srcpadvideo = gst_element_get_request_pad(videotee, "src%d");
  sinkpadvideo = gst_element_get_pad(bin, "videosink");
  audiotee = gst_bin_get_by_name (GST_BIN(pipeline), "audiotee");
  srcpadaudio = gst_element_get_request_pad(audiotee, "src%d");
  sinkpadaudio = gst_element_get_pad(bin, "audiosink");

  // Link src pad of tees to sink pads of bin
  gst_bin_add(GST_BIN(pipeline), bin);
  gst_pad_link(srcpadvideo, sinkpadvideo);
  gst_pad_link(srcpadaudio, sinkpadaudio);

  gst_element_set_state (pipeline, GST_STATE_PLAYING);

Removing the branch from a running pipeline

The following code illustrates how to remove the sub-pipeline.

  //gst_element_set_state (pipeline, GST_STATE_PAUSED);
  // pause pipeline if no more bins left
  gst_element_set_state (bin, GST_STATE_NULL);

  gst_pad_unlink(srcpadvideo, sinkpadvideo);
  gst_pad_unlink(srcpadaudio, sinkpadaudio);

  gst_element_remove_pad(videotee, srcpadvideo);
  gst_element_remove_pad(audiotee, srcpadaudio);

  gst_bin_remove(GST_BIN(pipeline), bin);

  //gst_element_set_state (pipeline, GST_STATE_PLAYING);
  // resume pipeline if there are bins left

For the curious, I cache the above pointers in a GHashTable using port number as the key.

7 thoughts on “GStreamer pipeline with Tee

  1. Hi, I’m a newbie to gstreamer trying to understand tee. I came across this post. However, I’m not quite clear. Can’t the same thing be done without videotee and audiotee here? May be I’m wrong. I really appreciate if you could help me on this

  2. When you linked the source pad of the audio queue to the sink pad of the muxer, why did you have to explicitly build the link? Later, when you link the muxer to the sink, you just did gst_element_link().

    Couldn’t you have done that with the audio queue and video queue? ie,

    gst_element_link(audioq, muxer);
    gst_element_link(videoq, muxer);

    (sorry, I am new to gstreamer and am trying to understand linking)

    1. It has been a while. I am quite sure I tried linking using gst_element_link and it didn’t work then. It may be due to the webmmux element having audio and video capabilities (caps). gst_element_link_filtered may work though.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s