Tongfamily (Richtong.com Sync)

Well as I’m kind of someone who is knows how to use Play Doh trying to build the Empire State Building, but in a week or so of hacking and understanding Gstreamer. Here are some of the conclusions:

1. Wow, this thing really has no documentation. There is a huge command reference manual, but really the tutorial that covers how the thing works and is structured isn’t there.
2. The main things to understand are that it is a tool for testing, but in fact used for lots of scripting, so it pays to learn it.
3. It is fundamentally about a pipe, so everything has a source and a sink. The simplest possible pipe looks like a command line where ! is used to represent the idea of a pipe |
4. Note that you launch it with an included version number. gst-launch-0.10 are very old, you want at least 1.0. The stable version is 1.2, but 1.4 has openGL acceleration but is very new

gst-launch-1.0 fakesrc ! fakesink

1. There are many common places to get things and each filter can take a bunch of parameters. It is nearly impossible to figure out what the parameters are, so you have to look through, but in general a parameter looks like

gst-launch-1.0 filesrc location=/home/rich/videos/launch.mp4 ! fakesink

That is, it is parameter=value syntax. One of the strange things is that gstreamer allows casting of values and it in fact tries to figure out values. So you will see lots of code this is trying to get parentheses through bash:

gst-launch-1.0 filesrc location=(string)/home/rich/videos/launch.mp4 ! fakesink

Or people manage the quoting to make this work. The big problem here of course is that with bash variables, everything they are evaluated, you lose a level of quoting so you can just assign the string above to variable. Most of time, you don’t need to worry about this and just assume gat-launch will guess properly.

1. The thing has thousands of filters, but there are some special ones that are like the universal pocket protectors of the gstreamer world, so the following one liner plays just about everything:

gst-launch-1.0 filesrc location=/home/rich/videos/launch.mp4 ! decodebin ! autovideosink

Here are some of the most useful
– decodebin. This looks at the input and just does decoding
– videoconvert. This takes any video format on the input and converts it to what the downstream folks want.
– videoscale. Same idea with dimentsions
– playbin. This is thing breaks the rule, it is both a source and sync and will play to the active output.
– autovideosink. This figures out how to display to your screen
– filesrc. Reads from your file system
– fakesink. This is most useful for debugging, when you ask it to dump output as text with fakesink dump=

1. Debugging this stuff is really hard. They have 5 levels of logging and you can set each chain in the pipeline with different debugging. This normally suffices as parameters where you have 3 which means print warnings, -v for verbose and -e means that when you Ctrl-C it, gstreamer will attempt to gracefully end the output.

gstlaunch-1.0 —gst-debug-level 3 -v -e fakesrc ! fakesink

1. Gstreamer will go on forever, this isn’t a problem when the source is a file, but what if you are just testing, the num-buffers=100 is very useful. You say you want to see 100 and then the pipeline ends.

# This will eventually end
gst-launch-1.0 fakesrc num-buffers=100 ! fakesink

1. Capabilities. There is one filter which really looks weird syntactically, basically, gstreamer sometimes can’t guess what a stream is, so you have to give it a hint with something called a get capability. This is a comma delimited thing that looks like a pipe element but is really a list. So for instance, something coming out of a shared memory pipe might be raw bits with no header information and it might go out the same way, so you need capabilities and then do the right conversion and scaling. The format is a 4CC standard four character descriptor of the byte format so AGBR means alpha, green, blue and then red bits

gst-launch-1.0 -e -v shmsrc pipe=/tmp/input-pipe ! video/x-raw, width=1920, height=1080, format=AGBR
! videoconvert ! videoscale
! video/x-raw, width=192, height=108, format=RGB ! shmsink pipe=/tmp/output-pipe

1. A simple pipe isn’t very flexible, so you can actually set up an arbitrarily complex graph with tee (a la unix) to let muliple pipes see the same input and also merging so an element can see multiple pipes as input. The syntax is pretty confusing, so the best thing to do is to use new lines as much as you can like. Also note that you can name things for clarity.

The syntax to indicate a tee is that one leg continues on while there is a space indicating the end of a pipe and then you get a line in essence for every new tee.

gst-launch-1.0 fakesrc ! tee name=hello-dolly ! fakesink name=first-fake
hello-dolly. ! fakesink name=second-fake
hello-dolly. ! fakesink name=third-fake

Title
Grokking Gstreamer Grokking Gstreamer
Content
Well as I’m kind of someone who is knows how to use Play Doh trying to build the Empire State Building, but in a week or so of hacking and understanding Gstreamer. Here are some of the conclusions: Well as I’m kind of someone who is knows how to use Play Doh trying to build the Empire State Building, but in a week or so of hacking and understanding Gstreamer. Here are some of the conclusions:

1. Wow, this thing really has no documentation. There is a huge command reference manual, but really the tutorial that covers how the thing works and is structured isn’t there.
2. Wow, this thing really has no documentation. There is a huge command reference manual, but really the tutorial that covers how the thing works and is structured isn’t there.
3. The main things to understand are that it is a tool for testing, but in fact used for lots of scripting, so it pays to learn it.
4. The main things to understand are that it is a tool for testing, but in fact used for lots of scripting, so it pays to learn it.
5. It is fundamentally about a pipe, so everything has a source and a sink. The simplest possible pipe looks like a command line where ! is used to represent the idea of a pipe |
6. It is fundamentally about a pipe, so everything has a source and a sink. The simplest possible pipe looks like a command line where ! is used to represent the idea of a pipe |
7. Note that you launch it with an included version number. gst-launch-0.10 are very old, you want at least 1.0. The stable version is 1.2, but 1.4 has openGL acceleration but is very new
8. Note that you launch it with an included version number. gst-launch-0.10 are very old, you want at least 1.0. The stable version is 1.2, but 1.4 has openGL acceleration but is very new

gst-launch-1.0 fakesrc ! fakesink

gst-launch-1.0 fakesrc ! fakesink

1. There are many common places to get things and each filter can take a bunch of parameters. It is nearly impossible to figure out what the parameters are, so you have to look through, but in general a parameter looks like
2. There are many common places to get things and each filter can take a bunch of parameters. It is nearly impossible to figure out what the parameters are, so you have to look through, but in general a parameter looks like

gst-launch-1.0 filesrc location=/home/rich/videos/launch.mp4 ! fakesink

gst-launch-1.0 filesrc location=/home/rich/videos/launch.mp4 ! fakesink

That is, it is parameter=value syntax. One of the strange things is that gstreamer allows casting of values and it in fact tries to figure out values. So you will see lots of code this is trying to get parentheses through bash: That is, it is parameter=value syntax. One of the strange things is that gstreamer allows casting of values and it in fact tries to figure out values. So you will see lots of code this is trying to get parentheses through bash:

gst-launch-1.0 filesrc location=(string)/home/rich/videos/launch.mp4 ! fakesink

gst-launch-1.0 filesrc location=(string)/home/rich/videos/launch.mp4 ! fakesink

Or people manage the quoting to make this work. The big problem here of course is that with bash variables, everything they are evaluated, you lose a level of quoting so you can just assign the string above to variable. Most of time, you don’t need to worry about this and just assume gat-launch will guess properly. Or people manage the quoting to make this work. The big problem here of course is that with bash variables, everything they are evaluated, you lose a level of quoting so you can just assign the string above to variable. Most of time, you don’t need to worry about this and just assume gat-launch will guess properly.

1. The thing has thousands of filters, but there are some special ones that are like the universal pocket protectors of the gstreamer world, so the following one liner plays just about everything:
2. The thing has thousands of filters, but there are some special ones that are like the universal pocket protectors of the gstreamer world, so the following one liner plays just about everything:

gst-launch-1.0 filesrc location=/home/rich/videos/launch.mp4 ! decodebin ! autovideosink

gst-launch-1.0 filesrc location=/home/rich/videos/launch.mp4 ! decodebin ! autovideosink

Here are some of the most useful Here are some of the most useful
– decodebin. This looks at the input and just does decoding – decodebin. This looks at the input and just does decoding
– videoconvert. This takes any video format on the input and converts it to what the downstream folks want. – videoconvert. This takes any video format on the input and converts it to what the downstream folks want.
– videoscale. Same idea with dimentsions – videoscale. Same idea with dimentsions
– playbin. This is thing breaks the rule, it is both a source and sync and will play to the active output. – playbin. This is thing breaks the rule, it is both a source and sync and will play to the active output.
– autovideosink. This figures out how to display to your screen – autovideosink. This figures out how to display to your screen
– filesrc. Reads from your file system – filesrc. Reads from your file system
– fakesink. This is most useful for debugging, when you ask it to dump output as text with fakesink dump= – fakesink. This is most useful for debugging, when you ask it to dump output as text with fakesink dump=

1. Debugging this stuff is really hard. They have 5 levels of logging and you can set each chain in the pipeline with different debugging. This normally suffices as parameters where you have 3 which means print warnings, -v for verbose and -e means that when you Ctrl-C it, gstreamer will attempt to gracefully end the output.
2. Debugging this stuff is really hard. They have 5 levels of logging and you can set each chain in the pipeline with different debugging. This normally suffices as parameters where you have 3 which means print warnings, -v for verbose and -e means that when you Ctrl-C it, gstreamer will attempt to gracefully end the output.

gstlaunch-1.0 —gst-debug-level 3 -v -e fakesrc ! fakesink

gstlaunch-1.0 —gst-debug-level 3 -v -e fakesrc ! fakesink

1. Gstreamer will go on forever, this isn’t a problem when the source is a file, but what if you are just testing, the num-buffers=100 is very useful. You say you want to see 100 and then the pipeline ends.
2. Gstreamer will go on forever, this isn’t a problem when the source is a file, but what if you are just testing, the num-buffers=100 is very useful. You say you want to see 100 and then the pipeline ends.

# This will eventually end

# This will eventually end
gst-launch-1.0 fakesrc num-buffers=100 ! fakesink       gst-launch-1.0 fakesrc num-buffers=100 ! fakesink

1. Capabilities. There is one filter which really looks weird syntactically, basically, gstreamer sometimes can’t guess what a stream is, so you have to give it a hint with something called a get capability. This is a comma delimited thing that looks like a pipe element but is really a list. So for instance, something coming out of a shared memory pipe might be raw bits with no header information and it might go out the same way, so you need capabilities and then do the right conversion and scaling. The format is a 4CC standard four character descriptor of the byte format so AGBR means alpha, green, blue and then red bits
2. Capabilities. There is one filter which really looks weird syntactically, basically, gstreamer sometimes can’t guess what a stream is, so you have to give it a hint with something called a get capability. This is a comma delimited thing that looks like a pipe element but is really a list. So for instance, something coming out of a shared memory pipe might be raw bits with no header information and it might go out the same way, so you need capabilities and then do the right conversion and scaling. The format is a 4CC standard four character descriptor of the byte format so AGBR means alpha, green, blue and then red bits

gst-launch-1.0 -e -v shmsrc pipe=/tmp/input-pipe ! video/x-raw, width=1920, height=1080, format=AGBR

gst-launch-1.0 -e -v shmsrc pipe=/tmp/input-pipe ! video/x-raw, width=1920, height=1080, format=AGBR
! videoconvert ! videoscale
    ! videoconvert ! videoscale ! videorate
! video/x-raw, width=192, height=108, format=RGB ! shmsink pipe=/tmp/output-pipe        ! video/x-raw, width=192, height=108, format=RGB ! shmsink pipe=/tmp/output-pipe

1. A simple pipe isn’t very flexible, so you can actually set up an arbitrarily complex graph with tee (a la unix) to let muliple pipes see the same input and also merging so an element can see multiple pipes as input. The syntax is pretty confusing, so the best thing to do is to use new lines as much as you can like. Also note that you can name things for clarity.
2. A simple pipe isn’t very flexible, so you can actually set up an arbitrarily complex graph with tee (a la unix) to let muliple pipes see the same input and also merging so an element can see multiple pipes as input. The syntax is pretty confusing, so the best thing to do is to use new lines as much as you can like. Also note that you can name things for clarity.

The syntax to indicate a tee is that one leg continues on while there is a space indicating the end of a pipe and then you get a line in essence for every new tee.
The syntax to indicate a tee is that one leg continues on while there is a space indicating the end of a pipe and then you get a line in essence for every new tee. As a convention, I like to use initial capitals to indicate names like this

gst-launch-1.0 fakesrc ! tee name=hello-dolly ! fakesink name=first-fake

gst-launch-1.0 fakesrc ! tee name=Tee-out ! fakesink name=First-fake
 hello-dolly. ! fakesink name=second-fake
     Tee-out. ! fakesink name=Second-fake
 hello-dolly. ! fakesink name=third-fake
     Tee-out. ! fakesink name=Third-fake

1. Now there are multiplexer which can take multiple inputs. A classic mux is one that interleaves audio and video like say mastrokamux which takes audio and video and merges them. This is where naming helps. Basically, if you create a name, you can use that as the end of a pipe, so if you want to merge say 1 video and 2 audio feeds into an MKV file, it would look like And again, there are really three pipes here all separate by white space

gst-launch-1.0 videotestsrc ! mastrokamux name=MkvMux
         audiotestsrc ! MkvMux
         audiotestsrc pattern=sine ! MkvMux
        

1. So putting it all together for extra credit is an audio source from Linux Pulse audio system that tees itself and a video source for V4L2 linux device first encodes itself as X.264 and compensates for different video format (con. These then fee two mux which create difference file formats for MKV and for flash (FLV) simultaneously. What is happening is that we have four declarations of pipelines for the audio and video and then for the multiplexor. Note that we are using the `queue` super command because gstreamer is real time. Queues ensure that we don’t have dropped frames

gst-launch-1.0 -e -v
v4l2src ! videoconvert ! videorate ! x264enc ! tee name=VideoSrc
pulsesrc ! ffenc-aac ! tee name=AudioSrc
matrovskamux name=MkvSink ! filesink location=file.mkv
flvmux name=FlvSink ! filesink location=file.flv
VideoSrc. ! queue ! MkvSink.
AudioSrc. ! queue ! MkvSink.
VideoSrc. ! queue ! FlvSink.
AudioSrc. ! queue ! FlvSink.

This syntax I think is much clearer than the normal one although more wordy. Normally, you save a three lines because you continue the pipe onwards, so the below is equivalent, but I find it much more obscure when you look at it again because it isn’t very symmetric:

gst-launch-1.0 -e -v
v4l2src ! videoconvert ! videorate ! x264enc ! tee name=VideoSrc ! queue ! mastrovskamux name=MkvMux ! filesink location=file.mkv
pulsesrc ! ffenc-aac ! tee name=AudioSrc ! queue ! flvmux name=FlvMux ! filesink location=file.flv
VideoSrc. ! queue ! FlvMux.
AudioSrc. ! queue ! MkvMux.