5/27/2021

The Internals of VLC and How it works

VLC or VideoLAN client, retconned to just VLC.

It is (or was) planned as a media player in a suite of tools for sharing audio video media files from a Satellite downlink on a University campus in France.

It was a student led effort that evolved and led to it being open sourced in the year 2000.

Because not all students used a Mac or a PC or Linux desktop they chose to design it such that it was portable across all those platforms. It has since been ported to iOS and Android as well.

Basically the terminology changes depending on which era of programming, or desktop PC platorm your currently on but its a collection of 'mini" programs called "Modules" linked together in a chain by a "Core" interface being used.

Its mostly a "loader" with a user interface stuck on the front.

The Interface can be a command line string of commands like in a command processor script, or it can be a GUI user interface for a windowing system like wxWidgets, or Qt or Cocoa.

In the background however, once its started, its just making calls to load and start up "modules" and pass them commands parsed from the user instructions to configure them.

Its very similar to the way a GraphEdit Graph works in DirectShow to load "filters" but it does not use "filters" to achieve its goals. People often assume it should expose a filter graph using the graph builder interface on Windows.. but it does not. Instead all of its functions are handled by a series of monolithic "modules" which accept commands from their own type of user interfaces.

So basically VLC was not designed "only for Windows" and does not adopt the DirectShow method of creating filter graphs for performing its functions.

At its center VLC is a ("muxer") which means it takes audio and video streams in, performs some things on them and blends them together and then shoves them out in a new format. While its doing that it can also split off a duplicate stream and "Display" what is currently going through the stream.

Initially it was intended to accept UDP and TCP streams, later reading a DVD device or .ISO file were added, then taking in the Output from a Capture device or a TV Tuner.. these were all added one at a time.. to bring it where it is today.

Just like splitting off the Display duplicate stream, the Record button on the advanced User interface can direct a duplicate stream of the Input data straight to a file, which is automatically named and stored in a pre-configured directory for videos. No processing is performed on a Stream "Capture" as a Record file.. which is rather the exception than the rule with VLC.. and a default Capture/Record processing cannot be configured from the GUI interface.

Separate from a ["Capture/Record"] is a ["Capture/Convert" or "Save"].

Using the "Convert"  button instead of "Play" (which "Play" starts a stream already configured and stored in the Playlist, or completes the process of adding a stream to the stream list and then starts playing it) spawns a Wizard for configuring an Input and selecting a predefined "Profile" which will perform processing on a twin set of audio and video streams to mux them before sending them to a final directory and file destination.

Profiles are preconfigured as VLC default installed, but new ones can easily be created.

The same Convert "Wizard" can be used to complete a File, DVD, Network or Capture Card stream along with its mux processing profile.

The "Stream" button is basically the same thing but tailored more towards "re-streaming" as fast as possible .. possibly without "muxing"

The GUI user interface layers a conceptual way of looking at the the command line way of building up a 'filter graph' in DirectShow terminology even though.. it is not a DirectShow filter graph.. its basically the same thing. "Conceptually"

The VLC command line invocation does not replicate the GUI user interface, the GUI is task based oriented. Where the command line better represents the actual process of building up a single minded, single task process of configuring a input source, configuring the modules that will operate on those source with "VLC filters" and then multiplexing them into a single stream and delivering them to a file or other destination.

Because of the way VLC is built, and its core is libavc.. its sometimes desired to "use" VLC as a DirectShow "filter" itself.

Doing this is possible.

Sensorray opensourced a VideoLAN client bridge

The DirectShow filter appears as a Sink and is preconfigured to save a stream as a processed mux file, currently only as an MPEG1 or MPEG2 file.