In general, video files are made up of two main parts: 1) the actual audio and video data, which is encoded through through the use of codecs (most commonly H.264, VP8/VP9, Theora, AAC, Vorbis, Opus, I think) and 2) some „packaging” called container format, which is generally responsible for organising both video and audio into a single file as well as providing some meta data (common examples: MP4, AVI, WebM, Matroska). Accordingly, the main libraries to use when working with video files using FFmpeg are libavcodec and libavformat, which deal with those two parts, respectively. These are the only two libraries which you absolutely need to deal with directly when working with common video files. The other libraries perform various related functions:
- libavutil is a general utility library containing some math functions and other common stuff
- libavdevice is for working with devices such as audio capture and playback devices. I haven’t really looked into it, though
- libavfilter provides a filter graph for A/V data (think ffmpeg -filter / -filter_complex options)
- libswscale is for software-side rescaling and color space conversion of video data
- libswresample is for software-side audio resampling, mixing and sample format conversion
- Apparently there’s also libpostproc but it’s practically undocumented and I have no idea what exactly it does
Some fundamental terms you will come across:
- stream: audio and video data in container formats are organised into those. A typical video file has one video and one audio stream
- packet: contains one or more frames of a stream
- frame: pixel data of a single picture (in video streams) or a bunch of samples (in audio streams)
Most other fundamental terms (framerate, channels, samples, etc.) should be familiar to you if you know a little about digital media. Lastly, have a look at the API documentation and the examples for more orientation. I hope you find this useful for getting started.