User:Joger/FFmpeg

FFProbe

FFProbe is used to inspect media, for example to view format or stream information. You can use the following command line to view a file's format and stream information. It will print out information about the file's streams, such as video and audio streams, as well as the container format.

ffprobe -v error -show_streams -show_format -print_format json <filename_or_url>

Useful options

• -v error suppresses FFprobe's header and decoding information, so we only see the interesting output information.
• -print_format json makes the output easier to read.
• -select_streams v will only print video streams. It can be replaced by a to show only audio streams or s to show only subtitle streams.

FFMpeg

Low latency streaming

> mediamtx
> ffmpeg -filter_complex ddagrab=video_size=1024x890:output_idx=0:framerate=60,hwdownload,format=bgra -fflags nobuffer -vcodec libx264 -tune zerolatency -f rtsp rtsp://127.0.0.1:8554/webcam.h264
> ffplay rtsp://127.0.0.1:8554/webcam.h264 -fflags nobuffer -flags low_delay -framedrop

Streaming protocols

RTMP

The Real-Time Messaging Protocol (RTMP), a TCP-based protocol renowned for its low latency, was created by Macromedia/Adobe for use with Adobe Flash. Although it was popular until recently, it has not been updated to support new codecs. Traditionally, RTMP was utilized for both the ingestion and distribution of streaming media, but its use has been on the decline. It necessitates an additional browser plugin and an RTMP server. Despite the discontinuation of Flash support, RTMP remains widely used due to its low latency and is still the standard for creating streamed content on platforms like YouTube and Facebook Live.

HTTP

TCP-based streaming offers the broadest reach and is unlikely to be blocked in any location. It eliminates the need for a separate streaming server or browser plugin, thanks to HTML5 video support. Media Source Extensions (MSE) allow browsers to play HLS and MPEG-DASH, commonly used in JavaScript players. While HTTP is the standard on the distribution side for major streaming platforms, it is less common on the ingestion side due to its higher latency compared to RTMP, making it less favored for high-quality content ingestion.

SRT

Secure Reliable Transport (SRT) is a UDP-based protocol that operates without an initial handshake. It is recognized for its low latency and is faster than RTMP. Although it relies on UDP, SRT ensures reliable streaming through its error correction mechanisms. SRT is seldom used on the distribution end as it lacks browser support. However, its popularity is growing on the ingest side due to its low latency, positioning it as a potential replacement for RTMP ingest.

RTP

The Real-Time Transport Protocol (RTP) is a network protocol used for delivering audio and video over IP networks. It is designed to provide end-to-end, real-time transfer of streaming media, accommodating for packet loss, jitter, and out-of-order delivery, which are common in UDP transmissions. RTP is often used in systems that require streaming media, such as telephony, video teleconferencing, and web-based push-to-talk features. It typically runs over User Datagram Protocol (UDP) and is used in conjunction with the RTP Control Protocol (RTCP) for monitoring transmission statistics and quality of service.

RTSP

RTSP, or Real-Time Streaming Protocol, is a network control protocol designed for streaming media systems. It enables the control and delivery of real-time multimedia content, such as live video and audio, across a network. Developed by the Internet Engineering Task Force (IETF) in 1998, RTSP allows users to interact with media streams similarly to how they would with local media players. The protocol's primary function is to establish and control media sessions between endpoints, which can be multimedia servers or clients. RTSP is known for its low-latency performance, making it suitable for applications where real-time playback is crucial, such as live video streaming, teleconferencing, and surveillance.

RTP (Real-Time Transport Protocol) is used for the transport of real-time data, such as audio and video, and is designed to work over UDP/IP. It does not reserve bandwidth or guarantee Quality of Service (QoS). RTSP (Real-Time Streaming Protocol), on the other hand, is a control protocol used for commanding servers to set up, play, pause, or tear down streaming sessions. It is an application-level protocol that works in conjunction with RTP and RSVP to provide complete streaming services over the internet.

WebRTC

WebRTC, short for Web Real-Time Communication, is an open-source project that enables real-time communication (RTC) via application programming interfaces (APIs) in web browsers and mobile applications. It is both an API and a protocol, allowing for secure, bi-directional, real-time communication between two WebRTC agents. The protocol is a set of rules for these agents to negotiate this communication, while the API allows developers to implement the protocol in JavaScript, with other languages and APIs also supporting it. WebRTC is known for its open standard, mandatory encryption, NAT traversal, and sub-second latency, making it a powerful tool for peer-to-peer communication.

HLS

HLS, which stands for HTTP Live Streaming, is a widely-used media streaming protocol that delivers audio and visual content to viewers over the internet. Initially developed by Apple, HLS streams are broken down into smaller, downloadable HTTP files that are delivered using the HTTP protocol. This method is used for both live and on-demand video streaming, providing a flexible and reliable way to distribute content across various devices and network conditions.

DASH

Dynamic Adaptive Streaming over HTTP (DASH), also known as MPEG-DASH, is an adaptive bitrate streaming technique that enables high-quality streaming of media content over the internet from conventional HTTP web servers. Like Apple's HTTP Live Streaming (HLS), it breaks the content into small segments, served over HTTP. Each segment contains a short interval of playback time, and the content is available at various bit rates. An MPEG-DASH client adapts to changing network conditions by selecting the segment with the highest bit rate possible for smooth playback without re-buffering events.

SDP

The Session Description Protocol (SDP) is a format used to describe multimedia communication sessions for the purposes of announcement and invitation. Its primary use is in supporting streaming media applications, such as Voice over IP (VoIP) and video conferencing. SDP itself does not transport any media streams; rather, it is used for negotiation between endpoints to establish network metrics, media types, and other session-related properties, collectively known as a session profile. SDP is designed to be extensible to accommodate new media types and formats.

Container formats

Media container formats are file types that store digital video and audio data, often along with metadata such as subtitles and related information. Examples of media container formats include:

MP4

A widely-used format that supports a variety of codecs. Its advantages include broad compatibility with many devices and systems, and it supports streaming. However, it is subject to patent issues which can limit its use in certain contexts. The MP4 format is broadly supported by all common operating systems, consumer electronic devices, and many software media players, including open source players such as VLC.

Matroska

The Matroska format, commonly known as MKV, is an open-source multimedia container format. It can store an unlimited number of video, audio, picture, or subtitle tracks within a single file. This flexibility makes it a popular choice for combining multiple streams of content, such as movies with several audio tracks and subtitles in different languages. Many modern operating systems include native Matroska capabilities, enabling playback of .mkv files without the need for additional software. Additionally, there are several players and components available for download that support Matroska files across different platforms.

WEBM

An open-source format designed for the web. It offers high-quality video and audio streams and is supported by all major browsers. Its downside is that it's less supported by hardware devices compared to other formats.

WEBP

Despite its name's similarity to WEBM, the WEBP format is not a container format but an image format. Developed by Google, WEBP is a modern image file type designed to provide superior lossless and lossy compression for images on the web. This format enables smaller, richer images that improve web performance. WEBP supports both still and animated images, featuring transparency and animation. Google introduced it in 2010 as an alternative to JPEG, PNG, and GIF formats, with the goal of reducing file size without sacrificing quality. The difference between WEBP and other formats like Matroska or MP4 lies in their purposes: WEBP is for images, while Matroska and MP4 are container formats primarily used for audio and video files.

MP3

Primarily an audio format, MP3 is known for its efficient compression which allows for smaller file sizes. The trade-off is a loss in audio quality compared to lossless formats like FLAC.

AVI

An older format that supports multiple streaming audio and video, but it's less efficient in terms of compression and doesn't support modern codec options as well as newer container formats do.

MOV

Developed by Apple, it's compatible with a wide range of Apple's software and hardware. However, it's not as widely supported on non-Apple devices and may require conversion for use on other platforms.

MPEG

The MPEG container format is a standard digital container format used for the transmission and storage of audio, video, and Program and System Information Protocol (PSIP) data. It is utilized in broadcast systems such as DVB, ATSC, and IPTV. The format encapsulates packetized elementary streams, which include error correction and synchronization features to maintain integrity over degraded communication channels. The MPEG container format is distinct from the MPEG program stream, as it is designed for less reliable transmission environments and can carry multiple programs.

MPEGTS

The MPEG Transport Stream (MPEG-TS, MTS) is a standard digital container format used for the transmission and storage of audio, video, and Program and System Information Protocol (PSIP) data. It is commonly utilized in broadcast systems like DVB, ATSC, and IPTV. MPEG-TS is designed to work in less reliable transmission environments, such as terrestrial or satellite broadcasting, and can carry multiple programs within a single stream. The format encapsulates packetized elementary streams, which include error correction and synchronization features to maintain the integrity of the transmission even when the communication channel is degraded.

FLV

FLV, or Flash Video, is a media container format primarily used to deliver digital video content over the internet. It can hold various types of data within a single file, including audio, video, and text data such as subtitles or captions. The format supports several codecs, like Sorenson Spark and VP6 for video, and MP3 and AAC for audio. Developed by Macromedia, which was later acquired by Adobe Systems, FLV was popular for streaming due to its ability to deliver high-quality video at low bitrates. Despite the decline of Flash, FLV remains relevant for its compatibility with older systems and software.

OGG

The OGG container format is a multimedia file structure that can encapsulate raw compressed data, allowing for the interleaving of audio and video streams within a single file. It is the native format for Xiph.org multimedia codecs and is designed as an open and free-to-use format. OGG supports packet framing, error detection, and periodic timestamps for seeking, which contributes to its efficiency in streaming and processing pipelines.