Scalable Video Coding (SVC) Extension for WebRTC

W3C Working Draft 08 May 2024

Abstract

This document defines a set of ECMAScript APIs in WebIDL to extend the WebRTC specification to enable configuration of encoding parameters for Scalable Video Coding (SVC). Discovery of SVC encoder and decoder capabilities is handled by the Media Capabilities specification.

Status of This Document

This section describes the status of this document at the time of its publication. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

The API is based on preliminary work done in the W3C ORTC Community Group.

This document was published by the Web Real-Time Communications Working Group as a Working Draft using the Recommendation track.

Publication as a Working Draft does not imply endorsement by W3C and its Members.

This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 03 November 2023 W3C Process Document.

This section is non-normative.

This specification extends the WebRTC specification [WEBRTC] to enable configuration of encoding parameters for Scalable Video Coding (SVC). Discovery of SVC encoder and decoder capabilities is handled by the Media Capabilities API [Media-Capabilities].

This specification does not change the behavior of WebRTC objects and methods. Therefore, restrictions relating to Offer/Answer negotiation and encoding parameters remain, as described in [WEBRTC] Section 5.2: "setParameters() does not cause SDP renegotiation and can only be used to change what the media stack is sending or receiving within the envelope negotiated by Offer/Answer."

As a result, this specification can be used to configure encoding parameters within codecs that do not negotiate SVC support within Offer/Answer. The VP8 [RFC6386], VP9 [VP9] and AV1 [AV1] codecs fit within this restriction. Configuraton of temporal scalability can also be supported for the H.264 [ITU-T-REC-H.264] and H.265 [ITU-T-REC-H.265] codecs.

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words MAY, MUST, and SHOULD in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Conformance requirements phrased as algorithms or specific steps may be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to follow, and not intended to be performant.

Implementations that use ECMAScript to implement the APIs defined in this specification MUST implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [WEBIDL], as this specification uses that specification and terminology.

The term "simulcast envelope" is defined in [WEBRTC] Section 5.4.1.

This specification references objects, methods, internal slots and dictionaries defined in [WEBRTC].

For Scalable Video Coding (SVC), the terms "single-session transmission" (SST) and "multi-session transmission" (MST) are defined in [RFC6190]. This specification only supports SST but not MST.

The term "Single Real-time Transport Protocol stream Single Transport" (SRST), defined in [RFC7656] Section 3.7, refers to SVC implementations that transmit all layers within a single transport, using a single Real-time Transport Protocol (RTP) stream and synchronization source (SSRC). The term "Multiple RTP stream Single Transport" (MRST), also defined in [RFC7656] Section 3.7, refers to implementations that transmit all layers within a single transport, using multiple RTP streams with a distinct SSRC for each layer. This specification only supports SRST, not MRST. Codecs with RTP payload specifications supporting SRST include VP8 [RFC7741], VP9 [VP9-PAYLOAD], AV1 [AV1-RTP-SPEC], H.264 [RFC6184] and H.265 [RFC7798].

The term "S mode" refers to a scalability mode in which multiple encodings are sent on the same SSRC. This includes the "S2T1", "S2T1h", "S2T2", "S2T2h", "S2T3", "S2T3h", "S3T1", "S3T1h", "S3T2", "S3T2h", "S3T3" and "S3T3h" scalabilityMode values.

The term Selective Forwarding Middlebox (SFM) is defined in Section 3.7 of [RFC7667].

This specification enables the configuration of encoding parameters for SVC by extending the RTCRtpEncodingParameters dictionary.

WebIDLpartial dictionary RTCRtpEncodingParameters {
             DOMString scalabilityMode;
};

scalabilityMode of type DOMString: A case-sensitive identifier of the scalability mode to be used for this stream. Scalability modes are defined in Section 6.

[WEBRTC] describes error handling in addTransceiver() (Section 5.1) and setParameters() (Section 5.2), including use of RTCError to indicate a "hardware-encoder-error" due to an unsupported encoding parameter, as well as other errors. Implementations utilize RTCError and other errors in the prescribed manner when an invalid scalabilityMode value is provided to setParameters() or addTransceiver().

[WEBRTC] Section 5.1 describes validation of sendEncodings within addTransceiver(). To validate scalabilityMode, add the following steps after step 3 of addTransceiver sendEncodings validation steps:

If sendEncodings contains any encoding with a RTCRtpEncodingParameters.codec value codec exists and where the same encoding's scalabilityMode value is not supported by codec, throw an OperationError.
Else if sendEncodings contains any encoding whose scalabilityMode value is not supported by any codec in the list of implemented send codecs for kind, throw an OperationError.
If RTCRtpEncodingParameters stored in sendEncodings contains more than 1 encoding with an active member with a value of true and sendEncodings contains any encoding whose scalabilityMode value represents an "S mode" and whose active member has a value of true, throw an OperationError.

When the addTransceiver() and setCodecPreferences() methods are called prior to conclusion of the Offer/Answer negotiation, the negotiated codec and its capabilities may not be known. In this situation the scalabilityMode values configured in sendEncodings may not be supported by the eventually negotiated codec. However, an error will only result if the requested scalabilityMode value is invalid for any supported codec, or if mixed simulcast transport is requested.

So as to ensure that the desired scalabilityMode values can be applied, setCodecPreferences() can be used to prefer or only include codecs supporting the desired configuration. For example, if temporal scalability is desired along with spatial simulcast, when addTransceiver() is called, sendEncodings can be configured to send multiple simulcast streams with different resolutions, with each stream utilizing temporal scalability. If only the VP8, VP9 and AV1 codec implementations support temporal scalability, setCodecPreferences() can be used to remove the H.264/AVC codec from the Offer, improving the chances that a codec supporting temporal scalability is negotiated.

When sendEncodings is used to request the sending of multiple simulcast streams using addTransceiver(), an "S mode" cannot be requested. The browser may only be configured to send simulcast encodings with multiple SSRCs and RIDs, or alternatively, to send all simulcast encodings on a single RTP stream. Simultaneously using both simulcast transport techniques is not permitted.

[WEBRTC] Section 5.2 describes validation of parameters within setParameters(). Insert the following conditions under which the operation causes a promise rejected with an InvalidModificationError (step 4) of setParameters validation steps:

If encodings contains any encoding with an existng RTCRtpEncodingParameters.codec value codec, where the same encoding's scalabilityMode value is not supported by codec.
Else if sender.[[SendCodecs]] is empty and encodings contains any encoding whose scalabilityMode value is not supported by any codec in the list of implemented send codecs for kind.
Else if sender.[[SendCodecs]] is not empty and encodings contains an encoding whose scalabilityMode value is not supported by the codec used for the encoding's RTP stream.
If RTCRtpEncodingParameters stored in encodings contains more than one encoding with an active member with a value of true and encodings contains any encoding whose scalabilityMode value represents an "S mode" and whose active member has a value of true.

The "L1T1" scalability mode enables SVC encoding to be turned off using setParameters(). If "L1T1" is set using setParameters() then it will be returned in response to getParameters().

Before the initial negotiation has completed, getParameters() returns the scalabilityMode value for each encoding in encodings, as last set by addTransceiver() or setParameters(). If no scalabilityMode value was provided for an encoding in encodings, or if a value was not successfully set, then getParameters() will not return a scalabilityMode value for that encoding.

After the initial negotiation has completed, getParameters() returns the currently configured scalabilityMode value for each encoding in encodings which had a value before the initial negotiation. This MAY be different from the values requested in addTransceiver() or setParameters(). For example, if the codecs selected during negotiation do not include an encoder supporting the desired scalabilityMode value, the user agent MAY select another value. If the configuration is not satisfactory, setParameters() can be used to change it.

If addTransceiver() or setParameters() did not provide a scalabilityMode value for an encoding in encodings, then after the initial negotiation has completed, getParameters() will not return a scalabilityMode value and the encoder will use the default scalabilityMode of the codec for that encoding's RTP stream. The default scalabilityMode for each codec is implementation dependent. The default scalabilityMode SHOULD be one of the temporal scalability modes (e.g. "L1T1","L1T2","L1T3", etc.).

This section is non-normative.

The [Media-Capabilities] API enables discovery of encoder and decoder support for scalable video coding. scalabilityMode is used to query whether an encoder supports a particular scalabilityMode value. The API indicates whether the scalabilityMode value is "supported", "smooth" and "power efficient".

The [Media-Capabilities] API also provides information on decoder support for spatial scalablity modes. spatialScalability indicates whether a decoder has the ability to support spatial prediction, which requires the ability to use frames of a resolution different than the current resolution as a dependency. If spatialScalability is set to true, the decoder can decode any scalabilityMode value supported by the encoder. If spatialScalability is set to false or is absent, the decoder cannot decode spatial scalability modes, but can can decode all other scalabilityMode values supported by the encoder.

This section is non-normative.

The SFM can provide information on the codecs and scalability modes it can decode by providing its receiver capabilities. After exchanging capabilities, the application can compute the intersection of codecs and scalabilityMode values supported by the browser's RTCRtpSender and the SFM's receiver. This can be used to determine the arguments passed to the browser's addTransceiver() and setParameters() methods.

There are situations where an SFM may only support reception of a subset of codecs and scalability modes. For example, an SFM that parses codec payloads may only support the H.264/AVC codec without scalability and the VP8 codec with temporal scalability. On the other hand, the browser may be able to encode VP8 with temporal scalability, VP9 with temporal and spatial scalability and or H.264/AVC with temporal scalability. In such a situation, an application desiring to use SVC would only be able to encode VP8 with temporal scalability.

Since sending simulcast encodings on a single stream is not negotiated within Offer/Answer, an application using SDP signaling needs to determine whether single stream simulcast transport is supported prior to the Offer/Answer negotiation. This can be handled by having the SFM send it's receiver capabilities to the application prior to Offer/Answer. This allows the application to determine whether single stream simulcast is supported, and if so, what scalability modes the SFM can handle. For example, an SFM that can only support reception of a maximum of 2 simulcast encodings on a single SSRC with the AV1 codec would only indicate support for the "S2T1" and "S2T1h" scalability modes in its receiver capabilities.

For an SFM the supported scalabilityMode values may depend on the negotiated RTP header extensions. For example, if the SFM cannot parse codec payloads (either because it is not designed to do so, or because the payloads are encrypted), then negotiation of an RTP header extension (such as the AV1 Dependency Descriptor defined in Appendix A of [AV1-RTP-SPEC]) could be a prerequisite for the SFM to forward a scalabilityMode value. As a result, the scalabilityMode values supported by an SFM may not be determined until completion of the Offer/Answer negotiation.

The scalabilityMode values supported in this specification, as well as their associated identifiers and characteristics, are provided in the table below. The names of the scalabilityMode values (which are case sensitive) are provided, along with the scalability mode identifiers assigned in [AV1] Section 6.7.5, and links to dependency diagrams provided in Section 10.

While the [AV1] and VP9 [VP9] specifications support all the scalabilityMode values defined in the table, other codec specifications do not. For example, VP8 [RFC6386], H.264 [RFC6184] and H.265 [RFC7798] only support temporal scalability (e.g. "L1T2", "L1T3"). Also VP8 [RFC6386], H.264 [RFC6184] and H.265 [RFC7798] only permit transport of simulcast on distinct SSRCs, so that "S" modes (where multiple encodings are transported on a single RTP stream) are not supported.

Scalability Mode Identifier	Spatial Layers	Resolution Ratio	Temporal Layers	Inter-layer dependency	AV1 scalability_mode_idc
"L1T1"	1		1		N/A
"L1T2"	1		2		SCALABILITY_L1T2
"L1T3"	1		3		SCALABILITY_L1T3
"L2T1"	2	2:1	1	Yes	SCALABILITY_L2T1
"L2T2"	2	2:1	2	Yes	SCALABILITY_L2T2
"L2T3"	2	2:1	3	Yes	SCALABILITY_L2T3
"L3T1"	3	2:1	1	Yes	SCALABILITY_L3T1
"L3T2"	3	2:1	2	Yes	SCALABILITY_L3T2
"L3T3"	3	2:1	3	Yes	SCALABILITY_L3T3
"L2T1h"	2	1.5:1	1	Yes	SCALABILITY_L2T1h
"L2T2h"	2	1.5:1	2	Yes	SCALABILITY_L2T2h
"L2T3h"	2	1.5:1	3	Yes	SCALABILITY_L2T3h
"L3T1h"	3	1.5:1	1	Yes
"L3T2h"	3	1.5:1	2	Yes
"L3T3h"	3	1.5:1	3	Yes
"S2T1"	2	2:1	1	No	SCALABILITY_S2T1
"S2T2"	2	2:1	2	No	SCALABILITY_S2T2
"S2T3"	2	2:1	3	No	SCALABILITY_S2T3
"S2T1h"	2	1.5:1	1	No	SCALABILITY_S2T1h
"S2T2h"	2	1.5:1	2	No	SCALABILITY_S2T2h
"S2T3h"	2	1.5:1	3	No	SCALABILITY_S2T3h
"S3T1"	3	2:1	1	No	SCALABILITY_S3T1
"S3T2"	3	2:1	2	No	SCALABILITY_S3T2
"S3T3"	3	2:1	3	No	SCALABILITY_S3T3
"S3T1h"	3	1.5:1	1	No	SCALABILITY_S3T1h
"S3T2h"	3	1.5:1	2	No	SCALABILITY_S3T2h
"S3T3h"	3	1.5:1	3	No	SCALABILITY_S3T3h
"L2T2_KEY"	2	2:1	2	Yes	SCALABILITY_L3T2_KEY
"L2T2_KEY_SHIFT"	2	2:1	2	Yes	SCALABILITY_L3T2_KEY_SHIFT
"L2T3_KEY"	2	2:1	3	Yes	SCALABILITY_L3T3_KEY
"L2T3_KEY_SHIFT"	2	2:1	3	Yes	SCALABILITY_L3T3_KEY_SHIFT
"L3T1_KEY"	3	2:1	1	Yes
"L3T2_KEY"	3	2:1	2	Yes	SCALABILITY_L4T5_KEY
"L3T2_KEY_SHIFT"	3	2:1	2	Yes	SCALABILITY_L4T5_KEY_SHIFT
"L3T3_KEY"	3	2:1	3	Yes	SCALABILITY_L4T7_KEY
"L3T3_KEY_SHIFT"	3	2:1	3	Yes	SCALABILITY_L4T7_KEY_SHIFT

When proposing a scalabilityMode value, the following principles should be followed:

The proposed scalabilityMode MUST define entries to the table in Section 6, including values for the Scalabilty Mode Identifier, spatial and temporal layers, Resolution Ratio, Inter-layer dependency and the corresponding AV1 scalability_mode_idc value (if assigned).
The Scalability Mode Identifier SHOULD be consistent with the existing naming scheme, which utilizes LxTy to denote a scalabilityMode with x spatial layers using a 2:1 resolution ratio and y temporal layers. LxTyh denotes x spatial layers with a 1.5:1 resolution ratio and y temporal layers. SxTy denotes a scalabilityMode with x simulcast encodings with a 2:1 resolution ratio, with each simulcast encoding containing y temporal layers. SxTyh denotes a 1.5:1 resolution ratio. LxTy_KEY denotes a scalabilityMode with x spatial layers using a 2:1 resolution ratio and y temporal layers in which spatial layers only depend on lower spatial layers at a key frame. LxTy_KEY_SHIFT modes denotes a scalabilityMode with x spatial layers using a 2:1 resolution ratio and y temporal layers in which spatial layers only depend on lower spatial layers at a key frame and subsequent frames have their temporal identifier shifted upward.
A dependency diagram MUST be supplied, in the format provided in Section 10.

This section is non-normative.

This example extends [WEBRTC] Section 7.1 (Example 1) to demonstrate sending three spatial simulcast layers each with three temporal layers, using an SSRC and RID for each simulcast layer. Only the "sendEncodings" attribute is changed from the original example.

Example 1

const signaling = new SignalingChannel(); // handles JSON.stringify/parse
const constraints = {audio: true, video: true};
const configuration = {'iceServers': [{'urls': 'stun:stun.example.org'}]};
let pc;

// call start() to initiate
async function start() {
  pc = new RTCPeerConnection(configuration);

  // let the "negotiationneeded" event trigger offer generation
  pc.onnegotiationneeded = async () => {
    try {
      await pc.setLocalDescription();
      // send the offer to the other peer
      signaling.send({description: pc.localDescription});
    } catch (err) {
      console.error(err);
    }
  };

  try {
    // get a local stream, show it in a self-view and add it to be sent
    const stream = await navigator.mediaDevices.getUserMedia(constraints);
    selfView.srcObject = stream;
    pc.addTransceiver(stream.getAudioTracks()[0], {direction: 'sendonly'});
    pc.addTransceiver(stream.getVideoTracks()[0], {
      direction: 'sendonly',
      sendEncodings: [
        {rid: 'q', scaleResolutionDownBy: 4.0, scalabilityMode: 'L1T3'},
        {rid: 'h', scaleResolutionDownBy: 2.0, scalabilityMode: 'L1T3'},
        {rid: 'f', scalabilityMode: 'L1T3'}
      ]
    });
  } catch (err) {
    console.error(err);
  }
}

signaling.onmessage = async ({data: {description, candidate}}) => {
  try {
    if (description) {
      await pc.setRemoteDescription(description);
      // if we got an offer, we need to reply with an answer
      if (description.type == 'offer') {
        await pc.setLocalDescription();
        signaling.send({description: pc.localDescription});
      }
    } else if (candidate) {
      await pc.addIceCandidate(candidate);
    }
  } catch (err) {
    console.error(err);
  }
};

This is an example with two spatial layers (with a 2:1 ratio) and three temporal layers.

Example 2

let sendEncodings = [
  {scalabilityMode: 'L2T3'}
];

This is an example of mixed codec simulcast, with each simulcast layer having 3 temporal layers.

Example 3

let sendEncodings = [
  {rid: 'q', codec: {clockRate: 90000, mimeType: 'video/AV1'}, scaleResolutionDownBy: 4.0, scalabilityMode: 'L1T3'},
  {rid: 'h', codec: {clockRate: 90000, mimeType: 'video/VP8'}, scaleResolutionDownBy: 2.0, scalabilityMode: 'L1T3'},
  {rid: 'f', codec: {clockRate: 90000, mimeType: 'video/VP8'}, scalabilityMode: 'L1T3'}
];

This is an example with three spatial simulcast layers each with three temporal layers on a single SSRC.

Example 4

let sendEncodings = [
  {scalabilityMode: 'S3T3'}
]

This section is non-normative.

This is an example of encodingInfo(configuration) returned by a browser implementing [WEBRTC] and [Media-Capabilities].

Example 5

const contentType = 'video/VP9';

const configuration = {
  type: 'webrtc',
  video: {
    contentType,
    width: 640,
    height: 480,
    bitrate: 10000,
    framerate: 29.97,
    scalabilityMode: 'L3T3_KEY'
  }
};

try {
  const info = await navigator.mediaCapabilities.encodingInfo(configuration);

  if (!info.supported) {
    console.log(`${contentType} is unsupported.`);
    return;
  }
  console.log(`${contentType} is ${info.smooth || 'NOT '}smooth, and ` +
              `${info.powerEfficient || 'NOT '}power efficient`);
} catch (err) {
  console.error(err, ' caused encodingInfo to fail');
}

This section is non-normative.

This is an example of receiver capabilities returned by an SFM that only supports forwarding of VP8, VP9 and AV1 temporal scalability modes.

Example 6

 "codecs": [
    {
      "clockRate": 90000,
      "mimeType": "video/VP8",
      "scalabilityModes": [
        "L1T1",
        "L1T2",
        "L1T3"
      ]
    },
    {
      "clockRate": 90000,
      "mimeType": "video/VP9",
      "scalabilityModes": [
        "L1T1",
        "L1T2",
        "L1T3"
      ]
    },
    {
      "clockRate": 90000,
      "mimeType": "video/AV1",
      "scalabilityModes": [
        "L1T1",
        "L1T2",
        "L1T3"
      ]
    }
]

This section is non-normative.

This section is non-normative; it specifies no new behaviour, but instead summarizes information already present in other parts of the specification. The privacy considerations for the WebRTC APIs are described in [WEBRTC] Section 13.

In WebRTC, the use of scalable coding tools is not negotiated between peers, so neither supported scalabilityMode values nor decoder support for spatial prediction is exposed in SDP.

By attempting to set scalabilityMode values for each codec using the setParameters() API, an application can determine the values supported by the encoder, by noting which configuration attempts succeed and which ones fail. However, this does not indicate whether a scalabilityMode value is supported by a hardware or software encoder (or both). Since setParameters() is not supported for the RTCRtpReceiver, equivalent experiments cannot be run to determine decoder support.

Since the scalabilityMode values supported by software encoders are typically a superset of those supported in hardware, the information available from these experiments has a high correlation with the browser in use, which is already available to web pages. Once media is flowing, information on performance characteristics or whether a scalabilityMode value is decodable for the codec in use can be obtained, which provides more information on hardware capabilities.

As noted in [Media-Capabilities] Section 3.1, the Media Capabilities API "will likely provide more accurate and consistent information" than is available from the WebRTC-SVC API. Media Capabilities provides information on encoder and decoder capabilities, indicating whether a proposed configuration (including a scalabilityMode value) is "supported", "smooth" and "power efficient". [Media-Capabilities] API also indicates whether the decoder supports spatial prediction. As noted in [Media-Capabilities] Section 3.1, "This information is expected to have a high correlation with other information already available to the web pages as a given class of device is expected to have very similar decoding/encoding capabilities."

This section is non-normative.

This section is non-normative; it specifies no new behaviour, but instead summarizes information already present in other parts of the specification. WebRTC protocol security considerations are described in [RFC8827] and the security and privacy considerations for the WebRTC APIs are described in [WEBRTC] Section 13.

Dependency diagrams for the scability modes defined in this specification are provided below.

L1T1: a single layer — Figure 1 L1T1: 1-layer encoding

L1T2: 2-layer temporal scalability encoding — Figure 2 L1T2: 1-layer spatial and 2-layer temporal scalability encoding

L1T3: 3-layer temporal scalability encoding — Figure 3 L1T3: 1-layer spatial and 3-layer temporal scalability encoding

Figure 4 L2T1 and L2T1h: 2-layer spatial and 1-layer temporal scalability encoding

Figure 5 L2T1_KEY: 2-layer spatial and 1-layer temporal scalability K-SVC encoding

Figure 6 L2T2 and L2T2h: 2-layer spatial and 2-layer temporal scalability encoding

Figure 7 L2T2_KEY: 2-layer spatial and 2-layer temporal scalability K-SVC encoding

L2T2_KEY_SHIFT: 2-layer spatial and 2-layer temporal scalability K-SVC shifted encoding with temporal shift — Figure 8 L2T2_KEY_SHIFT: 2-layer spatial and 2-layer temporal scalability K-SVC encoding with temporal shift

Figure 9 L2T3 and L2T3h: 2-layer spatial and 3-layer temporal scalability encoding

Figure 10 L2T3_KEY: 2-layer spatial and 3-layer temporal scalability K-SVC encoding

L2T3_KEY_SHIFT: 2-layer spatial and 3-layer temporal scalability K-SVC shifted encoding with temporal shift — Figure 11 L2T3_KEY_SHIFT: 2-layer spatial and 3-layer temporal scalability K-SVC encoding with temporal shift

Figure 12 L3T1 and L3T1h: 3-layer spatial and 1-layer temporal scalability encoding

Figure 13 L3T1_KEY: 3-layer spatial and 1-layer temporal scalability K-SVC encoding

Figure 14 L3T2 and L3T2h: 3-layer spatial and 2-layer temporal scalability encoding

Figure 15 L3T2_KEY: 3-layer spatial and 2-layer temporal scalability K-SVC encoding

Figure 16 L3T2_KEY_SHIFT: 3-layer spatial and 2-layer temporal scalability K-SVC with temporal shift

Figure 17 L3T3 and L3T3h: 3-layer spatial and 3-layer temporal scalability encoding

Figure 18 L3T3_KEY: 3-layer spatial and 3-layer temporal scalability K-SVC encoding

Figure 19 L3T3_KEY_SHIFT: 3-layer spatial and 3-layer temporal scalability K-SVC with temporal shift

Figure 20 S2T1 and S2T1h: 2-layer spatial simulcast encoding

Figure 21 S2T2 and S2T2h: 2-layer spatial simulcast and 2-layer temporal scalability encoding

Figure 22 S2T3 and S2T3h: 2-layer spatial simulcast and 3-layer temporal scalability encoding

Figure 23 S3T1 and S3T1h: 3-layer spatial simulcast encoding

Figure 24 S3T2 and S3T2h: 3-layer spatial simulcast and 2-layer temporal scalability encoding

Figure 25 S3T3 and S3T3h: 3-layer spatial simulcast and 3-layer temporal scalability encoding

The editors wish to thank Robin Raymond, Michael Horowitz, Harald Alvestrand, Chris Cunningham, Danil Chapovalov, Florent Castelli and Henrik Boström for their contributions to this specification, which evolved from the ORTC API developed in the W3C ORTC CG.

[infra]: Infra Standard. Anne van Kesteren; Domenic Denicola. WHATWG. Living Standard. URL: https://infra.spec.whatwg.org/
[RFC2119]: Key words for use in RFCs to Indicate Requirement Levels. S. Bradner. IETF. March 1997. Best Current Practice. URL: https://www.rfc-editor.org/rfc/rfc2119
[RFC7656]: A Taxonomy of Semantics and Mechanisms for Real-Time Transport Protocol (RTP) Sources. J. Lennox; K. Gross; S. Nandakumar; G. Salgueiro; B. Burman, Ed.. IETF. November 2015. Informational. URL: https://www.rfc-editor.org/rfc/rfc7656
[RFC7667]: RTP Topologies. M. Westerlund; S. Wenger. IETF. November 2015. RFC. URL: https://datatracker.ietf.org/doc/html/rfc7667
[RFC8174]: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words. B. Leiba. IETF. May 2017. Best Current Practice. URL: https://www.rfc-editor.org/rfc/rfc8174
[WEBIDL]: Web IDL Standard. Edgar Chen; Timothy Gu. WHATWG. Living Standard. URL: https://webidl.spec.whatwg.org/
[WEBRTC]: WebRTC: Real-Time Communication in Browsers. Cullen Jennings; Florent Castelli; Henrik Boström; Jan-Ivar Bruaroey. W3C. 6 March 2023. W3C Recommendation. URL: https://www.w3.org/TR/webrtc/

[AV1]: AV1 Bitstream & Decoding Process Specification. Peter de Rivaz; Jack Haughton. AOM. 8 January 2019. Standard. URL: https://aomediacodec.github.io/av1-spec/av1-spec.pdf
[AV1-RTP-SPEC]: RTP Payload Format For AV1 (v1.0). Alliance for Open Media. Draft Deliverable. URL: https://aomediacodec.github.io/av1-rtp-spec/
[ITU-T-REC-H.264]: H.264 : Advanced video coding for generic audiovisual services. ITU. June 2019. URL: https://www.itu.int/rec/T-REC-H.264
[ITU-T-REC-H.265]: H.265 : High efficiency video coding. ITU. August 2021. URL: https://www.itu.int/rec/T-REC-H.265
[Media-Capabilities]: Media Capabilities. Jean-Yves Avenard. W3C. 12 February 2024. W3C Working Draft. URL: https://www.w3.org/TR/media-capabilities/
[RFC6184]: RTP Payload Format for H.264 Video. Y.-K. Wang; R. Even; T. Kristensen; R. Jesup. IETF. May 2011. Proposed Standard. URL: https://www.rfc-editor.org/rfc/rfc6184
[RFC6190]: RTP Payload Format for Scalable Video Coding. S. Wenger; Y.-K. Wang; T. Schierl; A. Eleftheriadis. IETF. May 2011. Proposed Standard. URL: https://www.rfc-editor.org/rfc/rfc6190
[RFC6386]: VP8 Data Format and Decoding Guide. J. Bankoski; J. Koleszar; L. Quillio; J. Salonen; P. Wilkins; Y. Xu. IETF. November 2011. Informational. URL: https://www.rfc-editor.org/rfc/rfc6386
[RFC7741]: RTP Payload Format for VP8 Video. P. Westin; H. Lundin; M. Glover; J. Uberti; F. Galligan. IETF. March 2016. Proposed Standard. URL: https://www.rfc-editor.org/rfc/rfc7741
[RFC7798]: RTP Payload Format for High Efficiency Video Coding (HEVC). Y.-K. Wang; Y. Sanchez; T. Schierl; S. Wenger; M. M. Hannuksela. IETF. March 2016. Proposed Standard. URL: https://www.rfc-editor.org/rfc/rfc7798
[RFC8827]: WebRTC Security Architecture. E. Rescorla. IETF. January 2021. Proposed Standard. URL: https://www.rfc-editor.org/rfc/rfc8827
[VP9]: VP9 Bitstream & Decoding Process Specification. A. Grange; P. de Rivaz; J. Hunt. Google. February 2016. Version 0.6. URL: https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp9-bitstream-specification-v0.6-20160331-draft.pdf
[VP9-PAYLOAD]: RTP Payload Format for VP9 Video. J. Uberti; S. Holmer; M. Flodman; J. Lennox; D. Hong. IETF. 10 June 2021. Internet Draft (work in progress). URL: https://datatracker.ietf.org/doc/html/draft-ietf-payload-vp9

Scalable Video Coding (SVC) Extension for WebRTC

Abstract

Status of This Document

1. Introduction

2. Conformance

3. Terminology

4. Configuration

4.1 RTCRtpEncodingParameters Dictionary Extensions

Dictionary RTCRtpEncodingParameters Members

4.2 Behavior

4.2.1 addTransceiver()

4.2.2 setParameters()

4.2.3 getParameters()

5. Discovery

5.1 Negotiation with an SFM

6. Scalability modes

6.1 Guidelines for addition of scalabilityMode values

7. Examples

7.1 Spatial Simulcast and Temporal Scalability

7.2 SVC Encoder Capabilities

7.3 SFM Capabilities

8. Privacy Considerations

8.1 Persistent information

9. Security Considerations

10. Scalability Mode Dependency Diagrams

10.1 L1T1

10.2 L1T2

10.3 L1T3

10.4 L2T1 and L2T1h

10.5 L2T1_KEY

10.6 L2T2 and L2T2h

10.7 L2T2_KEY

10.8 L2T2_KEY_SHIFT

10.9 L2T3 and L2T3h

10.10 L2T3_KEY

10.11 L2T3_KEY_SHIFT

10.12 L3T1 and L3T1h

10.13 L3T1_KEY

10.14 L3T2 and L3T2h

10.15 L3T2_KEY

10.16 L3T2_KEY_SHIFT

10.17 L3T3 and L3T3h

10.18 L3T3_KEY

10.19 L3T3_KEY_SHIFT

10.20 S2T1 and S2T1h

10.21 S2T2 and S2T2h

10.22 S2T3 and S2T3h

10.23 S3T1 and S3T1h

10.24 S3T2 and S3T2h

10.25 S3T3 and S3T3h

A. Acknowledgements

B. References

B.1 Normative references

B.2 Informative references

4.1 `RTCRtpEncodingParameters` Dictionary Extensions

Dictionary `RTCRtpEncodingParameters` Members

4.2.1 `addTransceiver()`

4.2.2 `setParameters()`

4.2.3 `getParameters()`

6.1 Guidelines for addition of `scalabilityMode` values