Media Capabilities

2.1.1. MediaConfiguration

dictionary MediaConfiguration {
  VideoConfiguration video;
  AudioConfiguration audio;
};

dictionary MediaDecodingConfiguration : MediaConfiguration {
  required MediaDecodingType type;
  MediaCapabilitiesKeySystemConfiguration keySystemConfiguration;
};

dictionary MediaEncodingConfiguration : MediaConfiguration {
  required MediaEncodingType type;
};

The input to the decoding capabilities is represented by a MediaDecodingConfiguration dictionary and the input of the encoding capabilities by a MediaEncodingConfiguration dictionary.

For a MediaConfiguration to be a valid MediaConfiguration, all of the following conditions MUST be true:

1. audio and/or video MUST exist.
2. audio MUST be a valid audio configuration if it exists.
3. video MUST be a valid video configuration if it exists.

For a MediaDecodingConfiguration to be a valid MediaDecodingConfiguration, all of the following conditions MUST be true:

1. It MUST be a valid MediaConfiguration.
2. If keySystemConfiguration exists:
   1. The type MUST be media-source or file.
   2. If keySystemConfiguration.audio exists, audio MUST also exist.
   3. If keySystemConfiguration.video exists, video MUST also exist.

For a MediaDecodingConfiguration to describe [ENCRYPTED-MEDIA], a keySystemConfiguration MUST exist.

2.1.2. MediaDecodingType

enum MediaDecodingType {
  "file",
  "media-source",
  "webrtc"
};

A MediaDecodingConfiguration has three types:

• file is used to represent a configuration that is meant to be used for a plain file playback.
• media-source is used to represent a configuration that is meant to be used for playback of a MediaSource as defined in the [media-source] specification.
• webrtc is used to represent a configuration that is meant to be received using RTCPeerConnection as defined in [webrtc]).

2.1.3. MediaEncodingType

enum MediaEncodingType {
  "record",
  "webrtc"
};

A MediaEncodingConfiguration can have one of two types:

• record is used to represent a configuration for recording of media, e.g. using MediaRecorder as defined in [mediastream-recording].
• webrtc is used to represent a configuration that is meant to be transmitted using RTCPeerConnection as defined in [webrtc]).

2.1.4. MIME types

In the context of this specification, a MIME type is also called content type. A valid media MIME type is a string that is a valid MIME type per [mimesniff].

Please note that the definition of MIME subtypes and parameters is context dependent. For file, media-source, and record, the MIME types are specified as defined for HTTP, whereas for webrtc the MIME types are specified as defined for RTP.

A valid audio MIME type is a string that is a valid media MIME type and for which the type per [RFC9110] is either audio or application.

A valid video MIME type is a string that is a valid media MIME type and for which the type per [RFC9110] is either video or application.

2.1.5. VideoConfiguration

dictionary VideoConfiguration {
  required DOMString contentType;
  required unsigned long width;
  required unsigned long height;
  required unsigned long long bitrate;
  required double framerate;
  boolean hasAlphaChannel;
  HdrMetadataType hdrMetadataType;
  ColorGamut colorGamut;
  TransferFunction transferFunction;
  DOMString scalabilityMode;
  boolean spatialScalability;
};

The contentType member represents the MIME type of the video track.

To check if a VideoConfiguration configuration is a valid video configuration, the following steps MUST be run:

1. If configuration’s contentType is not a valid video MIME type, return false and abort these steps.
2. If framerate is not finite or is not greater than 0, return false and abort these steps.
3. If an optional member is specified for a MediaDecodingType or MediaEncodingType to which it’s not applicable, return false and abort these steps. See applicability rules in the member definitions below.
4. Return true.

The width and height members represent respectively the visible horizontal and vertical encoded pixels in the encoded video frames.

The bitrate member represents the average bitrate of the video track given in units of bits per second. In the case of a video stream encoded at a constant bit rate (CBR) this value should be accurate over a short term window. For the case of variable bit rate (VBR) encoding, this value should be usable to allocate any necessary buffering and throughput capability to provide for the un-interrupted decoding of the video stream over the long-term based on the indicated contentType.

The framerate member represents the framerate of the video track. The framerate is the number of frames used in one second (frames per second). It is represented as a double.

The hasAlphaChannel member represents whether the video track contains alpha channel information. If true, the encoded video stream can produce per-pixel alpha channel information when decoded. If false, the video stream cannot produce per-pixel alpha channel information when decoded. If undefined, the UA should determine whether the video stream encodes alpha channel information based on the indicated contentType, if possible. Otherwise, the UA should presume that the video stream cannot produce alpha channel information.

If present, the hdrMetadataType member represents that the video track includes the specified HDR metadata type, which the UA needs to be capable of interpreting for tone mapping the HDR content to a color volume and luminance of the output device. Valid inputs are defined by HdrMetadataType. hdrMetadataType is only applicable to MediaDecodingConfiguration for types media-source and file.

If present, the colorGamut member represents that the video track is delivered in the specified color gamut, which describes a set of colors in which the content is intended to be displayed. If the attached output device also supports the specified color, the UA needs to be able to cause the output device to render the appropriate color, or something close enough. If the attached output device does not support the specified color, the UA needs to be capable of mapping the specified color to a color supported by the output device. Valid inputs are defined by ColorGamut. colorGamut is only applicable to MediaDecodingConfiguration for types media-source and file.

If present, the transferFunction member represents that the video track requires the specified transfer function to be understood by the UA. Transfer function describes the electro-optical algorithm supported by the rendering capabilities of a user agent, independent of the display, to map the source colors in the decoded media into the colors to be displayed. Valid inputs are defined by TransferFunction. transferFunction is only applicable to MediaDecodingConfiguration for types media-source and file.

If present, the scalabilityMode member represents the scalability mode as defined in [webrtc-svc]. If absent, the implementer defined default mode for this contentType is assumed (i.e., the mode you get if you don’t specify one via setParameters()). scalabilityMode is only applicable to MediaEncodingConfiguration for type webrtc.

If present, the spatialScalability member represents the ability to do spatial prediction, that is, using frames of a resolution different than the current resolution as dependencies. If absent, spatialScalability will default to false. spatialScalability is closely coupled to scalabilityMode in the sense that streams encoded with modes using spatial scalability (e.g. "L2T1") can only be decoded if spatialScalability is supported. spatialScalability is only applicable to MediaDecodingConfiguration for types media-source, file, and webrtc.

2.1.6. HdrMetadataType

enum HdrMetadataType {
  "smpteSt2086",
  "smpteSt2094-10",
  "smpteSt2094-40"
};

If present, HdrMetadataType describes the capability to interpret HDR metadata of the specified type.

The VideoConfiguration may contain one of the following types:

• smpteSt2086, representing the static metadata type defined by [SMPTE-ST-2086].
• smpteSt2094-10, representing the dynamic metadata type defined by [SMPTE-ST-2094].
• smpteSt2094-40, representing the dynamic metadata type defined by [SMPTE-ST-2094].

2.1.7. ColorGamut

enum ColorGamut {
  "srgb",
  "p3",
  "rec2020"
};

The VideoConfiguration may contain one of the following types:

• srgb, representing the [sRGB] color gamut.
• p3, representing the DCI P3 Color Space color gamut. This color gamut includes the srgb gamut.
• rec2020, representing the ITU-R Recommendation BT.2020 color gamut. This color gamut includes the p3 gamut.

2.1.8. TransferFunction

enum TransferFunction {
  "srgb",
  "pq",
  "hlg"
};

The VideoConfiguration may contain one of the following types:

• srgb, representing the transfer function defined by [sRGB].
• pq, representing the "Perceptual Quantizer" transfer function defined by [SMPTE-ST-2084].
• hlg, representing the "Hybrid Log Gamma" transfer function defined by BT.2100.

2.1.9. AudioConfiguration

dictionary AudioConfiguration {
  required DOMString contentType;
  DOMString channels;
  unsigned long long bitrate;
  unsigned long samplerate;
  boolean spatialRendering;
};

The contentType member represents the MIME type of the audio track.

To check if a AudioConfiguration configuration is a valid audio configuration, the following steps MUST be run:

1. If configuration’s contentType is not a valid audio MIME type, return false and abort these steps.
2. Return true.

The channels member represents the audio channels used by the audio track. channels is only applicable to the decoding types media-source, file, and webrtc and the encoding type webrtc.

The channels needs to be defined as a double (2.1, 4.1, 5.1, ...), an unsigned short (number of channels) or as an enum value. The current definition is a placeholder.

The bitrate member represents the number of average bitrate of the audio track. The bitrate is the number of bits used to encode a second of the audio track.

The samplerate represents the samplerate of the audio track in. The samplerate is the number of samples of audio carried per second. samplerate is only applicable to the decoding types media-source, file, and webrtc and the encoding type webrtc.

The samplerate is expressed in Hz (ie. number of samples of audio per second). Sometimes the samplerates value are expressed in kHz which represents the number of thousands of samples of audio per second.
44100 Hz is equivalent to 44.1 kHz.

The spatialRendering member indicates that the audio SHOULD be renderered spatially. The details of spatial rendering SHOULD be inferred from the contentType. If it does not exist, the UA MUST presume spatialRendering is not required. When true, the user agent SHOULD only report this configuration as supported if it can support spatial rendering *for the current audio output device* without failing back to a non-spatial mix of the stream. spatialRendering is only applicable to MediaDecodingConfiguration for types media-source and file.

2.3. Algorithms

2.4. Navigator and WorkerNavigator extension

[Exposed=Window]
partial interface Navigator {
  [SameObject] readonly attribute MediaCapabilities mediaCapabilities;
};

[Exposed=Worker]
partial interface WorkerNavigator {
  [SameObject] readonly attribute MediaCapabilities mediaCapabilities;
};

2.5. Media Capabilities Interface

[Exposed=(Window, Worker)]
interface MediaCapabilities {
  [NewObject] Promise<MediaCapabilitiesDecodingInfo> decodingInfo(MediaDecodingConfiguration configuration);
  [NewObject] Promise<MediaCapabilitiesEncodingInfo> encodingInfo(MediaEncodingConfiguration configuration);
};

The decodingInfo() method method MUST run the following steps:

1. If configuration is not a valid MediaDecodingConfiguration, return a Promise rejected with a newly created TypeError.
2. If configuration.keySystemConfiguration exists, run the following substeps:
   1. If the global object is of type WorkerGlobalScope, return a Promise rejected with a newly created DOMException whose name is InvalidStateError.
   2. If the result of running Is the environment settings object settings a secure context? [secure-contexts] with the global object’s relevant settings object is not "Secure", return a Promise rejected with a newly created DOMException whose name is SecurityError.
3. Let p be a new promise.
4. In parallel, run the Create a MediaCapabilitiesDecodingInfo algorithm with configuration and resolve p with its result.
5. Return p.

Note, calling decodingInfo() with a keySystemConfiguration present may have user-visible effects, including requests for user consent. Such calls should only be made when the author intends to create and use a MediaKeys object with the provided configuration.

The encodingInfo() method MUST run the following steps:

1. If configuration is not a valid MediaConfiguration, return a Promise rejected with a newly created TypeError.
2. Let p be a new promise.
3. In parallel, run the Create a MediaCapabilitiesEncodingInfo algorithm with configuration and resolve p with its result.
4. Return p.

4.1. Query playback capabilities with decodingInfo()

The following example shows how to use decodingInfo() to query media playback capabilities when using Media Source Extensions [media-source].

<script>
  const contentType = 'video/mp4;codecs=avc1.640028';

  const configuration = {
    type: 'media-source',
    video: {
      contentType: contentType,
      width: 640,
      height: 360,
      bitrate: 2000,
      framerate: 29.97
    }
  };

  navigator.mediaCapabilities.decodingInfo(configuration)
    .then((result) => {
      console.log('Decoding of ' + contentType + ' is'
        + (result.supported ? '' : ' NOT') + ' supported,'
        + (result.smooth ? '' : ' NOT') + ' smooth and'
        + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
    })
    .catch((err) => {
      console.error(err, ' caused decodingInfo to reject');
    });
</script>

The following examples show how to use decodingInfo() to query WebRTC receive capabilities [webrtc].

<script>
  const contentType = 'video/VP8';

  const configuration = {
    type: 'webrtc',
    video: {
      contentType: contentType,
      width: 640,
      height: 360,
      bitrate: 2000,
      framerate: 25
    }
  };

  navigator.mediaCapabilities.decodingInfo(configuration)
    .then((result) => {
      console.log('Decoding of ' + contentType + ' is'
        + (result.supported ? '' : ' NOT') + ' supported,'
        + (result.smooth ? '' : ' NOT') + ' smooth and'
        + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
    })
    .catch((err) => {
      console.error(err, ' caused decodingInfo to reject');
    });
</script>

<script>
  const contentType = 'video/H264;level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f';

  const configuration = {
    type: 'webrtc',
    video: {
      contentType: contentType,
      width: 640,
      height: 360,
      bitrate: 2000,
      framerate: 25
    }
  };

  navigator.mediaCapabilities.decodingInfo(configuration)
    .then((result) => {
      console.log('Decoding of ' + contentType + ' is'
        + (result.supported ? '' : ' NOT') + ' supported,'
        + (result.smooth ? '' : ' NOT') + ' smooth and'
        + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
    })
    .catch((err) => {
      console.error(err, ' caused decodingInfo to reject');
    });
</script>

4.2. Query recording capabilities with encodingInfo()

<script>
  const contentType = 'video/VP9';

  const configuration = {
    type: 'webrtc',
    video: {
      contentType: contentType,
      width: 640,
      height: 480,
      bitrate: 10000,
      framerate: 29.97,
      scalabilityMode: "L3T3_KEY"
    }
  };

  navigator.mediaCapabilities.encodingInfo(configuration)
    .then((result) => {
      console.log(contentType + ' is:'
        + (result.supported ? '' : ' NOT') + ' supported,'
        + (result.smooth ? '' : ' NOT') + ' smooth and'
        + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
    })
    .catch((err) => {
      console.error(err, ' caused encodingInfo to reject');
    });
</script>

<script>
  const contentType = 'video/webm;codecs=vp8';

  const configuration = {
    type: 'record',
    video: {
      contentType: contentType,
      width: 640,
      height: 480,
      bitrate: 10000,
      framerate: 29.97
    }
  };

  navigator.mediaCapabilities.encodingInfo(configuration)
    .then((result) => {
      console.log(contentType + ' is:'
        + (result.supported ? '' : ' NOT') + ' supported,'
        + (result.smooth ? '' : ' NOT') + ' smooth and'
        + (result.powerEfficient ? '' : ' NOT') + ' power efficient');
    })
    .catch((err) => {
      console.error(err, ' caused encodingInfo to reject');
    });
</script>