Simulcast

Simulcast is a useful feature for broadcasting video in conferences where participants are in different network conditions and use various devices.

In a typical conference, media are sent from the sender to the Selective Forwarding Unit (SFU), and then to all conference participants. The bandwidth of each of the receivers is often different due to network conditions and device capability issues. Using non-scalable streams, the sender transmits a media stream with a bit rate adjusted to the participant that has the lowest bandwidth. Therefore, all other participants receive lower quality, even if they have higher bandwidth.

Simulcast solves this problem by allowing the sender to simultaneously send multiple video streams with different resolutions (HD, MD, SD). Therefore, the SFU adjusts the bit rate according to various receivers’ bandwidth limitations. Based on the received Real-time Transport Control Protocol (RTCP) feedback packets, the SFU forwards the most suitable stream to each receiver. If one participant has lower bandwidth, this participant receives lower-quality streams, without impacting the rest of participants.

2574

The Simulcast mechanism

Automatic and manual stream selection

The Dolby.io Communications APIs allow receivers to choose the quality of the received stream. For example, participants can choose to receive a lower quality stream through a wider bandwidth to display a small video during a group conference call. In case of a situation when the bit rate of the selected stream is beyond the receiver's available bandwidth, the system automatically selects a lower-quality stream that can be properly received.

If a user joins a conference where one or more clients send Simulcast streams, the system automatically adapts to the best quality stream that suits the receiver's network condition. It does not require any additional programming or configuration on the client side.

Resolution and bitrate

For the sender

Currently, only desktop web users can send video Simulcast streams. Senders can transmit specific Simulcast streams listed in the following table:

StreamsResolution
HD, MD, SD1280 x 720
HD, MD, SD960 x 540
MD, SD640 x 360
MD, SD480 x 270
SD320 x 180

The stream selection is based on the RTCP feedback packets sent by SFU, which can be either:

  • REMB packets, in cases where the sender's network bandwidth estimation is performed by SFU and sent back to the sender
  • TransportCC packets, in cases where the sender's network bandwidth estimation is performed by the sender, based on TrasnportCC packet data

The sender's network bandwidth estimation cannot be higher than 2 Mbps per stream. The number of layers (one, two, or three) is selected based on the WebRTC algorithm using the bandwidth estimation.

For the receiver

Conference participants receive the highest-quality streams, based on the following variables:

  • The stream quality sent by the sender
  • The stream quality requested by the receiver
  • The stream quality estimated based on the receiver's available bandwidth
  • The number of received video streams, as listed in the following table:
Number of received video streamsMaximum resolution
1 - 3HD
4 - 8MD
9 and moreSD

The stream selection first estimates the available bandwidth of all conference participants using the REMB packets. Then, it subtracts from these values the bit rate that is needed for transmitting standard streams (such as screen sharing and non-Simulcast video transmission). The results of this calculation are divided by the number of Simulcast participants. Based on these results, the highest quality streams are selected.

A participant cannot receive a video if their available bandwidth is lower than a bit rate of the sent SD Simulcast stream.

Available APIs

To allow sending Simulcast streams, the Web SDK allows setting Simulcast preferences before joining a conference via JoinOptions.

let options = { simulcast: true };

VoxeetSDK.conference
  .join(conference, options)
  .then((info) => {})
  .catch((error) => {});

To receive Simulcast streams, all SDKs offer the simulcast method (Web, iOS, Android) that allow conference participants to set the stream quality they would like to receive.

VoxeetSDK.conference.simulcast([
  {
    id: participantId,
    quality: "hd",
  },
]);
VoxeetSDK.shared.conference.simulcast(
  requested: [.init(id: "quality", quality: .hd)]
) { error in }
VoxeetSDK.conference()
    .simulcast(Arrays.asList(new ParticipantQuality("quality", Quality.HD)))
    .then(result -> { })
    .error(error -> { });

Requirements

The following table lists operating systems and browsers that support Simulcast for sending and receiving:

Operating system Supported browser Sending Receiving
Windows Chrome
Firefox
Edge
macOS Safari
Chrome
Firefox
iOS Mobile Safari X
Native iOS SDK X
Android Mobile Chrome X
Native Android SDK X

Due to the Firefox limitation, Simulcast on Firefox does not work with the H264 video codec and requires using VP8. On other browsers, the Dolby.io Communications APIs platform supports H264 and VP8 video codecs.

The supported versions of browsers are listed in the Supported Environments document.