Video Forwarding

The Video Forwarding feature solves a common problem that may occur in multi-party conferences. In such conferences, receiving a large number of video streams from other participants may negatively affect audio quality by overloading or slowing down an application. Video Forwarding allows the local participant to receive video streams only from selected participants. This feature is especially useful for participants who experience network connection issues or have limited device processing power and want to reduce bandwidth and CPU consumption.

The Video Forwarding functionality allows configuring the following elements:

  • The maximum number of received video streams. The valid values are between 0 and 49.
  • The strategy for selecting participants whose video streams should be forwarded to the local participant. Participants can be selected either based on audio volume or a distance from the local participant:
    • The lastSpeaker strategy allows receiving video streams only from active speakers.
    • The closestUser strategy is available only for participants who enabled spatial audio. The strategy allows receiving video streams only from participants who are located the closest to the local participant.
  • The list of prioritized participants whose video streams need to be forwarded to the local participant, regardless of their audio volume or location. For example, in the case of virtual classes, this option allows participants to pin the teacher's video tile and see the teacher even if the teacher is not the active speaker.

Participants can enable the Video Forwarding functionality when they join a conference and during the conference.
To join a conference with a defined maximum number of received video streams, participants need to use the maxVideoForwarding parameter that is available in join options of the join method (Web, iOS, Android) or listen options of the listen method(Web, iOS, Android). These options also contain the videoForwardingStrategy parameter that allows selecting the video forwarding strategy.

const maxVideoForwarding = 10;
const videoForwardingStrategy = 'closestUserStrategy';
await VoxeetSDK.conference.join({
  conference: conferenceObject,
  params: {
    maxVideoForwarding: maxVideoForwarding,
    forwardingStrategy: videoForwardingStrategy,
  },
});
let options = VTJoinOptions()
options.maxVideoForwarding = 2
options.videoForwardingStrategy = .closestUser

VoxeetSDK.shared.conference.join(
    conference: conferenceObject,
    options: options
) { conference in } fail: { error in }
VoxeetSDK.conference().join(
  new ConferenceJoinOptions.Builder(conference)
    .setMaxVideoForwarding(number)
    .setVideoForwardingStrategy(strategy)
    .build())
  .then(conference -> {
    // manage the success here
  })
  .error(error -> {
    // manage the error here
  });
const options: ConferenceJoinOptions = {
  maxVideoForwarding: 4,
  videoForwardingStrategy: VideoForwardingStrategy.CLOSEST_USER
};

await CommsAPI.conference.join(
  conference,
  options
);
var options = ConferenceJoinOptions();
options.maxVideoForwarding = 2;
options.videoForwardingStrategy = VideoForwardingStrategy.closestUser;

await _dolbyioCommsSdkFlutterPlugin.conference.join(conference, options);

To modify Video Forwarding during a conference, participants need to use the videoForwarding method (Web, iOS, Android). The method uses the VideoForwardingOptions model (Web, iOS, Android) that allows setting the maximum number of received video streams, selecting the video forwarding strategy, and providing a list of prioritized participants.

const participants = [{ externalId: participant1ExternalId }, { externalId: participant2ExternalId }];
const max = 16;
const strategy = 'lastSpeakerStrategy';
const videoForwardingOptions = { strategy, max, participants };
await VoxeetSDK.conference.videoForwarding(videoForwardingOptions);
let participant = VTParticipant()
let options = VideoForwardingOptions(
    strategy: .closestUser,
    max: 2,
    participants: [participant]
)

VoxeetSDK.shared.conference.videoForwarding(options: options) { error in }
VoxeetSDK.conference().videoForwarding(
  new VideoForwardingOptions.Builder()
    .setMaxVideoForwarding(number)
    .setVideoForwardingStrategy(strategy)
    .setParticipants(participants)
    .build()
);
  const options: VideoForwardingOptions = {
    max: 4,
    strategy: VideoForwardingStrategy.CLOSEST_USER,
    participants: [participant1, participant2]
  };

  await CommsAPI.conference.videoForwarding(options);
  await _dolbyioCommsSdkFlutterPlugin.conference
    .setVideoForwarding(VideoForwardingStrategy.lastSpeaker, 4, [participant1, participant2]);

Based on the specified parameters, the Dolby.io platform calculates the required streams for each conference participant. If the number of elements in the participants array is smaller than the value of the max parameter, SDK displays videos of additional participants who are not specified in the participants parameter. In Dolby Voice conferences, the selection is based on the video forwarding strategy. If the number of elements in the participants array is greater than the value of the max parameter, SDK selects only the max number of the prioritized participants, the rest of the selected participants is not displayed.

3865

A graphic that presents Video Forwarding

The application can continue to rely on the streamAdded (Web, iOS, Android), streamUpdated (Web, iOS, Android), and streamRemoved (Web, iOS, Android) events to get notified about the video forwarding status for a particular participant.