Integrating Shared Spatial Audio

This guide explains how to integrate shared spatial audio into an application one step at a time, using a virtual world use case to describe this process.

To use the shared spatial audio feature in your application you must:

  • Enable shared spatial audio on the conference
  • Configure the spatial environment using the SDK
  • Provide relevant position updates for the spatial audio locations

See Summary for the complete code snippet for integrating shared spatial audio.

Enable shared spatial audio

To enable shared spatial audio on a conference, create the conference with spatialAudioStyle (Web, iOS, Android, C++) set to shared and dolbyVoice (Web, iOS, Android, C++) set to true.

When joining the conference, enable spatial audio by setting the spatialAudio (Web, iOS, Android, C++) flag to true.

// Create a Dolby Voice conference with spatialAudioStyle set to shared
const createOptions = {
  alias: "conference name",
  params: {
    spatialAudioStyle: "shared"
  }
};
const conference = await VoxeetSDK.conference.create(createOptions);

// Join the conference with spatial audio enabled
const joinOptions = {
  spatialAudio: true
};
await VoxeetSDK.conference.join(conference, joinOptions);
// Create a Dolby Voice conference with spatialAudioStyle set to shared
let options = VTConferenceOptions()
options.alias = "conference name"
options.spatialAudioStyle = .shared

VoxeetSDK.shared.conference.create(options: options) { conference in
  // Join the conference with spatial audio enable
  let options = VTJoinOptions()
  options.spatialAudio = true

  VoxeetSDK.shared.conference.join(
    conference: conference,
    options: options
  ) { conference in
    // Success
  } fail: { error in }
} fail: { error in }
ParamsHolder paramsHolder = new ParamsHolder();
paramsHolder.setSpatialAudioStyle(SpatialAudioStyle.SHARED);

ConferenceCreateOptions conferenceCreateOptions = new ConferenceCreateOptions.Builder()
        .setConferenceAlias("conference name")
        .setParamsHolder(paramsHolder)
        .build();

// Create a Dolby Voice conference with spatialAudioStyle set to individual
VoxeetSDK.conference().create(conferenceCreateOptions)
        .then((ThenPromise<Conference, Conference>) conference -> {
            ConferenceJoinOptions conferenceJoinOptions = new ConferenceJoinOptions.Builder(conference)
                    .setSpatialAudio(true)
                    .build();

            // Join the conference using spatial audio
            return VoxeetSDK.conference().join(conferenceJoinOptions);
        })
        .then(conference -> {
            // Success
        })
        .error((error_in) -> {
            // Error
        });
// Create Shared Spatial Audio Conference  
dolbyio::comms::services::conference::conference_options conference_options{};
conference_options.alias = "conference name";
conference_options.params.spatial_audio_style =
      dolbyio::comms::spatial_audio_style::shared;

// The wait call will block until the asynchronous operation completes
auto conf_info = wait(sdk->conference().create(conference_options));

// Join the Shared Spatial Conference
dolbyio::comms::services::conference::join_options join_options{};
join_options.constraints.audio = true;
join_options.constraints.video = false;
join_options.connection.spatial_audio = true;

// The wait call will block until the asynchronous operation completes
wait(sdk->conference().join(conf_info, join_options));

Configure the spatial audio scene

To configure the scene, tell the audio renderer what units the application uses for its position coordinates and what directions are forward, right, and up. These parameters vary by application.

For directions, a typical use would have:

  • The positive X-axis pointing to the right
  • The positive Y-axis pointing upwards
  • The positive Z-axis pointing into the screen

In this case, the directions should be defined using the following code:

// Configure the spatial audio scene
const right = { x: 1, y: 0, z: 0 };
const up = { x: 0, y: 1, z: 0 };
const forward = { x: 0, y: 0, z: 1 };
// Configure the spatial audio scene
let right = VTSpatialPosition(x: 1, y: 0, z: 0)!
let up = VTSpatialPosition(x: 0, y: 1, z: 0)!
let forward = VTSpatialPosition(x: 0, y: 0, z: 1)!
// Configure the spatial audio scene
SpatialPosition right = new SpatialPosition(1, 0, 0);
SpatialPosition up = new SpatialPosition(0, 1, 0);
SpatialPosition forward = new SpatialPosition(0, 0, 1);
// Negative Y axis is heard in the forwards direction, so tell the SDK forward has -1 Y
dolbyio::comms::spatial_position right{1, 0, 0};

// Upwards axis is unimportant for this case, you can set it to either Z = +1 or Z -1,
// never provide a Z position
dolbyio::comms::spatial_position up{0, 1, 0};

// Positive X axis is heard in the right-hand direction, so tell the SDK right has +1 X
dolbyio::comms::spatial_position forward{0, 0, 1};

Configure the spatial environment scale

With spatial audio, a conference participant’s volume is dependent on how close they are to the local participant. A participant who is one meter away will be at full volume. A participant who is 100 meters or further away will not be heard. The spatial environment scale allows you to tell the audio renderer how to convert from the units that your application is using to set positions.

Many applications would use meters, especially the ones that present a virtual environment emulating the real world. In this case, the scale can be set to (1.0, 1.0, 1.0).

const scale = { x: 1, y: 1, z: 1 };

VoxeetSDK.conference.setSpatialEnvironment(scale, forward, up, right);
// Configure the spatial environment scale
let scale = VTSpatialScale(x: 1, y: 1, z: 1)!

VoxeetSDK.shared.conference.setSpatialEnvironment(
  scale: scale,
  forward: forward,
  up: up,
  right: right
)
// Configure the spatial environment scale
SpatialScale scale = new SpatialScale(1, 1, 1);

VoxeetSDK.conference().setSpatialEnvironment(scale, forward, up, right);
// Configure the spatial environment scale
dolbyio::comms::spatial_scale scale{1, 1, 1};

// Update the Spatial Environment
dolbyio::comms::spatial_audio_batch_update batch_update;
batch_update.set_spatial_environment(scale, right, up, forward);
sdk->conference()
    .update_spatial_audio_configuration(std::move(batch_update))
    .then([]() { std::cout << "Spatial Environment Set!\n"; })
    .on_error([](std::exception_ptr&&) {
       std::cerr << "Failed to Set Spatial Environment!";
    });

In an application with a different measurement unit, the scale needs to be calculated. For example, a virtual space using a pixel art graphic style may have a “floor space” of 5000 × 5000 pixels. Using a scale, it is possible to establish that someone on the extreme left is positioned at the edge of the hearing range of someone on the extreme right. In this scenario, 5000 pixels should equal 100 meters, with the same scale for all axis:

// Configure the spatial environment scale
const axis_scale = 5000 / 100;
const scale = { x: axis_scale, y: axis_scale, z: axis_scale };

VoxeetSDK.conference.setSpatialEnvironment(scale, forward, up, right);
// Configure the spatial environment scale
let axisScale = 5000 / 100.0
let scale = VTSpatialScale(x: axisScale, y: axisScale, z: axisScale)!

VoxeetSDK.shared.conference.setSpatialEnvironment(
  scale: scale,
  forward: forward,
  up: up,
  right: right
)
// Configure the spatial environment scale
double axis_scale = 5000 / 100;
SpatialScale scale = new SpatialScale(axis_scale, axis_scale, axis_scale);
        
VoxeetSDK.conference().setSpatialEnvironment(scale, forward, up, right);
// Configure the spatial environment scale
double axis_scale = 5000 / 100;
dolbyio::comms::spatial_scale scale{axis_scale, axis_scale, axis_scale};

// Update the Spatial Environment
dolbyio::comms::spatial_audio_batch_update batch_update;
batch_update.set_spatial_environment(scale, right, up, forward);
sdk->conference()
    .update_spatial_audio_configuration(std::move(batch_update))
    .then([]() { std::cout << "Spatial Enviroment Set!\n"; })
    .on_error([](std::exception_ptr&&) {
       std::cerr << "Failed to Set Spatial Environment!";
    });

Set the participant’s position

In a shared scene, each participant is responsible for setting their own position. Without a position, a participant will not hear other participants and will not be heard. As a participant moves around the virtual space, their position needs to be updated.

Assuming that the position for the local participant is in a variable named myPosition, the position can be set using the following code:

// Set the local participant’s position
const myPosition = ...

VoxeetSDK.conference.setSpatialPosition(VoxeetSDK.session.participant, myPosition);
// Set the local participant’s position
let myPosition = ...

VoxeetSDK.shared.conference.setSpatialPosition(
  participant: VoxeetSDK.shared.session.participant!,
  position: myPosition
)
// Set the local participant’s position
SpatialPosition myPosition = ...
        
VoxeetSDK.conference().setSpatialPosition(VoxeetSDK.session().getParticipant(), myPosition);
// Set the local participant’s position
dolbyio::comms::spatial_position position{1, 1, 1};

// Update the local participant's position
dolbyio::comms::spatial_audio_batch_update position_update;
position_update.set_spatial_position(session_info.participant_id.value(),
                                     position);
sdk->conference()
   .update_spatial_audio_configuration(std::move(position_update))
   .then([]() { std::cout << "Spatial Position Set!\n"; })
   .on_error([](std::exception_ptr&&) {
     std::cerr << "Failed to Set Spatial Position!";
   });

With positions being updated as people move, you should now have a working shared spatial scene conference.

Summary

The above steps combined might contain the following code snippet:

// Create a Dolby Voice conference with spatialAudioStyle set to shared
const createOptions = {
  alias: "conference name",
  params: {
    spatialAudioStyle: "shared"
  }
};

const conference = await VoxeetSDK.conference.create(createOptions);

// Join the conference with spatial audio enabled
const joinOptions = {
 spatialAudio: true
};

await VoxeetSDK.conference.join(conference, joinOptions);

// Configure the spatial audio scene and scale
const right = { x: 1, y: 0, z: 0 };
const up = { x: 0, y: 1, z: 0 };
const forward = { x: 0, y: 0, z: 1 };

const axis_scale = 5000 / 100;
const scale = { x: axis_scale, y: axis_scale, z: axis_scale };

VoxeetSDK.conference.setSpatialEnvironment(scale, forward, up, right);

// Set the local participant’s position
const myPosition = ...

VoxeetSDK.conference.setSpatialPosition(VoxeetSDK.session.participant, myPosition);
// Create a Dolby Voice conference with spatialAudioStyle set to shared
let options = VTConferenceOptions()
options.alias = "conference name"
options.spatialAudioStyle = .shared

VoxeetSDK.shared.conference.create(options: options) { conference in
  // Join the conference with spatial audio enable
  let options = VTJoinOptions()
  options.spatialAudio = true

  VoxeetSDK.shared.conference.join(
    conference: conference,
    options: options
  ) { conference in
    // Success
  } fail: { error in }
} fail: { error in }

// Configure the spatial audio scene and scale
let right = VTSpatialPosition(x: 1, y: 0, z: 0)!
let up = VTSpatialPosition(x: 0, y: 1, z: 0)!
let forward = VTSpatialPosition(x: 0, y: 0, z: 1)!

let axisScale = 5000 / 100.0
let scale = VTSpatialScale(x: axisScale, y: axisScale, z: axisScale)!

VoxeetSDK.shared.conference.setSpatialEnvironment(
  scale: scale,
  forward: forward,
  up: up,
  right: right
)

// Set the local participant’s position
let myPosition = ...

VoxeetSDK.shared.conference.setSpatialPosition(
  participant: VoxeetSDK.shared.session.participant!,
  position: myPosition
)
ParamsHolder paramsHolder = new ParamsHolder();
paramsHolder.setSpatialAudioStyle(SpatialAudioStyle.SHARED);

ConferenceCreateOptions conferenceCreateOptions = new ConferenceCreateOptions.Builder()
        .setConferenceAlias("conference name")
        .setParamsHolder(paramsHolder)
        .build();

// Create a Dolby Voice conference with spatialAudioStyle set to individual
VoxeetSDK.conference().create(conferenceCreateOptions)
        .then((ThenPromise<Conference, Conference>) conference -> {
            ConferenceJoinOptions conferenceJoinOptions = new ConferenceJoinOptions.Builder(conference)
                    .setSpatialAudio(true)
                    .build();

            // Join the conference using spatial audio
            return VoxeetSDK.conference().join(conferenceJoinOptions);
        })
        .then(conference -> {
            // Success
        })
        .error((error_in) -> {
            // Error
        });

// Configure the spatial audio scene
SpatialPosition right = new SpatialPosition(1, 0, 0);
SpatialPosition up = new SpatialPosition(0, 1, 0);
SpatialPosition forward = new SpatialPosition(0, 0, 1);

// Configure the spatial environment scale
double axis_scale = 5000 / 100;
SpatialScale scale = new SpatialScale(axis_scale, axis_scale, axis_scale);
        
VoxeetSDK.conference().setSpatialEnvironment(scale, forward, up, right);

// Set the local participant’s position
SpatialPosition myPosition = ...
        
VoxeetSDK.conference().setSpatialPosition(VoxeetSDK.session().getParticipant(), myPosition);
// Create Shared Spatial Audio Conference  
dolbyio::comms::services::conference::conference_options conference_options{};
conference_options.alias = "conference name";
conference_options.params.spatial_audio_style =
      dolbyio::comms::spatial_audio_style::shared;

// The wait call will block until the asynchronous operation completes
auto conf_info = wait(sdk->conference().create(conference_options));

// Join the Shared Spatial Conference
dolbyio::comms::services::conference::join_options join_options{};
join_options.constraints.audio = true;
join_options.constraints.video = false;
join_options.connection.spatial_audio = true;

// The wait call will block until the asynchronous operation completes
wait(sdk->conference().join(conf_info, join_options));

// Configure the spatial audio scene
dolbyio::comms::spatial_position right{1, 0, 0};
dolbyio::comms::spatial_position up{0, 1, 0};
dolbyio::comms::spatial_position forward{0, 0, 1};

// Configure the spatial environment scale
double axis_scale = 5000 / 100;
dolbyio::comms::spatial_scale scale{axis_scale, axis_scale, axis_scale};

// Set the Spatial Environment
dolbyio::comms::spatial_audio_batch_update batch_update;
batch_update.set_spatial_environment(scale, right, up, forward);

// Set the local participant’s position
dolbyio::comms::spatial_position position{1, 1, 1};
batch_update.set_spatial_position(session_info.participant_id.value(),
                                  position);

// Update the position and environment asynchronously
sdk->conference()
   .update_spatial_audio_configuration(std::move(batch_update))
   .then([]() { std::cout << "Spatial Environment and Local Position Set!\n"; })
   .on_error([](std::exception_ptr&&) {
     std::cerr << "Failed to Set Spatial Position and Environment!";
   });

Did this page help you?