Understanding the Unity Publisher and Subscriber

Overview

This document explains the Dolby.io Real-time Streaming Publisher and Player plugins for Unity McPublisher and McSubscriber scripts. Specifically, we will look at:

• Adding the McPublisher script and understanding the UI fields
• Understanding how Credentials work
• Customizing publisher video settings
• Understanding the McSubscriber UI fields
• Rendering onto Materials

The McPublisher script

Assuming you have added the Dolby.io Real-time Streaming Publisher and Player plugins for Unity to the project, open a new scene, and add the McPublisher script to the Main Camera via the Add Component button in the Inspector on the right:

• Stream Name: Once you create a publishing token in the streaming dashboard, you will obtain a stream name that you can use to publish with. If you have a wild-card token (.*), you can use any stream name you would like, which is the preferred method since you can use the same credentials to publish multiple streams with different stream names from within your game.

• Credentials: A Scriptable Object of type McCredentials that contains the necessary information to publish and subscribe. It will be discussed in detail in the Credentials section.

• Video Config Data: A Scriptable Object of type 'VideoConfiguration' that allows it's users to customize the video aspects of the McPublisher, like the resolution, bitrate, codec, etc. We will elaborate more about Video Configuration in this tutorial.

• Use Audio Listener As Source: If you add the McPublisher to an object that contains an AudioListener, this option allows you to stream whatever audio AudioListener to the Dolby.io Real-time Streaming service. A common use case is attaching this to the main character in the game, so the viewers can hear whatever the character is hearing.

• VideoSource: A camera to publish with. Viewers will be able to view the scene that the camera is capturing.

• AudioSource: An Audio Source to publish with. This audio source has to contain an audio clip. Viewers will be able to hear whatever the audio source is playing. If you set Use Audio Listener As Source to true, this audio source will be ignored and only Audio Listener's audio will be transmitted.

• Publish On Start: Tick this option to true to start publishing as soon as the game starts. In the next tutorial, we will learn how to use event handlers to control publishing.

Credentials

McCredentials is a Scriptable Object, that acts as a data store for your Dolby.io Credentials. You can reuse these credentials between multiple instances of the McPublisher and McSubscriber, and across different scenes. To create an instance of McCredentials, go to Assets > Create > Millicast > Credentials:

Video Configuration

The McPublisher also allows you to adjust the streaming resolution, video codec to encode with, the bitrates and maximum FPS. To create a Video Configuration Scriptable Object, go to Assets > Create > Millicast > Video Configuration:

• Codec Type: You can choose between VP8, VP9, H264 & AV1.

• Resolution: The video resolution. You can stream in 720p, 1080p, 2K & 4K. Since H264 is hardware accelerated, certain platforms might not support all resolutions with it, therefore, if you choose a resolution that is not supported, it will automatically switch to a supported one.

• Max Bitrate: The maximum encoding bitrate. Bitrate depends on the network bandwidth. The target bitrate might not always reach the maximum.

• Min Bitrate: The minimum bitrate. Similarly, this depends on the network bandwidth, and the target bitrate might drop below the minimum if the network bandwidth is poor.

• Framerate Option: This refers to the maximum framerate. Setting this value will cap the framerate of the video stream.

• Scale Down Option: Scale down the chosen resolution. For example, selecting 2.0 will halve the chosen resolution.

To set the publisher's video configuration, use the Video Config Data field.

If you'd like to learn more about changing the video configuration from within the scripting API, refer to the API documentation. You can also refer to the Reference Sample Scenes > Scenes > VideoConfigExample when you import the Samples.

The Millicast Subscriber

The 3D karting game tutorial demonstrates how to add the McSubscriber to render a first person view of the game character onto a UI RawImage.

• Stream Name: The stream name to subscribe to. This field must match the same stream name you published to.

• Credentials: The same credentials object as in the McPublisher.

• Subscribe On Start: Enables you to subscribe as soon as the game (or scene) starts.

• Update Mesh Renderer Material: If you add the subscriber to an object that contains a mesh renderer, i.e any 3D object, then checking this box allows the subscriber to render the incoming stream onto the object. In the next section, we will explain the different rendering approaches.

• Render Materials: A list of Materials to render the incoming video stream on. Once a material is added, any mesh renderer using that material will render the same stream.

• Render Images: A list of RawImages to render the incoming video stream on. This is intended to render onto UI, like in the Karting game HUD example earlier.

• Render Audio Source: A list of AudioSources to render the incoming audio stream on.

Rendering onto Materials

Start with a basic 3D scene. Add a Sphere:

Create a new Material from the Assets > Create > Materials tab. Let us call it DemoRenderMaterial:

Change the Sphere Mesh Renderer material to DemoRenderMaterial:

Add an McSubscriber component to the Sphere, and add DemoRenderMaterial to the list of Render Materials:

Do not forget to add your Credentials, your stream name, and enable Subscribe On Start. Note: here we are publishing directly from the web dashboard.

Note: If you are unsure how to publish from the web viewer, see: How to broadcast in dashboard

Now that we have all the pieces of the puzzle, start the scene and observe the sphere:

We can adjust the material's properties as well. For example, we can make it more metallic and smoother:

Note: The difference between using Render Materials list, and using Update Mesh Renderer Material, is that the latter will update the Mesh Renderer Material instance directly, applying the effect onto the containing objects instance. This means, other objects using the same material will not be affected. This is useful if you just want the object you added the McSubscriber to render the incoming video. In the prior case, adding a material to Render Materials will update all objects whose Mesh Renderer utilizes that Material. This is useful if you have multiple objects, on which you want to render the same stream, and is more efficient.

Note: if you are using URP (Universal Render Pipeline), using the Standard Lit shader on the material will cause issues. Make sure to choose a Shader that works with your scene, otherwise the stream won't render. For example, using the Unlit texture will work on both URP and non-URP projects.