test

Vidyo.io Group Layout

When you are designing an application that supports group video chat, you are faced with the several questions:

  1. How to draw your video tiles on the screen?
  2. What sizes should each video tile be?
  3. Can I show all of them? Or should I show a subset of the participants?
  4. How do I handle different aspect ratios?

For some scenarios, such as team collaboration use cases, you might want to equally distribute the space for all video tiles. While in other scenarios, such as a primary speaker use case, you may want to optimize the space for one source. The Vidyo.io framework allows you to choose the right layout for your scenario.

We recently covered the topic of custom vs. composited layouts in one of our getting started webinars. Here is the key tutorial:


With Vidyo.io, the video is routed through the Vidyo.io cloud and is delivered to the endpoint as individual streams. The video is not composited on the server but instead delivered to the endpoint, allowing a great degree of control over how the video is displayed.  Your client application can choose how to composite the video streams. The Vidyo Client SDKs gives you two approaches to displaying the layouts and works for WebRTC, Android, iOS, Windows, and macOS based apps.

Two Layout Options for Vidyo Client SDK

  1. Composite Layout – default layout provided by SDK
  2. Custom Layout – layout determined by your app

Composite Layout

With this approach the Vidyo client library composites the video streams for your application. Your application just assigns one single view or window where the video streams will get drawn and passes it to the Vidyo library. Your application can control the size of this view, but it does not have worry about individual video streams. The Vidyo Client SDK will take care of laying them out. You can specify the maximum number of participants you want to see on the screen and the SDK takes care of the rest.

VC.CreateVidyoConnector({
viewId: "renderer", // Div ID where the composited video will be rendered
viewStyle: "VIDYO_CONNECTORVIEWSTYLE_Default", // Visual style of the composited renderer
remoteParticipants: 16, // Maximum number of participants to render
logFileFilter: "warning",
logFileName:"",
userData:""
}).then(function(vc) {
// Success
}).catch(function() {
// Handle failure
});

Using this option makes the application code very simple. All you have to do is assign a view to the video conference UI and then you are done. You still get to decide what is the size of that view and you get the freedom to integrate that view with the rest of your application’s UI. While you don’t have to worry about how many video tiles are actually drawn within that view. The Vidyo SDK takes care of dynamically adding and removing video tiles as they come and go. It will resize the individual tiles accordingly.

One Person Call (Just Preview)

preview

Two Person Call

two people

Three Person Call

three people

Custom Layout

While the composited layout takes away all the resize complexity from the developer. Sometimes the scenarios requires specific layout configurations. For example you may have a virtual classroom where you would like the teachers video to take a bigger share of the view space compared to other student video tiles.

custom layout

In such cases the default composited layout options may not be suitable for your application. In which case you can choose to layout the individual video tiles yourself. The Vidyo SDK will still take care of rendering the video, you can still choose which remote participant video to render, at what size and where it will lie in your application UI.

To choose this option, when you create the VidyoConnector object, you need to pass null (or nil depending on your platform) as the view ID.

VC.CreateVidyoConnector({
viewId: null, // Set to null in order to create a custom layout
viewStyle: "VIDYO_CONNECTORVIEWSTYLE_Default", // Will be ignored
remoteParticipants: 16, // Will be ignored
logFileFilter: "warning",
logFileName: "VidyoConnector.log",
userData: 0
}).then(function(vc) {
// Success
}).catch(function(err) {
// Handle error
});

Once you pass null in your VidyoConnector constructor, you are basically telling the SDK that you will be handling each video tile yourself. To get notified when these remote participants and their videos come online, you need to register to the camera events.

vidyoConnector.RegisterRemoteCameraEventListener({
onAdded: function(camera, participant) {
// Callback when a remote participant shares their camera
},
onRemoved: function(camera, participant) {
// Callback when a remote participant stops sharing camera
},
onStateUpdated: function(camera, participant, state) {
// Camera state was updated
}
}).then(function() {
console.log("RegisterRemoteCameraEventListener Success");
}).catch(function() {
console.error("RegisterRemoteCameraEventListener Failed");
});

Along with the remote participant videos, you need to also register to local camera events so that you can show a preview of the self view.

vidyoConnector.RegisterLocalCameraEventListener({
onAdded: function(localCamera) {
// New local camera added
},
onRemoved: function(localCamera) {
// Local camera removed
},
onSelected: function(localCamera) {
// Camera was selected/unselected by you or automatically
}
}).then(function() {
console.log("RegisterLocalCameraEventListener Success");
}).catch(function() {
console.error("RegisterLocalCameraEventListener Failed");
});

Once you get the local or remote camera you need to assign a view/window for it –

// Assign view to local camera
vidyoConnector.AssignViewToLocalCamera({
viewId: "localRenderer", // view ID specific to render local camera
localCamera: localCamera,
displayCropped: true,
allowZoom: false
}).then(function() {

}).catch(function(e) {

});
// Assign View to remote camera
vidyoConnector.AssignViewToRemoteCamera({
viewId: "remoteRenderer", // view ID specific to this remote camera
remoteCamera: camera,
displayCropped: true, // Show video letterboxed or cropped to fit
allowZoom: false
}).then(function(retValue) {

}).catch(function() {

});

Once assigned if you change the size of the view you can call ShowViewAt to update the video size.

vidyoConnector.ShowViewAt(viewId, x, y, width, height);

Audio Devices Setup

For composited layout, the audio devices (speaker and microphone) is automatically setup, but for custom layout the Vidyo Client SDK waits for the go ahead from the application to start audio devices as well. To configure audio devices in custom layout, all you need to do is call register device events.

// Handle appearance and disappearance of microphone devices in the system
vidyoConnector.RegisterLocalMicrophoneEventListener({
onAdded: function(localMicrophone) {
// New microphone is available
},
onRemoved: function(localMicrophone) {
// Existing microphone became unavailable
},
onSelected: function(localMicrophone) {
// Microphone was selected/unselected by you or automatically
},
onStateUpdated: function(localMicrophone, state) {
// Microphone state was updated
}
}).then(function() {
console.log("RegisterLocalMicrophoneEventListener Success");
}).catch(function() {
console.error("RegisterLocalMicrophoneEventListener Failed");
});

// Handle appearance and disappearance of speaker devices in the system
vidyoConnector.RegisterLocalSpeakerEventListener({
onAdded: function(localSpeaker) {
// New speaker is available
},
onRemoved: function(localSpeaker) {
// Existing speaker became unavailable
},
onSelected: function(localSpeaker) {
// Speaker was selected/unselected by you or automatically
},
onStateUpdated: function(localSpeaker, state) {
// Speaker state was updated
}
}).then(function() {
console.log("RegisterLocalSpeakerEventListener Success");
}).catch(function() {
console.error("RegisterLocalSpeakerEventListener Failed");
});

Note that you don’t have to do anything in the callbacks to have basic audio setup, just register for it.

Summary

For most applications, the SDK provided composited layout will work fine. The default composited layout  is very simple to use and yet provides a sophisticated group video conference layout. But when you have a specific layout requirement or would like to tune your video layouts, you can always use the custom option.

To see how custom layouts are handled in the platform of your choice, just download the package from https://developer.vidyo.io/packages and check out the samples.

Try vidyo.io for free