Streaming to external networks

[JanHalozan]

I'm wondering how can this sample be used to stream to external networks.

For example I have an AWS instance running the USD Viewer app template. And the streaming locally works perfectly.

However I'm having trouble exposing the stream to external networks. I've opened the required ports on my security policy (TCP & UDP: 80, 5173, 49100, 8211, 5173, 443, 8111, 8443) and I can see that some connections are coming through however the stream doesn't start.

In case the two are related: I'm having the same issue with using Isaac Sim and the WebRTC client extension which loads the website but the stream doesn't start. I've checked the docs regarding streaming across networks but nothing seems to work.

Appreciate the help!

[quantyverse]

I am having th same issue and tried a lot. I set up coturn for webrtc etc. but I am unable to get the stream to work. Did you find a solution ?

[qmiinh]

I have the same issue

[winner6]

Same issue on my side. If I provide public ip(instead of 127.0.0.1) inside stream.config.json:
"local": {
"$comment": "Required props if source is set to 'local'.",
"server": "some ip"
}

On the client web browser I see next error:
WebSocket connection to 'ws://34.141.91.144:49100/sign_in?peer_id=peer-8129201440&version=2' failed.

If I start web viewer on https I'm getting another error:
https://some ip:49100/v2/session?keyboardLayout=en-US&languageCode=en_US net::ERR_CONNECTION_REFUSED

[toolmanson]

The computer providing the kit app stream and the computer running the client needs to be able to access each other. We recently added documentation on ports: https://docs.omniverse.nvidia.com/embedded-web-viewer/latest/create/run-the-solution.html#one-workstation-to-another

[toolmanson]

Are the 2 computers on the same network?

[JanHalozan]

I did manage to get it working eventually. As the docs mention a TURN server might be required. In my case I installed coturn and then edited the extensions configuration file and added the TURN server credentials in the ice_servers section.

It is quite a pain in Windows - I did it via WSL. If anyone wants detailed instructions on how to do it let me know but the general checklist is:

• Get a TURN server working on the host machine
• Edit the extension.toml file of the omni.services.streamclient.webrtc extension
• Add the IP of your machine and TURN credentials
• Start omniverse extension and it should hopefully work

[ThaiLe-gt]

@JanHalozan I tried your suggestion, but it didn't work on my end ( I installed a KIT 106 version onto an Azure VM). and It seems that omni.kit.livestream.webrtc doesn't support configuring a TURN server through ice_servers

@toolmanson. qmiinh and I are working together. We can stream locally, but we cannot stream the KIT from the cloud server, even though we installed the OVAS system as described in the OVAS Our KIT app still isn't streaming to the client, with the logs as qmiinh shared.

[JanHalozan]

@ThaiLe-gt Sorry I meant omni.services.streamclient.webrtc. You should see an ice_servers section in the config.

[ThaiLe-gt]

Thank you, @JanHalozan. Your suggestion with omni.services.streamclient.webrtc worked!

Regarding the new OVAS system, we contacted a solutions architect from NVIDIA today. Our memcache configuration in Azure cloud took a lot of time to compile shaders, which caused the client to time out on the request. We increased the request timeout on the client, and fortunately, it worked.

[quantyverse]

Thank you for the provided information. I am trying the above but it is not working. Are you also using the the kit-app-template and creating an app based on the USD Viewer and starting the template with the streaming suffix? When looking into the dependencies i can not find omni.services.streamclient.webrtc to set the ice_servcer settings. Where should it be located (I thought in "extcache") ?

Also when trying to connect from a client from another computer from outside the network i got the following messages on the kit application:

←[33mmain: thread_init: already added for thread
←[;m←[33mmain: thread_init: already added for thread
←[;m2024-10-22 12:40:08 [171,444ms] [Warning] [carb.livestream-rtc.plugin] onClientEventRaised: NVST_CCE_DISCONNECTED when m_connectionCount 4294967292 != 1
←[33mmain: thread_init: already added for thread
←[;m←[33mmain: thread_init: already added for thread
←[;m2024-10-22 12:40:31 [195,339ms] [Warning] [carb.livestream-rtc.plugin] onClientEventRaised: NVST_CCE_DISCONNECTED when m_connectionCount 4294967291 != 1

I guess I am missing something. Does somebody has an idea ?

[toolmanson]

omniverse-webrtc-streaming-library 3.2.0 has been released and it allows setting number of connection attempts to kit app stream. Note that the API has changed so you'll need to revisit the AppStreamer use. Working on updating sample. In the meantime:

AppStreamer.connect({
            streamSource: 'direct',
            streamConfig: {
                videoElementId: 'remote-video',
                audioElementId: 'remote-audio',
                signalingserver: ovasServer.signalingserver,
                signalingport: ovasServer.signalingport,
                mediaserver: ovasServer.mediaserver,
                mediaport: ovasServer.mediaport,
                accessToken: ovasServer.accessToken,
                sessionid: ovasServer.sessionid,
                authenticate: false,
                maxReconnects: 20,
                connectivityTimeout: 120000,
                nativeTouchEvents: true,
                width: 1920,
                height: 1080,
                fps: 60,
                cursor: 'free',
                autolaunch: true,
                onUpdate: handleStreamUpdate,
                onStart: handleStreamStart,
                onCustomEvent: handleStreamCustomEvent,
                onStop: handleStreamStop,
                onTerminate: handleStreamTerminate,
            }
        })
            .then((result: StreamEvent) => {
                console.info('AppStreamer Setup Success:', result);
            })
            .catch((error: StreamEvent) => {
                console.error('AppStreamer Setup Error:', error);
            });

[toolmanson]

@quantyverse those kit log entries may be fine - what does the client console/log look like?

[toolmanson]

@quantyverse what kit version re you using?

[quantyverse]

@toolmanson thank you. i cloned the newest kit app template which is 106.1. I am going to have a look into the console/log of the client. Where can i find the AppStreamer.connect in the web viewer sample ?

[toolmanson]

@quantyverse the most recent (main branch) is Kit 106.2

Where can i find the AppStreamer.connect in the web viewer sample ?

it's not out yet - please see above #3 (comment)

[toolmanson]

you can see kit version here https://github.com/NVIDIA-Omniverse/kit-app-template/blob/main/tools/deps/kit-sdk.packman.xml

[toolmanson]

you would need to refactor here https://github.com/NVIDIA-Omniverse/web-viewer-sample/blob/main/src/AppStream.tsx#L90

[toolmanson]

another thought: have you tried streaming without loading an asset - or just loading an asset that just has a cube or something super simple in it? Just to rule out that kit isn't getting into some weird state because of asset

[quantyverse]

@toolmanson i got the following in the webclient:

and from the kit application on the server:

I have kit version 106.2.0.
i now tried to use kit-base-editor with streaming and used viewport only option in the App.tsx from the sample.

the kit app is running on an aws vm.

[toolmanson]

@quantyverse

1. client is using omniverse-webrtc-streaming-library 3.2.0?
2. how many maxReconnects do you have set?
3. and you wait for all attempts to happen - and it still does not work?

[quantyverse]

@toolmanson

1. yes - version 3.2.0 now
2. 20
3. yes i wait and it does not work

Maybe I understand something wrong. Since the Omniverse Streaming App is running on AWS EC2 instance and the client is running on my local computer I thought that I need to use a TURN server (coturn on ubuntu aws machine) ? But i can not find any settings to set up iceserver settings with username and password in the StreamProps (RagnarokConfig) when doing Appstreamer.connect. Or is it somehow already configured internally ?

[toolmanson]

see this post: https://forums.developer.nvidia.com/t/streaming-of-kit-app-template-docker-container/311815/3
I think you'll need to use the OV App Streaming API

[quantyverse]

@toolmanson thank you for your help. I am going to have a look into that.