How to send multiple streams on one PeerConnection ?

[acevest]

hi, i want to send multiple streams on one peerconnection.

i have tried this between webpage and webpage. it works.

but when i tried pion/webrtc communicate with webpage, it seems not work.

newVideoStream(iceConnectedCtx, peerConnection, videoFileName, "video_a", "pion_a")
newVideoStream(iceConnectedCtx, peerConnection, videoFileName, "video_b", "pion_b")

i can only got one stream pion_a on the webpage side，stream pion_b does not exist.

here is newVideoStream

func newVideoStream(iceConnectedCtx context.Context, peerConnection *webrtc.PeerConnection, fileName, id, streamID string) {
	videoTrack, videoTrackErr := webrtc.NewTrackLocalStaticSample(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeH264}, id, streamID)
	if videoTrackErr != nil {
		panic(videoTrackErr)
	}

	rtpSender, videoTrackErr := peerConnection.AddTrack(videoTrack)
	if videoTrackErr != nil {
		panic(videoTrackErr)
	}

	go func() {
		rtcpBuf := make([]byte, 1500)
		for {
			if _, _, rtcpErr := rtpSender.Read(rtcpBuf); rtcpErr != nil {
				return
			}
		}
	}()

	go func() {
		// Open a H264 file and start reading using our IVFReader
		file, h264Err := os.Open(fileName)
		if h264Err != nil {
			panic(h264Err)
		}

		h264, h264Err := h264reader.NewReader(file)
		if h264Err != nil {
			panic(h264Err)
		}

		// Wait for connection established
		<-iceConnectedCtx.Done()
		log.Printf("fucking video ...")
		ticker := time.NewTicker(h264FrameDuration)
		for ; true; <-ticker.C {
			nal, h264Err := h264.NextNAL()
			if h264Err == io.EOF {
				fmt.Printf("All video frames parsed and sent")
				os.Exit(0)
			}
			if h264Err != nil {
				panic(h264Err)
			}

			if h264Err = videoTrack.WriteSample(media.Sample{Data: nal.Data, Duration: time.Second}); h264Err != nil {
				panic(h264Err)
			}
		}
	}()
}

and init the peerconnection like this:

	// Enable Extension Headers needed for Simulcast
	m := &webrtc.MediaEngine{}
	if err := m.RegisterDefaultCodecs(); err != nil {
		panic(err)
	}
	for _, extension := range []string{
		"urn:ietf:params:rtp-hdrext:sdes:mid",
		"urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id",
		"urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id",
	} {
		if err := m.RegisterHeaderExtension(webrtc.RTPHeaderExtensionCapability{URI: extension}, webrtc.RTPCodecTypeVideo); err != nil {
			panic(err)
		}
		if err := m.RegisterHeaderExtension(webrtc.RTPHeaderExtensionCapability{URI: extension}, webrtc.RTPCodecTypeAudio); err != nil {
			panic(err)
		}
	}

	// Create a InterceptorRegistry. This is the user configurable RTP/RTCP Pipeline.
	// This provides NACKs, RTCP Reports and other features. If you use `webrtc.NewPeerConnection`
	// this is enabled by default. If you are manually managing You MUST create a InterceptorRegistry
	// for each PeerConnection.
	i := &interceptor.Registry{}

	// Use the default set of Interceptors
	if err := webrtc.RegisterDefaultInterceptors(m, i); err != nil {
		panic(err)
	}

	config := webrtc.Configuration{
		ICEServers: []webrtc.ICEServer{
			{
				URLs: []string{"stun:stun.l.google.com:19302"},
			},
		},
	}

	// Create a new RTCPeerConnection
	//peerConnection, err := webrtc.NewPeerConnection(config)
	peerConnection, err := webrtc.NewAPI(webrtc.WithMediaEngine(m), webrtc.WithInterceptorRegistry(i)).NewPeerConnection(config)
	if err != nil {
		panic(err)
	}

so, my question is:

1. does pion/webrtc support send multiple streams on one peerconnection ?
2. if support, how to do it ?

[Sean-Der]

Hey @acevest

pion/webrtc does support multiple audio/video tracks per PeerConnection.

I am not exactly sure what the problem is. I would try play-from-disk-renegotation. This example shows adding/removing multiple tracks to a single PeerConnection.

[acevest]

thank you @Sean-Der

with the help of this code play-from-disk-renegotation , i found a solution:

add this javascript code to the webpage

pc.addTransceiver('video');

and then i got the stream pion_b on the webpage webrtc client side.

but, it's a little different from

pion/webrtc trigger pc.ontrack twice, each time the event.streams.length == 1

webrtc between webpage and webpage, trigger pc.ontrack once, with the event.streams.length == 2

do you know why ?

[Sean-Der]

When you are doing webpage <-> webpage you are doing a MediaStream with two MediaStreamTracks

When you are doing pion <-> webpage you are doing two MediaStreams each with one MediaStreamTrack.

You can re-create the webpage <-> webpage experience by creating two tracks with the same streamId

[tuananhhust05]

Dear @acevest , i am new with webrtc, I'm working with a sfu project and have some issue with multiple stream, i sent multiple streams( exact 2) to server and restore it on array, after i tried send streams to an other client but this client just receive 1 stream. Can you give me some suggestions or keywords to resolve this problem ( receive mutiple stream with 1 peer connection) ,
Thank you very much!