Handling device rotation in "preview" mode

[replicant1]

Hi Pedro,

I have a long-running problem I'm hoping you can shed some light on. I am using an OpenGlView as the basis for showing video. I am finding that in "preview" mode, if I rotate the device to a new orientation, the preview sometimes appears correctly but other times is distorted as if it has not adapted to the new aspect ratio correctly. I have tried everything to fix this, but I can't get the preview to handle an orientation change consistently correctly.

The best I have found is to register a SurfaceHolder.Callback and in the surfaceChanged() method call camera1.stopPreview() and then camera1.startPreview() to "hard reset" the preview. This gives correct behaviour about 50 percent of the time.

Other things I have tried (unsuccessfully) include:

• Listening for the orientationChange() in the containing Activity and using that to trigger the stopPreview() then startPreview()
• Callinginvalidate()and requestLayout() on the OpenGlView
• Doing the above a few seconds after the orientation change, to give things time to "settle down"
• Too many other desperate things to remember

The only saving grace I have is that when streaming starts and prepareVideo() is called, that always makes the video appear correctly with respect to the current device orientation. I don't know why prepareVideo() always works, but nothing else will work consistently.

Have you ever encountered problems with the OpenGlView like this before, and do you have any suggestions on how I could fix this?

I looked at your sample app, which does handle device rotation correctly, but I note that you're using SurfaceView and AutoFitTextureView. I can only find one example of OpenGlView (in FromFileActivity.kt) but this Activity doesn't handle device rotations. Have you intentionally omitted device rotation from the FromFileActivity because of problems with OpenGlView handling it?

Anyway - sorry for the long and rambling post. I would appreciate any suggestions you might have on how to handle device rotation correctly when previewing. Is there something I can do to get the OpenGlView handling it correctly, or should I give up on OpenGlView and switch to another type of view like SurfaceView or AutoFitTextureView? Do I need to call prepareVideo() after an orientation change to get correct behaviour.

Thanks in advance for your patience.

Rod

[pedroSG94]

Hello,

If you want handle device rotation, currently, StreamBase clases is the recommended way (this is because you can handle device orientation easier). Anyway, OpenGlView should works without problems. Currently I haven't an example rotation with OpenGlView for that reason.
You have an old example here (should works exactly the same way, you only need replace ConnectCheckerRtmp to ConnectChecker):
https://github.com/pedroSG94/RootEncoder/blob/eca8783a07fb55613b36e0caf7bf92b11cbc18af/app/src/main/java/com/pedro/streamer/openglexample/OpenGlRtmpActivity.java
Remember lock the screen while you are streaming or recording or the stream will stop automatically with that code.

If the problem persist share me a video about your case and code modified to do it. Also, tell me your device screen size to fully reproduce your case.

[replicant1]

Thanks for your help! I'll look at the code you've linked to and follow its example.

[replicant1]

Hi Pedro,

Quick followup. Looking at the difference between your code and mine, I think I've narrowed down the difference. In the manifest file, I have the following:

<activity 
   android:name = ".MainActivity"
   android:configChanges="orientation|screenSize"
   ...
/>

The OpenGlView will NOT handle device orientation changes correctly while this android:configChanges parameter is set. But if I remove the android:configChanges parameter all together, the OpenGlView DOES handle device orientation changes correctly.

The android:configChanges governs whether the Activity is recreated upon a configuration change. In other words, the OpenGlView only behaves correctly when the Activity hosting it (.MainActivity in this case) IS recreated on orientation change.

Is this what you would expect? If you would like to see code that displays a minimal example of this, check out this repo:

https://github.com/replicant1/OpenGLViewRootEncoderProblem

I'm not sure if I am able to remove theandroid:configChangesparameter in my own app without having unwanted side effects for the rest of the app - I'm yet to look into this.

Thanks for your time.

[pedroSG94]

Hello,

Your are right, OpenGlView can't handle orientation automatically with that parameter but it is expected for your case because OpenGlView was designed to be recreated on activity rotation. You can fix it stopping preview if preview is active when the OpenGlView size change:

    override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
        if (rtmpCamera1.isOnPreview) rtmpCamera1.stopPreview()
        rtmpCamera1.startPreview(CameraHelper.Facing.BACK, 1280, 720)
    }

This is working for me in the project that you shared but, I recommend you use RtmpStream for your case if your min sdk version is 21+:

class InstructorStreamFragment : Fragment(), ConnectChecker, SurfaceHolder.Callback {
    private lateinit var binding: FragmentInstructorStreamBinding
    private lateinit var rtmpCamera1: RtmpStream
    private lateinit var surfaceView: SurfaceView

    override fun onCreateView(
        inflater: LayoutInflater,
        container: ViewGroup?,
        savedInstanceState: Bundle?,
    ): View {
        super.onCreate(savedInstanceState)

        binding = FragmentInstructorStreamBinding.inflate(inflater, container, false)

        surfaceView = binding.surfaceView
        rtmpCamera1 = RtmpStream(requireContext(), this).apply {
            getGlInterface().autoHandleOrientation = true
        }

       //RtmpStream need configure audio and video before start preview
        val rotation = 0 //0 for landscape resolution stream (1280x720), 90 for portrait resolution (720x1280)
        rtmpCamera1.prepareAudio(44100, true, 128 * 1000)
        rtmpCamera1.prepareVideo(1280, 720, 3000 * 1000, rotation = rotation)

        surfaceView.holder.addCallback(this)
        return binding.root
    }

    override fun onDestroyView() {
        super.onDestroyView()
        rtmpCamera1.release()
    }

    override fun onAuthError() {

    }

    override fun onAuthSuccess() {

    }

    override fun onConnectionFailed(reason: String) {

    }

    override fun onConnectionStarted(url: String) {

    }

    override fun onConnectionSuccess() {

    }

    override fun onDisconnect() {

    }

    override fun onNewBitrate(bitrate: Long) {

    }

    override fun surfaceCreated(holder: SurfaceHolder) {
        rtmpCamera1.startPreview(surfaceView)
    }

    override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
        rtmpCamera1.getGlInterface().setPreviewResolution(width, height)
    }

    override fun surfaceDestroyed(holder: SurfaceHolder) {
        rtmpCamera1.stopPreview()
    }

}

[replicant1]

Hi Pedro - thank you very much for your comprehensive answer. Much appreciated!