Thursday, September 11, 2025
HomeiOS DevelopmentInconsistent UI Updates with CameraPermissionManager in iOS on Compose Multiplatform

Inconsistent UI Updates with CameraPermissionManager in iOS on Compose Multiplatform


I am engaged on a Compose Multiplatform challenge the place I’ve a display screen applied in commonMain that makes use of a CameraPermissionManager class, which is platform-specific. The display screen works fantastic on Android, however I am working into points on iOS the place the UI is not persistently updating primarily based on state adjustments.

This is a quick overview of my setup:
In commonMain:

I’ve a Composable perform App() that units up the UI. It features a ReadTextScreen composable the place the digicam preview is began and the detected textual content is processed.

@Composable
enjoyable App() {
    SosMilkTheme {
        ReadTextScreen()
    }
}

@Composable
personal enjoyable ReadTextScreen() {
    val cameraPermissionManager = bear in mind { CameraPermissionManager() }
    val snackBarHostState = bear in mind { SnackbarHostState() }
    var milkCondition by bear in mind { mutableStateOf(null) }
    var detected by rememberSaveable { mutableStateOf("") }

    cameraPermissionManager.RequestCameraPermission(
        onPermissionGranted = {
            Scaffold(
                snackbarHost = {
                    SnackbarHost(hostState = snackBarHostState)
                },
                topBar = { /* ... */ },
                bottomBar = {
                    FooterDetection(
                        title = "Detected Phrases:",
                        desc = detected,
                    )
                }
            ) { innerPadding ->
                Field {
                    cameraPermissionManager.StartCameraPreview { detectedText ->
                        val res = processDetectedTextUseCase.invoke(detectedText)
                        detected = detectedText
                        milkCondition = res
                    }
                    // different UI components...
                }
            }
        },
        onPermissionDenied = {
            BodyMedium(textual content = "Digital camera entry is required.")
        }
    )
}

In iosMain:

My CameraPermissionManager class is chargeable for dealing with digicam permissions and beginning the digicam preview. The textual content detection makes use of Apple’s Imaginative and prescient framework.

@OptIn(ExperimentalForeignApi::class)
precise class CameraPermissionManager {

    @Composable
    precise enjoyable RequestCameraPermission(
        onPermissionGranted: @Composable () -> Unit,
        onPermissionDenied: @Composable () -> Unit
    ) {
        // Permission dealing with logic
    }

@OptIn(ExperimentalForeignApi::class)
@Composable
precise enjoyable StartCameraPreview(onTextDetected: (String) -> Unit) {
    val system = AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo)
        .firstOrNull { system -> (system as AVCaptureDevice).place == AVCaptureDevicePositionBack } as? AVCaptureDevice

    if (system == null)  No digicam discovered")
        return
    

    val enter = AVCaptureDeviceInput.deviceInputWithDevice(system, null)
    if (enter == null)  No digicam enter discovered")
        return
    

    val videoOutput = AVCaptureVideoDataOutput()

    videoOutput.alwaysDiscardsLateVideoFrames = true
    videoOutput.videoSettings = mapOf("PixelFormatType" to kCVPixelFormatType_32BGRA)

    val session = AVCaptureSession()

    session.sessionPreset = AVCaptureSessionPresetHigh
    session.addInput(enter)
    session.addOutput(videoOutput)

    val cameraPreviewLayer = bear in mind { AVCaptureVideoPreviewLayer(session = session) }

    UIKitView(
        modifier = Modifier.fillMaxSize(),
        background = Colour.Black,
        manufacturing unit = {
            val container = UIView()
            container.layer.addSublayer(cameraPreviewLayer)
            cameraPreviewLayer.videoGravity = AVLayerVideoGravityResizeAspectFill
            cameraPreviewLayer.body = container.bounds
            dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT.toLong(), 0u)) {
                session.startRunning()
            }
            container
        },
        onResize = { container: UIView, rect: CValue ->
            CATransaction.start()
            CATransaction.setValue(true, kCATransactionDisableActions)
            container.layer.setFrame(rect)
            cameraPreviewLayer.setFrame(rect)
            CATransaction.commit()
        })

    val delegate = object : NSObject(), AVCaptureVideoDataOutputSampleBufferDelegateProtocol {
        override enjoyable captureOutput(
            output: AVCaptureOutput,
            didOutputSampleBuffer: CMSampleBufferRef?,
            fromConnection: AVCaptureConnection
        ) {
            didOutputSampleBuffer?.let { processSampleBuffer(it, onTextDetected) }
        }
    }

    videoOutput.setSampleBufferDelegate(delegate, dispatch_get_main_queue())
}

personal enjoyable processSampleBuffer(
    sampleBuffer: CMSampleBufferRef,
    onTextDetected: (String) -> Unit
) {
    val pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)

    if (pixelBuffer == null)  Pixel buffer is null")
        return
    

    val handler = VNImageRequestHandler(pixelBuffer, choices = mapOf())

    val request = VNRecognizeTextRequest { request, error ->
        if (error != null)  Error recognizing textual content: $error")
            return@VNRecognizeTextRequest
        

        val observations = request?.outcomes?.filterIsInstance()

        observations?.forEach { remark ->
            val topCandidates = remark.topCandidates(1u)
            topCandidates.firstOrNull()?.let { candidate ->
                val recognizedText = candidate as? VNRecognizedText
                recognizedText?.string?.let { textual content ->
                    onTextDetected(textual content)
                } ?: run  Acknowledged textual content is null")
                
            }
        }
    }

    attempt {
        handler.performRequests(listOf(request), null)
    } catch (e: Exception) 
}

The Drawback:
On iOS, the textual content detected by the digicam (detectedText) is just not reliably updating the UI components just like the FooterDetection and the milkCondition. Nevertheless, I can see the right values being logged, so the textual content is being detected correctly. The difficulty appears to be associated to how the UI state updates are dealt with, doubtlessly attributable to threading.

Potential Trigger:

I am conscious that on iOS, UI updates have to happen on the principle thread. I’ve already wrapped the onTextDetected callback in dispatch_async(dispatch_get_main_queue()) to make sure it is working on the principle thread. Regardless of this, the UI updates are nonetheless inconsistent, and I am not sure if there’s one thing else I am lacking associated to string dealing with or Compose-specific habits on iOS.

Query:
Has anybody else encountered comparable points with Compose Multiplatform on iOS? Is there one thing particular to Compose on iOS that I would like to think about for guaranteeing constant UI updates? Any steering or options could be tremendously appreciated!

Thanks prematurely on your assist!

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments