Using supervision with onnxruntime

[acode-x]

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Hi,
I am having yolo11.onnx model. How to use supervision along with onnxruntime?

 def callback(frame: np.ndarray, _: int) -> np.ndarray:
    onnx_model_path = "./yolo11n.onnx"
    session = ort.InferenceSession(onnx_model_path, providers=['CPUExecutionProvider'])
  
    input_tensor = preprocess(frame) # do something here?
    outputs = session.run(None, {"images": input_tensor})
    detections = postprocess(outputs) # do something here?
    detections = tracker.update_with_detections(detections)
    
sv.process_video(source_path="people-walking.mp4", target_path="result.mp4", callback=callback)

I couldn't use torch / ultralytics because I had to run inference on edge device and it was too heavy.
Appreciate if any help on same.

Thanks!

Additional

No response

[onuralpszr]

@acode-x 👋 Hello, We don't have onnxruntime yet to be supported but for edge cases I understand you wanted to be minimal, I made a rather small collab for you to run onnxruntime with supervision I hope you enjoy

https://colab.research.google.com/drive/1eDM_MFuMgvb3znAXq31GtwQHYHPcNaL6?usp=sharing

PS : I used yolov11 but If you want to use yolov8 it should also work without any code. change as well. Good luck ! :)

[acode-x]

Thank you so much @onuralpszr
I was successfully able to run inference using the notebook you provided!

I also tried below object tracking tutorial using the same YOLO11 class you provided: https://supervision.roboflow.com/develop/notebooks/count-objects-crossing-the-line/#run-object-detection

https://colab.research.google.com/drive/1hEfzobxNHGNsfP97Iz8_D3LHoD9hGpij?usp=sharing

Got this error. Anything obvious I could be missing.

[onuralpszr]

You got empty xyxy detection. Basically, you can do none check

[onuralpszr]

Thank you so much @onuralpszr
I was successfully able to run inference using the notebook you provided!

I also tried below object tracking tutorial using the same YOLO11 class you provided: https://supervision.roboflow.com/develop/notebooks/count-objects-crossing-the-line/#run-object-detection

https://colab.research.google.com/drive/1hEfzobxNHGNsfP97Iz8_D3LHoD9hGpij?usp=sharing

Got this error. Anything obvious I could be missing.

Btw please select as answer for my first reply for your main question as well

[acode-x]

Thanks a lot for your quick support! Object tracker is also working now.

[acode-x]

Hello @onuralpszr,
Are you aware of tflite implementation for same use case? i.e. run yolov8n_int8.tflite along with tflite-runtime.
I found an existing implementation (but for images): https://github.com/ultralytics/ultralytics/blob/main/examples/YOLOv8-OpenCV-int8-tflite-Python/main.py

Just curious if I can reuse same implementation.

[onuralpszr]

Hello @onuralpszr, Are you aware of tflite implementation for same use case? i.e. run yolov8n_int8.tflite along with tflite-runtime. I found an existing implementation (but for images): https://github.com/ultralytics/ultralytics/blob/main/examples/YOLOv8-OpenCV-int8-tflite-Python/main.py

Just curious if I can reuse same implementation.

Please ask new question to keep this discussion clean :))