Semantic segmentation is the task of clustering parts of an image together which belong to the same object class. It is a form of pixel-level prediction because each pixel in an image is classified according to a category.
This example shows how the DeepLab V3 MobileNet V2, from TFLite models may be implemented by using the WebNN API. (Note: the DeepLab V3 MobileNet V2 model is trained on the Pascal VOC 2012 dataset.)
The model referenced in this example only provides frozen graph, we use following tools and commands to convert which to TFLite and ONNX model.
Use tensorflow's optimize_for_inference tool to generate a stripped frozen graph:
python3 -m tensorflow.python.tools.optimize_for_inference \
--input=./frozen_inference_graph.pb \
--output=./frozen_inference_graph_stripped.pb \
--frozen_graph=True \
--input_names="sub_7" \
--output_names="ArgMax"
Use 'TensorFlow Lite converter' tool to convert stripped frozen graph to TFLite model:
tflite_convert \
--graph_def_file=frozen_inference_graph_stripped.pb \
--output_file=deeplab_mobilenetv2.tflite \
--output_format=TFLITE \
--input_format=TENSORFLOW_GRAPHDEF \
--input_arrays=sub_7 \
--output_arrays=ArgMax
Use 'tf2onnx' tool to convert TFLite model to ONNX model:
python3 -m tf2onnx.convert --opset 13 --tflite deeplab_mobilenetv2.tflite --output deeplab_mobilenetv2.onnx
Choose device, model and layout, in a very few second you will see the predict result for the test image be presented on the page. Operate the renderer menus to experience variance partitioning effects.
You could also click 'Pick Image' button to choose your local image to partition it.
Here we partition every frame in a live camera, then combined the results. Click 'LIVE CAMERA' tab, allow the browser to use your local camera if there's a prompt, then you will see real-time partitioning results on the page.
Switch model or layout to check variance predict result.