TensorFlow Lite Operator Support Query

[mtobin-tdab]

Hello!

I’ve been working with TensorFlow Lite models in tract and have come across a model which cannot be loaded. I've compared the operators used in themodel with those mentioned in the README for TFLite support and have identified the following operators from my model and was wondering if you could confirm which (if not all) are not currently supported by Tract:

DELEGATE
BATCH_MATMUL
VAR_HANDLE
READ_VARIABLE
CALL_ONCE
MAX_POOL_2D
If any of these operators have aliases or are supported in a beta release, I’d appreciate the clarification. I’m trying to determine if any unsupported operators are preventing my model from running properly with Tract.

Thanks in advance for your time and any insights you can provide!

[kali]

Hello, thanks for your interest in tract :)

In this list, MAX_POOL_2D and BATCH_MATMUL should be easy, as their semantics are already well represented in tract-core, so it is probably just a matter of translating/mapping/plugging stuff together.

The other ops are more worrisome. VAR_HANDLE and READ_VARIABLE seem to deal with variables. We have some support for variable and state in tract, but I don't know if their semantics will match tflite.

I can just guess what DELEGATE and CALL_ONCE are supposed to do :) DELEGATE is particularly scary as it hints at some kind of callback/extension point.

As you may have noticed, tflite is not a big priority for us, there is no active development on this binding. I would find the time to bind MAX_POOL_2D and BATCH_MATMUL if it helps, but I will not invest serious time sorting out the rest. I would of course help somebody trying to figure out what's needed and contributing support for these extra ops assuming it does not require a massive refactor in tract.

[mtobin-tdab]

Hi @kali sorry for my delayed response, I have been away.

Thanks for the detail. I think based on what you've said I will look to convert my TFLite model into ONNX format. My initial attempts at this have resulted in a model which produces invalid predictions when running inference with tract but I have not done any further investigation at this point.

[mtobin-tdab]

Hi again @kali, it appears that our model architecture requires the MAX_POOL_2D operator and the others are less strictly necessary.

Would you still be able to find time to bind MAX_POOL_2D and BATCH_MATMUL?

[kali]

Hey, as far as I can tell checking the code, both MAX_POOL_2D and BATCH_MATMUL are supported to some extent. It's hard to say how complete tract coverage is, as tflite provides neither formal spec nor test suite for validation of third party implementation.

tract tests translate many examples from the ONNX test suite into tflite files before 1/ running them in tfile and 2/ re-loading them again in tract, to check that tflite and tract implementation results are consistent with ONNX test suite. Both tflite MAX_POOL_2D and BATCH_MATMUL seem to receive some coverage from this.

So I think it's time for you to give it a shot and tell me how it goes.