What is the error "Clashing resolution for expression. 1=1 != 8."?

[barakugav]

I created a model using Pytorch, saved it with torch.onnx.export, and when i try to run it from Rust i get an error.
Python code saving the model:

model.eval()
with torch.no_grad():
    torch.onnx.export(
        model,
        torch.randn(input_shape),
        path,
        verbose=False,
        input_names=["planes"],
        output_names=["value", "policy"],
    )

Rust code loading and running the model:

type OnnxModel = TypedRunnableModel<TypedModel>;
fn load_model(model_path: &str) -> TractResult<OnnxModel> {
    tract_onnx::onnx()
        .model_for_path(model_path)?
        .into_optimized()?
        .into_runnable()
}
let model = load_model(model_path).unwrap();

let input: Array4<f32> = ...;
let input: Tensor = input.into();
let outputs_res = self.model.run(tvec!(input.into()));
let outputs = outputs_res.unwrap(); // error occurred here

The error:

called `Result::unwrap()` on an `Err` value: Clashing resolution for expression. 1=1 != 8.

What does the error mean? How can I fix it? I think the error should be more informative in any case.

[kali]

Hello, thanks for tour interest in tract. In this case, it looks like tract can not make sense of the dimensions of tensor appearing in the network. It may be a tract bug, or something wrong with the model encoding. tract may be more picky than other frameworks with shape determination.

To get more details, you can try to dump the Debug form ("{:?}") of the error instead of just using unwrap: see https://github.com/sonos/tract/blob/main/cli/src/main.rs#L291 . it should give you some information about the context at least.

[barakugav]

Hey, thanks for the quick replay. When uwrap() fails it panic and format the error with {:?} (std code):

#[cfg(not(feature = "panic_immediate_abort"))]
#[inline(never)]
#[cold]
#[track_caller]
fn unwrap_failed(msg: &str, error: &dyn fmt::Debug) -> ! {
    panic!("{msg}: {error:?}")
}

Tried it manually and the printed error is the same.

Now that you mention that tract is more picky with shapes i see the problem, when I create the model the input shape is always (B=1,C,H,W) but in inference the batch size may be bigger.
Does the library only support the exact same shape? Its a bit limiting.
As I mentioned above, I think it worth adding some info to the error anyway

[kali]

There is something I dont get with the error formatting here. Tract error uses anyhow, so the contexts usually stack and are displayed in the Debug form. Unless this is a case where there is just no context specificied in the code, but it will be difficult to find from here without the model.

You need to use ONNX "dynamic axes" to export your model with a flexible batch size. In some cases, you can also "fix" this inside tract after loading the model, before calling into_optimized().

let mut model = tract_onnx::onnx().model_for_path(model_path)?;
let batch = model.symbol_table.sym("batch");
model.set_input_fact(0, f32::fact(dims!(batch, 3, 224, 224)).into())?
let model = model.into_optimized()?.into_runnable()?;
...

[barakugav]

Thanks! Modified my export code in Python and now it works:

torch.onnx.export(
  model,
  torch.randn(input_shape),
  path,
  verbose=False,
  input_names=["planes"],
  output_names=["policy", "value"],
  dynamic_axes={"planes": {0: "batch"}}, # New code
)

Thanks again for the support! Love the library :)