[Bug]: Benchmarking on distilbert / distilbert -base-uncased-finetuned-sst-2-english stuck at Step 10 / 11

[dgks0n]

OpenVINO Version

2022.3.2

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

ONNX

Model used

distilbert / distilbert -base-uncased-finetuned-sst-2-english

Issue description

I used benchmark_app to benchmark model distilbert / distilbert -base-uncased-finetuned-sst-2-english but it stuck at step 10 / 11 and never end.

Step-by-step reproduction

Command line is:

benchmark_app -m "/home/ubuntu/Workspaces/Projects/SuperModelOptimization/OpenVINO/workbench/wb/data/models/419/original/distilbert_distilbert
-base-uncased-finetuned-sst-2-english.xml"  -d "CPU"   -t "20" -shape "[1,128],[1,128]" -nstreams "0" --report_type "no_counters" --report_folder "/home/ubuntu/Workspaces/Projects/SuperModelOptimization/OpenVINO/workbench/wb/data/profiling_artifacts/305/job_artifacts/0_0" -hint none -pc --exec_graph_path "/home/ubuntu/Workspaces/Projects/SuperModelOptimization/OpenVINO/workbench/wb/data/profiling_artifacts/305/job_artifacts/0_0/exec_graph.xml"

Relevant log output

Here is output logs:

[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO:
[ INFO ] Build ................................. 2023.2.0-13089-cfd42bd2cb0-HEAD
[ INFO ] 
[ INFO ] Device info:
[ INFO ] CPU
[ INFO ] Build ................................. 2023.2.0-13089-cfd42bd2cb0-HEAD
[ INFO ] 
[ INFO ] 
[Step 3/11] Setting device configuration
[Step 4/11] Reading model files
[ INFO ] Loading model files
[ INFO ] Read model took 15.21 ms
[ INFO ] Original model I/O parameters:
[ INFO ] Model inputs:
[ INFO ]     input_ids (node: input_ids) : i64 / [N,C] / [1,100]
[ INFO ]     attention_mask (node: attention_mask) : i64 / [N,C] / [1,100]
[ INFO ] Model outputs:
[ INFO ]     logits (node: logits) : f32 / [...] / [1,2]
[Step 5/11] Resizing model to match image sizes and given batch
[ INFO ] Model batch size: 1
[ INFO ] Reshaping model: 'input_ids': [1,128], 'attention_mask': [1,128]
[ INFO ] Reshape model took 5.05 ms
[Step 6/11] Configuring input of the model
[ INFO ] Model inputs:
[ INFO ]     input_ids (node: input_ids) : i64 / [N,C] / [1,128]
[ INFO ]     attention_mask (node: attention_mask) : i64 / [N,C] / [1,128]
[ INFO ] Model outputs:
[ INFO ]     logits (node: logits) : f32 / [...] / [1,2]
[Step 7/11] Loading the model to the device
[ INFO ] Compile model took 346.85 ms
[Step 8/11] Querying optimal runtime parameters
[ INFO ] Model:
[ INFO ]   NETWORK_NAME: torch_jit
[ INFO ]   OPTIMAL_NUMBER_OF_INFER_REQUESTS: 0
[ INFO ]   NUM_STREAMS: 0
[ INFO ]   AFFINITY: Affinity.CORE
[ INFO ]   INFERENCE_NUM_THREADS: 0
[ INFO ]   PERF_COUNT: True
[ INFO ]   INFERENCE_PRECISION_HINT: <Type: 'float32'>
[ INFO ]   PERFORMANCE_HINT: PerformanceMode.LATENCY
[ INFO ]   EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE
[ INFO ]   PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ]   ENABLE_CPU_PINNING: True
[ INFO ]   SCHEDULING_CORE_TYPE: SchedulingCoreType.ANY_CORE
[ INFO ]   ENABLE_HYPER_THREADING: True
[ INFO ]   EXECUTION_DEVICES: ['CPU']
[ INFO ]   CPU_DENORMALS_OPTIMIZATION: False
[ INFO ]   CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1.0
[Step 9/11] Creating infer requests and preparing input tensors
[ WARNING ] No input files were given for input 'input_ids'!. This input will be filled with random values!
[ WARNING ] No input files were given for input 'attention_mask'!. This input will be filled with random values!
[ INFO ] Fill input 'input_ids' with random values 
[ INFO ] Fill input 'attention_mask' with random values 
[Step 10/11] Measuring performance (Start inference asynchronously, 0 inference requests using 0 streams for CPU, limits: 20000 ms duration)
[ INFO ] Benchmarking in inference only mode (inputs filling are not included in measurement loop).

### Issue submission checklist

- [X] I'm reporting an issue. It's not a question.
- [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
- [X] There is reproducer code and related data files such as images, videos, models, etc.

[dgks0n]

How can I solve this problem?

[Iffa-Intel]

@dgks0n could you provide:

1. Steps that you did to convert the model into OpenVINO format (xml and bin)
2. Relevant model files if possible
3. Have you consider of using a newer OpenVINO version? the one you are using is outdated (latest is version 2024.x)
4. Fyi, this distilbert/distilbert-base-cased-distilled-squad is officially supported by OpenVINO and the model in OV format is available for download & can be used directly with benchmark_app