Skip to content

Commit

Permalink
tolerance tweaks
Browse files Browse the repository at this point in the history
  • Loading branch information
@lintool
lintool committed Sep 15, 2024
1 parent 5996a8e commit 33ed276
Show file tree
Hide file tree
Showing 48 changed files with 88 additions and 88 deletions.
2 changes: 1 addition & 1 deletion src/main/resources/regression/beir-v1.0.0-bioasq.bge-base-en-v1.5.flat-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,6 @@ models:
nDCG@10:
- 0.004
R@100:
- 0.001
- 0.004
R@1000:
- 0.001
4 changes: 2 additions & 2 deletions src/main/resources/regression/beir-v1.0.0-bioasq.bge-base-en-v1.5.flat-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ models:
- 0.8059
tolerance:
nDCG@10:
- 0.003
- 0.004
R@100:
- 0.002
- 0.003
R@1000:
- 0.002
2 changes: 1 addition & 1 deletion src/main/resources/regression/beir-v1.0.0-climate-fever.bge-base-en-v1.5.hnsw.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,6 @@ models:
nDCG@10:
- 0.001
R@100:
- 0.001
- 0.002
R@1000:
- 0.002
2 changes: 1 addition & 1 deletion ...in/resources/regression/beir-v1.0.0-dbpedia-entity.bge-base-en-v1.5.flat-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,6 @@ models:
nDCG@10:
- 0.001
R@100:
- 0.001
- 0.004
R@1000:
- 0.002
2 changes: 1 addition & 1 deletion src/main/resources/regression/beir-v1.0.0-hotpotqa.bge-base-en-v1.5.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ models:
R@100:
- 0.025
R@1000:
- 0.025
- 0.03
4 changes: 2 additions & 2 deletions src/main/resources/regression/beir-v1.0.0-nq.bge-base-en-v1.5.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ models:
- 0.9859
tolerance:
nDCG@10:
- 0.003
- 0.004
R@100:
- 0.006
R@1000:
- 0.006
- 0.007
2 changes: 1 addition & 1 deletion src/main/resources/regression/beir-v1.0.0-robust04.bge-base-en-v1.5.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,6 @@ models:
nDCG@10:
- 0.001
R@100:
- 0.003
- 0.004
R@1000:
- 0.004
4 changes: 2 additions & 2 deletions src/main/resources/regression/beir-v1.0.0-robust04.bge-base-en-v1.5.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ models:
- 0.5981
tolerance:
nDCG@10:
- 0.003
R@100:
- 0.004
R@100:
- 0.005
R@1000:
- 0.006
4 changes: 2 additions & 2 deletions src/main/resources/regression/beir-v1.0.0-signal1m.bge-base-en-v1.5.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ models:
- 0.5331
tolerance:
nDCG@10:
- 0.006
- 0.025
R@100:
- 0.02
R@1000:
- 0.03
- 0.035
6 changes: 3 additions & 3 deletions src/main/resources/regression/beir-v1.0.0-signal1m.bge-base-en-v1.5.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ models:
- 0.5331
tolerance:
nDCG@10:
- 0.006
- 0.02
R@100:
- 0.015
- 0.02
R@1000:
- 0.03
- 0.035
2 changes: 1 addition & 1 deletion src/main/resources/regression/beir-v1.0.0-trec-covid.bge-base-en-v1.5.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ models:
R@100:
- 0.001
R@1000:
- 0.001
- 0.002
2 changes: 1 addition & 1 deletion src/main/resources/regression/beir-v1.0.0-trec-news.bge-base-en-v1.5.flat-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ models:
R@100:
- 0.005
R@1000:
- 0.001
- 0.002
2 changes: 1 addition & 1 deletion src/main/resources/regression/beir-v1.0.0-trec-news.bge-base-en-v1.5.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,6 @@ models:
nDCG@10:
- 0.015
R@100:
- 0.01
- 0.015
R@1000:
- 0.02
2 changes: 1 addition & 1 deletion .../resources/regression/beir-v1.0.0-webis-touche2020.bge-base-en-v1.5.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ models:
- 0.8298
tolerance:
nDCG@10:
- 0.004
- 0.008
R@100:
- 0.002
R@1000:
Expand Down
2 changes: 1 addition & 1 deletion ...in/resources/regression/beir-v1.0.0-webis-touche2020.bge-base-en-v1.5.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ models:
- 0.8298
tolerance:
nDCG@10:
- 0.005
- 0.008
R@100:
- 0.003
R@1000:
Expand Down
4 changes: 2 additions & 2 deletions src/main/resources/regression/dl19-passage.bge-base-en-v1.5.flat-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ models:
AP@1000:
- 0.004
nDCG@10:
- 0.005
R@100:
- 0.006
R@100:
- 0.007
R@1000:
- 0.005
6 changes: 3 additions & 3 deletions src/main/resources/regression/dl19-passage.bge-base-en-v1.5.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ models:
AP@1000:
- 0.008
nDCG@10:
- 0.015
R@100:
- 0.02
R@1000:
R@100:
- 0.025
R@1000:
- 0.03
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl19-passage.bge-base-en-v1.5.hnsw.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ models:
- 0.8472
tolerance:
AP@1000:
- 0.001
- 0.002
nDCG@10:
- 0.015
R@100:
Expand Down
4 changes: 2 additions & 2 deletions src/main/resources/regression/dl19-passage.cohere-embed-english-v3.0.hnsw.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,6 @@ models:
nDCG@10:
- 0.015
R@100:
- 0.007
- 0.01
R@1000:
- 0.025
- 0.03
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl19-passage.cos-dpr-distil.flat-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,6 @@ models:
nDCG@10:
- 0.002
R@100:
- 0.002
- 0.003
R@1000:
- 0.001
6 changes: 3 additions & 3 deletions src/main/resources/regression/dl19-passage.cos-dpr-distil.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8201
tolerance:
AP@1000:
- 0.003
- 0.015
nDCG@10:
- 0.015
R@100:
- 0.009
- 0.025
R@1000:
- 0.015
- 0.025
8 changes: 4 additions & 4 deletions src/main/resources/regression/dl19-passage.cos-dpr-distil.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8201
tolerance:
AP@1000:
- 0.003
nDCG@10:
- 0.015
nDCG@10:
- 0.02
R@100:
- 0.009
- 0.025
R@1000:
- 0.015
- 0.025
6 changes: 3 additions & 3 deletions src/main/resources/regression/dl19-passage.cos-dpr-distil.hnsw.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8201
tolerance:
AP@1000:
- 0.007
- 0.015
nDCG@10:
- 0.025
R@100:
- 0.015
R@1000:
- 0.02
R@1000:
- 0.025
6 changes: 3 additions & 3 deletions src/main/resources/regression/dl19-passage.cos-dpr-distil.hnsw.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8201
tolerance:
AP@1000:
- 0.007
- 0.015
nDCG@10:
- 0.025
R@100:
- 0.015
R@1000:
- 0.02
R@1000:
- 0.025
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl20-passage.bge-base-en-v1.5.flat-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ models:
- 0.8503
tolerance:
AP@1000:
- 0.003
- 0.004
nDCG@10:
- 0.002
R@100:
Expand Down
6 changes: 3 additions & 3 deletions src/main/resources/regression/dl20-passage.bge-base-en-v1.5.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8503
tolerance:
AP@1000:
- 0.007
- 0.008
nDCG@10:
- 0.008
R@100:
- 0.015
- 0.02
R@1000:
- 0.025
- 0.03
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl20-passage.bge-base-en-v1.5.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ models:
- 0.8503
tolerance:
AP@1000:
- 0.009
- 0.01
nDCG@10:
- 0.008
R@100:
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl20-passage.cohere-embed-english-v3.0.flat-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ models:
AP@1000:
- 0.001
nDCG@10:
- 0.003
- 0.004
R@100:
- 0.001
R@1000:
Expand Down
4 changes: 2 additions & 2 deletions src/main/resources/regression/dl20-passage.cohere-embed-english-v3.0.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8682
tolerance:
AP@1000:
- 0.003
- 0.005
nDCG@10:
- 0.001
R@100:
- 0.015
- 0.02
R@1000:
- 0.02
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl20-passage.cohere-embed-english-v3.0.hnsw.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ models:
- 0.8682
tolerance:
AP@1000:
- 0.004
- 0.005
nDCG@10:
- 0.001
R@100:
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl20-passage.cos-dpr-distil.flat-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ models:
AP@1000:
- 0.002
nDCG@10:
- 0.002
- 0.005
R@100:
- 0.004
R@1000:
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl20-passage.cos-dpr-distil.flat-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ models:
AP@1000:
- 0.001
nDCG@10:
- 0.004
- 0.005
R@100:
- 0.004
R@1000:
Expand Down
4 changes: 2 additions & 2 deletions src/main/resources/regression/dl20-passage.cos-dpr-distil.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8533
tolerance:
AP@1000:
- 0.007
- 0.009
nDCG@10:
- 0.006
R@100:
- 0.015
R@1000:
- 0.015
- 0.02
4 changes: 2 additions & 2 deletions src/main/resources/regression/dl20-passage.cos-dpr-distil.hnsw-int8.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8533
tolerance:
AP@1000:
- 0.007
- 0.009
nDCG@10:
- 0.006
R@100:
- 0.015
R@1000:
- 0.015
- 0.02
8 changes: 4 additions & 4 deletions src/main/resources/regression/dl20-passage.cos-dpr-distil.hnsw.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8533
tolerance:
AP@1000:
- 0.005
- 0.008
nDCG@10:
- 0.001
- 0.004
R@100:
- 0.008
- 0.015
R@1000:
- 0.01
- 0.015
8 changes: 4 additions & 4 deletions src/main/resources/regression/dl20-passage.cos-dpr-distil.hnsw.onnx.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ models:
- 0.8533
tolerance:
AP@1000:
- 0.005
- 0.008
nDCG@10:
- 0.001
- 0.005
R@100:
- 0.008
- 0.015
R@1000:
- 0.01
- 0.015
4 changes: 2 additions & 2 deletions src/main/resources/regression/dl20-passage.openai-ada2.hnsw-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,6 @@ models:
nDCG@10:
- 0.015
R@100:
- 0.002
- 0.003
R@1000:
- 0.006
- 0.01
2 changes: 1 addition & 1 deletion src/main/resources/regression/dl20-passage.openai-ada2.hnsw.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,4 @@ models:
R@100:
- 0.003
R@1000:
- 0.004
- 0.007
2 changes: 1 addition & 1 deletion src/main/resources/regression/msmarco-v1-passage.bge-base-en-v1.5.flat-int8.cached.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,6 @@ models:
RR@10:
- 0.002
R@100:
- 0.001
- 0.002
R@1000:
- 0.001
Loading

0 comments on commit 33ed276

Please sign in to comment.