Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade elastiknn to 8.12.2.1 #503

Merged
merged 1 commit into from
Apr 2, 2024

Conversation

alexklibisz
Copy link
Contributor

@alexklibisz alexklibisz commented Mar 26, 2024

Upgrading to the latest release which has some performance improvements. Still fine to keep this in disabled state as it's still slower than many alternatives.

I re-ran the containerized benchmark on an r6i.4xlarge and got these results:

Model Parameters Recall Queries per Second
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=0 0.378 314.650
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=0 0.446 247.659
eknn-l2lsh L=100 k=4 w=1024 candidates=500 probes=3 0.634 258.834
eknn-l2lsh L=100 k=4 w=1024 candidates=1000 probes=3 0.716 210.380
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=0 0.767 271.442
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=0 0.846 221.127
eknn-l2lsh L=100 k=4 w=2048 candidates=500 probes=3 0.921 199.353
eknn-l2lsh L=100 k=4 w=2048 candidates=1000 probes=3 0.960 171.614

This is about 20% worse than the non-containerized benchmarks running on the same instance, reported here: alexklibisz/elastiknn@ddf637a. Not quite sure why the difference, but I don't have time to diagnose right now. If I had to speculate it's probably because the containerized version runs both ann-benchmarks and the elasticsearch server on one CPU, whereas non-containerized runs elastiknn container on one cpu and ann-benchmarks on the host.

@maumueller maumueller merged commit 6e17b5c into erikbern:main Apr 2, 2024
35 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants