Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat] add benchmarking page. #142

Merged
merged 15 commits into from
Dec 21, 2024
Merged

[Feat] add benchmarking page. #142

merged 15 commits into from
Dec 21, 2024

Conversation

a-ghorbani
Copy link
Owner

@a-ghorbani a-ghorbani commented Dec 18, 2024

Description

This PR introduces a new benchmark page that enables users to compare the performance of models directly on their phones. It provides performance metrics and the ability to share results with the community.

Benchmark Page

  • Prompt Processing (PP) tokens per second
  • Text Generation (TG) tokens per second
  • RAM peak usage

Community Sharing
Legit users, verified through app check, can share their benchmark results with the community. In the future we need to find a way to let users outside Play Store/Google Play be able to share too.

Results can be shared to the "AI Phone Leaderboard."

Simulator Screenshot

Fixes #141

Platform Affected

  • iOS
  • Android

Checklist

  • Necessary comments have been made.
  • I have tested this change on:
    • iOS Simulator/Device
    • Android Emulator/Device
  • Unit tests and integration tests pass locally.

@a-ghorbani
Copy link
Owner Author

builds will fail if no firebase configs are available. This need to be fixed #143

@a-ghorbani a-ghorbani marked this pull request as ready for review December 21, 2024 07:58
@a-ghorbani a-ghorbani merged commit f8dcd16 into main Dec 21, 2024
2 of 3 checks passed
@a-ghorbani a-ghorbani deleted the feat/add-bench branch December 21, 2024 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feat]: Model Benchmarking
1 participant