Skip to content

Issues: bigcode-project/bigcodebench

[Roadmap] BigCodeBench Q3 2024 Roadmap
#45 opened Sep 10, 2024 by terryyz
Open
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

9 Open 37 Closed
9 Open 37 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Evaluator fails with UnboundLocalError: local variable 'pass_at_k' referenced before assignment. bug Something isn't working
#62 opened Dec 4, 2024 by DevonPeroutky
@terryyz
9
Evaluation setting for StarCoder2-15B-Instruct-v0.1 in the leaderboard question Further information is requested
#53 opened Oct 23, 2024 by renmengjie7
@terryyz
12
[Roadmap] BigCodeBench Q3 2024 Roadmap enhancement New feature or request
#45 opened Sep 10, 2024 by terryyz
5 of 12 tasks
[docs] Clarify risks of running evals locally. documentation Improvements or additions to documentation
#41 opened Sep 3, 2024 by hvaara
@terryyz
[docs] Add working end-to-end example for a local model documentation Improvements or additions to documentation
#40 opened Sep 3, 2024 by hvaara
1
🐛 [TestRemoval/TestRepair] - 211, 215- include status code in mock response bug Something isn't working
#33 opened Jul 26, 2024 by dmelcer9
1
@terryyz
3
🤗 [REQUEST] - InternLM2.5 7B
#32 opened Jul 25, 2024 by ethanc8
2 tasks done
1
Problem Categories question Further information is requested
#31 opened Jul 25, 2024 by normster
@terryyz
4
🤗 [REQUEST] - multi-token-prediction
#16 opened Jul 4, 2024 by ethanc8
2 tasks done
@terryyz
ProTip! Follow long discussions with comments:>50.