Skip to content

Pull requests: huggingface/lighteval

New pull request New
36 Open 561 Closed
36 Open 561 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

use inspect-ai to evaluate aime25, gsm8k and IFEval
#1010 opened Oct 7, 2025 by NathanHB Loading…
4
Fixing mixeval
#1006 opened Oct 4, 2025 by clefourrier Loading…
1
Wrap vllm inputs to compatible with VLLM>=0.10.2
#1003 opened Oct 2, 2025 by JIElite Loading…
1
Fix caching logic
#994 opened Sep 25, 2025 by jxmorris12 Loading…
6
Cleanup tasks to easely remove suites
#993 opened Sep 25, 2025 by NathanHB Loading…
1 of 2 tasks
1
fix lighteval task inspect command and tiny bench task bug ignore-for-release
#992 opened Sep 25, 2025 by NathanHB Loading…
1
4
Fix deberta overflow error bug
#990 opened Sep 24, 2025 by amstu2 Loading…
6
Fixing naming for sample evals + adding reqs in aime24 science-team task-update
#989 opened Sep 23, 2025 by clefourrier Loading…
2
run slow tests aginst vllm and transformers main
#985 opened Sep 23, 2025 by NathanHB Loading…
2
Split up enhancement and features in release notes template enhancement ignore-for-release
#984 opened Sep 23, 2025 by NathanHB Loading…
1
vllm 0.10.2 makes integration tests fail bug ignore-for-release
#980 opened Sep 22, 2025 by NathanHB Loading…
2
added HELMET task with json datasets and tests new-task
#971 opened Sep 17, 2025 by nayana1729 Loading…
Add ChartQA new-task
#954 opened Sep 11, 2025 by 0xjunhao Loading…
3
adds concurrent_requests parameter for litellm backend enhancement
#953 opened Sep 10, 2025 by anupam-dewan Loading…
2
[RFC] Rework the suites to be inferred breaking feature refacto
#952 opened Sep 9, 2025 by LysandreJik Loading…
3
Initial implementation for chat template parameters
#904 opened Aug 6, 2025 by LysandreJik • Draft
5
Debug continuous batching
#900 opened Aug 5, 2025 by clefourrier • Draft
1
Fix #742
#860 opened Jul 14, 2025 by mcleish7 Loading…
1
add additional_messages parameters to forward OpenAi style messages to inference providers
#859 opened Jul 10, 2025 by mfuntowicz Loading…
5
Save and output number of samples of each task
#851 opened Jul 3, 2025 by itsmejul Loading…
1
2
Multilingual Overhaul
#833 opened Jun 25, 2025 by hynky1999 Loading…
Adding a range of multilingual evals
#832 opened Jun 25, 2025 by clefourrier • Draft
3
Added fallback for incomplete configs for vlm models launched as llms
#828 opened Jun 24, 2025 by clefourrier Loading…
1
@clefourrier
1
Add support for vLLM KV-cache quantization
#773 opened May 22, 2025 by eldarkurtic Loading…
1
Add Chinese (zh) Translation of Documentation documentation
#744 opened May 19, 2025 by CassiopeiaCode Loading…
13
ProTip! Add no:assignee to see everything that’s not assigned.