Repository navigation

Add img-understanding task group (lmms-eval) - #125

Open

harshraj172 wants to merge 11 commits into

mainfrom

harsh/vqa-task-suite

Open

Add img-understanding task group (lmms-eval)#125
harshraj172 wants to merge 11 commits into
mainfrom
harsh/vqa-task-suite

Commits on Sep 30, 2026
- Add vqa and mmmu task groups via lmms-eval
  
  harshraj172
  and
  claude
  committed
  740b46e
- Fix bugs found by live-testing the lmms-eval branch on real SLURM jobs
  
  harshraj172
  and
  claude
  committed
  7b1a5a4
Commits on Oct 1, 2026
- Merge vqa and mmmu into one img-understanding task group
  
  harshraj172
  and
  claude
  committed
  45060dc
- Trim comments down to one line each
  
  harshraj172
  and
  claude
  committed
  0da3b34
- Remove comments added for img-understanding
  
  harshraj172
  and
  claude
  committed
  3241d6e
- Tighten the img-understanding VENV.md section
  
  harshraj172
  and
  claude
  committed
  595f458
- Pass trust_remote_code and n_shot through to lmms_eval, fix HF_TOKEN doc gap
  
  harshraj172
  and
  claude
  committed
  9c9d0cd
- Only pass trust_remote_code to llava_hf, not the Qwen VL model classes
  
  harshraj172
  and
  claude
  committed
  6133c13
Commits on Oct 2, 2026
- Skip directories matching *.json in collect_results
  
  harshraj172
  and
  claude
  committed
  c6d7fb7
- Add seed2_omni model class for mixturevitae2's omni checkpoints
  
  harshraj172
  and
  claude
  committed
  ba1aafe
Commits on Oct 5, 2026
- Merge branch 'main' into harsh/vqa-task-suite
  haideraltahan
  authored
  205f40b