Issues

New issue

generate diagram as part of the cli
#80 · rom1504 opened on Feb 3, 2023
1
Make a training companion eval script
#2 · rom1504 opened on Jul 31, 2022

Proposal: optional audit manifest for benchmark result provenance
LAION-AI/CLIP_benchmark#153 · opened on May 10, 2026
CTMinfo
LAION-AI/CLIP_benchmark#152 · opened on Apr 12, 2026
Zeroshot-classification accuracy function dimensionality bug
LAION-AI/CLIP_benchmark#147 · opened on Jan 22, 2026
Mismatch between MS-COCO val2014 images and annotations
LAION-AI/CLIP_benchmark#145 · opened on Oct 26, 2025
Eval custom open_clip architecture – Add model config
LAION-AI/CLIP_benchmark#144 · opened on Sep 13, 2025
Possibility to save weights after training with linear probing
LAION-AI/CLIP_benchmark#143 · opened on Aug 15, 2025
Implement LP++ for better linear probing results
LAION-AI/CLIP_benchmark#142 · opened on Jul 31, 2025
Webdatasets for Linear Probe evaluation
LAION-AI/CLIP_benchmark#127 · opened on May 27, 2024
6
additional argument not recognized in MODEL_TYPES
LAION-AI/CLIP_benchmark#123 · opened on Mar 21, 2024
Update babel image net
LAION-AI/CLIP_benchmark#117 · opened on Jan 17, 2024
Add support of zero-shot classification/retrieval evaluation of captioning models using log-likelihood scoring
LAION-AI/CLIP_benchmark#115 · opened on Jan 4, 2024
Support computing metrics using test time distribution normalisation
LAION-AI/CLIP_benchmark#112 · opened on Nov 1, 2023
Add compositionality benchmarks
LAION-AI/CLIP_benchmark#97 · opened on Jun 30, 2023
10
generate diagram as part of the cli
LAION-AI/CLIP_benchmark#80 · opened on Feb 3, 2023
1
Attribute based eval
LAION-AI/CLIP_benchmark#72 · opened on Jan 26, 2023
4
Evaluate mse aligned multilingual clip
LAION-AI/CLIP_benchmark#69 · opened on Jan 20, 2023
2
Organize CLI arguments into groups
LAION-AI/CLIP_benchmark#66 · opened on Jan 13, 2023
Add forensic/exif tasks
LAION-AI/CLIP_benchmark#65 · opened on Jan 12, 2023
Variation in the benchmark
LAION-AI/CLIP_benchmark#59 · opened on Dec 26, 2022
1
add multi30k for eval
LAION-AI/CLIP_benchmark#46 · opened on Nov 28, 2022
1
add xm3600 for multilingual eval
LAION-AI/CLIP_benchmark#44 · opened on Nov 23, 2022
Support ImageNet-X
LAION-AI/CLIP_benchmark#41 · opened on Nov 21, 2022
add multilingual task from mural
LAION-AI/CLIP_benchmark#35 · opened on Nov 19, 2022
add chinese retrieval tasks
LAION-AI/CLIP_benchmark#33 · opened on Nov 18, 2022
1
Add XMRec multilingual retrieval benchmark
LAION-AI/CLIP_benchmark#31 · opened on Nov 15, 2022