HU
HuggingFaceH4/zephyr-7b-beta
text-generationtransformersmit7.2B params · updated Oct 2024
Evaluation
Scores as recorded on the registry record, each with its source. Bars are relative to the best score listed for this checkpoint. Self-reported numbers are the publisher’s claim, not an OSAI-Models measurement.
AI2 Reasoning Challenge (25-Shot) · normalized accuracy62.0307167235494825-shotmodel-index, self-reported by the repository owner
HellaSwag (10-Shot) · normalized accuracy84.3557060346544510-shotmodel-index, self-reported by the repository owner
Drop (3-Shot) · f1 score9.662437080536913-shotmodel-index, self-reported by the repository owner
TruthfulQA (0-shot) · mc257.449169427628550-shotmodel-index, self-reported by the repository owner
GSM8k (5-shot) · accuracy12.7369219105382875-shotmodel-index, self-reported by the repository owner
MMLU (5-Shot) · accuracy61.075-shotmodel-index, self-reported by the repository owner
Winogrande (5-shot) · accuracy77.74269928966065-shotmodel-index, self-reported by the repository owner
AlpacaEval · win rate0.9060-shotmodel-index, self-reported by the repository owner
MT-Bench · score7.340-shotmodel-index, self-reported by the repository owner