Performance and methodology glossary
How the numbers on a model card are defined, weighted and measured. Read the definition for what a term means; read the methodology column for how it is computed.
18 terms across 6 groups · index methodology v4.3 · source: AI Performance and Methodology Glossary
Composite indices
A composite metric that assesses AI model quality across reasoning, coding, knowledge, scientific analysis and agentic capabilities on a 0 to 100 scale.
Aggregates four pillars: Agents (30%), General (30%), Coding (20%), Science (20%). The v4.3 index incorporates ten evaluations: AA-Briefcase, GDPval-AA v2, AutomationBench-AA, Terminal-Bench v4.0, SciCode, Humanity’s Last Exam, GDP.pdf, CritPt, AA-Omniscience and AA-LCR v1.1.
A standardized metric quantifying the accessibility, usage rights and technical transparency of AI models on a 0 to 100 scale.
Calculated from Model Availability (up to 6 points) and Model Transparency (up to 12 points). Measures weight accessibility for download, self-hosting capability and licence details.
A dimension of the Openness Index measuring developer access and legal freedom of use.
Two sub-components: Model Weights Access (0–3 points) and Model Weights Licence (0–3 points).
A dimension of the Openness Index measuring whether scientific and engineering methods are inspectable.
Evaluates training-data disclosures, architecture and technical methodology, and availability of code or pipelines (max 12 points).
Access and licensing
A classification for AI models whose underlying weights are accessible for download.
Enables self-hosting and customisation via fine-tuning. Further categorised by licence as ‘Commercial Use Restricted’ (limited conditions) or ‘Non-commercial’ (prohibits business use).
Scale and architecture
The total number of trainable weights and biases in a model, usually expressed in billions (B).
Defines storage size, knowledge capacity and VRAM requirements. Tiers: Tiny (≤ 4B), Small (4B–40B), Medium (40B–150B), Large (> 150B).
The subset of total parameters actually executed during a single inference forward pass.
In dense models, active equals total. In Mixture-of-Experts models a routing mechanism selects specific experts, making active parameters a fraction of the total — this determines compute cost (FLOPs) and latency.
An architecture where a routing mechanism selects only a fraction of specialised “experts” for any given token.
Allows high total parameter capacity for knowledge storage while maintaining low active parameter counts for computational efficiency.
The maximum number of combined input and output tokens a model can process in a single session.
Measured in tokens (128k, 1M). Critical for retrieval-augmented generation; evaluated via AA-LCR and AA-Omniscience to check for degradation across the window.
A classification for models optimised for complex problem-solving or chain-of-thought processing.
Indicated in model names (for example Flash Reasoning) and associated with higher effort levels or specific evaluation tasks.
Size classifications
A size classification for the smallest AI models in terms of resource demand.
Less than or equal to 4 billion (≤ 4B) parameters.
A size classification for AI models with low to moderate parameter counts.
Fewer than 40 billion (< 40B) parameters.
A size classification for mid-range AI models with substantial parameter counts.
Between 40 billion and 150 billion (40B–150B) parameters.
A size classification for the most resource-intensive and high-capacity AI models.
Over 150 billion (> 150B) parameters.
Evaluation methodology
An evaluation category focusing on autonomous problem-solving and tool use.
Weighted at 30% of the Intelligence Index; uses benchmarks like AA-Briefcase (long-horizon) and AutomationBench-AA (SaaS workflows).
Evaluations built using non-public test data to prevent contamination.
Used for approximately 45% of the Intelligence Index weight, to ensure models have not memorised answers during training.
Cost and throughput
The cost associated with processing one million tokens through a model’s API.
Calculated in USD (for example $0.90) to allow comparison of cost-efficiency across providers.
A performance metric measuring the speed of text generation at inference.
Quantitative values indicating the throughput speed of the model under specific testing conditions.