Table of Contents - F.748.74 (06/2026) - Requirements and framework for evaluation of multimodal foundation models

1	Scope
2	References
3	Definitions
	3.1	Terms defined elsewhere
	3.2	Terms defined in this Recommendation
4	Abbreviations and acronyms
5	Conventions
6	Framework for the evaluation of multimodal foundation models
	6.1	Test scenarios
	6.2	Test datasets
	6.3	Test tools
	6.4	Test workflow
7	Requirements of multimodal foundation models
	7.1	Multimodal perception
	7.2	Multimodal understanding
	7.3	Multimodal generation
	7.4	Multimodal retrieval
	7.5	Multimodal reasoning
Annex A – Evaluation metric definitions and calculations
	A.1	Character error rate:
	A.2	Precision:
	A.3	Frame-level precision:
	A.4	Word error rate:
	A.5	BLEU-4 score:
	A.6	False acceptance rate:
	A.7	Frechet image distance:
	A.8	Frechet video distance:
	A.9	Pronunciation accuracy:
	A.10	Chamfer distance:
	A.11	Top-10 hit rate:
	A.12	Accuracy:
	A.13	Reasoning path:
	A.14	F1 score:
Bibliography