version 0.0.9
Read Latest Definition Version
See FAQs
Endorse the OSAID
Leave comments for this text
Checklist to evaluate machine learning systems
This checklist is based on the paper The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency and Usability in AI - DOI published Mar 21, 2024. The framework is implemented on the Model Openness Tool website.
As used herein:
- Available under OSD-conformant terms means any license or agreement that ensures that all materials are available for use under terms that conform with the Open Source Definition.
- Available under OSD-compliant license means any license that ensures that all materials are available for use under terms that comply with the Open Source Definition.
For more details, see also the Open Source AI FAQ.
Table of default required components
| Required components | Legal frameworks |
|---|---|
| Data | |
| At least one of these data components is required, in decreasing order of importance | |
| – Datasets | Available under OSD-compliant license |
| – Research paper | Available under OSD-compliant license |
| – Technical report | Available under OSD-compliant license |
| – Data card | Available under OSD-compliant license |
| Code | |
| All of these components are required | |
| – Data pre-processing | Available under OSI-approved license |
| – Training, validation and testing | Available under OSI-approved license |
| – Inference | Available under OSI-approved license |
| – Supporting libraries and tools | Available under OSI-approved license |
| Model | |
| All of these components are required | |
| – Model architecture | Available under OSI-approved license |
| – Model parameters | Available under OSD-conformant terms |
The other components listed in the Model Openness Framework are optional.
| Optional components | Legal frameworks |
|---|---|
| Data | |
| – Evaluation data | Available under OSD-compliant license |
| – Evaluation results | Available under OSD-compliant license |
| Code | |
| – Code used to perform inference for benchmark tests | Available under OSI-approved license |
| – Evaluation code | Available under OSI-approved license |
| Model | |
| – Model card | Available under OSD-compliant license |
| – Sample model outputs | Available under OSD-compliant license |
| – Model metadata | Available under OSD-compliant license |