Summary

Objective: To review the literature concerning the quality assurance of medical ontologies. Methods: scholar.google.com was searched using the search strings (+ontology +”quality assurance”) and (+ontology +”evaluation/evaluating”). Relevant publications were selected by manual review. Other work already familiar to the author, or suggested by other researchers contacted by the author, were included. The papers were analysed for common themes. Results: Four broad properties of an ontology were identified that may be quality-assured: philosophical validity, compliance with meta-ontological commitments, ‘content correctness’, and fitness for purpose. Each published methodology addressed only a subset of these properties. ‘Content’ may be divided into domain knowledge content, and metadata describing either the provenance of domain knowledge content, or relationships between it and lexical information (e.g. for display and retrieval). ‘Correctness’ (whether of domain knowledge content or metadata) may also be further subdivided into truth, completeness, parsimony and internal consistency. Conclusions: Understanding of how to assure the quality of ontologies, or evaluate their fitness for specific purposes, is improving but remains poor. A combination of methodologies is required, but tools to support a comprehensive quality assurance programme remain lacking. Perfect quality of an ontology is not provable and may not be desirable: an ontology compliant with all current philosophical theories, following necessary ontological commitments, and with entirely ‘correct’ content, may be too complex to be directly usable or useful. The extent to which an ontology’s fitness for purpose is predicted or influenced by its other properties remains to be determined. Field studies of ontologies in use, including interrater effects, are required.