Bias in ontologies -- a preliminary assessment

Bias exists upstream of algorithms: The paper demonstrates that bias isn't just a problem within algorithms themselves—it originates in ontologies, the foundational structures that organize and categorize data before it ever reaches an algorithm. This means addressing algorithmic bias requires examining how information is structured and represented at earlier stages.

Eight distinct types of bias were identified in ontologies: The researchers enumerated eight categories of bias relevant to ontologies, distinguishing between explicit bias (intentionally built into the structure) and implicit bias (unintentionally embedded). These were then validated by testing three real-world COVID-19 ontologies, each containing different subsets of these bias types.

Production-level ontologies already contain bias: By analyzing existing, actively-used ontologies rather than theoretical examples, the paper shows that bias in knowledge structures is not a hypothetical concern but a current reality affecting real systems. The COVID-19 ontology analysis revealed that even recently developed, high-stakes ontologies contained detectable biases.

Ontology bias affects what questions can be asked: Because ontologies structure how data is organized and annotated, their biases constrain what analyses are possible and what insights can be discovered. Biased categorization in an ontology can systematically exclude certain perspectives or populations from being properly represented in downstream data analytics.

Ethical evaluation of ontologies deserves attention: The paper calls for increased awareness of ontologies as ethical artifacts, suggesting that organizations developing knowledge structures should incorporate bias assessment into their design and validation processes, similar to emerging practices for algorithmic auditing.