Ethics & Algorithms Toolkit

Overview

The Ethics & Algorithms Toolkit is a risk management framework designed to help governments and organizations understand, evaluate, and mitigate risks associated with algorithmic decision-making. Developed collaboratively by GovEx, the City and County of San Francisco, Harvard DataSmart, and Data Community DC, the toolkit provides a practical, repeatable process for managing algorithmic risk through a structured approach familiar to government institutions.

Key Principles

The toolkit is built on several core assumptions:

• Algorithm use in government is inevitable
• All data contains bias
• All algorithms contain bias
• All people contain bias
• Risk management is an effective framework for addressing algorithmic concerns

The toolkit emphasizes that data collection and data analysis are typically separate efforts with different intentions, requiring coordinated evaluation.

How to Apply

The toolkit comprises five main documents:

Overview and Introduction - Provides level-setting background information, real-life scenarios, and definitions of key concepts including types of machine learning (supervised, unsupervised, etc.).

Part 1: Assess Algorithm Risk - Contains six major steps to characterize an algorithm and identify risks. Includes a companion worksheet to track responses and calculate overall risk values. This section requires stakeholder collaboration and typically involves extended discussion.

Part 2: Manage Algorithm Risk - Identifies specific mitigation techniques to address risks identified in Part 1. Recommendations are customized based on Part 1 findings and may require executive and political support.

Appendices - Provides supplementary context including detailed questions for understanding data and additional background on bias.

All documents are available individually or as a downloadable zip file.

Target Audience

The toolkit is designed for:

• Government leaders and staff implementing algorithms
• Organizations seeking to manage algorithmic risk