Hidden Errors in Big Data: The Case of Property Records

Data errors are widespread in commercial property datasets: The researchers found that 1-2% of property sales in Cook County had prices differing from official records by more than 5%, and 12-15% of transactions had missing or incorrectly reported information, meaning these errors affect millions of transactions nationwide.

Errors systematically bias research on inequality: Because these brokered datasets are commonly used to study gentrification, inequality, and property taxes, the documented errors produce significantly different conclusions depending on which data source researchers use—potentially leading to flawed policy recommendations.

Errors are shared across competing data brokers: Rather than random mistakes, the researchers found that multiple commercial data brokers make identical errors for the same transactions, suggesting they copy from each other rather than independently verify information, which means the problem cannot be solved by simply using multiple data sources.

AI models inherit these hidden errors: Since property valuation models and other AI systems used in lending and real estate decisions rely on these brokered datasets as inputs, the documented errors propagate into automated decision-making systems that affect people's access to credit and property assessments.

Transparency and public data are essential safeguards: The paper emphasizes that open access to official administrative records and greater disclosure from data brokers about how they collect and process information are critical for catching errors before they influence research, policy, and automated decisions.