Trustworthy Human Computation: A Survey

Human computation works best when AI and humans trust each other mutually. The paper emphasizes that trustworthy human computation requires "two-way trust" where both humans and AI systems build confidence in one another to accomplish difficult tasks together, rather than treating trust as a one-directional relationship.

AI systems need to be reliable, available, and maintainable—just like traditional computer systems. Using the RAS (Reliability, Availability, and Serviceability) framework from conventional computing, the survey argues that human computation systems should be evaluated on whether they consistently work, remain accessible, and can be serviced when problems arise.

Fairness, privacy, and transparency are essential ethical requirements for human computation systems. The paper identifies these three AI ethics principles as critical for establishing social trustworthiness with users and participants, ensuring that human computation systems treat people equitably and respect their data and autonomy.

Human computation fills a real gap where AI alone falls short. The survey recognizes that many problems are "difficult using AI only," making human-AI collaboration not just beneficial but necessary for solving complex challenges that neither humans nor machines can handle independently.

Trustworthy human computation requires ongoing research and development. The paper concludes by highlighting future challenges and research directions, indicating that establishing trustworthiness in human computation systems is an evolving field requiring continued attention and innovation.