Themis: An explainable AI-enabled framework for Reinforcement Learning with Human Feedback

Explainability and human feedback together improve AI safety: Themis combines two key defenses against unwanted AI behaviors—making systems transparent through explainability and aligning them with human values through feedback—addressing a gap where no existing public framework previously integrated both approaches.

Human preferences can match or exceed automated reward signals: The framework successfully trained reward models using human preferences that performed as well as or better than the environment's built-in reward signals, suggesting human judgment is a viable alternative for guiding AI behavior when properly structured.

Practical scalability for human-in-the-loop AI development: Themis demonstrated the ability to support 1,000 simultaneous users conducting experiments on standard commercial hardware, making human feedback collection feasible at scale for organizations developing RL systems without requiring specialized infrastructure.

Broad applicability across AI environments: By supporting over 200 widely-used environments and offering easy configuration options, Themis provides researchers and practitioners with a flexible tool for testing AI safety across diverse applications rather than requiring custom solutions for each use case.

Cloud-based infrastructure reduces development barriers: The platform's user-friendly design and auto-scalable cloud component eliminate the need for additional development overhead when managing large participant groups, potentially democratizing access to responsible AI testing methods.