SMITE: Enhancing Fairness in LLMs through Optimal In-Context Example Selection via Dynamic Validation

LLMs can exhibit unfairness in practical applications: The paper demonstrates that Large Language Models used for real-world tasks like data classification can produce biased or unfair outputs, highlighting the need for deliberate interventions to ensure equal representation across different groups.

In-context examples significantly influence model fairness: By carefully selecting which examples are shown to an LLM before asking it to perform a task, researchers can meaningfully improve both accuracy and fairness—suggesting that how we prompt these systems matters as much as the models themselves.

Dynamic validation outperforms traditional static approaches: Rather than using a fixed set of validation examples, the SMITE algorithm uses examples that change alongside the test data, resulting in "significantly improved" fairness and accuracy across four different LLMs tested.

This is a scalable solution for responsible AI deployment: The approach works across multiple different LLMs without requiring retraining or modification of the underlying models, making it a practical tool for organizations already using these systems who want to reduce bias in their outputs.

Fairness improvements don't require sacrificing accuracy: The research shows that enhancing fairness through better example selection simultaneously improves predictive accuracy, demonstrating that responsible AI practices and performance are not inherently in conflict.