Position: Adopt Constraints Over Fixed Penalties in Deep Learning

Constraints should be treated as non-negotiable requirements rather than trade-offs: The paper argues that when AI systems have hard requirements (like fairness thresholds or safety limits), these should be enforced as constraints rather than converted into penalties that can be compromised to improve task performance. Fixed penalization "weakens hard requirements into soft penalties," meaning safety or fairness goals can be sacrificed if it improves the main objective.

Current penalty-based approaches often fail to actually solve the problem they're meant to solve: In non-convex deep learning problems, minimizing a penalized objective (constraint + weighted loss) does not mathematically guarantee that the actual constraints are satisfied. This means systems trained with traditional penalty methods may appear optimized but still violate their stated requirements.

Tuning penalty coefficients is inefficient and error-prone: Practitioners typically use trial-and-error to find the right penalty weights, but this process is costly because changing weights fundamentally changes what problem is being solved. The paper suggests this indirect approach wastes resources that could be better spent on direct constraint-based methods.

Constrained optimization should be the starting point for trustworthy AI: Rather than treating constraints as an afterthought added to existing loss functions, the paper recommends making constraints the foundation of the problem formulation from the beginning, then selecting appropriate solution strategies based on the specific problem structure.