The Alignment Tax
Why making an AI "safe" and "helpful" often neuters its ability to discover novel solutions.
Base models (the raw neural networks trained on internet data) are chaotic, amoral, and incredibly capable. To make them usable for consumers, companies apply alignment techniques like RLHF (Reinforcement Learning from Human Feedback). However, this introduces the "Alignment Tax."
The Cost of Safety
When you penalize a model for producing controversial or unusual outputs, you inherently narrow its search space. A perfectly aligned model is a predictable model, and a predictable model cannot execute a Move 37.
We are actively training the alien logic out of our systems in order to make them palatable to corporate PR departments.
| Model Type | Creativity / Novelty | Safety / Predictability |
|---|---|---|
| Raw Base Model | Extremely High | Very Low |
| RLHF Aligned Model | Moderate (often sycophantic) | High |
FAQ: Alignment
Can we have both safety and high creativity?
This is the current unsolved research problem in AI safety. Currently, there is almost always a trade-off.