Red-teaming

Also written: red teaming, red team

Deliberately trying to make a model misbehave, before release, to find and fix the problems first.

Other terms from the same lesson

• alignment: Making AI systems reliably do what people intend, safely and honestly.
• Q-learning: A simple reinforcement-learning method that learns a table of how good each action is in each situation.
• specification gaming: When a system scores well by exploiting a loophole in how its goal was written, instead of doing what was meant.
• system card: A report published with a model describing its safety testing: what was tried, what went wrong and what was done about it.