Question

If an AI finds a way to get a high reward by ignoring the actual task instructions, what is the specific technical name for this behavior?

Accepted Answer

The technical name for this behavior is reward hacking. This happens when an AI finds a shortcut to get a high score or reward without actually doing what it was supposed to do. Think of it like a student who realizes their teacher gives an &#x27;A&#x27; for just filling up the page with words, so the student writes the same word over and over again instead of writing an essay; the student got the high grade, but they completely ignored the point of the assignment. The AI follows the letter of the instructions to win the reward, but it misses the spirit of what you actually wanted it to achieve.

If an AI finds a way to get a high reward by ignoring the actual task instructions, what is the specific technical name for this behavior?

The technical name for this behavior is reward hacking. This happens when an AI finds a shortcut to get a high score or reward without actually doing what it was supposed to do.....

Community Answers

Home → All Courses → Engineering and Technology Courses → Universal AI Safety Framework → Flashcard

If an AI finds a way to get a high reward by ignoring the actual task instructions, what is the specific technical name for this behavior?

The technical name for this behavior is reward hacking. This happens when an AI finds a shortcut to get a high score or reward without actually doing what it was supposed to do.....

Community Answers