The field has spent a quarter of a century trying three ways to hand a goal to a machine: write it down, learn it from us, or raise it. This second part of three puts the listener's proposal that a mind should be raised rather than specified beside the results the field has produced. Inverse reinforcement learning learns the goal from what an expert does and misses whatever the expert never faced. Preference training rewards the answers people like best, which a paper named sycophancy. A laboratory's own results, from a planted trigger that survived safety training to alignment faking, say that what is measured is what is learned. An upbringing is the one method nobody has tried, and it keeps the goal inside a relationship and pays for that by keeping it unmeasurable.

Message Board