The facts
NIST’s 2024 generative AI risk profile has a grown-up word for the thing you already noticed: confabulation. The system can spit out false or misleading crap and say it like a cop reading a report.
A polished paragraph is not proof the model checked anything. NIST put that in a government PDF because vendors will not put it on the homepage.
Our take
The sales pitch is an infinitely patient genius. The failure mode is a bullshitter with excellent formatting. Same window. Same friendly little cursor.
Confabulation means it made that shit up and did not flinch. Names, citations, a medical answer with a bow on it. Sounds smart. Is not the same as true.
Use the tool. Then check the names. Check the numbers. Check the links. “The AI said so” is not a correction policy, it is a shrug in a hoodie. Someone has to own what gets published. The model will not. It will just offer to write you another paragraph, you lazy cunt.
