Greg, this is one of my favorite pieces of yours -- so on point!
Accountability and transparency is the biggest issue for me. We wouldn't sign our lives away to a human, why do so with a machine?
Clearly, declaring AI an infallible god is nothing more than a ploy for profit, license and power -- a tactic for getting ahead of your competition and evading oversight.
It's like giving a PhD to someone whose thesis is an entirely novel, undefended, controversial, suspicious, unsupported claim.
On the other hand, well-founded caution shouldn't ignore "opportunity costs." At the end of the day, trusting AI over humans is a pragmatic decision -- which choice delivers the greater benefit? If letting AI make certain medical decisions proves to save more lives than human decisions, then what is the dispositive argument against AI in such a case?
Thank you for shining such a clear, focused light on these issues!