I am afraid that this will just lead down the path to what twitter.com/ESYudkowsky/status/17186541… was mocking. We're dictating solutions to today's threats, leaving tomorrow to its own devices. But what will tomorrow bring? As Sam Altman warns in twitter.com/sama/status/171697281596096…, superhuman persuasion is likely to be next. What does that mean? We've already had the problem of social media echo chambers leading to extremism, and online influencers creating cult-like followings. jonathanhaidt.substack.com/p/mental-hea…... is a sober warning about the dangers to mental health from this. These are connected humans accidentally persuading each other. Now imagine AI being able to drive that intentionally to a particular political end. Then remember that China controls TikTok. Will Biden's order keep China from developing that capability? Will we develop tools to identify how that might be being actively used against us? I doubt both. Instead, we'll almost certainly get security theater leading to a regulatory moat. Which is almost certain to help profit margins at established AI companies. But is unlikely to address the likely future problems that haven't materialized yet.

> He was simply ERP-ing and his characteristic doesn't suggest any serious problems with his intelligence. i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes twitter.com/sama/status/171697281596096… / archive.md/RWDNn

Well he did say it would be persuasive.. maybe he asked GPT-5 how to get out from under the non-profit. x.com/sama/status/1716972815960961174

they have the potential to take over the control systems of the explosive stuff. including the decision making processes of humans themselves. to quote none other than sama, oct 2023: "i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes" twitter.com/sama/status/171697281596096…