OpenAI has banned a cluster of ChatGPT accounts originating in Russia that it said were being used to support a covert online influence campaign to spread misinformation.
The company discovered the issue after investigating AI-generated social media posts, which led it to a "much broader influence operation," OpenAI said in a blog post on Tuesday.
The campaign was "built around a website containing copied and misattributed academic work, a 'sovereignty' index that cast Russia in a favourable light, and efforts to disguise the operators' Russian origins," the company added.
It's the latest incident of pro-Russia actors using AI to spread misinformation. OpenAI does not allow access to its models in Russia, but the company said VPNs were used to disguise location.
In February, OpenAI said it had shut ChatGPT accounts that were connected to Rybar, a pro-Russia media company the UK government describes as "partially co-ordinated by the Presidential Administration," funded by Russian state corporation Rostec and working with members of the Russian intelligence services.
CNBC has contacted the Russian embassy in London and Russia's Foreign Ministry for comment.
Pro-Russia influence campaign
The operators prompted in Russian to generate social media comments that were posted on Substack, Telegram, X, Facebook and LinkedIn, and instructed ChatGPT to hide any linguistic clues that they were Russian, OpenAI said.
Much of the content generated promoted an organization called the International Burke Institute (IBI), the website of which features articles copied from real academic writing, sometimes with false attribution.
The influence campaign also created a "sovereignty index," OpenAI said, which praised Russia and denigrated Western countries.
While the immediate impact of the operation appeared limited, the company added, the "significance of the operation lies less in the audience it reached, however, than in the infrastructure it had built."
ChatGPT-produced posts pointed to an "otherwise credible-appearing institution, complete with purported experts, republished academic work and a purported proprietary risk index," OpenAI said.
"This illustrates how influence actors can use AI as a supporting tool within a broader effort to manufacture authority, obscure the source of favored narratives, and establish assets that could be scaled over time," the company added. "It also illustrates how their supporting use of AI can lead to the broader operation being exposed."
