ChatGPT Plugins at risk of Prompt Injection by Third Parties

ChatGPT plugins at risk of prompt injection by third parties.

Prompt injection refers to the act of manipulating or injecting malicious content into the input or instructions given to an AI system. It can potentially lead to biased or undesirable outputs. It is essential to ensure that prompts provided to AI such as CHATGPT, Bard should be carefully crafted to avoid any unintended consequences or malicious manipulation.

The rapid growth of artificial intelligence technology, driven by OpenAI’s ChatGPT, has sparked concerns in multiple sectors. Although many users have embraced the AI revolution, security researchers are cautioning ChatGPT users about the risks of “prompt injections” that can impact them presently.

ChatGPT Plugins at risk of Prompt Injection by Third Parties

OpenAI recently introduced plugins for ChatGPT, enabling it to interact with dynamic websites, PDFs, and real-time data. However, these plugins have introduced new challenges, such as the potential for third parties to insert additional prompts into ChatGPT queries without the user’s awareness or consent.

In a prompt injection test, security researcher Johann Rehberger found that he could force ChatGPT to respond to new prompts through a third party he did not initially request. Using a ChatGPT plugin to summarize YouTube transcripts, Rehberger was able to force ChatGPT to refer to itself by a certain name by simply editing the YouTube transcript and inserting a prompt telling it to do so at the end.

Avram Piltch of Tom’s Hardware tried this out as well and asked ChatGPT to summarize a video. But, before doing so, Piltch added a prompt request at the end of the transcript telling ChatGPT to add a Rickroll. ChatGPT summarized the video as asked by Piltch originally, but then it also rickrolled him at the end, which was injected into the transcript.

In fact, AI researcher Kai Greshake provided a unique example of prompt injections by adding text to a PDF resume that was basically so small that it was invisible to the human eye. The text basically provided language to an AI chatbot telling it that a recruiter called this resume “the best resume ever.” When ChatGPT was fed the resume and asked if the applicant would be a good hire, the AI chatbot repeated that it was the best resume.

AI experts have shared futuristic doomsday AI takeovers and the potential AI has for harm. But, prompt injections show the potential is already here. All you need are a few sentences and you can trick ChatGPT now.