Risks of Sensitive Data with OpenAI’s Custom GPTs

Concerns have been raised following the announcement of the opening of OpenAI’s new GPT Store, including the possible disclosure of underlying data. Various parties have recognised the system’s sensitivity to rapid injection attacks, raising the possibility that sensitive information might be disclosed.

Northwestern University researchers ran comprehensive testing on over 200 user-designed GPT models with hostile prompts. Their findings show that these systems are vulnerable to prompt injections, a known flaw that may be used to harvest sensitive information or influence the model’s output. Prompt injection is one of several vulnerabilities connected with language models, including prompt leakage and jail breaking.

Prompt injection is the creation of special inputs or ‘prompts’ by an attacker in order to alter the behaviour of Large Language Models (LLMs) such as GPT. The study team revealed that prompt injection allows an adversary to get customised system prompts as well as access submitted data.

Yesterday, I built a custom GPT for http://Levels.fyi, using a portion of our data as a knowledge source (RAG). People replied to the suggestions with incredible ingenuity, and the response was fantastic. However, I immediately saw that the source data file was accessible. — Zuhayeer Musa

Our findings emphasise the vital need of effective security mechanisms in both the design and deployment phases of customisable GPT models. We emphasise that, although customisation opens up new opportunities for AI utility by allowing the building of bespoke models without coding knowledge, it also opens up new channels for possible security concerns.

“Our assessment identified considerable security concerns related with rapid injection, exposing weaknesses in the existing custom GPT ecosystem. Our comprehensive testing revealed that these prompts could successfully disclose system prompts and retrieve uploaded data from the vast majority of bespoke GPTs.” This reveals a significant vulnerability in current custom GPTs that allows for system prompt extraction and file disclosure.

The researchers issue a strong call to action at the end of their paper: “Our findings highlight the immediate need for heightened security measures in the dynamic realm of customisable AI.” We hope that this spurs additional discussion on the topic.” They emphasise the critical balance between innovation and security as AI technologies advance.

Adversa AI has revealed that GPTs can mistakenly disclose information regarding their development, such as prompts, API names, metadata, and content from uploaded documents. OpenAI recognised and corrected the issues identified by Adversa AI researchers.

“We are continuously enhancing the safety and resilience of our models and products against adversarial attacks, like prompt injections, while preserving their utility and task performance.” – The response from OpenAI to Wired.

For individuals interested in learning more about Northwestern University’s study, the technical article is available on Arxiv, and current debates may be found on social media.

Leave a Reply

Your email address will not be published. Required fields are marked *