METHOD AND APPARATUS FOR DETECTING PROMPT INJECTIONS IN LLM-INTEGRATED APPLICATIONS
卷宗概要
发明人
Jesper SCHLEGEL
IPC 分类
CPC 分类
Method for recognizing manipulation attacks in the form of prompt injections on large-language model-integrated applications which generate a prompt in order to pass this as input to the LLM, which generates a model output based on the prompt, the prompt comprising a trusted part which contains the instructions defined by the LLM-integrated application, over which an attacker by definition has no influence, and wherein the prompt consists of an untrusted part comprising the data to be processed, over which the attacker by definition has full control, characterized in that a prompt injection is present if the trust score of the trusted part on the model output is below a threshold, wherein the trust score of the trusted part for the model output is calculated using an input feature saliency method.
原文（中文）
Method for recognizing manipulation attacks in the form of prompt injections on large-language model-integrated applications which generate a prompt in order to pass this as input to the LLM, which generates a model output based on the prompt, the prompt comprising a trusted part which contains the instructions defined by the LLM-integrated application, over which an attacker by definition has no influence, and wherein the prompt consists of an untrusted part comprising the data to be processed, over which the attacker by definition has full control, characterized in that a prompt injection is present if the trust score of the trusted part on the model output is below a threshold, wherein the trust score of the trusted part for the model output is calculated using an input feature saliency method.