METHOD AND APPARATUS FOR DETECTING PROMPT INJECTIONS IN LLM-INTEGRATED APPLICATIONS
개요
발명자
Jesper SCHLEGEL
IPC 분류
CPC 분류
Method for recognizing manipulation attacks in the form of prompt injections on large-language model-integrated applications which generate a prompt in order to pass this as input to the LLM, which generates a model output based on the prompt, the prompt comprising a trusted part which contains the instructions defined by the LLM-integrated application, over which an attacker by definition has no influence, and wherein the prompt consists of an untrusted part comprising the data to be processed, over which the attacker by definition has full control, characterized in that a prompt injection is present if the trust score of the trusted part on the model output is below a threshold, wherein the trust score of the trusted part for the model output is calculated using an input feature saliency method.
원문 (중국어)
Method for recognizing manipulation attacks in the form of prompt injections on large-language model-integrated applications which generate a prompt in order to pass this as input to the LLM, which generates a model output based on the prompt, the prompt comprising a trusted part which contains the instructions defined by the LLM-integrated application, over which an attacker by definition has no influence, and wherein the prompt consists of an untrusted part comprising the data to be processed, over which the attacker by definition has full control, characterized in that a prompt injection is present if the trust score of the trusted part on the model output is below a threshold, wherein the trust score of the trusted part for the model output is calculated using an input feature saliency method.