DETERMINING ANOMALOUS TOOL INVOCATIONS BY APPLICATIONS USING LARGE GENERATIVE MODELS

원문 (중국어)

This disclosure relates to utilizing a threat detection system to detect anomalous actions provided by a compromised large generative language model (LLM). For instance, the threat detection system utilizes a detection-based large generative model to process select communication between an application system and the LLM and determine when the LLM may have been potentially compromised. In various implementations, utilizing the detection-based large generative model, the threat detection system determines when an LLM is improperly instructing an application system to invoke tools to perform unapproved actions. Furthermore, when an LLM becomes compromised, the threat detection system intelligently safeguards the detection-based large generative model against similar threats that seek to evade detection or compromise the detection-based large generative model.