SYSTEM, METHOD, AND COMPUTER PROGRAM FOR EVOLVING MULTI-TURN CHATBOT DIALOGS
개요
발명자
Liad Levi-Raz; Laila Sprejer; Andrew Baxter; Mary Tsilov
IPC 분류
CPC 분류
As described herein, an LLM-based chatbot is evolved over at least one iteration. The iteration includes presenting, by a LLM-based evaluator, a question to a LLM-based chatbot during a dialog with the LLM-based chatbot comprised of a sequence of question and answer pairs. The iteration includes receiving, by the LLM-based evaluator, an answer to the question from the LLM-based chatbot. The iteration includes evaluating, by the LLM-based evaluator, the answer according to one or more evaluation metrics and a ground truth. The iteration includes determining, by the LLM-based evaluator, that a result of the evaluation is unsatisfactory. The iteration includes presenting, by the LLM-based evaluator, a follow-up question to the LLM-based chatbot designed to encourage a new answer of the LLM-based chatbot to be satisfactory with respect to the ground truth and to cause an optimization of the LLM-based chatbot.
원문 (중국어)
As described herein, an LLM-based chatbot is evolved over at least one iteration. The iteration includes presenting, by a LLM-based evaluator, a question to a LLM-based chatbot during a dialog with the LLM-based chatbot comprised of a sequence of question and answer pairs. The iteration includes receiving, by the LLM-based evaluator, an answer to the question from the LLM-based chatbot. The iteration includes evaluating, by the LLM-based evaluator, the answer according to one or more evaluation metrics and a ground truth. The iteration includes determining, by the LLM-based evaluator, that a result of the evaluation is unsatisfactory. The iteration includes presenting, by the LLM-based evaluator, a follow-up question to the LLM-based chatbot designed to encourage a new answer of the LLM-based chatbot to be satisfactory with respect to the ground truth and to cause an optimization of the LLM-based chatbot.