SYSTEM, METHOD, AND COMPUTER PROGRAM FOR EVOLVING MULTI-TURN CHATBOT DIALOGS

原文（中国語）

As described herein, an LLM-based chatbot is evolved over at least one iteration. The iteration includes presenting, by a LLM-based evaluator, a question to a LLM-based chatbot during a dialog with the LLM-based chatbot comprised of a sequence of question and answer pairs. The iteration includes receiving, by the LLM-based evaluator, an answer to the question from the LLM-based chatbot. The iteration includes evaluating, by the LLM-based evaluator, the answer according to one or more evaluation metrics and a ground truth. The iteration includes determining, by the LLM-based evaluator, that a result of the evaluation is unsatisfactory. The iteration includes presenting, by the LLM-based evaluator, a follow-up question to the LLM-based chatbot designed to encourage a new answer of the LLM-based chatbot to be satisfactory with respect to the ground truth and to cause an optimization of the LLM-based chatbot.