ITERATIVE DATA PROCESSING OPTIMIZATION ENGINE IN A DATA INTELLIGENCE SYSTEM

原文（中国語）

Methods, systems, and computer storage media for providing iterative data processing optimization using an iterative data processing optimization engine in a data intelligence system are described. Iterative data processing refers to handling data where the processing steps are repeated multiple times, across multiple views or modalities, to train machine learning models, filter and score data or generate output. The iterative data processing optimization engine employs expectation step machine learning models that are simple but with fast language models to efficiently and effectively probe and analyze data, while iteratively refining maximization step machine learning models that are optimized and fast to approximate the probing mechanism of the expectation step machine learning models more efficiently, for example, using metadata, external information, and compressed representation. The iterative data processing optimization engine can operate based on an agentic framework using lightweight artificial intelligence (AI) agents to perform model fitting, featurization, and report generation autonomously.