INTELLIGENT SYSTEM FOR SCALABLE NETWORK DATA TRANSMISSION
INTELLIGENT SYSTEM FOR SCALABLE NETWORK DATA TRANSMISSION
Systems, computer program products, and methods are described herein for scalable network data transmission within a network environment. An example system receives a plurality of ETL tasks and task characteristics associated with each ETL task. In addition, the example system captures network environment configuration at periodic intervals. Using a machine learning (ML) model, the example system may dynamically determine additional secondary ETL pipelines to process the plurality of ETL tasks. Once the appropriate secondary pipelines are identified, the system executes the ETL tasks using both the primary and the secondary pipelines, ensuring efficient and adaptable task processing within a network environment.