PIPELINED EXECUTION OF DATABASE QUERIES PROCESSING STREAMING DATA
Dossier Overview
Applicant
Databricks, Inc.
Inventor
Michael Paul Armbrust; Alexander Balikov; Boyang Peng
IPC Classification
CPC Classification
A database system performs pipelined execution of queries that process batches of streaming data. The database system compiles a database query to generate an execution plan and determines a set of stages based on the execution plan. The database query processes streaming data comprising batches. A scheduler schedules pipelined execution stages of the database query. Accordingly, the database system performs execution of a particular stage processing a batch of the streaming data in parallel with subsequent stages of the database query processing previous batches of the streaming data. The system further maintains watermarks for different stages of the database query.
Original (Chinese)
A database system performs pipelined execution of queries that process batches of streaming data. The database system compiles a database query to generate an execution plan and determines a set of stages based on the execution plan. The database query processes streaming data comprising batches. A scheduler schedules pipelined execution stages of the database query. Accordingly, the database system performs execution of a particular stage processing a batch of the streaming data in parallel with subsequent stages of the database query processing previous batches of the streaming data. The system further maintains watermarks for different stages of the database query.
External Resources