ARBITRARILY LOW-LATENCY INTERFERENCE WITH COMPUTATIONALLY INTENSIVE MACHING LEARNING VIA PRE-FETCHING

원문 (중국어)

Methods for providing a machine learning (ML) final inference to a user, wherein an ML model and a computer-based content generation system (CGS) receives possible inputs and generates possible inferences, which are stored in association with the possible inputs to a memory so that they may be recalled based on the possible inputs. After receiving an actual input and an acceptability criterion, the CGS identifies a possible input that acceptably matches the actual input by satisfying the acceptability criterion. If a match is identified, the CGS substitutes the matching possible input in place of the actual input and outputs the possible inference corresponding to the matching possible input as the final inference to a user or to a second ML model. When a match is identified, inference is never performed on the actual input and the possible inferences are generated prior to receipt of the actual input.