EXPERT LOAD BALANCING IN TRANSFORMER MODELS
개요
출원인
ADVANCED MICRO DEVICES, INC.; Xilinx, Inc.
발명자
Venkata Pavan Kumar Miriyala; Lucian Petrica; Kenneth O'Brien
IPC 분류
CPC 분류
In response to one or more conditions, a processing system determines whether transferring one or more experts to different processing units would improve load balancing at the processing system. The processing system determines an amount of variance between the utilization for each expert relative to the average utilization of all experts at their currently-assigned processing units. The processing system then measures the amount of variance under one or more different configurations of expert-processing unit assignments. If so, the processing system transfers one or more of the experts to different processing units.
원문 (중국어)
In response to one or more conditions, a processing system determines whether transferring one or more experts to different processing units would improve load balancing at the processing system. The processing system determines an amount of variance between the utilization for each expert relative to the average utilization of all experts at their currently-assigned processing units. The processing system then measures the amount of variance under one or more different configurations of expert-processing unit assignments. If so, the processing system transfers one or more of the experts to different processing units.