EXPERT LOAD BALANCING IN TRANSFORMER MODELS
卷宗概要
申请人
ADVANCED MICRO DEVICES, INC.; Xilinx, Inc.
发明人
Venkata Pavan Kumar Miriyala; Lucian Petrica; Kenneth O'Brien
IPC 分类
CPC 分类
In response to one or more conditions, a processing system determines whether transferring one or more experts to different processing units would improve load balancing at the processing system. The processing system determines an amount of variance between the utilization for each expert relative to the average utilization of all experts at their currently-assigned processing units. The processing system then measures the amount of variance under one or more different configurations of expert-processing unit assignments. If so, the processing system transfers one or more of the experts to different processing units.
原文（中文）
In response to one or more conditions, a processing system determines whether transferring one or more experts to different processing units would improve load balancing at the processing system. The processing system determines an amount of variance between the utilization for each expert relative to the average utilization of all experts at their currently-assigned processing units. The processing system then measures the amount of variance under one or more different configurations of expert-processing unit assignments. If so, the processing system transfers one or more of the experts to different processing units.