COARSE-TO-FINE FUSION METHOD AND APPARATUS FOR VIRTUAL SPACE NAVIGATION BASED ON LANGUAGE COMMANDS
卷宗概要
申请人
INDUSTRY ACADEMY COOPERATION FOUNDATION OF SEJONG UNIVERSITY
发明人
Yong Guk KIM; Soo Mi CHOI; Anh Hoang Vo; Thanh Tin Nguyen
IPC 分类
CPC 分类
Disclosed are a coarse-to-fine fusion method and apparatus for virtual space navigation based on language commands. The coarse-to-fine fusion method for virtual space navigation based on language commands includes: (a) applying an input image to an encoder to extract first to nth visual feature maps having a hierarchical structure; (b) applying an instruction having a length of N to a language model to extract a text feature map; and (c) fusing each of the first to nth visual feature maps with the text feature map to generate an attention map.
原文（中文）
Disclosed are a coarse-to-fine fusion method and apparatus for virtual space navigation based on language commands. The coarse-to-fine fusion method for virtual space navigation based on language commands includes: (a) applying an input image to an encoder to extract first to nth visual feature maps having a hierarchical structure; (b) applying an instruction having a length of N to a language model to extract a text feature map; and (c) fusing each of the first to nth visual feature maps with the text feature map to generate an attention map.