REGION OF INTEREST PROMPT PROCESSING FOR LARGE MULTIMODAL MODELS
개요
출원인
Microsoft Technology Licensing, LLC
발명자
Shubham VERMA; Sanjay RAMANUJAN; Rakesh KELKAR; Ashwini KATARIA; Sagar TANEJA
IPC 분류
CPC 분류
A method for processing a multimodal prompt. The method includes receiving a multimodal prompt including a media file and information related to a region of interest (ROI) of the media file. The method further includes determining a ROI of the media file based on the information related to the media file and generating a plurality of media tiles of interest associated with the ROI. The method further includes encoding the plurality of media tiles of interest and using a large multimodal model (LMM) to process the encoded plurality of media tiles of interest according to a natural-language input of the prompt to generate a response.
원문 (중국어)
A method for processing a multimodal prompt. The method includes receiving a multimodal prompt including a media file and information related to a region of interest (ROI) of the media file. The method further includes determining a ROI of the media file based on the information related to the media file and generating a plurality of media tiles of interest associated with the ROI. The method further includes encoding the plurality of media tiles of interest and using a large multimodal model (LMM) to process the encoded plurality of media tiles of interest according to a natural-language input of the prompt to generate a response.