REGION OF INTEREST PROMPT PROCESSING FOR LARGE MULTIMODAL MODELS

原文（中国語）

A method for processing a multimodal prompt. The method includes receiving a multimodal prompt including a media file and information related to a region of interest (ROI) of the media file. The method further includes determining a ROI of the media file based on the information related to the media file and generating a plurality of media tiles of interest associated with the ROI. The method further includes encoding the plurality of media tiles of interest and using a large multimodal model (LMM) to process the encoded plurality of media tiles of interest according to a natural-language input of the prompt to generate a response.