Qwen-MM-Plugins Core

You have qwen-mm-plugins-core MCP tools available. Use them to read and visualize any file (images, videos, documents, code, data, 3D models, notebooks, etc.) and to call vision-language APIs. Prefer these MCP tools over manual scripting.

Check the qwen-mm-plugins-core tools in your tool list for full schemas and parameters.

When to Use Which Tool

Native reading (feeds content directly to you):

• See a file (PDF, Office, CSV, code, notebook, 3D, ...) → visualize
• Read an image with dynamic resolution → read_image
• Read a video (extract frames) → read_video
• Save specific frame(s) of a video to file (for image_search) → save_view (pass times=[...])

Producing / annotating (writes an image file):

• Crop a rectangular region from an image → crop
• Draw bounding boxes on an image (e.g. to visualize grounding output) → draw_bbox

External API calls (DashScope):

• Call an external VLM about images/videos → vision_chat
• Transcribe speech from audio/video → transcribe_audio
• Detect/locate objects in an image → grounding
• Extract text from an image → ocr
• Segment objects in an image → segmentation
• Reverse image search to identify an entity from a frame → image_search
• Search the web for facts → web_search
• Read a web page in depth → web_extractor

Visualize — Supported Formats

Use pages for page ranges, budget for resolution, max_pages to cap output.

Tips

Resolution budgets: small for preview, normal (~1024) default, large for fine detail.

Video strategy: fps=0 auto-selects. Skim with fps=1, budget="normal" in 5-min chunks, then detail with fps=2, budget="large" on interesting segments. Use start_time/end_time for windowing.

Vision chat: pass images/videos + text prompt. Default model qwen3.7-plus. Use dry_run=true to inspect payloads. Details in references/vision_chat.md.

Video knowledge & search (any video question that needs external knowledge — identify a specific thing, OR a fact about what's shown that you can't confirm from the frames alone): watch with read_video — for a long video do a low-fps overview first (~32 frames, fps≈32/duration) then zoom, and don't run ffmpeg/montage yourself — grab a frame with save_view (times=[...]), and you MUST confirm with image_search (reverse-search the frame) and/or web_search before answering — never commit from frames alone. Details in references/video_search.md.

ASR: accepts audio or video, auto-chunks long files. Formats: srt (default), text, json. Needs DASHSCOPE_API_KEY.

Segmentation: needs a SAM3 server (SAM3_SERVER_URL). To stand one up, run references/launch_sam3_server.py (multi-GPU HTTP server; see its header for prerequisites).