capabilities / local visual analysis

Local Visual Analysis

MiniCPM-V can provide local visual analysis for screenshots, images, and video frames.

Developer Previewllms.txtSource evidence
Local-First Visual Analysis Visual analysis can route to a local MiniCPM-V runtime even when the selected text model lacks native vision. Local-First Visual Analysis Visual analysis can route to a local MiniCPM-V runtime even when the selected text model lacks native vision. captures pixelsanalyzes locallyreturns schemacontrols risk Vision bridge local runtimeschemarepair 1 Input ImageVideo frameScreen state 2 Local runtime MiniCPM-Vllama.cppSchema output 3 Agent Text modelTool resultFollow-up 4 Control RepairTimeoutEndpoint review
Visual analysis can route to a local MiniCPM-V runtime even when the selected text model lacks native vision.
Product screenshots

Settings

Settings search keeps provider, permission, and runtime configuration discoverable.

Ambient Desktop settings search.

Why This Matters

Some strong text models do not handle visual input natively. Ambient can still do local-first visual analysis by routing image or frame understanding to MiniCPM-V and returning structured observations to the agent.

Safety And Repair

  • Local runtime checks identify missing binaries, model assets, timeouts, invalid structured output, and memory pressure.
  • Hosted or remote visual endpoints require explicit security review before they are enabled.
  • This is a routing building block for local-first operation.