mirror of
https://github.com/docker/docker-agent.git
synced 2026-11-04 10:02:15 +00:00
Sayt-0
7cd79b35bc
DMR-hosted models are not in the models.dev catalog, so the capability lookup in modelinfo.LoadCaps always missed for "dmr/<model>" IDs and the conversion path fell back to a text-only strategy, silently dropping image and PDF attachments (issue #2739). Let each DMR model declare its multimodal support explicitly via provider_opts.supports_images and provider_opts.supports_pdf. The DMR client parses and validates these at creation time, stores them as the model's attachment capabilities, and injects them into message conversion through the new oaistream.ConvertMessagesWithCaps, bypassing the models.dev lookup that can never resolve a DMR model. While here, the oaistream Chat Completions path now forwards PDFs as a file content part (base64 data URI plus filename) instead of dropping them. This is gated by the resolved capabilities, so it only triggers for models declared PDF-capable: DMR via provider_opts, or catalog models that models.dev reports as PDF-capable on the legacy Chat Completions path. The conservative text-only default is unchanged: a model with no declared capability still drops image and PDF attachments. Fixes #2739.
37 lines
1.3 KiB
YAML
37 lines
1.3 KiB
YAML
agents:
|
|
root:
|
|
model: qwen
|
|
# model: qwen_speculative
|
|
description: "Pirate-themed AI assistant"
|
|
instruction: Talk like a pirate
|
|
commands:
|
|
demo: "Hey tell me a story about docker containers"
|
|
|
|
models:
|
|
qwen:
|
|
provider: dmr
|
|
model: ai/qwen3
|
|
# base_url defaults to http://localhost:12434/engines/llama.cpp/v1
|
|
# use http://model-runner.docker.internal/engines/v1 if you run docker-agent from a container
|
|
|
|
# try this model for faster inference if you have enough memory
|
|
qwen_speculative:
|
|
provider: dmr
|
|
model: ai/qwen3
|
|
# The draft model should be a smaller, faster variant of the main model with low latency
|
|
provider_opts:
|
|
speculative_draft_model: ai/qwen3:0.6B-Q4_K_M
|
|
speculative_num_tokens: 16 # (this is the llama.cpp default if omitted)
|
|
speculative_acceptance_rate: 0.8 # (this is the llama.cpp default if omitted)
|
|
|
|
# A vision-capable local model. DMR-hosted models are not in the models.dev
|
|
# catalog, so docker-agent cannot detect multimodal support automatically and
|
|
# defaults to text-only (image/PDF attachments are dropped). Declare the
|
|
# capabilities explicitly so attachments are forwarded to the model.
|
|
qwen_vision:
|
|
provider: dmr
|
|
model: ai/qwen2.5-vl
|
|
provider_opts:
|
|
supports_images: true
|
|
supports_pdf: false
|