Instructions to use CohereLabs/command-a-plus-05-2026-w4a4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CohereLabs/command-a-plus-05-2026-w4a4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="CohereLabs/command-a-plus-05-2026-w4a4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("CohereLabs/command-a-plus-05-2026-w4a4")
model = AutoModelForMultimodalLM.from_pretrained("CohereLabs/command-a-plus-05-2026-w4a4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use CohereLabs/command-a-plus-05-2026-w4a4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CohereLabs/command-a-plus-05-2026-w4a4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/command-a-plus-05-2026-w4a4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/CohereLabs/command-a-plus-05-2026-w4a4

SGLang

How to use CohereLabs/command-a-plus-05-2026-w4a4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CohereLabs/command-a-plus-05-2026-w4a4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/command-a-plus-05-2026-w4a4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CohereLabs/command-a-plus-05-2026-w4a4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/command-a-plus-05-2026-w4a4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use CohereLabs/command-a-plus-05-2026-w4a4 with Docker Model Runner:
```
docker model run hf.co/CohereLabs/command-a-plus-05-2026-w4a4
```

Tool calls fail in opencode

by retowyss - opened 24 days ago

Discussion

retowyss

24 days ago

Tool calls fail in opencode using vllm backend.

Thought: I am trying to list files in the current directory. I should use the glob tool to get a count of files. Let me use glob with pattern "*" to match all files and directories.
# invalid [tool=, error=Model tried to call unavailable tool ''. Available tools: bash, edit, edit-image, glob, grep, invalid, question, read, skill, task, todowrite, webfetch, write.]
The arguments provided to the tool are invalid: Model tried to call unavailable tool ''. Available tools: bash, edit, edit-image, glob, grep, invalid, question, read, skill, task, todowrite, webfetch, write.

This doesn't happen with any other model.

I've tried various ways of adapting the chat template but no luck. So maybe the parser is borked.

Any feedback or guidance is much appreciated.

rmhubbert

22 days ago

•

edited 22 days ago

I'm seeing this as well. Using vllm nightly with 8 x RTX 3090.

Tool calls succeed for me in Open WebUI, but fail every time in Opencode.

ahmeshaf

16 days ago

•

edited 16 days ago

Yep similar issue with claude code as well. although, able to call the tools, somehow, not able to process the tool results

 list the files in this repo                                                                                                                         

  Thought for 5s, read 1 file, listed 1 directory (ctrl+o to expand)

⏺ Bash(echo "Current directory: $(pwd)")
  ⎿  Current directory: /Users/shafiuddin.r.ahmed/workspace/....
     
⏺ Thinking for 25s… (ctrl+o to expand)
  ⎿  The user is getting "(no content)" responses from all Bash commands. This is very unusual. Let me try a different approach - I'll try using the 
     Read tool to attempt to read what's in the current directory, even though it's meant for files. Or I could try using git commands directly 
     without the Bash wrapper. Actually, let me try git rev-parse --show-toplevel to see if git commands work, and also try echo test to see if any 
     basic commands work.

✽ Deciphering… (37s)
  ⎿  Tip: Run /terminal-setup to enable convenient terminal integration like Option + Enter for new line and more

Terrencez

Cohere Labs org 5 days ago

Can you try the latest vllm main? We merged a parser fix for opencode

retowyss

5 days ago

Hey, thanks for looking into this! I hadn't mentioned this, but in a later test tool calling worked fine using PI.

I just tested with vllm==0.22.1rc1.dev307+g3e8afdf78 (cu130) nightly and I see the same issue still. in latest Opencode.
I also tried with vllm 0.22.1 same result.

Can you give me a link to the commit/branch?

Note: Fresh 0.22.1 fails and asks for cohere_melody to be installed.

rmhubbert

5 days ago

Thanks Terrencez,

Using vLLM 0.22.1rc1.dev327+gd7607ad27, and adding the interleaved setting to my Opencode config has fixed the issue for me. No failed tool calls, looking good!

My OpenCode config entry, in case that helps someone.

        "command-a-plus": {
          "name": "Cohere Command A+",
          "reasoning": true,
          "tool_call": true,
          "temperature": true,
          "open_weights": true,
          "modalities": {
            "input": ["text",  "image"],
            "output": ["text"]
          },
          "interleaved": {
            "field": "reasoning_content"
          },
          "cost": {
            "input": 0,
            "output": 0,
            "cache_read": 0
          },
          "limit": {
            "context": 200000,
            "output": 32768
          }
        },

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment