Docs/Integrations/Together & OSS Models
Together & OSS Models
Any OpenAI-compatible host — Together, Groq, vLLM, Ollama, OpenRouter — works as an upstream and as a replay candidate.
OpenAI-compatible hosts
The gateway isn't limited to the big two providers. Any host that speaks the OpenAI API shape can be configured as an upstream and addressed by its path prefix — hosted platforms like Together, Groq, and OpenRouter, or your own vLLM and Ollama deployments.
upstreams: openai: https://api.openai.com anthropic: https://api.anthropic.com together: https://api.together.xyz # groq: https://api.groq.com/openai # vllm: http://10.0.0.12:8000 # self-hosted
Together AI
With a togetherupstream configured, point the OpenAI SDK at it — Together's API is OpenAI-shaped, so the client is identical:
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["TOGETHER_API_KEY"], # your Together key
base_url="https://gateway.valite.ai/together/v1",
default_headers={"x-valite-api-key": "vk_..."},
)
resp = client.chat.completions.create(
model="meta-llama/Llama-4-Maverick",
messages=[{"role": "user", "content": "Hello"}],
)OSS models as candidates
The bigger win: open-weight models as replay candidates for traffic currently on frontier models. In a replay request, candidate models resolve three ways — a bare name (gpt-4o-mini), a catalog ID via an OpenRouter upstream (meta-llama/llama-4-maverick), or pinned to a specific upstream (together:meta-llama/Llama-4-Maverick).
curl -X POST \
https://gateway.valite.ai/valite/api/workflows/support-triage/jobs/replay \
-H "x-valite-api-key: vk_..." -H "Content-Type: application/json" \
-d '{"models": ["together:meta-llama/Llama-4-Maverick"], "budget": 5.00}'Note