Tokenizer - Python SDK

Tokenizer method reference

The Python SDK and docs are currently in beta. Report issues on GitHub.

Tokenizer type used by the model

Values

Name	Value
`ROUTER`	Router
`MEDIA`	Media
`OTHER`	Other
`GPT`	GPT
`CLAUDE`	Claude
`GEMINI`	Gemini
`GEMMA`	Gemma
`GROK`	Grok
`COHERE`	Cohere
`NOVA`	Nova
`QWEN`	Qwen
`YI`	Yi
`DEEP_SEEK`	DeepSeek
`MISTRAL`	Mistral
`LLAMA2`	Llama2
`LLAMA3`	Llama3
`LLAMA4`	Llama4
`PA_LM`	PaLM
`RWKV`	RWKV
`QWEN3`	Qwen3