Rafael dos Santos SilvaandRoman Rizzi d792919ddf DEV: Move tokenizers to a gem (#1481) ...

Also renames the Mixtral tokenizer to Mistral.

See gem at github.com/discourse/discourse_ai-tokenizers


Co-authored-by: Roman Rizzi <[email protected]>

2025-07-02 14:43:03 -03:00

..

chat_gpt.rb

FEATURE: optionally support OpenAI responses API (#1423)

2025-06-11 17:12:25 +10:00

claude_tools.rb

…

claude.rb

…

cohere_tools.rb

…

command.rb

FIX: Strip uploads from msg when searching for rag fragments (#1475)

2025-06-30 15:03:17 -03:00

dialect.rb

DEV: Move tokenizers to a gem (#1481)

2025-07-02 14:43:03 -03:00

fake.rb

…

gemini.rb

…

mistral.rb

…

nova_tools.rb

…

nova.rb

FIX: Strip uploads from msg when searching for rag fragments (#1475)

2025-06-30 15:03:17 -03:00

ollama_tools.rb

…

ollama.rb

FIX: Strip uploads from msg when searching for rag fragments (#1475)

2025-06-30 15:03:17 -03:00

open_ai_compatible.rb

…

open_ai_tools.rb

FEATURE: optionally support OpenAI responses API (#1423)

2025-06-11 17:12:25 +10:00

xml_tools.rb

…