Since translations only require a single key back, there is little point in using structured output. This PR also includes some prompt updates dealing with quotes, details, and code.
Related: #1502
This does mean reverting discourse/discourse-translator#257, but we can see how it goes.
Structured outputs are prone to formatting issue, especially around newlines
and custom pieces of text that need escaping.
This avoids using it for the automation reporting.
Particularly previous to this fix o4-mini based reports were broken
This commit
- normalizes locales like en_GB and variants to en. With this, the feature will not translate en_GB posts to en (or similarly pt_BR to pt_PT)
- consolidates whether the feature is enabled in `DiscourseAi::Translation.enabled?`
- similarly for backfill in `DiscourseAi::Translation.backfill_enabled?`
- turns off backfill if `ai_translation_backfill_max_age_days` is 0 to keep true to what it says. Set it to a high number to backfill everything
This update fixes a regression from https://github.com/discourse/discourse-ai/pull/1484, which caused AI helper title suggestions to begin suggesting numerous non-unique titles because it was looping through structured responses incorrectly.
* FIX: make AI helper more robust
- If JSON is broken for structured output then lean on a more forgiving parser
- Gemini 2.5 flash does not support temp, support opting out
- Evals for assistant were broken, fix interface
- Add some missing LLMs
- Translator was not mapped correctly to the feature - fix that
- Don't mix XML in prompt for translator
* lint
* correct logic
* simplify code
* implement best effort json parsing direct in the structured output object
Keegan George
Natalie Tay
Sam
Roman Rizzi
Rafael dos Santos Silva