* FEATURE: simplify streaming implementation - rush last update
Previous to this change we would simply "flash" the final update
on the screen, this amends it so we quickly update the UI in about
1 second in the end with all the final update.
This makes the UI feel more interactive to end users.
* DEV: Updates...
- Remove unnecessary comments
- Add doc style comments for all methods
- Organize methods (private at bottom)
- Update some variable names
---------
Co-authored-by: Keegan George <[email protected]>
Related: https://github.com/discourse/discourse-translator/pull/310
This commit includes all the jobs and event hooks to localize posts, topics, and categories.
A few notes:
- `feature_name: "translation"` because the site setting is `ai-translation` and module is `Translation`
- we will switch to proper ai-feature in the near future, and can consider using the persona_user as `localization.localizer_user_id`
- keeping things flat within the module for now as we will be moving to ai-feature soon and have to rearrange
- Settings renamed/introduced are:
- ai_translation_backfill_rate (0)
- ai_translation_backfill_limit_to_public_content (true)
- ai_translation_backfill_max_age_days (5)
- ai_translation_verbose_logs (false)
Since we enable/disable `ai_spam_detection_enabled` setting in a custom Spam tab UI in AI, we want to ensure we retain the setting and logging features. To preserve that, we want to update the controller to use `SiteSetting.set_and_log` instead of setting the value directly.
## 🔍 Overview
We want to unhide `ai_spam_detection_enabled` setting so that we can retain staff action log features. However, we also want to ensure users cannot enable spam detection without having `AiModerationSetting.spam` present in the database.
In this update we unhide the setting, but also add a validator to ensure the necessary configuration is in place before allowing the setting to be enabled.
## 🔍 Overview
When you have a post with audio being played and you open and close the AI post helper menu, it re-renders the entire post DOM, causing the audio to be interrupted and stop playing.
The reason for this is because we highlight the selected text when opening the AI post helper menu and we replace the entire post back with the original post HTML when closing the menu. This fix ensures that we do not re-render the entire post DOM and instead only remove the highlighted section that was added.
## 🔗 Context
https://meta.discourse.org/t/ai-helper-interrupting-uploaded-mp3-audio-stream/366817?u=keegan
* Small fix, reasoning is now available on Claude 4 models
* fix invalid filters should raise, topic filter not working
* fix spec so we are consistent
* FIX: apply diffs more consistently
1. Do escaping direct in diff streamer, that way HTML tags and other unsafe chars can be displayed and fixed
2. Add safeguard to ensure streaming always stops when it was terminated elsewhere
* lint
* bug unsubscribe should unsubscribe
We were getting an error in this logic causing Ember to fail to render the non-bot-topic that we navigate to.
I believe this is because the getter of participants is re-calculating (due to this.header.topicInfo being updated) before the args to this connector changes. Adding some safe navigation here fixes the issue.
* FIX: Improve MessageBus efficiency and correctly stop streaming
This commit enhances the message bus implementation for AI helper streaming by:
- Adding client_id targeting for message bus publications to ensure only the requesting client receives streaming updates
- Limiting MessageBus backlog size (2) and age (60 seconds) to prevent Redis bloat
- Replacing clearTimeout with Ember's cancel method for proper runloop management, we were leaking a stop
- Adding tests for client-specific message delivery
These changes improve memory usage and make streaming more reliable by ensuring messages are properly directed to the requesting client.
* composer suggestion needed a fix as well.
* backlog size of 2 is risky here cause same channel name is reused between clients
**This update includes a variety of small fixes to the AI composer helper:**
- ensures there is no flash of no text when diff modal is triggered
- escapes selected text prior to diff streaming
- uses monospace font in diff modal since text rendered is raw markdown
* FIX: ensures stream update object is scoped to its initial topic
Before this commit you could end up in this situation where a `post-updater` is constructed for a specific topic, but the user changes topic mid steam and it ends up updating the same post number but in a different topic as we were only checking for `post_number` and not the combination of `topic_id` + `post_number`.
* we should have the topic
Previous to this fix we would diff the entire body of text, this
could lead to situations where a diff presented to a user was
wildly off matching areas in the text that should not have been
tested.
New algorithm only checks a portion of the string, this ensures
that during streaming there is no chance for wild mistakes
The structured output JSON comes embedded inside the API response, which is also a JSON. Since we have to parse the response to process it, any control characters inside the structured output are unescaped into regular characters, leading to invalid JSON and breaking during parsing. This change adds a retry mechanism that escapes
the string again if parsing fails, preventing the parser from breaking on malformed input and working around this issue.
For example:
```
original = '{ "a": "{\\"key\\":\\"value with \\n newline\\"}" }'
JSON.parse(original) => { "a" => "{\"key\":\"value with \n newline\"}" }
# At this point, the inner JSON string contains an actual newline.
```
Alan Guo Xiang Tan
David Taylor
SamandKeegan George
Natalie Tay
Guhyoun Nam
Roman Rizzi
Discourse Translator Bot
Mark VanLandingham
Jordan Vidrine
Joffrey JAFFEUX
Kris