Previous to this change we reused channels for proofreading progress and
ai helper progress
The new changeset ensures each POST to stream progress gets a dedicated
message bus channel
This fixes a class of issues where the wrong information could be displayed
to end users on subsequent proofreading or helper calls
* fix tests
* fix implementation (got to subscribe at 0)
- allows features to have multiple llms and multiple personas
- sorts module list
- adds Bot as a first class module
- fixes issue where search module was always configured
- some tests
* FIX: A typo in bot filtration in ai-bot-header-icon
* FIX: Show header icon when there's only one persona with a default LLM set
---------
Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
Introduces import/export feature for tools and personas.
Uploads are omitted for now, and will be added in a future PR
* **Backend:**
* Adds `import` and `export` actions to `Admin::AiPersonasController` and `Admin::AiToolsController`.
* Introduces `DiscourseAi::PersonaExporter` and `DiscourseAi::PersonaImporter` services to manage JSON serialization and deserialization.
* The export format for a persona embeds its associated custom tools. To ensure portability, `AiTool` references are serialized using their `tool_name` rather than their internal database `id`.
* The import logic detects conflicts by name. A `force=true` parameter can be passed to overwrite existing records.
* **Frontend:**
* `AiPersonaListEditor` and `AiToolListEditor` components now include an "Import" button that handles file selection and POSTs the JSON data to the respective `import` endpoint.
* `AiPersonaEditorForm` and `AiToolEditorForm` components feature an "Export" button that triggers a download of the serialized record.
* Handles import conflicts (HTTP `409` for tools, `422` for personas) by showing a `dialog.confirm` prompt to allow the user to force an overwrite.
* **Testing:**
* Adds comprehensive request specs for the new controller actions (`#import`, `#export`).
* Includes unit specs for the `PersonaExporter` and `PersonaImporter` services.
* Persona import and export implemented
This update replaces deprecated integer duration with standardized duration of either `short` or `long` throughout the plugin usages of `FloatKit` toasts.
This update fixes an issue where the AI user preferences tab was not appearing unless `SiteSetting.ai_helper_enabled` was `true`. This is because we previously checked for it's presence when user preferences only had a single setting related to Helper. However, since then, we've also added search discoveries setting there too. As such, we don't want it to depend on Helper. We also sneak in this update a modernization of converting the preferences template from `.hbs` to `.gjs`.
## 🔍 Overview
This update fixes an issue where message bus streaming related specs
were not working correctly. To do so we pass the `last_id` when
subscribing to `MessageBus` which allows us to unskip those broken
tests.
---------
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
The current menu was rendering inside the post text toolbar (on desktop). This is not ideal as the post text toolbar rendering is conditioned on the presence of text selection, when you click a button on the toolbar, by design of the web browsers you will lose your text selection, making all of this super tricky.
This commit makes desktop and mobile behave in the same way by rendering their own menu and capturing the quote state when we render the post text selection toolbar, this allows us to reason a much simpler way about the AI helper.
This commit also removes what appears to be an unused file and corrects which was seemingly copy/paste mistakes.
⚠️ Technical note, this commit is correcting the message bus subscription which amongst other things allows to write specs which are not flaky. However due to the current implementation we have a channel per post, which means we need to serialize on last message bus id per post.
We have two possible solutions here:
- subscribe at the topic level
- refactor the code to be able to use `MessageBus.last_ids` to be able to grab multiple posts at once instead of having to call `MessageBus.last_id` and done one Redis call per post
---------
Co-authored-by: Keegan George <kgeorge13@gmail.com>
* FIX: implement max_output tokens (anthropic/openai/bedrock/gemini/open router)
Previously this feature existed but was not implemented
Also updates a bunch of models to in our preset to point to latest
* implementing in base is safer, simpler and easier to manage
* anthropic 3.5 is getting older, lets use 4.0 here and fix spec
This update ensures that we reset the tracked properties when closing the DiffModal so that the state doesn't leak when triggering the AI suggestions again. We also reset before suggesting new changes, thus if regeneration is called there shouldn't be any leaks either.
No tests in this PR as tests currently not working great due to streaming/animation issues. Will do a broader PR following up with various specs to improve test coverage here.
In hybrid mode ai artifacts can optionally automatically run.
This is useful for cases where you may want to embed a survey and so on.
Additionally, artifacts now allow for better fidelity around display:
<div class="ai-artifact" data-ai-artifact-id="501" data-ai-artifact-height="300px" data-ai-artifact-autorun data-ai-artifact-seamless></div>
User can supply height and seamless mode to be seamlessly rendered with no box shadow and show full screen button.
* FIX: improve transition logic in forms
previously back button would take you back to the /new route
* FIX: enum selection not working for persona tools
* seed information correctly in the DB
* fix broken spec
* Update assets/javascripts/discourse/components/ai-tool-editor-form.gjs
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
---------
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
## 🔍 Overview
When the title suggestions return no suggestions, there is no indication in the UI. In the tag suggester we show a toast when there aren't any suggestions but the request was a success. In this update we make a similar UI indication with a toast for both category and title suggestions. Additionally, for all suggestions we add a "Try again" button to the toast so that suggestions can be generated again if the results yield nothing the first time.
Introduces a persistent, user-scoped key-value storage system for
AI Artifacts, enabling them to be stateful and interactive. This
transforms artifacts from static content into mini-applications that can
save user input, preferences, and other data.
The core components of this feature are:
1. **Model and API**:
- A new `AiArtifactKeyValue` model and corresponding database table to
store data associated with a user and an artifact.
- A new `ArtifactKeyValuesController` provides a RESTful API for
CRUD operations (`index`, `set`, `destroy`) on the key-value data.
- Permissions are enforced: users can only modify their own data but
can view public data from other users.
2. **Secure JavaScript Bridge**:
- A `postMessage` communication bridge is established between the
sandboxed artifact `iframe` and the parent Discourse window.
- A JavaScript API is exposed to the artifact as `window.discourseArtifact`
with async methods: `get(key)`, `set(key, value, options)`,
`delete(key)`, and `index(filter)`.
- The parent window handles these requests, makes authenticated calls to the
new controller, and returns the results to the iframe. This ensures
security by keeping untrusted JS isolated.
3. **AI Tool Integration**:
- The `create_artifact` tool is updated with a `requires_storage`
boolean parameter.
- If an artifact requires storage, its metadata is flagged, and the
system prompt for the code-generating AI is augmented with detailed
documentation for the new storage API.
4. **Configuration**:
- Adds hidden site settings `ai_artifact_kv_value_max_length` and
`ai_artifact_max_keys_per_user_per_artifact` for throttling.
This also includes a minor fix to use `jsonb_set` when updating
artifact metadata, ensuring other metadata fields are preserved.
Additional changes:
Adds a "#features" method in AiPersona to find which features are using that persona.
Serializes a basic version of a LlmModel in the persona's "#default_llm" serializer attribute.
* FEATURE: Display features that rely on multiple personas.
This change makes the previously hidden feature page visible while displaying features, like the AI helper, which relies on multiple personas.
* Fix system specs
This commit changes the RAG uploader form elements to
be @format="full" instead of doing a hardcoded 500px width,
which was causing a horizontal scrollbar in the tools form
on mobile.
Also, it moves the 80% max width for the tools form into the
new viewport CSS API, and only applies it on desktop, because
this was also causing width issues on mobile.
* FEATURE: add inferred concepts system
This commit adds a new inferred concepts system that:
- Creates a model for storing concept labels that can be applied to topics
- Provides AI personas for finding new concepts and matching existing ones
- Adds jobs for generating concepts from popular topics
- Includes a scheduled job that automatically processes engaging topics
* FEATURE: Extend inferred concepts to include posts
* Adds support for concepts to be inferred from and applied to posts
* Replaces daily task with one that handles both topics and posts
* Adds database migration for posts_inferred_concepts join table
* Updates PersonaContext to include inferred concepts
Co-authored-by: Roman Rizzi <rizziromanalejandro@gmail.com>
Co-authored-by: Keegan George <kgeorge13@gmail.com>
This is showing as the most expensive CSS selector in Discourse at the moment. Adding specific classes and dropping the general `span` selector will make this much cheaper.
This update includes a variety of small refinements to the AI composer helper:
- prevent height jump when going from loading text placeholder → proofreading text streaming
- update padding on AI helper options list to be more suitable with typical Discourse menu design
- for composer helper results that are not `showResultAsDiff` (i.e. translation):
- update before/after diff design to be more subtle
- results should be in normal font (as the text is cooked and not raw markdown)
- fix: smooth streaming animation stuck showing dot icon even after smooth streaming is done
* FEATURE: simplify streaming implementation - rush last update
Previous to this change we would simply "flash" the final update
on the screen, this amends it so we quickly update the UI in about
1 second in the end with all the final update.
This makes the UI feel more interactive to end users.
* DEV: Updates...
- Remove unnecessary comments
- Add doc style comments for all methods
- Organize methods (private at bottom)
- Update some variable names
---------
Co-authored-by: Keegan George <kgeorge13@gmail.com>
Related: https://github.com/discourse/discourse-translator/pull/310
This commit includes all the jobs and event hooks to localize posts, topics, and categories.
A few notes:
- `feature_name: "translation"` because the site setting is `ai-translation` and module is `Translation`
- we will switch to proper ai-feature in the near future, and can consider using the persona_user as `localization.localizer_user_id`
- keeping things flat within the module for now as we will be moving to ai-feature soon and have to rearrange
- Settings renamed/introduced are:
- ai_translation_backfill_rate (0)
- ai_translation_backfill_limit_to_public_content (true)
- ai_translation_backfill_max_age_days (5)
- ai_translation_verbose_logs (false)
## 🔍 Overview
When you have a post with audio being played and you open and close the AI post helper menu, it re-renders the entire post DOM, causing the audio to be interrupted and stop playing.
The reason for this is because we highlight the selected text when opening the AI post helper menu and we replace the entire post back with the original post HTML when closing the menu. This fix ensures that we do not re-render the entire post DOM and instead only remove the highlighted section that was added.
## 🔗 Context
https://meta.discourse.org/t/ai-helper-interrupting-uploaded-mp3-audio-stream/366817?u=keegan
* FIX: apply diffs more consistently
1. Do escaping direct in diff streamer, that way HTML tags and other unsafe chars can be displayed and fixed
2. Add safeguard to ensure streaming always stops when it was terminated elsewhere
* lint
* bug unsubscribe should unsubscribe
We were getting an error in this logic causing Ember to fail to render the non-bot-topic that we navigate to.
I believe this is because the getter of participants is re-calculating (due to this.header.topicInfo being updated) before the args to this connector changes. Adding some safe navigation here fixes the issue.
* FIX: Improve MessageBus efficiency and correctly stop streaming
This commit enhances the message bus implementation for AI helper streaming by:
- Adding client_id targeting for message bus publications to ensure only the requesting client receives streaming updates
- Limiting MessageBus backlog size (2) and age (60 seconds) to prevent Redis bloat
- Replacing clearTimeout with Ember's cancel method for proper runloop management, we were leaking a stop
- Adding tests for client-specific message delivery
These changes improve memory usage and make streaming more reliable by ensuring messages are properly directed to the requesting client.
* composer suggestion needed a fix as well.
* backlog size of 2 is risky here cause same channel name is reused between clients
**This update includes a variety of small fixes to the AI composer helper:**
- ensures there is no flash of no text when diff modal is triggered
- escapes selected text prior to diff streaming
- uses monospace font in diff modal since text rendered is raw markdown
Joffrey JAFFEUX