This commit enhances the AI image generation functionality by adding support for:
1. OpenAI's GPT-based image generation model (gpt-image-1)
2. Image editing capabilities through the OpenAI API
3. A new "Designer" persona specialized in image generation and editing
4. Two new AI tools: CreateImage and EditImage
Technical changes include:
- Renaming `ai_openai_dall_e_3_url` to `ai_openai_image_generation_url` with a migration
- Adding `ai_openai_image_edit_url` setting for the image edit API endpoint
- Refactoring image generation code to handle both DALL-E and the newer GPT models
- Supporting multipart/form-data for image editing requests
* wild guess but maybe quantization is breaking the test sometimes
this increases distance
* Update lib/personas/designer.rb
Co-authored-by: Alan Guo Xiang Tan <[email protected]>
* simplify and de-flake code
* fix, in chat we need enough context so we know exactly what uploads a user uploaded.
* Update lib/personas/tools/edit_image.rb
Co-authored-by: Alan Guo Xiang Tan <[email protected]>
* cleanup downloaded files right away
* fix implementation
---------
Co-authored-by: Alan Guo Xiang Tan <[email protected]>
# Preview
https://github.com/user-attachments/assets/3fe3ac8f-c938-4df4-9afe-11980046944d
# Details
- Group pms by `last_posted_at`. In this first iteration we are group by `7 days`, `30 days`, then by month beyond that.
- I inject a sidebar section link with the relative (last_posted_at) date and then update a tracked value to ensure we don't do it again. Then for each month beyond the first 30days, I add a value to the `loadedMonthLabels` set and we reference that (plus the year) to see if we need to load a new month label.
- I took the creative liberty to remove the `Conversations` section label - this had no purpose
- I hid the _collapse all sidebar sections_ carrot. This had no purpose.
- Swap `BasicTopicSerializer` to `ListableTopicSerializer` to get access to `last_posted_at`
In the last commit, I introduced a topic_custom_field to determine if a PM is indeed a bot PM.
This commit adds a migration to backfill any PM that is between 1 real user, and 1 bot. The correct topic_custom_field is added for these, so they will appear on the bot conversation sidebar properly.
We can also drop the joining to topic_users in the controller for sidebar conversations, and the isPostFromAiBot logic from the sidebar.
* FEATURE: display more places where AI is used
- Usage was not showing automation or image caption in llm list.
- Also: FIX - reasoning models would time out incorrectly after 60 seconds (raised to 10 minutes)
* correct enum not to enumerate non configured models
* FEATURE: implement chat streamer
This implements a basic chat streamer, it provides 2 things:
1. Gives feedback to the user when LLM is generating
2. Streams stuff much more efficiently to client (given it may take 100ms or so per call to update chat)
This PR is a retry of: #1135, where we migrate AiTools form to FormKit. The previous PR accidentally removed code related to setting enum values, and as a result was reverted. This update includes enums correctly along with the previous updates.
Overview
This PR introduces a Bot Homepage that was first introduced at https://ask.discourse.org/.
Key Features:
Add a bot homepage: /discourse-ai/ai-bot/conversations
Display a sidebar with previous bot conversations
Infinite scroll for large counts
Sidebar still visible when navigation mode is header_dropdown
Sidebar visible on homepage and bot PM show view
Add New Question button to the bottom of sidebar on bot PM show view
Add persona picker to homepage
This update adds metrics for estimated spending in AI usage. To make use of it, admins must add cost details to the LLM config page (input, output, and cached input costs per 1M tokens). After doing so, the metrics will appear in the AI usage dashboard as the AI plugin is used.
### 🔍 Overview
This update performs some enhancements to the LLM configuration screen. In particular, it renames the UI for the number of tokens for the prompt to "Context window" since the naming can be confusing to the user. Additionally, it adds a new optional field called "Max output tokens".
Followup fe7e73a6a4 and
1300cc8a36
Reintroduces proofreader specs and unskips ones skipped when
streaming composer suggestions were added. We just need to make sure
the jobs run immediately in specs because the job is what sends the
MessageBus event to the UI.
Also adds a pageobject method to confirm a diff is shown
in the modal before pressing Confirm button.
* FEATURE: Update model names and specs
- not a bug, but made it explicit that tools and thinking are not a chat thing
- updated all models to latest in presets (Gemini and OpenAI)
* allow larger context windows
This core PR must be merged for this to work
https://github.com/discourse/discourse/pull/32280
We want to make sure we don't have regressions with this
functionality in the rich editor, which uses contendeditable
and other APIs than the old composer.
1. Add age of post to topic context (1 month ago, 1 year ago, etc)
2. Refactor code for simplicity
3. Fix handling of post context in DMs which was not using new handling of uploads
Previous to this fix we assumed the name field contained usernames
when in fact it was stored in the id field.
This fixes the context contruction and also adds some basic user
information to the context to assist responders in understanding
the cast of chars
In this feature update, we add the UI for the ability to easily configure persona backed AI-features. The feature will still be hidden until structured responses are complete.
This update fixes a few issues in the split topic suggester. It fixes an issue where not all the category suggestions were appearing in the client. It also fixes an issue where the `move-post` request fails when creating a new topic with only one tag suggestion.
Add API methods to AI tools for reading and updating personas, enabling
more flexible AI workflows. This allows custom tools to:
- Fetch persona information through discourse.getPersona()
- Update personas with modified settings via discourse.updatePersona()
- Also update using persona.update()
These APIs enable new use cases like "trainable" moderation bots, where
users with appropriate permissions can set and refine moderation rules
through direct chat interactions, without needing admin panel access.
Also adds a special API scope which allows people to lean on API
for similar actions
Additionally adds a rather powerful hidden feature can allow custom tools
to inject content into the context unconditionally it can be used for memory and similar features
Previously, allowing "everyone" to access gists meant anons would see them too.
With the move to Personas, we used "[]" to reflect that.
With discourse/discourse#32199 adding the "everyone" option to the personas-allowed
groups, we are switching back to the original behavior.
Leaving allowed groups empty should always mean nobody can use the feature.
We started used a callback as a buffer in FoldContent, so the Fake endpoint is attempting
to emulate delays in the streaming. However, we don't care about that in these specs.
* REFACTOR: Move personas into it's own module.
* WIP: Use personas for summarization
* Prioritize persona default LLM or fallback to newest one
* Simplify summarization strategy
* Keep ai_sumarization_model as a fallback
This feature update allows for continuing the conversation with Discobot Discoveries in an AI bot chat. After discoveries gives you a response to your search you can continue with the existing context.
This change moves all the personas code into its own module. We want to treat them as a building block features can built on top of, same as `Completions::Llm`.
The code to title a message was moved from `Bot` to `Playground`.
* DEV: refactor bot internals
This introduces a proper object for bot context, this makes
it simpler to improve context management as we go cause we
have a nice object to work with
Starts refactoring allowing for a single message to have
multiple uploads throughout
* transplant method to message builder
* chipping away at inline uploads
* image support is improved but not fully fixed yet
partially working in anthropic, still got quite a few dialects to go
* open ai and claude are now working
* Gemini is now working as well
* fix nova
* more dialects...
* fix ollama
* fix specs
* update artifact fixed
* more tests
* spam scanner
* pass more specs
* bunch of specs improved
* more bug fixes.
* all the rest of the tests are working
* improve tests coverage and ensure custom tools are aware of new context object
* tests are working, but we need more tests
* resolve merge conflict
* new preamble and expanded specs on ai tool
* remove concept of "standalone tools"
This is no longer needed, we can set custom raw, tool details are injected into tool calls
* wip: more dynamic availableForcedTools
* FIX: forced tools wasn't set correctly when tool has no options.
---------
Co-authored-by: Roman Rizzi <[email protected]>
The AI Tool spec that checks that the tool runner will not timeout on slow HTTP request is flaky. In this PR we attempt to resolve the flakiness by:
Ensuring stub_request runs before the request
Increasing the timeout for CI env
This PR adds support for disabling further tool calls by setting tool_choice to :none across all supported LLM providers:
- OpenAI: Uses "none" tool_choice parameter
- Anthropic: Uses {type: "none"} and adds a prefill message to prevent confusion
- Gemini: Sets function_calling_config mode to "NONE"
- AWS Bedrock: Doesn't natively support tool disabling, so adds a prefill message
We previously used to disable tool calls by simply removing tool definitions, but this would cause errors with some providers. This implementation uses the supported method appropriate for each provider while providing a fallback for Bedrock.
Co-authored-by: Natalie Tay <[email protected]>
* remove stray puts
* cleaner chain breaker for last tool call (works in thinking)
remove unused code
* improve test
---------
Co-authored-by: Natalie Tay <[email protected]>
* REFACTOR: Migrate Personas' form to FormKit
We re-arranged fields into sections so we can better differentiate which options are specific to the AI bot.
* few form-kit improvements
https://github.com/discourse/discourse/pull/31934
---------
Co-authored-by: Joffrey JAFFEUX <[email protected]>
Sérgio Saquetim
Mark VanLandingham
SamandAlan Guo Xiang Tan
Isaac Janzen
Joffrey JAFFEUX
Keegan George
Martin Brennan
Loïc Guitaut
Roman Rizzi
David Taylor
Natalie Tay
Régis Hanol