Commit Graph
1050 Commits
Author SHA1 Message Date
Kris 956efba8cb UX: set usage as first AI admin tab (#1089) 2025-01-24 10:14:52 +11:00
Rafael dos Santos Silva 67a1257b89 FEATURE: Gemini Tokenizer (#1088) 2025-01-23 18:20:35 -03:00
Roman Rizzi 5a97752117 FIX: Always raise the single exception/Open AI models migration (#1087) 2025-01-23 15:30:06 -03:00
Penar Musaraj d5cf53e8e0 UX: Fix composer helper z-index (#1086)
Followup to https://github.com/discourse/discourse-ai/pull/1064

That commits adds a higher z-index due to core changes, we no longer need
an iPad-specific z-index.
2025-01-23 10:07:27 -05:00
Sam 8bf350206e FEATURE: track duration of AI calls (#1082)
* FEATURE: track duration of AI calls

* annotate
2025-01-23 11:32:12 +11:00
Roman Rizzi faa8e6e873 FIX: Embeddings backfill rake task was using old code (#1084) 2025-01-22 14:00:26 -03:00
Roman Rizzi e2e753d73c FEATURE: Formalize support for matryoshka dimensions. (#1083)
We have a flag to signal we are shortening the embeddings of a model.
Only used in Open AI's text-embedding-3-*, but we plan to use it for other services.
2025-01-22 11:26:46 -03:00
我秦始皇 654f90f1cd FIX: convert provider_params hash to json before db insert (#1081)
* FIX: convert provider_params hash to json before db insert

* FIX: lint issues in config migration

* FIX: simplify provider_params json conversion
2025-01-22 09:55:41 -03:00
Roman Rizzi a5e5ae72a8 FIX: Open AI embedding shortening is only available for some models (#1080) 2025-01-21 17:50:40 -03:00
Roman Rizzi 3b66fb3e87 FIX: Restore the accidentally deleted query prefix. (#1079)
Additionally, we add a prefix for embedding generation.
Both are stored in the definitions table.
2025-01-21 14:10:31 -03:00
Roman Rizzi f5cf1019fb FEATURE: configurable embeddings (#1049)
* Use AR model for embeddings features

* endpoints

* Embeddings CRUD UI

* Add presets. Hide a couple more settings

* system specs

* Seed embedding definition from old settings

* Generate search bit index on the fly. cleanup orphaned data

* support for seeded models

* Fix run test for new embedding

* fix selected model not set correctly
2025-01-21 12:23:19 -03:00
Discourse Translator Bot fad4b65d4f Update translations (#1078) 2025-01-21 15:55:40 +01:00
Sam 2c609e165b FEATURE: Add user location info to spam scanner context (#1076)
This adds registration and last known IP information and email to scanning context.

This provides another hint for spam scanner about possible malicious users.

For example registered in India, replying from Australia or
email is clearly a throwaway email address.
2025-01-21 17:51:21 +11:00
Kelv 7957796e56 DEV: update all suffix alt icon names (#1075) 2025-01-20 17:33:25 +08:00
David Taylor 890b85bff3 DEV: Update icons for FA6 (#1074) 2025-01-17 10:12:57 +00:00
Roman Rizzi 4784e7fe43 FIX: Set default for existing records. (#1073)
We'll later copy the correct value from content_range. 1 should be the min highest post number a topic has.
2025-01-16 10:38:53 -03:00
Roman Rizzi 46fcdb6ba5 FIX: Make summaries backfill job more resilient. (#1071)
To quickly select backfill candidates without comparing SHAs, we compare the last summarized post to the topic's highest_post_number. However, hiding or deleting a post and adding a small action will update this column, causing the job to stall and re-generate the same summary repeatedly until someone posts a regular reply. On top of this, this is not always true for topics with `best_replies`, as this last reply isn't necessarily included.

Since this is not evident at first glance and each summarization strategy picks its targets differently, I'm opting to simplify the backfill logic and how we track potential candidates.

The first step is dropping `content_range`, which serves no purpose and it's there because summary caching was supposed to work differently at the beginning. So instead, I'm replacing it with a column called `highest_target_number`, which tracks `highest_post_number` for topics and could track other things like channel's `message_count` in the future.

Now that we have this column when selecting every potential backfill candidate, we'll check if the summary is truly outdated by comparing the SHAs, and if it's not, we just update the column and move on
2025-01-16 09:42:53 -03:00
Rafael dos Santos Silva f9aa2de413 FIX: AWS Bedrock non-streaming calls response log (#1072) 2025-01-15 18:51:25 -03:00
Sam 81b952d56e FIX: only hide posts detected explicitly as spam (#1070)
When enabling spam scanner it there may be old unscanned posts
this can create a risky situation where spam scanner operates
on legit posts during false positives

To keep this a lot safer we no longer try to hide old stuff by
the spammers.
2025-01-15 16:50:41 +11:00
Natalie Tay c881f8b361 DEV: Add rake task to send topics or posts to spam scanner (#1059) 2025-01-15 11:48:57 +08:00
Rafael dos Santos Silva 92f122c54d SECURITY: Fix XSS on Shared AI Conversations local Onebox (#1069) 2025-01-14 18:05:37 -03:00
Roman Rizzi cd03874b4d FIX: Missing table check in post_migration (#1068) 2025-01-14 17:33:01 -03:00
Roman Rizzi 65456c8b30 DEV: Migration to remove old embeddings tables~ (#1067)
* DEV: Migration to remove old embeddings tables~

* Check for table existence
2025-01-14 17:13:34 -03:00
Roman Rizzi c4d2b7de1d PERF: Optimize backfill query to prevent statement timeouts (#1066) 2025-01-14 15:39:19 -03:00
Roman Rizzi 6721c6751d FIX: Do batches for backfilling huge embeddings tables (#1065) 2025-01-14 14:42:40 -03:00
Keegan George bbae790c2b FIX: Composer helper not appearing on tablets (#1064)
This update fixes an issue when the composer helper menu was not being shown on tablets in desktop mode. Updating the `z-index` to use the modal-dialog case is more appropriate here.
2025-01-14 09:35:31 -08:00
Roman Rizzi 356ea77201 FIX: Split backfill into separate migrations to use independent transactions (#1063) 2025-01-14 13:30:52 -03:00
Roman Rizzi 09ca123757 FIX: Split statements to avoid timeout (#1062) 2025-01-14 12:54:18 -03:00
Discourse Translator Bot c029bc8979 Update translations (#1060) 2025-01-14 16:20:00 +01:00
Roman Rizzi 65bbcd71fc DEV: Embedding tables' model_id has to be a bigint (#1058)
* DEV: Embedding tables' model_id has to be a bigint

* Drop old search_bit indexes

* copy rag fragment embeddings created during deploy window
2025-01-14 10:53:06 -03:00