FEATURE: Share conversations with AI via a URL (#521)

This allows users to share a static page of an AI conversation with
the rest of the world.

By default this feature is disabled, it is enabled by turning on
ai_bot_allow_public_sharing via site settings

Precautions are taken when sharing

1. We make a carbonite copy
2. We minimize work generating page
3. We limit to 100 interactions
4. Many security checks - including disallowing if there is a mix
of users in the PM.

* Bonus commit, large PRs like this PR did not work with github tool
large objects would destroy context


Co-authored-by: Martin Brennan <[email protected]>

lib/ai_bot/tools/github_pull_request_diff.rb

@@ -4,6 +4,8 @@ module DiscourseAi
   module AiBot
     module Tools
       class GithubPullRequestDiff < Tool
+        LARGE_OBJECT_THRESHOLD = 30_000
+
         def self.signature
           {
             name: name,
@@ -56,6 +58,7 @@ module DiscourseAi
 
           if response.code == "200"
             diff = response.body
+            diff = sort_and_shorten_diff(diff)
             diff = truncate(diff, max_length: 20_000, percent_length: 0.3, llm: llm)
             { diff: diff }
           else
@@ -66,6 +69,46 @@ module DiscourseAi
         def description_args
           { repo: repo, pull_id: pull_id, url: url }
         end
+
+        private
+
+        def sort_and_shorten_diff(diff, threshold: LARGE_OBJECT_THRESHOLD)
+          # This regex matches the start of a new file in the diff,
+          # capturing the file paths for later use.
+          file_start_regex = /^diff --git.*/
+
+          prev_start = 0
+          prev_match = nil
+
+          split = []
+
+          diff.scan(file_start_regex) do |match|
+            match_start = $~.offset(0)[0] # Get the start position of this match
+
+            if prev_start != 0
+              full_diff = diff[prev_start...match_start]
+
+              split << [prev_match, full_diff]
+            end
+
+            prev_match = match
+            prev_start = match_start
+          end
+
+          split << [prev_match, diff[prev_start..-1]] if prev_match
+
+          split.sort! { |x, y| x[1].length <=> y[1].length }
+
+          split
+            .map do |x, y|
+              if y.length < threshold
+                y
+              else
+                "#{x}\nRedacted, Larger than #{threshold} chars"
+              end
+            end
+            .join("\n")
+        end
       end
     end
   end