Sam e76fc77189 fixes (#53) ...

* Minor... use username suggester in case username already exists

* FIX: ensure we truncate long prompts

Previously we

1. Used raw length instead of token counts for counting length
2. We totally dropped a prompt if it was too long

New implementation will truncate "raw" if it gets too long maintaining
meaning.

2023-05-06 07:31:53 -03:00

..

tokenizer.rb

fixes (#53)

2023-05-06 07:31:53 -03:00