Problem with getting a response from the API in Russian

API
, ,
1

** Hello everyone!**
In my Laravel application, I ran into a strange problem.
I get a response from the api, but everything is in spaces in Russian, and everything is OK in English. Please tell me how to fix it? Below is a screenshot of the problem :heart:

Ju095
Ju0951280×339 34.9 KB

(chatcontroller.php):

<?php

namespace App\Http\Controllers;

use App\Http\Requests\MessageRequest;
use Illuminate\Support\Facades\Http;
use Illuminate\Http\Request;
use App\Models\Message;
use OpenAI\Laravel\Facades\OpenAI;
use Illuminate\Support\Facades\Auth;

class ChatController extends Controller
{
    private $openAIRequest;

    public function __construct()
    {
        $this->openAIRequest = Http::withoutVerifying()
            ->withOptions(['timeout' => 120])
            ->withHeaders([
                'Authorization' => 'Bearer ' . config('openai.api_key'),
                'Content-Type' => 'application/json',
            ]);
    }

    public function messages(Request $request, MessageRequest $messageRequest)
    {
        
        $prompt = $messageRequest->input('prompt');
        $proglang = $messageRequest->input('proglang');
        $framework = $messageRequest->input('framework');
        $size = $messageRequest->input('size');
        $lang = $messageRequest->input('lang');
        $message = 'Задокументируйте код на языке программирования: ' . $proglang . ' с использованием фреймворка: ' . $framework .
            ' Документация должна быть в формате: ' . $size . ' Объект - предназначение' . ' и написана на ' . $lang . ' языке.' .
            ' Код, который необходимо документировать: ' . $prompt . ' Если в поле для кода будет короткий код, то мы его всё равно описываем, а если код отсутствует, то сообщи об этом. Текже не повторяй в своем ответе мой запрос, сконкретизируся на документации';

       Message::create([
            'user_id' => Auth::user()->id,
            'prompt' => $prompt,
            'proglang' => $proglang,
            'framework' => $framework,
            'size' => $size,
            'lang' => $lang,
        ]);

        return $this->askToChatGPT($message);
    }

    private function askToChatGPT($message)
    {
        $question = $message;
        return response()->stream(function () use ($question) {
            $stream = OpenAI::completions()->createStreamed([
                'model' => 'gpt-3.5-turbo-instruct',
                'prompt' => $question,
                'max_tokens' => 2000,
                'temperature' => 0.3,
            ]);
    
            foreach ($stream as $response) {
                $text = trim($response->choices[0]->text);
                if (connection_aborted()) {
                    break;
                }
    
                echo "\n";
                echo $text;
                echo "\n\n";
                ob_flush();
                flush();
            }
        }, 200, [
            'Cache-Control' => 'no-cache',
            'X-Accel-Buffering' => 'no',
            'Content-Type' => 'text/event-stream; charset=utf-8',
        ]);
    }    

}

As well as the script of the page with the API:

   <script>
        $(document).on("click", "#submit-btn", function(e) {
            e.preventDefault();

            var form = $("#ask-chatgpt-form");
            var formData = form.serialize();
            var textarea = form.find('textarea[name="prompt"]');

            var selectedproglang = $('select[name="proglang"]').val();
            formData += '&proglang=' + selectedproglang;


            var selectedframework = $('select[name="framework"]').val();
            formData += '&framework=' + selectedframework;

            var selectedsize = $('select[name="size"]').val();
            formData += '&size=' + selectedsize;

            var selectedlang = $('select[name="lang"]').val();
            formData += '&lang=' + selectedlang;

            generateChatBoxMessage(false, textarea.val());
            textarea.val('');
            function generateChatBoxMessage(isChatGPTMessage, messageText) {
            var template = isChatGPTMessage ? $("#chatgpt-message-box-template").html() : $("#user-message-box-template").html();
            var messageBox = $(template);
            messageBox.find('#message-box-text').text(messageText);
            $('#chat-box').append(messageBox);
            }
            $.ajax({
                type: "GET",
                url: 'chatgpt/ask',
                data: formData,
                success: function(answer) {
                    generateChatBoxMessage(true, answer);
                },
            });
        });
        const source = new EventSource('chatgpt/ask' + encodeURIComponent(input));
        source.addEventListener('update', function(event) {
            if (event.data === "<END_STREAMING_SSE>") {
                source.close();
                return;
            }
            result.innerText += event.data
        });
        // Обновляем скрытое поле при выборе значения в с-електоре
        $('select').change(function() {
            var selectedValue = $(this).val();
            $(this).siblings('input[type="hidden"]').val(selectedValue);
        });     
    </script>

P.s. If you don’t have enough information or code, then write in the comments and I will definitely supplement the question!

2

How consistent is the strange spaces?

If the AI happens to make a “mistake” and output a space, it can quickly train itself that the rest needs random space character tokens also.

While the temperature has been lowered, it doesn’t prevent improbable tokens from being output. You might also include 'top_p' => 0.3, as an API sampling parameter. That gives token choices only from the top 30% probability mass.

Then you’d have to look at what could be triggering this in input language if it persists. There should be no extra spaces or carriage returns in your messages; user inputs should be stripped of leading and trailing whitespace.