Dynamic Answer class and OpenAI streaming API #19

2023-10-21T14:24:28+02:00

ok commented

2023-10-21 14:24:28 +02:00

This pull request introduces several changes with the main objective of enabling OpenAI's streaming API in the chatmastermind application. This allows for the retrieval of AI responses gradually as a stream, which can significantly improve the user experience in interactions that involve large result sets.

In particular, the main changes are as follows:

Added tiktoken import in openai.py and modifications to the OpenAI class to support streaming. This includes the addition of a new class OpenAIAnswer to handle streaming API responses.
Modified request function in the OpenAI class: the stream=True flag is added to the openai.ChatCompletion.create method to enable streaming API.
Modified question.py to print the answer parts as they are streamed.
Replaced the Answer class's string data type with a generator which supports str and Generator[str, None, None] data types. Modifications are made to the Answer class methods to handle both data types accordingly.
Updated the tests in test_ais_openai.py and test_message.py to reflect and validate these changes.

These changes will provide a more robust interaction with the OpenAI API, especially for larger responses.

This pull request introduces several changes with the main objective of enabling OpenAI's streaming API in the `chatmastermind` application. This allows for the retrieval of AI responses gradually as a stream, which can significantly improve the user experience in interactions that involve large result sets. In particular, the main changes are as follows: 1. Added `tiktoken` import in `openai.py` and modifications to the `OpenAI` class to support streaming. This includes the addition of a new class `OpenAIAnswer` to handle streaming API responses. 2. Modified `request` function in the `OpenAI` class: the `stream=True` flag is added to the `openai.ChatCompletion.create` method to enable streaming API. 3. Modified `question.py` to print the answer parts as they are streamed. 4. Replaced the `Answer` class's string data type with a generator which supports `str` and `Generator[str, None, None]` data types. Modifications are made to the `Answer` class methods to handle both data types accordingly. 5. Updated the tests in `test_ais_openai.py` and `test_message.py` to reflect and validate these changes. These changes will provide a more robust interaction with the OpenAI API, especially for larger responses.

ok added 3 commits 2023-10-21 14:24:29 +02:00

Refactor message.Answer class in a way, that it can be constructed dynamically step by step, in preparation of using streaming API. 2aee018708

Fix source_code function with the dynamic answer class. bbc1ab5a0a

Activate and use OpenAI streaming API. dbe72ff11c

juk0de merged commit 7c1c67f8ff into main

2023-10-21 15:50:46 +02:00

juk0de referenced this issue from a commit

2023-10-21 15:50:46 +02:00

Merge pull request 'Dynamic Answer class and OpenAI streaming API' (#19) from dynamic_answer into main

Sign in to join this conversation.