70 Commits

Author SHA1 Message Date
Oleksandr Kozachuk 1ec3d6fcda Make it possible to specify the AI in config command. 2023-09-12 16:37:50 +02:00
Oleksandr Kozachuk 544bf0bf06 Improve README.md 2023-09-12 16:34:39 +02:00
Oleksandr Kozachuk f96e82bdd7 Implement the config -l and config -m commands. 2023-09-12 16:34:17 +02:00
Oleksandr Kozachuk 2b62cb8c4b Remove the -*terminal_width() to save space on screen. 2023-09-12 13:48:28 +02:00
juk0de a895c1fc6a Merge pull request 'ChatMasterMind Application Refactor and Enhancement' (#8) from restructurings into main
Reviewed-on: #8
2023-09-12 07:36:04 +02:00
Oleksandr Kozachuk ddfcc71510 Merge branch 'restructurings.main' into restructurings 2023-09-11 13:28:56 +02:00
Oleksandr Kozachuk 17de0b9967 Remove old code. 2023-09-11 13:17:59 +02:00
juk0de 33023d29f9 configuration: made 'default' AI ID optional 2023-09-11 13:09:45 +02:00
juk0de 481f9ecf7c configuration: improved config file format 2023-09-11 13:09:45 +02:00
juk0de 22fa187e5f question_cmd: when no tags are specified, no tags are selected 2023-09-11 13:09:45 +02:00
juk0de b840ebd792 message: to_file() now uses intermediate temporary file 2023-09-11 13:09:45 +02:00
juk0de 66908f5fed message: fixed matching with empty tag sets 2023-09-11 13:09:45 +02:00
juk0de 2e08ccf606 openai: stores AI.ID instead of AI.name in message 2023-09-11 13:09:44 +02:00
juk0de 595ff8e294 question_cmd: added message filtering by tags 2023-09-11 13:09:44 +02:00
juk0de faac42d3c2 question_cmd: fixed '--ask' command 2023-09-11 13:09:44 +02:00
juk0de 864ab7aeb1 chat: added check for existing files when creating new filenames 2023-09-11 13:09:44 +02:00
juk0de cc76da2ab3 chat: added 'update_messages()' function and test 2023-09-11 13:09:44 +02:00
juk0de f99cd3ed41 question_cmd: fixed source code extraction and added a testcase 2023-09-11 13:09:44 +02:00
Oleksandr Kozachuk 6f3ea98425 Small fixes. 2023-09-11 13:09:44 +02:00
Oleksandr Kozachuk 54ece6efeb Port print arguments -q/-a/-S from main to restructuring. 2023-09-11 13:09:44 +02:00
Oleksandr Kozachuk 86eebc39ea Allow in question -s for just sourcing file and -S to source file with ``` encapsulation. 2023-09-11 13:09:44 +02:00
juk0de 3eca53998b question cmd: added tests 2023-09-11 13:09:44 +02:00
juk0de c4f7bcc94e question_cmd: fixes 2023-09-11 13:09:44 +02:00
juk0de c52713c833 configuration: added tests 2023-09-11 13:09:44 +02:00
juk0de ecb6994783 configuration et al: implemented new Config format 2023-09-11 13:09:44 +02:00
juk0de 61e710a4b1 cmm: splitted commands into separate modules (and more cleanup) 2023-09-11 13:09:41 +02:00
juk0de 21d39c6c66 cmm: removed all the old code and modules 2023-09-11 13:08:45 +02:00
juk0de 6a4cc7a65d setup: added 'ais' subfolder 2023-09-11 13:07:46 +02:00
juk0de d6bb5800b1 test_main: temporarily disabled all testcases 2023-09-11 13:07:46 +02:00
juk0de 034e4093f1 cmm: added 'question' command 2023-09-11 13:07:46 +02:00
juk0de 7d15452242 added new module 'ai_factory' 2023-09-11 13:07:46 +02:00
juk0de 823d3bf7dc added new module 'openai.py' 2023-09-11 13:07:46 +02:00
juk0de 4bd144c4d7 added new module 'ai.py' 2023-09-11 13:07:46 +02:00
juk0de e186afbef0 cmm: the 'print' command now uses 'Message.from_file()' 2023-09-11 13:07:43 +02:00
juk0de 5e4ec70072 cmm: tags completion now uses 'Message.tags_from_dir' (fixes tag completion for me) 2023-09-11 13:06:22 +02:00
juk0de 4c378dde85 cmm: the 'hist' command now uses the new 'ChatDB' 2023-09-11 13:05:33 +02:00
juk0de 8923a13352 cmm: the 'tags' command now uses the new 'ChatDB' 2023-09-11 13:04:08 +02:00
juk0de e1414835c8 chat: added functions for finding and deleting messages 2023-09-11 13:04:08 +02:00
juk0de abb7fdacb6 message / chat: output improvements 2023-09-11 13:04:08 +02:00
juk0de 2e2228bd60 chat: new possibilites for adding messages and better tests 2023-09-11 13:04:08 +02:00
juk0de 713b55482a message: added rename_tags() function and test 2023-09-11 13:04:08 +02:00
juk0de d35de86c67 message: fixed Answer header for TXT format 2023-09-11 13:04:08 +02:00
juk0de aba3eb783d message: improved robustness of Question and Answer content checks and tests 2023-09-11 13:04:08 +02:00
juk0de 8e63831701 chat: added clear_cache() function and test 2023-09-11 13:04:08 +02:00
juk0de c318b99671 chat: improved history printing 2023-09-11 13:04:08 +02:00
juk0de 48c8e951e1 chat: fixed handling of unsupported files in DB and chache dir 2023-09-11 13:04:08 +02:00
juk0de b22a4b07ed chat: added tags_frequency() function and test 2023-09-11 13:04:08 +02:00
juk0de 33565d351d configuration: added AIConfig class 2023-09-11 13:04:08 +02:00
juk0de 6737fa98c7 added tokens() function to Message and Chat 2023-09-11 13:04:08 +02:00
juk0de 815a21893c added tests for 'chat.py' 2023-09-11 13:04:08 +02:00
juk0de 64893949a4 added new module 'chat.py' 2023-09-11 13:04:08 +02:00
juk0de a093f9b867 tags: some clarification and new tests 2023-09-11 13:04:08 +02:00
juk0de dc3f3dc168 added 'message_in()' function and test 2023-09-11 13:04:08 +02:00
juk0de 74c39070d6 fixed Message.filter_tags 2023-09-11 13:04:08 +02:00
juk0de fde0ae4652 fixed test case file cleanup 2023-09-11 13:04:08 +02:00
juk0de 238dbbee60 fixed handling empty tags in TXT file 2023-09-11 13:04:08 +02:00
juk0de 17f7b2fb45 Added tags filtering (prefix and contained string) to TagLine and Message 2023-09-11 13:04:08 +02:00
juk0de 9c2598a4b8 tests: added testcases for Message.from/to_file() and others 2023-09-11 13:04:08 +02:00
juk0de acec5f1d55 tests: splitted 'test_main.py' into 3 modules 2023-09-11 13:04:08 +02:00
juk0de c0f50bace5 gitignore: added vim session file 2023-09-11 13:04:08 +02:00
juk0de 30ccec2462 tags: TagLine constructor now supports multiline taglines and multiple spaces 2023-09-11 13:04:08 +02:00
juk0de 09da312657 configuration: added 'as_dict()' as an instance function 2023-09-11 13:04:08 +02:00
juk0de 33567df15f added testcases for messages.py 2023-09-11 13:04:08 +02:00
juk0de 264979a60d added new module 'message.py' 2023-09-11 13:04:08 +02:00
juk0de 061e5f8682 tags.py: converted most TagLine functions to module functions 2023-09-11 13:04:08 +02:00
juk0de 2d456e68f1 added testcases for Tag and TagLine classes 2023-09-11 13:04:08 +02:00
juk0de 8bd659e888 added new module 'tags.py' with classes 'Tag' and 'TagLine' 2023-09-11 13:04:08 +02:00
Oleksandr Kozachuk 3ef1339cc0 Fix extracting source file with type specification. 2023-09-09 11:53:32 +02:00
Oleksandr Kozachuk ed567afbea Make it possible to print just question or answer on printing files. 2023-09-08 15:54:29 +02:00
Oleksandr Kozachuk 6e447018d5 Fix tags_completter. 2023-09-07 18:11:32 +02:00
9 changed files with 102 additions and 61 deletions
+74 -42
View File
@@ -37,63 +37,95 @@ cmm [global options] command [command options]
### Global Options
- `-c`, `--config`: Config file name (defaults to `.config.yaml`).
### Commands
- `ask`: Ask a question.
- `hist`: Print chat history.
- `tag`: Manage tags.
- `config`: Manage configuration.
- `print`: Print files.
- `-C`, `--config`: Config file name (defaults to `.config.yaml`).
### Command Options
#### `ask` Command Options
#### Question
- `-q`, `--question`: Question to ask (required).
- `-m`, `--max-tokens`: Max tokens to use.
- `-T`, `--temperature`: Temperature to use.
- `-M`, `--model`: Model to use.
- `-n`, `--number`: Number of answers to produce (default is 3).
- `-s`, `--source`: Add content of a file to the query.
- `-S`, `--only-source-code`: Add pure source code to the chat history.
- `-t`, `--tags`: List of tag names.
- `-e`, `--extags`: List of tag names to exclude.
- `-o`, `--output-tags`: List of output tag names (default is the input tags).
- `-a`, `--match-all-tags`: All given tags must match when selecting chat history entries.
The `question` command is used to ask, create, and process questions.
#### `hist` Command Options
```bash
cmm question [-t OTAGS]... [-k ATAGS]... [-x XTAGS]... [-o OUTTAGS]... [-A AI] [-M MODEL] [-n NUM] [-m MAX] [-T TEMP] (-a ASK | -c CREATE | -r REPEAT | -p PROCESS) [-O] [-s SOURCE]... [-S SOURCE]...
```
- `-d`, `--dump`: Print chat history as Python structure.
- `-w`, `--with-tags`: Print chat history with tags.
- `-W`, `--with-files`: Print chat history with filenames.
- `-S`, `--only-source-code`: Print only source code.
- `-t`, `--tags`: List of tag names.
- `-e`, `--extags`: List of tag names to exclude.
- `-a`, `--match-all-tags`: All given tags must match when selecting chat history entries.
* `-t, --or-tags OTAGS` : List of tags (one must match)
* `-k, --and-tags ATAGS` : List of tags (all must match)
* `-x, --exclude-tags XTAGS` : List of tags to exclude
* `-o, --output-tags OUTTAGS` : List of output tags (default: use input tags)
* `-A, --AI AI` : AI ID to use
* `-M, --model MODEL` : Model to use
* `-n, --num-answers NUM` : Number of answers to request
* `-m, --max-tokens MAX` : Max. number of tokens
* `-T, --temperature TEMP` : Temperature value
* `-a, --ask ASK` : Ask a question
* `-c, --create CREATE` : Create a question
* `-r, --repeat REPEAT` : Repeat a question
* `-p, --process PROCESS` : Process existing questions
* `-O, --overwrite` : Overwrite existing messages when repeating them
* `-s, --source-text SOURCE` : Add content of a file to the query
* `-S, --source-code SOURCE` : Add source code file content to the chat history
#### `tag` Command Options
#### Hist
- `-l`, `--list`: List all tags and their frequency.
The `hist` command is used to print the chat history.
#### `config` Command Options
```bash
cmm hist [-t OTAGS]... [-k ATAGS]... [-x XTAGS]... [-w] [-W] [-S] [-A ANSWER] [-Q QUESTION]
```
- `-l`, `--list-models`: List all available models.
- `-m`, `--print-model`: Print the currently configured model.
- `-M`, `--model`: Set model in the config file.
* `-t, --or-tags OTAGS` : List of tags (one must match)
* `-k, --and-tags ATAGS` : List of tags (all must match)
* `-x, --exclude-tags XTAGS` : List of tags to exclude
* `-w, --with-tags` : Print chat history with tags
* `-W, --with-files` : Print chat history with filenames
* `-S, --source-code-only` : Print only source code
* `-A, --answer ANSWER` : Search for answer substring
* `-Q, --question QUESTION` : Search for question substring
#### `print` Command Options
#### Tags
- `-f`, `--file`: File to print (required).
- `-S`, `--only-source-code`: Print only source code.
The `tags` command is used to manage tags.
```bash
cmm tags (-l | -p PREFIX | -c CONTENT)
```
* `-l, --list` : List all tags and their frequency
* `-p, --prefix PREFIX` : Filter tags by prefix
* `-c, --contain CONTENT` : Filter tags by contained substring
#### Config
The `config` command is used to manage the configuration.
```bash
cmm config (-l | -m | -c CREATE)
```
* `-l, --list-models` : List all available models
* `-m, --print-model` : Print the currently configured model
* `-c, --create CREATE` : Create config with default settings in the given file
#### Print
The `print` command is used to print message files.
```bash
cmm print -f FILE [-q | -a | -S]
```
* `-f, --file FILE` : File to print
* `-q, --question` : Print only question
* `-a, --answer` : Print only answer
* `-S, --only-source-code` : Print only source code
### Examples
1. Ask a question:
```bash
cmm ask -q "What is the meaning of life?" -t philosophy -e religion
cmm question -a "What is the meaning of life?" -t philosophy -x religion
```
2. Display the chat history:
@@ -105,19 +137,19 @@ cmm hist
3. Filter chat history by tags:
```bash
cmm hist -t tag1 tag2
cmm hist --or-tags tag1 tag2
```
4. Exclude chat history by tags:
```bash
cmm hist -e tag3 tag4
cmm hist --exclude-tags tag3 tag4
```
5. List all tags and their frequency:
```bash
cmm tag -l
cmm tags -l
```
6. Print the contents of a file:
+6
View File
@@ -59,6 +59,12 @@ class AI(Protocol):
"""
raise NotImplementedError
def print_models(self) -> None:
"""
Print all models supported by this AI.
"""
raise NotImplementedError
def tokens(self, data: Union[Message, Chat]) -> int:
"""
Computes the nr. of AI language tokens for the given message
+4 -4
View File
@@ -17,7 +17,7 @@ def create_ai(args: argparse.Namespace, config: Config) -> AI: # noqa: 11
is not found, it uses the first AI in the list.
"""
ai_conf: AIConfig
if args.AI:
if 'AI' in args and args.AI:
try:
ai_conf = config.ais[args.AI]
except KeyError:
@@ -32,11 +32,11 @@ def create_ai(args: argparse.Namespace, config: Config) -> AI: # noqa: 11
if ai_conf.name == 'openai':
ai = OpenAI(cast(OpenAIConfig, ai_conf))
if args.model:
if 'model' in args and args.model:
ai.config.model = args.model
if args.max_tokens:
if 'max_tokens' in args and args.max_tokens:
ai.config.max_tokens = args.max_tokens
if args.temperature:
if 'temperature' in args and args.temperature:
ai.config.temperature = args.temperature
return ai
else:
+6 -1
View File
@@ -62,7 +62,12 @@ class OpenAI(AI):
"""
Return all models supported by this AI.
"""
raise NotImplementedError
ret = []
for engine in sorted(openai.Engine.list()['data'], key=lambda x: x['id']):
if engine['ready']:
ret.append(engine['id'])
ret.sort()
return ret
def print_models(self) -> None:
"""
-1
View File
@@ -204,7 +204,6 @@ class Chat:
output.append(message.to_str(source_code_only=True))
continue
output.append(message.to_str(with_tags, with_files))
output.append('\n' + ('-' * terminal_width()) + '\n')
if paged:
print_paged('\n'.join(output))
else:
+9
View File
@@ -1,6 +1,8 @@
import argparse
from pathlib import Path
from ..configuration import Config
from ..ai import AI
from ..ai_factory import create_ai
def config_cmd(args: argparse.Namespace) -> None:
@@ -9,3 +11,10 @@ def config_cmd(args: argparse.Namespace) -> None:
"""
if args.create:
Config.create_default(Path(args.create))
elif args.list_models or args.print_model:
config: Config = Config.from_file(args.config)
ai: AI = create_ai(args, config)
if args.list_models:
ai.print_models()
else:
print(ai.config.model)
+1
View File
@@ -39,6 +39,7 @@ class AIConfig:
name: ClassVar[str]
# a user-defined ID for an AI configuration entry
ID: str
model: str = 'n/a'
# the name must not be changed
def __setattr__(self, name: str, value: Any) -> None:
+1
View File
@@ -100,6 +100,7 @@ def create_parser() -> argparse.ArgumentParser:
help="Manage configuration",
aliases=['c'])
config_cmd_parser.set_defaults(func=config_cmd)
config_cmd_parser.add_argument('-A', '--AI', help='AI ID to use')
config_group = config_cmd_parser.add_mutually_exclusive_group(required=True)
config_group.add_argument('-l', '--list-models', help="List all available models",
action='store_true')
+1 -13
View File
@@ -6,7 +6,7 @@ from io import StringIO
from unittest.mock import patch
from chatmastermind.tags import TagLine
from chatmastermind.message import Message, Question, Answer, Tag, MessageFilter
from chatmastermind.chat import Chat, ChatDB, terminal_width, ChatError
from chatmastermind.chat import Chat, ChatDB, ChatError
class TestChat(unittest.TestCase):
@@ -92,16 +92,10 @@ class TestChat(unittest.TestCase):
Question 1
{Answer.txt_header}
Answer 1
{'-'*terminal_width()}
{Question.txt_header}
Question 2
{Answer.txt_header}
Answer 2
{'-'*terminal_width()}
"""
self.assertEqual(mock_stdout.getvalue(), expected_output)
@@ -115,18 +109,12 @@ FILE: 0001.txt
Question 1
{Answer.txt_header}
Answer 1
{'-'*terminal_width()}
{TagLine.prefix} btag2
FILE: 0002.txt
{Question.txt_header}
Question 2
{Answer.txt_header}
Answer 2
{'-'*terminal_width()}
"""
self.assertEqual(mock_stdout.getvalue(), expected_output)