Commit Graph

77 Commits

Author SHA1 Message Date
Reinier van der Leer
4e761b49f3 Clean up logging 2023-08-22 07:29:56 +02:00
Reinier van der Leer
2c53530e99 Fix path processing (#5032)
* Fix and clean up path processing in logs module

* Fix path processing throughout the project

* Fix plugins test

* Fix borky pytest vs mkdir(exist_ok=True)

* Update docs and gitignore for new workspace location

* Fix borky pytest vol.2

* ok james
2023-07-21 11:36:15 -07:00
James Collins
98c3f6b781 Bugfix/remove breakpoint from embedding function (#5022)
* Add links to github issues in the README and clarify run instructions

* Remove breakpoint from embedding function
2023-07-20 09:24:14 -07:00
Reinier van der Leer
db95d4cb84 Agent loop v2: Planning & Task Management (part 1: refactoring) (#4799)
* Move rename module `agent` -> `agents`

* WIP: abstract agent structure into base class and port Agent

* Move command arg path sanitization to decorator

* Add fallback token limit in llm.utils.create_chat_completion

* Rebase `MessageHistory` class on `ChatSequence` class

* Fix linting

* Consolidate logging modules

* Wham Bam Boom

* Fix tests & linting complaints

* Update Agent class docstring

* Fix Agent import in autogpt.llm.providers.openai

* Fix agent kwarg in test_execute_code.py

* Fix benchmarks.py

* Clean up lingering Agent(ai_name=...) initializations

* Fix agent kwarg

* Make sanitize_path_arg decorator more robust

* Fix linting

* Fix command enabling lambda's

* Use relative paths in file ops logger

* Fix test_execute_python_file_not_found

* Fix Config model validation breaking on .plugins

* Define validator for Config.plugins

* Fix Config model issues

* Fix agent iteration budget in testing

* Fix declaration of context_while_think

* Fix Agent.parse_and_process_response signature

* Fix Agent cycle_budget usages

* Fix budget checking in BaseAgent.__next__

* Fix cycle budget initialization

* Fix function calling in BaseAgent.think()

* Include functions in token length calculation

* Fix Config errors

* Add debug thing to patched_api_requestor to investigate HTTP 400 errors

* If this works I'm gonna be sad

* Fix BaseAgent cycle budget logic and document attributes

* Document attributes on `Agent`

* Fix import issues between Agent and MessageHistory

* Improve typing

* Extract application code from the agent (#4982)

* Extract application code from the agent

* Wrap interaction loop in a function and call in benchmarks

* Forgot the important function call

* Add docstrings and inline comments to run loop

* Update typing and docstrings in agent

* Docstring formatting

* Separate prompt construction from on_before_think

* Use `self.default_cycle_instruction` in `Agent.think()`

* Fix formatting

* hot fix the SIGINT handler (#4997)

The signal handler in the autogpt/main.py doesn't work properly because
of the clean_input(...) func. This commit remedies this issue. The issue
is mentioned in
3966cdfd69 (r1264278776)

* Update the sigint handler to be smart enough to actually work (#4999)

* Update the sigint handler to be smart enough to actually work

* Update autogpt/main.py

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

* Can still use context manager

* Merge in upstream

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

* Fix CI

* Fix initial prompt construction

* off by one error

* allow exit/EXIT to shut down app

* Remove dead code

---------

Co-authored-by: collijk <collijk@uw.edu>
Co-authored-by: Cyrus <39694513+cyrus-hawk@users.noreply.github.com>
2023-07-20 17:34:49 +02:00
Lei Zhang
5ae044f53d Integrate plugin.handle_text_embedding hook (#2804)
* add feature custom text embedding in plugin

* black code format

* _get_embedding_with_plugin()

* Fix docstring & type hint

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-07-15 03:10:32 +02:00
ido777
c821b294c6 Fix orjson encoding text with UTF-8 surrogates (#3666)
* added lib ftfy (fixes text for you), to solve surrogates errors

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-07-15 00:23:59 +02:00
James Collins
c9adedf746 Refactor/rename agent subpackage to agents (#4961)
* Add links to github issues in the README and clarify run instructions

* Rename agent subpackage to agents

* Revert all unwanted changes

* Use relative import in `agents/__init__.py`

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-07-13 07:31:49 -07:00
Reinier van der Leer
e746a45158 Merge branch 'master' into release-v0.4.4 2023-07-11 23:51:48 +02:00
James Collins
9adcad8b8a Fix regression: restore api_base and organization configurability (#4933) 2023-07-10 04:32:04 +02:00
Reinier van der Leer
e8b6676b22 Restructure logs.py into a module; include log_cycle (#4921)
* Consolidate all logging stuff into one module

* Merge import statement for `logs` and `logs.log_cycle`

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-07-09 20:14:25 +02:00
Reinier van der Leer
1e1eff70bc Rebase MessageHistory on ChatSequence (#4922)
* Rebase `MessageHistory` on `ChatSequence`

* Process feedback & make mypy happy

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-07-09 19:52:59 +02:00
James Collins
f6da992b3e Drop AbstractSingleton as a parent class of the vector memory (#4901)
* Drop AbstractSingleton as a parent class of the vector memory

* Forgot pytest fixture in integration tests

* Revert typo
2023-07-08 09:27:43 -07:00
sagarishere
9a2a9f7439 Add docstring to function get_memory() in autogpt.memory.vector (#1296)
* Document function get_memory in ./scripts/memory/init.py

* Update get_memory docstring to current format

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-07-07 07:39:36 +02:00
Venkat Teja
0f538f6e2c Fix summarize_text usages and self-calls (#4863)
* Fix summarize_text function usage

* Update memory_item.py
2023-07-07 06:56:29 +02:00
Reinier van der Leer
bde007e6f7 Use GPT-4 in Agent loop by default (#4899)
* Use GPT-4 as default smart LLM in Agent

* Rename (smart|fast)_llm_model to (smart|fast)_llm everywhere

* Fix test_config.py::test_initial_values

* Fix test_config.py::test_azure_config

* Fix Azure config backwards compatibility
2023-07-07 03:42:18 +02:00
Jayden
ac17518663 Fix Azure OpenAI setup problems (#4875)
* [Fix] Recover the azure config load function

* [Style] Apply black, isort, mypy, autoflake

* [Fix] Rename the return parameter from 'azure_model_map' to 'azure_model_to_deployment_id_map'

* [Feat] Change the azure config file path to be dynamically configurable

* [Test] Add azure_config and azure deployment_id_for_model

* [Style] Apply black, isort, mypy, autoflake

* [Style] Apply black, isort, mypy, autoflake

* Refactor Azure configuration

- Refactor the `azure_config_file` attribute in the `Config` class to be optional.
- Refactor the `azure_model_to_deployment_id_map` attribute in the `Config` class to be optional and provide default values.
- Update the `get_azure_deployment_id_for_model` function to accept additional parameters.
- Update references to `get_azure_deployment_id_for_model` in `create_text_completion`, `create_chat_completion`, and `get_embedding` functions to pass the required parameters.

* Clean up process for azure

* Docstring

* revert some unneccessary fiddling

* Avoid altering args to models

* Retry on 404s

* Don't permanently change the environment

* Formatting

---------

Co-authored-by: Luke <2609441+lc0rp@users.noreply.github.com>
Co-authored-by: lc0rp <2609411+lc0rp@users.noreply.github.com>
Co-authored-by: collijk <collijk@uw.edu>
2023-07-06 17:51:59 -07:00
Erik Peterson
857d26d101 Add OpenAI function call support (#4683)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-22 04:52:44 +02:00
merwanehamadi
a7f805604c Pass config everywhere in order to get rid of singleton (#4666)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-18 19:05:41 -07:00
merwanehamadi
0b6fec4a28 Fix summarization happening in first cycle (#4719) 2023-06-16 18:17:47 -07:00
Reinier van der Leer
f0a5250da5 Implement loading MemoryItems from file in JSONFileMemory (#4703)
Further changes:
* remove `init` param from `get_memory()`, replace usages by `memory.clear()`
* make token length calculation optional in `MemoryItem.dump()`
2023-06-15 17:45:14 +02:00
James Collins
6e6e7fcc9a Extract openai API calls and retry at lowest level (#3696)
* Extract open ai api calls and retry at lowest level

* Forgot a test

* Gotta fix my local docker config so I can let pre-commit hooks run, ugh

* fix: merge artiface

* Fix linting

* Update memory.vector.utils

* feat: make sure resp exists

* fix: raise error message if created

* feat: rename file

* fix: partial test fix

* fix: update comments

* fix: linting

* fix: remove broken test

* fix: require a model to exist

* fix: BaseError issue

* fix: runtime error

* Fix mock response in test_make_agent

* add 429 as errors to retry

---------

Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-14 07:59:26 -07:00
Erik Peterson
07d9b584f7 Correct and clean up JSON handling (#4655)
* Correct and clean up JSON handling

* Use ast for message history too

* Lint

* Add comments explaining why we use literal_eval

* Add descriptions to llm_response_format schema

* Parse responses in code blocks

* Be more careful when parsing in code blocks

* Lint
2023-06-13 09:54:50 -07:00
Kinance
7bf39cbb72 Include the token length of the current summary (#4670)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-12 16:29:11 -07:00
Kinance
ff46c16805 Remove extra spaces in summarization prompt (#4660)
* Implement Batch Running Summarization to avoid max token error (#4652)

* Fix extra space in prompt

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-12 02:13:47 +02:00
Kinance
bc5dbb6692 Implement Batch Summarization in MessageHistory Class to manage context length under model's token limit (#4652)
* Implement Batch Running Summarization to avoid max token error

* Rename test func
2023-06-11 13:04:41 -07:00
Erik Peterson
0594ba33a2 Pass agent to commands instead of config (#4645)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

* Pass agent to commands instead of config

* Lint

* Fix merge error

* Fix memory challenge a

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 15:48:50 -07:00
Erik Peterson
6b9e3b21d3 Add config as attribute to Agent, rename old config to ai_config (#4638)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 14:47:26 -07:00
merwanehamadi
ee9f10a8d8 remove unused imports automatically (#4449)
* remove unused imports automatically

* add linters to pr template

* remove useless try statement
2023-05-28 05:50:50 -07:00
Reinier van der Leer
bfbe613960 Vector memory revamp (part 1: refactoring) (#4208)
Additional changes:

* Improve typing

* Modularize message history memory & fix/refactor lots of things

* Fix summarization

* Move memory relevance calculation to MemoryItem & improve test

* Fix import warnings in web_selenium.py

* Remove `memory_add` ghost command

* Implement overlap in `split_text`

* Move memory tests into subdirectory

* Remove deprecated `get_ada_embedding()` and helpers

* Fix used token calculation in `chat_with_ai`

* Replace Message TypedDict by dataclass

* Fix AgentManager singleton issues in tests

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
2023-05-25 20:31:11 +02:00
dominic-ks
21a202b655 Updated memory setup links (#3829)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-13 02:01:52 +03:00
Ashutosh Kataria
9c56b1beef Message about Pinecone initializing (#1194)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-01 15:31:28 +01:00
Richard Beales
06ae4684c8 replace 50+ occurrences of print() with logger (#3056)
Co-authored-by: James Collins <collijk@uw.edu>
Co-authored-by: Luke Kyohere <lkyohere@mfsafrica.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Media <12145726+rihp@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-29 23:40:57 -05:00
James Collins
3b74d2150e Organize all the llm stuff into a subpackage (#3436) 2023-04-28 12:00:54 -07:00
chyezh
65b6c2706e fix connection bug for zilliz uri on milvus (#3278)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-04-26 18:57:29 -07:00
James Collins
0ff471a49a Have api manager use singleton pattern (#3269)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-26 11:37:49 -05:00
vlad
1e71f952f9 Codecov - don't fail pipelines for project cov changes (#3327)
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
2023-04-26 09:54:22 -05:00
James Collins
2619740daa Extract OpenAI API retry handler and unify ADA embeddings calls. (#3191)
* Extract retry logic, unify embedding functions

* Add some docstrings

* Remove embedding creation from API manager

* Add test suite for retry handler

* Make api manager fixture

* Fix typing

* Streamline tests
2023-04-25 11:12:24 -07:00
James Collins
dfcbf6eee6 Refactor/move singleton out of config module (#3161) 2023-04-24 17:24:57 -05:00
scout9ll
89125376ba Fixed incorrect comment: Clear memory instead of Redis (#3092)
Co-authored-by: liaolin.qiu <liaolin.qiu@qingteng.cn>
2023-04-24 08:07:08 +01:00
James Collins
715916a5ba Merge branch 'master' into bugfix/make-local-memory-json-when-it-doesnt-exist 2023-04-23 16:44:59 -07:00
James Collins
a28b8906a6 Add tests in pytest 2023-04-23 16:40:53 -07:00
James Collins
aedd288dbe Refactor/collect embeddings code (#3060)
* Collect all embedding code into a single module

* Collect all embedding code into a single module

* actually, llm_utils is a better place

* Oh, and remove the module now that we don't use it

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-23 17:50:50 -05:00
James Collins
680c7b5aaa Make local json cache when it doesn't exist 2023-04-23 15:43:04 -07:00
Vwing
d6ef9d1b5d Make Auto-GPT aware of its running cost (#762)
* Implemented running cost counter for chat completions

This data is known to the AI as additional system context, and is printed out to the user

* Added comments to api_manager.py

* Added user-defined API budget.

The user is now prompted if they want to give the AI a budget for API calls. If they enter nothing, there is no monetary limit, but if they define a budget then the AI will be told to shut down gracefully once it has come within 1 cent of its limit, and to shut down immediately once it has exceeded its limit. If a budget is defined, Auto-GPT is always aware of how much it was given and how much remains to be spent.

* Chat completion calls are now done through api_manager. Total running cost is printed.

* Implemented api budget setting and tracking

User can now configure a maximum api budget, and the AI is aware of that and its remaining budget. The AI is instructed to shut down when exceeding the budget.

* Update autogpt/api_manager.py

Change "per token" to "per 1000 tokens" in a comment on the api cost

Co-authored-by: Rob Luke <code@robertluke.net>

* Fixed lint errors

* Include embedding costs

* Add embedding completion cost

* lint

* Added 'requires_api_key' decorator to test_commands.py, switched to a valid chat completions model

* Refactor API manager, add debug mode, and add tests

- Extract model costs to  to avoid duplication
- Add debug mode parameter to ApiManager class
- Move debug mode configuration to
- Log AI response and budget messages in debug mode
- Implement 'test_api_manager.py'

* Fixed test_setup failing. An extra user input is needed for api budget

* Linting

---------

Co-authored-by: Rob Luke <code@robertluke.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-23 16:04:31 -05:00
chyezh
da48f9c972 Fix Milvus module config import (#3036)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-23 18:32:17 +02:00
chyezh
cac1ea27e2 Support secure and authenticated Milvus memory backends (#2127)
Co-authored-by: Reinier van der Leer (Pwuts) <github@pwuts.nl>
2023-04-23 18:11:04 +02:00
non-adjective
4906e3d7ef update weaviate.py for weaviate compatibility (#2985)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/weaviate/schema/crud_schema.py", line 708, in _create_class_with_primitives
    raise UnexpectedStatusCodeException("Create class", response)
weaviate.exceptions.UnexpectedStatusCodeException: Create class! Unexpected status code: 422, with response body: {'error': [{'message': "'Auto-gpt' is not a valid class name"}]}.

GPT4:

The error message indicates that "Auto-gpt" is not a valid class name. In Weaviate, class names must start with a capital letter and can contain only alphanumeric characters.

Took advice and code and applying to weaviate.py to great result, programs runs now with no error!

Unable to reproduce easily. Might be related to switching memory between Local and Weaviate? Either way, the proposed solution works for MacOS using Docker + Weaviate.
2023-04-23 09:17:42 +01:00
Merwane Hamadi
da65bc3f68 black 2023-04-17 13:47:38 -07:00
Merwane Hamadi
cf9a94a8b6 isort implemented 2023-04-17 13:42:01 -07:00
Richard Beales
e849e4ff0b Merge pull request #1836 from cs0lar/fix/weaviate_index_to_classname
fixes Weaviate index name to classname conversion
2023-04-17 20:21:39 +01:00