Commit Graph

165 Commits

Author SHA1 Message Date
Reinier van der Leer
4e761b49f3 Clean up logging 2023-08-22 07:29:56 +02:00
Reinier van der Leer
3fe2246468 Agent loop v2: Prompting improvements & WIP planning (#5077)
* Add categories to command registry

* Fix tests

* Clean up prompt generation

* Rename Performance Evaluations to Best Practices
* Move specification of response format from system prompt to Agent.construct_base_prompt
* Clean up PromptGenerator class

* Add debug logging to AIConfig autogeneration

* Clarify prompting and add support for multiple thought processes to Agent

* WIP: PlanningAgent

* Disable message history by default on BaseAgent

* Add CommandOutput and ThoughtProcessOutput type aliases

* Fix interrupts in main.py

* Use custom exceptions and clean up exception/error handling

* Remove duplicate agent_history.py

* Update PlanningAgent from upstream

* WIP: Support for dynamic in-prompt context

* WIP: response formats for PlanningAgent three-stage cycle

* Remove browsing overlay & separate browsing from extraction code

* Fix human feedback

* Fix tests

* Include history in Agent prompt generation

* Code improvements in agent.py

* Add ask_user command and revise system prompt
2023-08-19 17:44:50 +02:00
Luke
3a2d08fb41 Pass TestSearch benchmark consistently (Add browse_website TOKENS_TO_TRIGGER_SUMMARY) (#5092)
* Added SUMMARIZATION_TRIGGER_LENGTH
browse_website won't summarize content that's shorter
than SUMMARIZATION_TRIGGER_LENGTH.
It defaults to 250 characters, which is approximately 50 tokens.

* Refactor BrowserOptions

* Use tokens instead of length
to trigger summarization

* Bugfix

* fix: Always return links even if not summarizing
feat: Increase the number of links returned from 5 to 20

---------

Co-authored-by: lc0rp <2609411+lc0rp@users.noreply.github.com>
Co-authored-by: James Collins <collijk@uw.edu>
2023-08-01 11:48:13 -07:00
Reinier van der Leer
b7f1df3e1d Fix execute_shell_popen 2023-07-31 01:21:09 +02:00
Reinier van der Leer
c9bf2ee48d Add categories to command registry (#5063)
* Add categories to command registry

* Fix tests
2023-07-29 13:38:25 -07:00
merwanehamadi
45c9566298 Remove append to file (#5051)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-07-29 10:55:19 -07:00
Reinier van der Leer
f87b01620d Merge branch 'master' into release-v0.4.6-sync-to-master 2023-07-27 01:01:42 +02:00
Merwane Hamadi
b6d9c3ab03 Remove delete file
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-07-24 10:15:25 -07:00
James Collins
669e66a1e7 Move all application code to an application subpackage (#5026)
* Move all application code to an application subpackage

* Remove main.py
2023-07-21 12:01:32 -07:00
Reinier van der Leer
2c53530e99 Fix path processing (#5032)
* Fix and clean up path processing in logs module

* Fix path processing throughout the project

* Fix plugins test

* Fix borky pytest vs mkdir(exist_ok=True)

* Update docs and gitignore for new workspace location

* Fix borky pytest vol.2

* ok james
2023-07-21 11:36:15 -07:00
Sohrab Saran
a758acef2c Fix execute_python_file workspace mount & Windows path formatting (#4996)
* fix for #4975

* Add TODO based on code comment.

* Use builtin `Path.as_posix()`

* Remove TODO

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-07-17 20:24:47 +02:00
James Collins
c9adedf746 Refactor/rename agent subpackage to agents (#4961)
* Add links to github issues in the README and clarify run instructions

* Rename agent subpackage to agents

* Revert all unwanted changes

* Use relative import in `agents/__init__.py`

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-07-13 07:31:49 -07:00
Reinier van der Leer
5e39dd1d26 Speed up CI (#4930)
* Match requests in cassette by hash

* Strip requests more extensively for VCR

* Sort JSON keys on cassette save

* Strip max_tokens from cassettes

* Improve logging in retry decorator

* Raise when quota exceeded

* Clean up @retry_api

* Fix @retry_api

* Remove dead tests/vcr/openai_filter.py

* Add debug logging to execute_python_file

* Make Docker CI pass
2023-07-10 17:26:13 +02:00
Reinier van der Leer
c562fbf4bc Move path argument sanitization for commands to a decorator (#4918)
* Move path argument sanitization for commands to a decorator

* Fix tests

* Add `@functools.wraps` to `@sanitize_path_arg` decorator

Co-authored-by: James Collins <collijk@uw.edu>

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-07-09 12:40:56 -07:00
lengweiping1983
8b8b3a2cdd Improve command system; add aliases for commands (#2635)
* Command name supports multiple names

* Separate CommandRegistry.commands and .command_aliases

* Update test_commands.py

* Add __contains__ operator to CommandRegistry

* Update error message for unknown commands

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-07-08 17:29:55 +02:00
WladBlank
32038c9f5b Fix up Python execution commands (#4756)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-21 09:38:39 -07:00
Merwane Hamadi
f4c000a547 Fixes LLM thinking command descriptions are orders
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-19 16:23:09 -07:00
Luke K (pr-0f3t)
eb2410ac6c Merge remote-tracking branch 'upstream/master' into Release-v0.4.1 2023-06-19 14:08:06 -04:00
Erik Peterson
9f737274b7 Fix issues with execute_python_code responses (#4738)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-18 20:30:08 -07:00
merwanehamadi
a7f805604c Pass config everywhere in order to get rid of singleton (#4666)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-18 19:05:41 -07:00
merwanehamadi
10d7747ae2 Use JSON format for commands signature (#4714)
* Use JSON for command signature

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Improve plugin backward compatibility (#4716)

* Fixed plugin test

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Fix Docker-CI

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Put back commands, clean typing and signatures

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com>
2023-06-17 08:39:17 -07:00
merwanehamadi
d923004e20 Remove app commands, audio text and playwright (#4711)
* Remove App Commands and Audio Text
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Remove self feedback

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
2023-06-15 15:04:51 -07:00
merwanehamadi
a30e5a85b2 Remove write_tests command (#4707)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
2023-06-15 13:32:20 -07:00
merwanehamadi
512d7ba208 Remove analyze_code (#4705)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
2023-06-15 12:55:50 -07:00
Erik Peterson
195a7fcad8 Refactor module layout of command classes (#4706) 2023-06-15 11:34:41 -07:00
Reinier van der Leer
f0a5250da5 Implement loading MemoryItems from file in JSONFileMemory (#4703)
Further changes:
* remove `init` param from `get_memory()`, replace usages by `memory.clear()`
* make token length calculation optional in `MemoryItem.dump()`
2023-06-15 17:45:14 +02:00
Richard Beales
c76c67a69c Introduce method to ignore unexpected command params (#3570)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: Luke K <2609441+lc0rp@users.noreply.github.com>
2023-06-14 06:03:11 -03:00
Erik Peterson
07d9b584f7 Correct and clean up JSON handling (#4655)
* Correct and clean up JSON handling

* Use ast for message history too

* Lint

* Add comments explaining why we use literal_eval

* Add descriptions to llm_response_format schema

* Parse responses in code blocks

* Be more careful when parsing in code blocks

* Lint
2023-06-13 09:54:50 -07:00
Reinier van der Leer
a9d177eeeb Remove unused function split_file from file_operations.py (#4658) 2023-06-12 02:20:39 +02:00
Erik Peterson
0594ba33a2 Pass agent to commands instead of config (#4645)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

* Pass agent to commands instead of config

* Lint

* Fix merge error

* Fix memory challenge a

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 15:48:50 -07:00
Erik Peterson
5fe600af9d Clean up and fix issues with env configuration and .env.template (#4630)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 15:28:30 -07:00
Erik Peterson
ff4e53d0e6 Streamline / clarify shell command control configuration (#4628)
* Streamline / clarify shell command control configuration

* Fix lint
2023-06-09 11:48:20 -07:00
Erik Peterson
94280b2d14 Add command for directly executing python code (#4581)
* Add command for directly executing python code

* Fix docstring

* Clarify / update filename references

---------

Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 10:40:32 -07:00
Erik Peterson
fdc6e12945 Improve logic and error messages for file reading and writing with Python code (#4567)
* Fix issues with file reading and writing with Python code

* Change error message, use Workspace.get_path

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 02:54:02 +02:00
merwanehamadi
1b04e5cafc Fix duckduckgo rate limiting (#4592)
* Fix duckduckgo rate limiting

* use list instead of loop

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 01:20:52 +02:00
Erik Peterson
055806e124 Fix inverted logic for deny_command (#4563) 2023-06-06 22:56:17 +02:00
Benny van der Lans
74e8a886e6 Add replace_in_file command (#4565)
Resubmission of #3643

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-04 19:37:35 +02:00
merwanehamadi
ee9f10a8d8 remove unused imports automatically (#4449)
* remove unused imports automatically

* add linters to pr template

* remove useless try statement
2023-05-28 05:50:50 -07:00
k-boikov
064e95b46b Allow spinning to be disabled (#4329) 2023-05-27 00:01:34 +03:00
Nicholas Tindle
acfd966aa4 Pass Configs to Commands and remove CFG = Config() in the commands/ folder (#4328)
* feat: pass config to call_ai_functions in coimmands

* feat: config for read_audio_from_file

* feat: file operations cfg

NOTE: we replaced the CFG in the command enable with TRUE b/c not sure how to handle this yet

* feat: git command conversion

* feat: google search

* feat: image generation

* feat: extract cfg from browser commands

* feat: remove cfg from execute code commands

* fix: file operation related tests

* fix: linting

* fix: tests for read_audio

* fix: test error

* feat: update cassettes

* fix: linting

* fix: test typechecking

* fix: google_search errors if unexpected kw arg is passed

* fix: pass config param to google search test

* fix: agent commands were broken + cassettes

* fix: agent test

* feat: cassettes

* feat: enable/disable logic for commands

* fix: some commands threw errors

* feat: fix tests

* Add new cassettes

* Add new cassettes

* ci: trigger ci

* Update autogpt/commands/execute_code.py

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

* fix prompt

* fix prompt + rebase

* add config remove useless imports

* put back CFG just for download file

* lint

* The signature should be mandatory in the decorator

* black isort

* fix: remove the CFG

* fix: non typed arg

* lint: type some args

* lint: add types for libraries

* Add new cassettes

* fix: windows compatibility

* fix: add config access to decorator

* fix: remove twitter mention

* DDGS search works at 3.0.2 version

* ci: linting

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: kinance <kinance@gmail.com>
2023-05-26 08:39:25 -07:00
Reinier van der Leer
bfbe613960 Vector memory revamp (part 1: refactoring) (#4208)
Additional changes:

* Improve typing

* Modularize message history memory & fix/refactor lots of things

* Fix summarization

* Move memory relevance calculation to MemoryItem & improve test

* Fix import warnings in web_selenium.py

* Remove `memory_add` ghost command

* Implement overlap in `split_text`

* Move memory tests into subdirectory

* Remove deprecated `get_ada_embedding()` and helpers

* Fix used token calculation in `chat_with_ai`

* Replace Message TypedDict by dataclass

* Fix AgentManager singleton issues in tests

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
2023-05-25 20:31:11 +02:00
Nicholas Tindle
43565b280f Remove Twitter Command (#4381) 2023-05-24 22:40:30 -05:00
Kinance
5168cb5219 Migrate google search to use DDGS.text function (#4383) 2023-05-24 19:36:23 +03:00
Robin Richtsfeld
77ee9f8119 Pass command line args as list (#1486)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-22 22:53:31 +03:00
sherif-med
31525dfef7 Text file loaders (#3031)
* adding requiered packages for loading pdf, docx, md, tex files (preferably pure python packages)

* adding text file utils providing function to load file based on extension && adding read_text_file command

* adding test cases for text file loading (pdf file creation is hardcoded due to external package requierment for creation (a sample file can be added))

* formatting

* changing command name from 'read_text_file' to 'parse_text_document'

* fallback to txtParser if file extension is not known to read script and code files

* adding extension respective parsers

* adding binary file check function

* adding file existance check && raising valueError for unsupported binary file formats

* adding check file type (binary) in test_parsers for specific extensions && fixing mock pdf generation to include null bytes

* adding .yml extension parser

* removal of .doc parser

* updating file loading commands names

* updating test (removing .doc mock function)

* fix: import sort

* new cassette for mem A

* feat: update Cassettes

* feat: consolidate commands

* feat: linting

* feat: updates to cassettes

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-21 14:48:40 -05:00
k-boikov
bcc32cc441 Fix split_file when overlap = 0, add test (#3599)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-20 20:20:03 -05:00
WladBlank
bf33f4a7b0 add command shell blacklist and whitelist (#3950)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-19 18:50:43 +01:00
k-boikov
2cd835e5a4 Added custom_search_engine_id as mandatory for official google search (#4228)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-16 22:20:02 -05:00
Kory Becker
f424fac1d8 Huggingface retry generate_image with delay (#2745)
Co-authored-by: Media <12145726+rihp@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
Co-authored-by; lc0rp
2023-05-16 10:02:55 -05:00
Boostrix
3c80e05e00 Improve error message by providing the name of the env file (#3964)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-16 02:26:13 +03:00