Commit Graph

100 Commits

Author SHA1 Message Date
Reinier van der Leer
a9d177eeeb Remove unused function split_file from file_operations.py (#4658) 2023-06-12 02:20:39 +02:00
Kinance
bc5dbb6692 Implement Batch Summarization in MessageHistory Class to manage context length under model's token limit (#4652)
* Implement Batch Running Summarization to avoid max token error

* Rename test func
2023-06-11 13:04:41 -07:00
Erik Peterson
0594ba33a2 Pass agent to commands instead of config (#4645)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

* Pass agent to commands instead of config

* Lint

* Fix merge error

* Fix memory challenge a

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 15:48:50 -07:00
Erik Peterson
6b9e3b21d3 Add config as attribute to Agent, rename old config to ai_config (#4638)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 14:47:26 -07:00
Erik Peterson
15c6b0c1c3 Implement directory-based plugin system (#4548)
* Implement directory-based plugin system

* Fix Selenium test

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-10 13:16:00 -07:00
Erik Peterson
5fe600af9d Clean up and fix issues with env configuration and .env.template (#4630)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 15:28:30 -07:00
Stefan Ayala
1e851ba3ea Feat set token limits based on model (#4498)
* feat: set max token limits for better user experience

* fix: use OPEN_AI_CHAT_MODELS max limits

* fix: use the old default of 8000

* fix: formatting so isort/black checks pass

* fix: avoid circular dependencies

* fix: use better to avoid circular imports

* feat: introduce soft limits and use them

* fix: circular import issue and missing field

* fix: move import to avoid overriding doc comment

* feat: DRY things up and set token limit for fast llm models too

* tests: make linter tests happy

* test: use the max token limits in config.py test

* fix: remove fast token limit from config

* feat: remove smart token limit from config

* fix: remove unused soft_token_limit var

* fix: remove unneeded tests, settings aren't in config anymore

---------

Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 10:16:53 +02:00
Erik Peterson
fdc6e12945 Improve logic and error messages for file reading and writing with Python code (#4567)
* Fix issues with file reading and writing with Python code

* Change error message, use Workspace.get_path

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 02:54:02 +02:00
Reinier van der Leer
dafbd11686 Rearrange tests & fix CI (#4596)
* Rearrange tests into unit/integration/challenge categories

* Fix linting + `tests.challenges` imports

* Fix obscured duplicate test in test_url_validation.py

* Move VCR conftest to tests.vcr

* Specify tests to run & their order (unit -> integration -> challenges) in CI

* Fail Docker CI when tests fail

* Fix import & linting errors in tests

* Fix `get_text_summary`

* Fix linting errors

* Clean up pytest args in CI

* Remove bogus tests from GoCodeo
2023-06-06 10:48:49 -07:00
Reinier van der Leer
576f24e3ae Merge branch 'master' into release-0.4.0 2023-06-05 16:08:35 +02:00
Benny van der Lans
74e8a886e6 Add replace_in_file command (#4565)
Resubmission of #3643

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-04 19:37:35 +02:00
merwanehamadi
af28510aba Fix test_web_selenium (#4554) 2023-06-04 16:38:32 +02:00
merwanehamadi
ee9f10a8d8 remove unused imports automatically (#4449)
* remove unused imports automatically

* add linters to pr template

* remove useless try statement
2023-05-28 05:50:50 -07:00
Ryan Johns
03036c1bd6 Added three more tests to check for edge cases in URL validation (#4441)
Co-authored-by: Ryan Johns <rkjohns@verisk.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-27 18:58:38 +03:00
Nicholas Tindle
acfd966aa4 Pass Configs to Commands and remove CFG = Config() in the commands/ folder (#4328)
* feat: pass config to call_ai_functions in coimmands

* feat: config for read_audio_from_file

* feat: file operations cfg

NOTE: we replaced the CFG in the command enable with TRUE b/c not sure how to handle this yet

* feat: git command conversion

* feat: google search

* feat: image generation

* feat: extract cfg from browser commands

* feat: remove cfg from execute code commands

* fix: file operation related tests

* fix: linting

* fix: tests for read_audio

* fix: test error

* feat: update cassettes

* fix: linting

* fix: test typechecking

* fix: google_search errors if unexpected kw arg is passed

* fix: pass config param to google search test

* fix: agent commands were broken + cassettes

* fix: agent test

* feat: cassettes

* feat: enable/disable logic for commands

* fix: some commands threw errors

* feat: fix tests

* Add new cassettes

* Add new cassettes

* ci: trigger ci

* Update autogpt/commands/execute_code.py

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

* fix prompt

* fix prompt + rebase

* add config remove useless imports

* put back CFG just for download file

* lint

* The signature should be mandatory in the decorator

* black isort

* fix: remove the CFG

* fix: non typed arg

* lint: type some args

* lint: add types for libraries

* Add new cassettes

* fix: windows compatibility

* fix: add config access to decorator

* fix: remove twitter mention

* DDGS search works at 3.0.2 version

* ci: linting

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: kinance <kinance@gmail.com>
2023-05-26 08:39:25 -07:00
k-boikov
f07fcdf0a7 Pull auto-gpt-plugin-template from GitHub instead of PyPI (#4402)
* switch from pypi to github for auto-gpt-plugin-template

* adapt tests to new plugin interface
2023-05-26 14:48:36 +02:00
Reinier van der Leer
bfbe613960 Vector memory revamp (part 1: refactoring) (#4208)
Additional changes:

* Improve typing

* Modularize message history memory & fix/refactor lots of things

* Fix summarization

* Move memory relevance calculation to MemoryItem & improve test

* Fix import warnings in web_selenium.py

* Remove `memory_add` ghost command

* Implement overlap in `split_text`

* Move memory tests into subdirectory

* Remove deprecated `get_ada_embedding()` and helpers

* Fix used token calculation in `chat_with_ai`

* Replace Message TypedDict by dataclass

* Fix AgentManager singleton issues in tests

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
2023-05-25 20:31:11 +02:00
k-boikov
bcc32cc441 Fix split_file when overlap = 0, add test (#3599)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-20 20:20:03 -05:00
Konrad
57ea7b5216 Fixed #4229 (#4278)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-21 02:45:27 +03:00
Omri Grossman
a08fc850f0 Used a regex expression for simple URL validation and added tests (#3763)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-19 19:47:04 +01:00
Media
c4b32e067c switching from unittest to pytest in test_json_parser (#3481) 2023-05-18 00:58:08 -05:00
Andres Caicedo
7508e9941f Implement Logging of Self-Feedback in logs/Debug Folder (#3868)
* Adds SELF_FEEDBACK_FILE_NAME

* Add self-feedback logging to logs/Debug folder

* Reformatting

* Uses JSON file

* Update agent.py

Changes position

* Update agent.py

* Adds PROMPT_FEEDBACK_FILE_NAME

* Update agent.py

* Update agent.py

* Reformatting

* Update agent.py

* Update agent.py

* Changes file names

* Update agent.py

* Reformatting

* Update agent.py

* Changes conts names

* Update agent_manager.py

* Update agent_manager.py

* HARD reset

* Update test_get_self_feedback.py

* Update test_get_self_feedback.py

---------

Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-05-17 04:38:42 -07:00
merwanehamadi
4143d212a5 Feature/basic proxy (#4164)
* basic proxy (#54)

* basic proxy (#55)

* basic proxy

* basic proxy

* basic proxy

* basic proxy

* add back double quotes

* add more specific files

* write file

* basic proxy

* Put back double quotes
2023-05-14 11:07:37 +12:00
AbTrax
e12438de41 Self feedback Improvement (#3680)
* Improved `Self-Feedback`

* minor tweak

* Test: Updated `test_get_self_feedback.py`
2023-05-05 09:09:21 -07:00
bszollosinagy
ea08050049 BUGFIX: Selenium Driver object reference was included in the browsing results for some reason (#3642)
* * there is really no need to return the  reference to the Selenium driver along with the text summary and list of links.

* * removing unused second return value from browse_website()

* * updated cassette

* * updated YAML cassette for test_browse_website

* * after requirements reinstall, another update YAML cassette for test_browse_website

* * another update YAML cassette for test_browse_website, only as a placholder commit to trigger re-testing due to some docker TCP timeout issue

* * another update YAML cassette for test_browse_website

---------

Co-authored-by: batyu <batyu@localhost>
2023-05-04 23:11:21 +01:00
kinance
4767fe63d3 Fix the maximum context length issue by chunking (#3222)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-05-01 20:13:24 +02:00
Bob
94ec4a4ea5 Fix file operations logger (#3489)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-05-01 17:37:30 +02:00
Richard Beales
c1329c92fd rename search_files to list_files (#3595) 2023-04-30 16:14:53 -05:00
merwanehamadi
dd96d98fa1 Feature/test summarization against memory challenge (#3567)
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
2023-04-30 09:56:57 +01:00
Media
4f72ee7815 Refactor test_spiunner to deprecate unittest in favor of pytest (#3532)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-29 12:40:32 -05:00
k-boikov
2d058feaf8 Extend & improve file operations tests (#3404)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-29 16:55:47 +02:00
James Collins
92009ceb32 More graceful browsing error handling (#3494) 2023-04-28 22:12:47 +01:00
James Collins
3b74d2150e Organize all the llm stuff into a subpackage (#3436) 2023-04-28 12:00:54 -07:00
Media
ee4043ae19 Refactor test_chat to use pytest instead of unittest (#3484)
* refactor_for_pytest

* formatting

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-04-28 11:27:52 -07:00
Eddie Cohen
5ce6da95fc Make y/n configurable (#3178)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-27 21:26:47 +02:00
Montana Flynn
7cd76b8d8e Add makedirs to file operations (#3289)
* Add makedirs to file operations

* Add new directory tests for file operations

* Fix wrong setUp test error

* Simplify makedirs and use correct nested path

* Fix linter error

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: James Collins <collijk@uw.edu>
2023-04-27 09:12:24 -07:00
merwanehamadi
02f546d2bc Run the integration tests in the CI pipeline BUT without API keys (#3359)
* integration tests in ci pipeline

* Update CONTRIBUTING.md

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-26 20:45:03 -05:00
Eddie Cohen
3b56716a68 Hotfix/validate url strips query params (#3370)
* reconstruct url in sanitize

* tests for url validation

---------

Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
2023-04-26 20:20:15 -05:00
James Collins
0ff471a49a Have api manager use singleton pattern (#3269)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-26 11:37:49 -05:00
merwanehamadi
58d84787f3 Test Agent.create_agent_feedback (#3209) 2023-04-25 08:41:57 -07:00
YOUNESS ZEMZGUI
45f2513a73 Adjust test_json_parser file (#1935)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-24 19:54:46 +02:00
k-boikov
3886afc825 fix test_search_files for windows (#3073)
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-04-24 17:42:08 +01:00
Andres Caicedo
f8dfedf1c6 Add function and class descriptions to tests (#2715)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-24 14:55:49 +02:00
Eddie Cohen
40a75c804c Validate URLs in web commands before execution (#2616)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-24 12:33:44 +02:00
coditamar
ec71075bfe Add tests for json_utils.json_fix_llm (#2952)
* config.py: make load_dotenv(override=True)

* Update Python version and benchmark file in benchmark.yml

* Refactor main function and imports in cli.py

* Update import statement in ai_config.py

* Add set_temperature and set_memory_backend methods in config.py

* Remove unused import in prompt.py

* Add goal oriented tasks workflow

* Added agent_utils to create agent

* added pytest and vcrpy

* added write file cassette

* created goal oriented task write file with cassettes to not pay openai tokens

* solve conflicts

* add ability set azure because github workflow needs it off

* solve conflicts in cli.py

* black because linter fails

* solve conflict

* setup github action to v3

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* fix conflicts

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Plugins: debug line always printed in plugin load

* add test for fix_json_using_multiple_techniques

* style

* style

* mocking try_ai_fix to avoid call_ai_function

* black style

* mock try_ai_fix to avoid calling the AI model

* removed mock, as we can add @requires_api_key("OPEN_API_KEY")

* style

* reverse merge conflict related files and changes

* bring back the mock for try_ai_fix

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
2023-04-23 16:29:40 -05:00
Vwing
d6ef9d1b5d Make Auto-GPT aware of its running cost (#762)
* Implemented running cost counter for chat completions

This data is known to the AI as additional system context, and is printed out to the user

* Added comments to api_manager.py

* Added user-defined API budget.

The user is now prompted if they want to give the AI a budget for API calls. If they enter nothing, there is no monetary limit, but if they define a budget then the AI will be told to shut down gracefully once it has come within 1 cent of its limit, and to shut down immediately once it has exceeded its limit. If a budget is defined, Auto-GPT is always aware of how much it was given and how much remains to be spent.

* Chat completion calls are now done through api_manager. Total running cost is printed.

* Implemented api budget setting and tracking

User can now configure a maximum api budget, and the AI is aware of that and its remaining budget. The AI is instructed to shut down when exceeding the budget.

* Update autogpt/api_manager.py

Change "per token" to "per 1000 tokens" in a comment on the api cost

Co-authored-by: Rob Luke <code@robertluke.net>

* Fixed lint errors

* Include embedding costs

* Add embedding completion cost

* lint

* Added 'requires_api_key' decorator to test_commands.py, switched to a valid chat completions model

* Refactor API manager, add debug mode, and add tests

- Extract model costs to  to avoid duplication
- Add debug mode parameter to ApiManager class
- Move debug mode configuration to
- Log AI response and budget messages in debug mode
- Implement 'test_api_manager.py'

* Fixed test_setup failing. An extra user input is needed for api budget

* Linting

---------

Co-authored-by: Rob Luke <code@robertluke.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-23 16:04:31 -05:00
James Collins
dcd6aa912b Add workspace abstraction (#2982)
* Add workspace abstraction

* Remove old workspace implementation

* Extract path resolution to a helper function

* Add api key requirements to new tests
2023-04-23 14:36:04 -05:00
Richard Beales
9ed2a7a2d2 Add missing test decorator (#2989)
* Mark test test_generate_aiconfig_automatic_typical  as @requires_api_key("OPENAI_API_KEY")

* missing import

* add missing decorator
2023-04-23 09:17:18 +01:00
Richard Beales
bb786461c7 Mark test test_generate_aiconfig_automatic_typical as @requires_api_… (#2981)
* Mark test test_generate_aiconfig_automatic_typical  as @requires_api_key("OPENAI_API_KEY")

* missing import
2023-04-23 07:35:17 +01:00
Toran Bruce Richards
f462674e32 Automatic prompting (#2896)
* Add automatic ai prompting

* Tweak the default prompt.

* Print agent info upon creation.

* Improve system prompt

* Switch to fast_llm_model by default

* Add format output command to user prompt.

This vastly improves formatting success rate.

* Add fallback to manual mode if llm output cannot be parsed (or other error).

* Add unit test to cover ai creation setup.

* Replace redundent prompt with manual mode instructions.

* Add missing docstrings and typing.

* Runs black on changes.

* Runs isort

* Update Python version and benchmark file in benchmark.yml

* Refactor main function and imports in cli.py

* Update import statement in ai_config.py

* Add set_temperature and set_memory_backend methods in config.py

* Remove unused import in prompt.py

* Add goal oriented tasks workflow

* Added agent_utils to create agent

* added pytest and vcrpy

* added write file cassette

* created goal oriented task write file with cassettes to not pay openai tokens

* solve conflicts

* add ability set azure because github workflow needs it off

* solve conflicts in cli.py

* black because linter fails

* solve conflict

* setup github action to v3

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* fix conflicts

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Plugins: debug line always printed in plugin load

* add decorator to tests

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* move decorator higher up

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* merge

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
2023-04-23 06:36:10 +01:00