Commit Graph

190 Commits

Author SHA1 Message Date
Media
ee4043ae19 Refactor test_chat to use pytest instead of unittest (#3484)
* refactor_for_pytest

* formatting

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-04-28 11:27:52 -07:00
k-boikov
c1f1da27e7 move remove_color_codes to utils and add tests (#3260)
* move remove_color_codes to utils and add tests

* Fix for ai_settings goals loaded as list(dict)

Some ai_settings formats can cause goals to load as list(dict)
not list(str)

Refactor code in utils.py to explicitly convert input type to string in
remove_color_codes() function.

- Updated remove_color_codes function to convert input argument
 to string type explicitly to avoid unexpected type errors.
- Test case added to check conversion of dict to string in
remove_color_codes function.

* Update tests/test_utils.py

Co-authored-by: James Collins <collijk@uw.edu>

* move remove_color_codes fn and tests to proper files

---------

Co-authored-by: Luke Kyohere <lkyohere@mfsafrica.com>
Co-authored-by: James Collins <collijk@uw.edu>
2023-04-28 11:13:30 -07:00
Media
aebe891489 Remove unittest in favor of pytest in the test_token_counter module (#3453)
* init remove unittest for pytest

* docstrings

* black

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-04-28 09:48:30 -07:00
Media
cf5fdabdfc Removing unitest in favor of pytest from test_config.py (#3417)
* removing unitest in favor of pytest

* remove singleton test and unnecessary fixture

---------

Co-authored-by: James Collins <collijk@uw.edu>
2023-04-28 09:32:11 -07:00
rickythefox
20ef130341 Add tests for code/shell execution & improve config fixture (#1268)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-28 14:51:29 +02:00
Eddie Cohen
5ce6da95fc Make y/n configurable (#3178)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-27 21:26:47 +02:00
James Collins
94dc6f19aa Add a regression test for the embedding (#3422) 2023-04-27 11:48:18 -07:00
Reinier van der Leer
3d89ed1787 Fix imports, type hints and fixtures for goal oriented tests (#3415) 2023-04-27 19:16:56 +02:00
merwanehamadi
adbb47fb65 scrape text regression test (#3387)
Co-authored-by: James Collins <collijk@uw.edu>
2023-04-27 09:27:15 -07:00
Montana Flynn
7cd76b8d8e Add makedirs to file operations (#3289)
* Add makedirs to file operations

* Add new directory tests for file operations

* Fix wrong setUp test error

* Simplify makedirs and use correct nested path

* Fix linter error

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: James Collins <collijk@uw.edu>
2023-04-27 09:12:24 -07:00
Robin Richtsfeld
76bd192f82 Set vcr_config scope to "session" (#3361)
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
2023-04-26 20:55:01 -05:00
merwanehamadi
02f546d2bc Run the integration tests in the CI pipeline BUT without API keys (#3359)
* integration tests in ci pipeline

* Update CONTRIBUTING.md

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-26 20:45:03 -05:00
Eddie Cohen
3b56716a68 Hotfix/validate url strips query params (#3370)
* reconstruct url in sanitize

* tests for url validation

---------

Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
2023-04-26 20:20:15 -05:00
merwanehamadi
a3195d84d3 remove do nothing (#3369) 2023-04-26 19:55:02 -05:00
merwanehamadi
7a006afb17 fix cassettes recording (#3342) 2023-04-26 13:11:08 -07:00
James Collins
83f11465f5 Clean up image generation tests (#3338) 2023-04-26 12:07:28 -07:00
merwanehamadi
109fa04c7c test image gen (#3287) 2023-04-26 10:23:05 -07:00
merwanehamadi
a6355a6bc8 use pytest-recording with VCR (#3283) 2023-04-26 09:57:05 -07:00
James Collins
0ff471a49a Have api manager use singleton pattern (#3269)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-26 11:37:49 -05:00
merwanehamadi
4241fbbbf0 mock openai in test image gen (#3285) 2023-04-26 09:11:31 -07:00
James Collins
eec86a7b82 Load .env in package init (#3251) 2023-04-25 14:53:13 -07:00
James Collins
2619740daa Extract OpenAI API retry handler and unify ADA embeddings calls. (#3191)
* Extract retry logic, unify embedding functions

* Add some docstrings

* Remove embedding creation from API manager

* Add test suite for retry handler

* Make api manager fixture

* Fix typing

* Streamline tests
2023-04-25 11:12:24 -07:00
merwanehamadi
58d84787f3 Test Agent.create_agent_feedback (#3209) 2023-04-25 08:41:57 -07:00
James Collins
83b91a31bc Remove dead permanent memory module (#3145)
* Remove dead permanent memory module

* Delete sqlite db that snuck in
2023-04-24 21:48:37 +01:00
James Collins
b984f985bc Hotfix/global agent manager workaround (#3157)
* Add indirection layer to entry point

* Get around singleton pattern for AgentManager to fix tests
2023-04-24 21:27:31 +01:00
Media
5de1025520 Agent and agent manager tests (#3116)
* Update Python version and benchmark file in benchmark.yml

* Refactor main function and imports in cli.py

* Update import statement in ai_config.py

* Add set_temperature and set_memory_backend methods in config.py

* Remove unused import in prompt.py

* Add goal oriented tasks workflow

* Added agent_utils to create agent

* added pytest and vcrpy

* added write file cassette

* created goal oriented task write file with cassettes to not pay openai tokens

* solve conflicts

* add ability set azure because github workflow needs it off

* solve conflicts in cli.py

* black because linter fails

* solve conflict

* setup github action to v3

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* fix conflicts

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Plugins: debug line always printed in plugin load

* add decorator to tests

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* move decorator higher up

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* init

* more tests

* passing tests

* skip gitbranch decorator on ci

* decorator skiponci

* black

* Update tests/utils.py decorator of skipping ci

Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>

* black

* I oopsed the name

* black

* finally

* simple tests for agent and manager

* ísort

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
2023-04-24 14:19:42 -05:00
Nicholas Tindle
97d2f417c7 Merge branch 'master' into bugfix/error-on-null-bytes-in-path-windows 2023-04-24 13:55:41 -05:00
YOUNESS ZEMZGUI
45f2513a73 Adjust test_json_parser file (#1935)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-24 19:54:46 +02:00
James Collins
17819e2a55 More robust null byte checking 2023-04-24 10:28:51 -07:00
k-boikov
3886afc825 fix test_search_files for windows (#3073)
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-04-24 17:42:08 +01:00
Andres Caicedo
f8dfedf1c6 Add function and class descriptions to tests (#2715)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-24 14:55:49 +02:00
Eddie Cohen
40a75c804c Validate URLs in web commands before execution (#2616)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-24 12:33:44 +02:00
James Collins
715916a5ba Merge branch 'master' into bugfix/make-local-memory-json-when-it-doesnt-exist 2023-04-23 16:44:59 -07:00
James Collins
a28b8906a6 Add tests in pytest 2023-04-23 16:40:53 -07:00
James Collins
aedd288dbe Refactor/collect embeddings code (#3060)
* Collect all embedding code into a single module

* Collect all embedding code into a single module

* actually, llm_utils is a better place

* Oh, and remove the module now that we don't use it

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-23 17:50:50 -05:00
Media
374f543bea Perm memory test cases (#2996)
* Update Python version and benchmark file in benchmark.yml

* Refactor main function and imports in cli.py

* Update import statement in ai_config.py

* Add set_temperature and set_memory_backend methods in config.py

* Remove unused import in prompt.py

* Add goal oriented tasks workflow

* Added agent_utils to create agent

* added pytest and vcrpy

* added write file cassette

* created goal oriented task write file with cassettes to not pay openai tokens

* solve conflicts

* add ability set azure because github workflow needs it off

* solve conflicts in cli.py

* black because linter fails

* solve conflict

* setup github action to v3

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* fix conflicts

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Plugins: debug line always printed in plugin load

* add decorator to tests

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* move decorator higher up

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* init

* more tests

* passing tests

* skip gitbranch decorator on ci

* decorator skiponci

* black

* Update tests/utils.py decorator of skipping ci

Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>

* black

* I oopsed the name

* black

* finally

* perm memory tests

* perm memory tests

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
2023-04-23 16:50:15 -05:00
coditamar
ec71075bfe Add tests for json_utils.json_fix_llm (#2952)
* config.py: make load_dotenv(override=True)

* Update Python version and benchmark file in benchmark.yml

* Refactor main function and imports in cli.py

* Update import statement in ai_config.py

* Add set_temperature and set_memory_backend methods in config.py

* Remove unused import in prompt.py

* Add goal oriented tasks workflow

* Added agent_utils to create agent

* added pytest and vcrpy

* added write file cassette

* created goal oriented task write file with cassettes to not pay openai tokens

* solve conflicts

* add ability set azure because github workflow needs it off

* solve conflicts in cli.py

* black because linter fails

* solve conflict

* setup github action to v3

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* fix conflicts

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Plugins: debug line always printed in plugin load

* add test for fix_json_using_multiple_techniques

* style

* style

* mocking try_ai_fix to avoid call_ai_function

* black style

* mock try_ai_fix to avoid calling the AI model

* removed mock, as we can add @requires_api_key("OPEN_API_KEY")

* style

* reverse merge conflict related files and changes

* bring back the mock for try_ai_fix

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
2023-04-23 16:29:40 -05:00
Vwing
d6ef9d1b5d Make Auto-GPT aware of its running cost (#762)
* Implemented running cost counter for chat completions

This data is known to the AI as additional system context, and is printed out to the user

* Added comments to api_manager.py

* Added user-defined API budget.

The user is now prompted if they want to give the AI a budget for API calls. If they enter nothing, there is no monetary limit, but if they define a budget then the AI will be told to shut down gracefully once it has come within 1 cent of its limit, and to shut down immediately once it has exceeded its limit. If a budget is defined, Auto-GPT is always aware of how much it was given and how much remains to be spent.

* Chat completion calls are now done through api_manager. Total running cost is printed.

* Implemented api budget setting and tracking

User can now configure a maximum api budget, and the AI is aware of that and its remaining budget. The AI is instructed to shut down when exceeding the budget.

* Update autogpt/api_manager.py

Change "per token" to "per 1000 tokens" in a comment on the api cost

Co-authored-by: Rob Luke <code@robertluke.net>

* Fixed lint errors

* Include embedding costs

* Add embedding completion cost

* lint

* Added 'requires_api_key' decorator to test_commands.py, switched to a valid chat completions model

* Refactor API manager, add debug mode, and add tests

- Extract model costs to  to avoid duplication
- Add debug mode parameter to ApiManager class
- Move debug mode configuration to
- Log AI response and budget messages in debug mode
- Implement 'test_api_manager.py'

* Fixed test_setup failing. An extra user input is needed for api budget

* Linting

---------

Co-authored-by: Rob Luke <code@robertluke.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-23 16:04:31 -05:00
James Collins
dcd6aa912b Add workspace abstraction (#2982)
* Add workspace abstraction

* Remove old workspace implementation

* Extract path resolution to a helper function

* Add api key requirements to new tests
2023-04-23 14:36:04 -05:00
Richard Beales
9ed2a7a2d2 Add missing test decorator (#2989)
* Mark test test_generate_aiconfig_automatic_typical  as @requires_api_key("OPENAI_API_KEY")

* missing import

* add missing decorator
2023-04-23 09:17:18 +01:00
Richard Beales
bb786461c7 Mark test test_generate_aiconfig_automatic_typical as @requires_api_… (#2981)
* Mark test test_generate_aiconfig_automatic_typical  as @requires_api_key("OPENAI_API_KEY")

* missing import
2023-04-23 07:35:17 +01:00
Toran Bruce Richards
f462674e32 Automatic prompting (#2896)
* Add automatic ai prompting

* Tweak the default prompt.

* Print agent info upon creation.

* Improve system prompt

* Switch to fast_llm_model by default

* Add format output command to user prompt.

This vastly improves formatting success rate.

* Add fallback to manual mode if llm output cannot be parsed (or other error).

* Add unit test to cover ai creation setup.

* Replace redundent prompt with manual mode instructions.

* Add missing docstrings and typing.

* Runs black on changes.

* Runs isort

* Update Python version and benchmark file in benchmark.yml

* Refactor main function and imports in cli.py

* Update import statement in ai_config.py

* Add set_temperature and set_memory_backend methods in config.py

* Remove unused import in prompt.py

* Add goal oriented tasks workflow

* Added agent_utils to create agent

* added pytest and vcrpy

* added write file cassette

* created goal oriented task write file with cassettes to not pay openai tokens

* solve conflicts

* add ability set azure because github workflow needs it off

* solve conflicts in cli.py

* black because linter fails

* solve conflict

* setup github action to v3

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* fix conflicts

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Plugins: debug line always printed in plugin load

* add decorator to tests

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* move decorator higher up

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* merge

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
2023-04-23 06:36:10 +01:00
Media
2b5852f7da Tests utils suite (#2961)
* Update Python version and benchmark file in benchmark.yml

* Refactor main function and imports in cli.py

* Update import statement in ai_config.py

* Add set_temperature and set_memory_backend methods in config.py

* Remove unused import in prompt.py

* Add goal oriented tasks workflow

* Added agent_utils to create agent

* added pytest and vcrpy

* added write file cassette

* created goal oriented task write file with cassettes to not pay openai tokens

* solve conflicts

* add ability set azure because github workflow needs it off

* solve conflicts in cli.py

* black because linter fails

* solve conflict

* setup github action to v3

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* fix conflicts

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Plugins: debug line always printed in plugin load

* add decorator to tests

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* move decorator higher up

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* init

* more tests

* passing tests

* skip gitbranch decorator on ci

* decorator skiponci

* black

* Update tests/utils.py decorator of skipping ci

Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>

* black

* I oopsed the name

* black

* finally

* black

* wrong file

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
2023-04-22 19:07:28 -05:00
Nicholas Tindle
986bdaab36 Merge pull request #2946 from merwanehamadi/feature/add-decorator-to-tests
add decorator to tests so it's skipped if api key required but not present in the environment
2023-04-23 00:45:55 +02:00
Merwane Hamadi
b7cd56f72b move decorator higher up
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-04-23 00:45:54 +02:00
Merwane Hamadi
eb5a8a87d8 add decorator to tests
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-04-23 00:45:53 +02:00
Nicholas Tindle
0410331ecd Merge pull request #2931 from riensen/fix/multiple-plugins
Enable support for loading multiple plugins per zip file
2023-04-23 00:45:50 +02:00
Merwane Hamadi
996a3b331a Add CI smoke test (#2461) 2023-04-23 00:23:45 +02:00
riensen
8173e4d139 Fix: Mulitple plugins per zip for Auto-GPT-Plugins 2023-04-22 18:31:04 +02:00
Toran Bruce Richards
e39cd1bf57 Fix(tests): restore config values after changing them in tests (#2904) 2023-04-22 12:14:18 +02:00