Commit Graph

5229 Commits

Author SHA1 Message Date
Krzysztof Czerwinski
fb8ed0b46b fix(agent, forge): Conform web_search.py to duckduckgo_search v5 (#7046)
Update web_search command for both autogpt and forge to adjust for breaking change in v5 of duckduckgo_search,
update duckduckgo_search to ^5.0.0

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-03-25 21:10:13 +01:00
Krzysztof Czerwinski
12640f7092 fix(agent, forge): Conform web_search.py to duckduckgo_search v5 (#7045)
Update `web_search` command for both autogpt and forge to adjust for breaking change in v5 of `duckduckgo_search`
2024-03-25 19:50:26 +01:00
Reinier van der Leer
5f9cc585b1 fix(agent): Fix type issue in test_s3_file_storage.py 2024-03-22 17:09:31 +01:00
Krzysztof Czerwinski
262771a69c fix(agent): Fix check when loading an existing agent (#7026)
Now the check also ensures the chosen agent number is within proper range
2024-03-22 14:55:49 +01:00
Matheus Oliveira
a1ffe15142 security(agent): Replace unsafe pyyaml loader with SafeLoader (#7035)
Co-authored-by: pixeebot[bot] <104101892+pixeebot[bot]@users.noreply.github.com>
2024-03-22 14:45:07 +01:00
Reinier van der Leer
30bc761391 ci(agent): Add macOS on M1 to AutoGPT CI matrix (#7041)
Use a `macos-14` runner to cover macOS on M1/arm64

- Add `macos-arm64` to `platform-os` matrix, and map it to `macos-14` runner
2024-03-22 14:26:16 +01:00
Reinier van der Leer
2a0e087461 ci(agent): Disable Python dependency caching on Windows
On Windows, unpacking cached dependencies takes longer than just installing them with `poetry install`. :')
2024-03-22 14:15:43 +01:00
Reinier van der Leer
828b81e5ef ci(agent): Fix Python dependency caching on macOS 2024-03-22 14:13:22 +01:00
Reinier van der Leer
fe3f835b3e fix(cli): Add timeout to agent start command
- Add `timeout` parameter (default 30) to `wait_until_conn_ready(..)` function
- Apply `isort` and `black` formatting
2024-03-22 13:25:23 +01:00
Reinier van der Leer
6dd76afad5 test(agent): Fix VCRpy request header filter for cross-platform cassette reuse (#7040)
- Move filtering logic from tests/vcr/__init__.py to tests/vcr/vcr_filter.py
- Ignore all `X-Stainless-*` headers for cassette matching, e.g. `X-Stainless-OS` and `X-Stainless-Runtime-Version`
- Remove deprecated OpenAI proxy logic
- Reorder methods in vcr_filter.py for readability
2024-03-22 13:08:15 +01:00
Reinier van der Leer
20041d65bf ci(agent): Fix Docker CI for PR runs from forks (vol. 2)
- Fix docker image tag format error when `secrets.DOCKER_USER` is not set
2024-03-22 12:57:29 +01:00
Krzysztof Czerwinski
028d2c319f feat(autogpt): Handle OpenAI API key exceptions gracefully (#6992)
* Better handle no API keys or invalid ones
* Handle exception and exit when invalid key is provided
* Handle any APIError exception when trying to get OpenAI models and exit

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-03-22 12:55:40 +01:00
Reinier van der Leer
9e39937072 ci(agent): Fix Docker CI for PR runs from forks
- Disable 'Log in to Docker hub' step for `pull_request` runs
2024-03-22 12:50:58 +01:00
Reinier van der Leer
07a3c1848c test(agent): Update VCRpy cassettes
Add new entries for `test_dalle`
2024-03-22 11:24:29 +01:00
Reinier van der Leer
dde0c70a81 ci(agent): Matrix CI tests across Linux, macOS and Windows (#7029)
* Matrix the AutoGPT Python CI's `test` job across Ubuntu, macOS and Windows

   - Set up MinIO in a step rather than specifying it under `jobs[test].services`, because services are only supported on Linux runners

   - Add Windows version of step to install Poetry

   - Add macOS compatibility patches to 'Install Poetry (Unix)' and `setup_git_auth` steps
  
   **Caveats:**
   - **No Docker on macOS or Windows**
      * Windows comes with Docker but only supports running Windows containers, while we're mainly interested in using Linux containers for code execution and/or running auxiliary services.
      * [The macOS runner doesn't come with Docker](https://github.com/actions/runner-images/issues/17). Setting it up is possible but takes ~3-4 minutes, and the performance of the Colima engine is poor: a `docker pull` that takes 2 seconds on Linux takes 45 seconds on macOS.

   - **No S3 service available on Windows**
   It seems that running a background process [isn't possible on Windows](https://github.com/actions/runner/issues/598#issuecomment-2011890429), and neither is running Linux-based Docker containers.

* Add `autogpt-agent` and OS-specific flags to Codecov upload step

* Improve caching of Python dependencies in CI by changing the cache key
   - Include hash of `poetry.lock` instead of `pyproject.toml` in key
   - Remove date component from key; it was included to avoid getting stuck to old cached versions of packages when we were still using `requirements.txt`. With `poetry.lock` that is no longer a concern.

* Fix skip check in test_s3_file_storage.py
2024-03-21 21:15:46 +01:00
Krzysztof Czerwinski
76d6e61941 feat(agent): Implement more fault tolerant json_loads function (#7016)
* Implement syntax fault tolerant `json_loads` function using `dem3json`
   - Add `dem3json` dependency

* Replace `json.loads` by `json_loads` in places where malformed JSON may occur

* Move `json_utils.py` to `autogpt/core/utils`

* Add tests for `json_utils`

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-03-21 18:11:36 +01:00
Reinier van der Leer
bca50310f6 fix(agent): Add check for Linux container support to is_docker_available 2024-03-20 21:46:09 +01:00
Reinier van der Leer
632686cfa5 fix(agent): Replace PromptToolkit with click.prompt
- Replace `session.prompt_async(..)` with `click.prompt(..)` in `clean_input` (autogpt/app/utils.py)
- Convert `clean_input` back to a synchronous function (and amend its usages accordingly)
- Remove `prompt-toolkit` dependency

This mitigates issues crashes in some shell environments on Windows.
2024-03-20 17:51:10 +01:00
Reinier van der Leer
1262b72f5c feat(agent): Allow boolean values for available param on @command 2024-03-20 17:24:11 +01:00
Reinier van der Leer
e985f7c105 test(agent): Add skip statements to test_execute_code.py for when Docker is not available 2024-03-20 17:23:14 +01:00
Reinier van der Leer
596487b9ad fix(agent): Windows-proof file_operations
Make file_operations and test_file_operations behave more consistently between Unix and Windows
2024-03-20 17:13:47 +01:00
Reinier van der Leer
a7c0440e9b fix(agent): Fix and windows-proof scan_plugins
- Improve error output for failure to load plugin
- Fix logic to determine qualified module name
- Use `importlib` rather than `__import__` magic function

This unbreaks `scan_plugins` on Windows.
2024-03-20 17:10:03 +01:00
Reinier van der Leer
03ffb50dcf ci(docs): Fix Netlify build ignore check 2024-03-20 11:49:41 +01:00
Krzysztof Czerwinski
e201f57861 feat(autogpt/cli): Display info if code execution is enabled (#6997)
Display if code execution is enabled or not on CLI startup, depending if Docker is available.
2024-03-18 20:09:56 +01:00
Krzysztof Czerwinski
fea62a77bc feat(autogpt/cli): Check if port is available before running server (#6996) 2024-03-16 12:10:43 +01:00
Sukka
dfad535dea fix(docs): Replace polyfill.io by Cloudflare mirror (#6952) 2024-03-15 13:01:30 +01:00
Nicholas Tindle
fa14865163 docs: Redirect AutoGPT users from Forge tutorial with warning (#7014)
Update forge/get-started.md
2024-03-15 11:38:52 +01:00
Krzysztof Czerwinski
ef35028ecb fix(autogpt/cli): Loop until non-empty task is provided by the user (#6995) 2024-03-14 18:06:58 +01:00
Reinier van der Leer
fb97e15e4b lint(agent): Remove unused import in autogpt/agents/base.py 2024-03-13 20:56:45 +01:00
Reinier van der Leer
da4f013a5d fix(agent): Fix & improve agent self-termination and resumption mechanism
- Add `AgentFinished` exception (subclass of `AgentTerminated`)
- Raise `AgentFinished` instead of `AgentTerminated` in `finish` method
- Remove resumption patch from PR #6990 in `BaseAgent`
- Clean up implementation of `finish` in `AgentProtocolServer`
- Add resumption mechanism in `run_auto_gpt` (main.py)
2024-03-13 20:42:12 +01:00
Krzysztof Czerwinski
fd2c26188f fix(agent): Handle action_history-related exceptions gracefully (#6990)
Fix resume-related exceptions

- CLI: prevent resumed agent to register action on already existing one
- Server: prevent trying to json() command without result
2024-03-12 23:05:30 +01:00
Krzysztof Czerwinski
89cf0154f4 feat(autogpt/forge): Send exception details in agent protocol endpoints (#7005)
Send exception details in agent protocol endpoints

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-03-12 17:39:03 +01:00
Krzysztof Czerwinski
cb1297ec74 fix(autogpt): Fix GCS and S3 root path issue (#7010)
Fix root path issue
2024-03-12 17:34:12 +01:00
Krzysztof Czerwinski
37904a0f80 feat(agent): Fully abstracted file storage access with FileStorage (#6931)
* Rename `FileWorkspace` to `FileStorage`
   - `autogpt.file_workspace` -> `autogpt.file_storage`
   - `LocalFileWorkspace` -> `LocalFileStorage`
   - `S3FileWorkspace` -> `S3FileStorage`
   - `GCSFileWorkspace` -> `GCSFileStorage`

* Rename `WORKSPACE_BACKEND` to `FILE_STORAGE_BACKEND`
* Rename `WORKSPACE_STORAGE_BUCKET` to `STORAGE_BUCKET`

* Rewrite `AgentManager` to use `FileStorage` rather than direct local file access
* Rename `AgentManager.retrieve_state(..)` method to `load_agent_state`
* Add docstrings to `AgentManager`

* Create `AgentFileManagerMixin` to replace `AgentFileManager`, `FileWorkspaceMixin`, `BaseAgent.attach_fs(..)`
* Replace `BaseAgentSettings.save_to_json_file(..)` method by `AgentFileManagerMixin.save_state()`
* Replace `BaseAgent.set_id(..)` method by `AgentFileManagerMixin.change_agent_id(..)`
* Remove `BaseAgentSettings.load_from_json_file(..)`
* Remove `AgentSettings.agent_data_dir`

* Update `AgentProtocolServer` to work with the new `FileStorage` system and `AgentFileManagerMixin`

* Make `agent_id` and `file_storage` parameters for creating an Agent:
   - `create_agent`, `configure_agent_with_state`, `_configure_agent`, `create_agent_state` in `autogpt.agent_factory.configurators`
   - `generate_agent_for_task` in `autogpt.agent_factory.generators`
   - `Agent.__init__(..)`
   - `BaseAgent.__init__(..)`
   - Initialize and pass in `file_storage` in `autogpt.app.main.run_auto_gpt(..)` and `autogpt.app.main.run_auto_gpt_server(..)`

* Add `clone_with_subroot` to `FileStorage`
* Add `exists`, `make_dir`, `delete_dir`, `rename`, `list_files`, `list_folders` methods to `FileStorage`

* Update `autogpt.commands.file_operations` to use `FileStorage` and `AgentFileManagerMixin` features

* Update tests for `FileStorage` implementations and usages
* Rename `workspace` fixture to `storage`
   * Update conftest.py
2024-03-11 22:26:14 +01:00
Joyce
6c18627b0f Create Security Policy (#6900)
* Create SECURITY.md

Signed-off-by: Joyce <joycebrum@google.com>

* Update SECURITY.md

Signed-off-by: Joyce <joycebrum@google.com>

* Update SECURITY.md

Signed-off-by: Joyce <joycebrum@google.com>

* Update SECURITY.md

Signed-off-by: Joyce <joycebrum@google.com>

---------

Signed-off-by: Joyce <joycebrum@google.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2024-03-09 13:20:06 -06:00
Reinier van der Leer
d5aa8d373b Update QUICKSTART.md
* Clarify instructions for entering the Arena
* Clarify that entering the Arena is optional
* Remove indent from top level content
2024-03-05 11:16:44 +01:00
Toran Bruce Richards
7bf31dad35 Update README.md
Adds dark-theme support to star-history graph.
2024-03-03 15:25:36 +00:00
Reinier van der Leer
29d390d54d ci: Disable annoying auto-message discouraging big PRs 2024-03-01 12:28:00 +01:00
Reinier van der Leer
b69f0b2cd0 fix(ci/arena): Fix requesting manual review
Three times the charm, right?
2024-03-01 12:22:05 +01:00
Reinier van der Leer
0308fb45be fix(ci/arena): Fix requesting manual review 2024-03-01 11:44:27 +01:00
Reinier van der Leer
0325370fed fix(ci/arena): Fix requesting manual review 2024-03-01 11:41:49 +01:00
Reinier van der Leer
1e4bd0388f fix(ci/arena): Skip checking file against itself for duplicates 2024-03-01 11:34:47 +01:00
Reinier van der Leer
d1b06f0be3 fix(ci/arena): Improve output format 2024-03-01 11:27:26 +01:00
Reinier van der Leer
3e40b35ef1 fix(ci/arena): Reverse check for pr.mergeable 2024-03-01 11:23:14 +01:00
Reinier van der Leer
70873906b7 fix(ci/arena): Make check for pr.mergeable more specific 2024-03-01 11:20:54 +01:00
Reinier van der Leer
f93a8a93b4 fix(ci/arena): Fix error accessing context & improve log output readability 2024-03-01 11:19:31 +01:00
Reinier van der Leer
4121d3712d fix(ci/arena): Fix syntax & formatting errors 2024-03-01 11:07:54 +01:00
Reinier van der Leer
4546dfdf17 feat(ci/arena): Add logging and debug output to workflow script 2024-03-01 11:02:41 +01:00
Reinier van der Leer
4011294da0 ci(arena): Fix arena-intake workflow
Sorry folks, it's been a while since I wrote javascript :')
2024-03-01 10:41:34 +01:00
Reinier van der Leer
48f6f83f05 ci(arena): Fix arena-intake workflow 2024-03-01 10:35:28 +01:00