- Move filtering logic from tests/vcr/__init__.py to tests/vcr/vcr_filter.py
- Ignore all `X-Stainless-*` headers for cassette matching, e.g. `X-Stainless-OS` and `X-Stainless-Runtime-Version`
- Remove deprecated OpenAI proxy logic
- Reorder methods in vcr_filter.py for readability
* Better handle no API keys or invalid ones
* Handle exception and exit when invalid key is provided
* Handle any APIError exception when trying to get OpenAI models and exit
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* Matrix the AutoGPT Python CI's `test` job across Ubuntu, macOS and Windows
- Set up MinIO in a step rather than specifying it under `jobs[test].services`, because services are only supported on Linux runners
- Add Windows version of step to install Poetry
- Add macOS compatibility patches to 'Install Poetry (Unix)' and `setup_git_auth` steps
**Caveats:**
- **No Docker on macOS or Windows**
* Windows comes with Docker but only supports running Windows containers, while we're mainly interested in using Linux containers for code execution and/or running auxiliary services.
* [The macOS runner doesn't come with Docker](https://github.com/actions/runner-images/issues/17). Setting it up is possible but takes ~3-4 minutes, and the performance of the Colima engine is poor: a `docker pull` that takes 2 seconds on Linux takes 45 seconds on macOS.
- **No S3 service available on Windows**
It seems that running a background process [isn't possible on Windows](https://github.com/actions/runner/issues/598#issuecomment-2011890429), and neither is running Linux-based Docker containers.
* Add `autogpt-agent` and OS-specific flags to Codecov upload step
* Improve caching of Python dependencies in CI by changing the cache key
- Include hash of `poetry.lock` instead of `pyproject.toml` in key
- Remove date component from key; it was included to avoid getting stuck to old cached versions of packages when we were still using `requirements.txt`. With `poetry.lock` that is no longer a concern.
* Fix skip check in test_s3_file_storage.py
* Implement syntax fault tolerant `json_loads` function using `dem3json`
- Add `dem3json` dependency
* Replace `json.loads` by `json_loads` in places where malformed JSON may occur
* Move `json_utils.py` to `autogpt/core/utils`
* Add tests for `json_utils`
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
- Replace `session.prompt_async(..)` with `click.prompt(..)` in `clean_input` (autogpt/app/utils.py)
- Convert `clean_input` back to a synchronous function (and amend its usages accordingly)
- Remove `prompt-toolkit` dependency
This mitigates issues crashes in some shell environments on Windows.
- Improve error output for failure to load plugin
- Fix logic to determine qualified module name
- Use `importlib` rather than `__import__` magic function
This unbreaks `scan_plugins` on Windows.
- Add `AgentFinished` exception (subclass of `AgentTerminated`)
- Raise `AgentFinished` instead of `AgentTerminated` in `finish` method
- Remove resumption patch from PR #6990 in `BaseAgent`
- Clean up implementation of `finish` in `AgentProtocolServer`
- Add resumption mechanism in `run_auto_gpt` (main.py)
Fix resume-related exceptions
- CLI: prevent resumed agent to register action on already existing one
- Server: prevent trying to json() command without result
* Rename `FileWorkspace` to `FileStorage`
- `autogpt.file_workspace` -> `autogpt.file_storage`
- `LocalFileWorkspace` -> `LocalFileStorage`
- `S3FileWorkspace` -> `S3FileStorage`
- `GCSFileWorkspace` -> `GCSFileStorage`
* Rename `WORKSPACE_BACKEND` to `FILE_STORAGE_BACKEND`
* Rename `WORKSPACE_STORAGE_BUCKET` to `STORAGE_BUCKET`
* Rewrite `AgentManager` to use `FileStorage` rather than direct local file access
* Rename `AgentManager.retrieve_state(..)` method to `load_agent_state`
* Add docstrings to `AgentManager`
* Create `AgentFileManagerMixin` to replace `AgentFileManager`, `FileWorkspaceMixin`, `BaseAgent.attach_fs(..)`
* Replace `BaseAgentSettings.save_to_json_file(..)` method by `AgentFileManagerMixin.save_state()`
* Replace `BaseAgent.set_id(..)` method by `AgentFileManagerMixin.change_agent_id(..)`
* Remove `BaseAgentSettings.load_from_json_file(..)`
* Remove `AgentSettings.agent_data_dir`
* Update `AgentProtocolServer` to work with the new `FileStorage` system and `AgentFileManagerMixin`
* Make `agent_id` and `file_storage` parameters for creating an Agent:
- `create_agent`, `configure_agent_with_state`, `_configure_agent`, `create_agent_state` in `autogpt.agent_factory.configurators`
- `generate_agent_for_task` in `autogpt.agent_factory.generators`
- `Agent.__init__(..)`
- `BaseAgent.__init__(..)`
- Initialize and pass in `file_storage` in `autogpt.app.main.run_auto_gpt(..)` and `autogpt.app.main.run_auto_gpt_server(..)`
* Add `clone_with_subroot` to `FileStorage`
* Add `exists`, `make_dir`, `delete_dir`, `rename`, `list_files`, `list_folders` methods to `FileStorage`
* Update `autogpt.commands.file_operations` to use `FileStorage` and `AgentFileManagerMixin` features
* Update tests for `FileStorage` implementations and usages
* Rename `workspace` fixture to `storage`
* Update conftest.py
Poetry recently released v1.8.x containing a fix for the issue we were having earlier:
https://github.com/python-poetry/poetry/issues/8548
This means unavailable optional directory dependencies no longer break the docker build.
* Fix unmasking of `azure_endpoint` in `OpenAICredentials.get_api_access_kwargs()`
* Amend `ApiManager.get_models` to use `AzureOpenAI` client when `api_type` is set to `azure`
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* By default, allow requests originating from http://localhost:{AP_SERVER_PORT} instead of all origins
* Allow configuring allowed CORS origins through `AP_SERVER_CORS_ALLOWED_ORIGINS`
* Mitigate shell injection in `MacOSTTS._speech` implementation
* Mitigate shell command control bypassing in `execute_shell` and `execute_shell_popen` commands
- Improve implementation and docstring of `validate_command` function
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
We have been seeing `AttributeError: 'NoneType' object has no attribute 'split'` in Sentry.
According to https://github.com/SergeyPirogov/webdriver_manager/issues/649, this occurs
when Chrome is not installed, or when no suitable ChromeDriver version is available for
the installed version of Chrome.
Instead of the `AttributeError`, we can print a better message for the agent and user to help them fix the issue.
---
Co-authored-by: kcze <kpczerwinski@gmail.com>
OpenAI requires the `id` property on tool calls in assistant messages.
We weren't storing it in the `AssistantChatMessage` that is created from the LLM's response,
causing an error when adding those messages back to the prompt.
Fix:
* Add `id` to `AssistantToolCall` and `AssistantToolCallDict` types in model_providers/schema.py
* Amend `_tool_calls_compat_extract_calls` to generate an ID for extracted tool calls
---
Co-authored-by: kcze <kpczerwinski@gmail.com>
OpenAI likes neither `tool_calls=[]` nor `tool_calls=None`. If no `tool_calls` are in the message, the key must be omitted.
This partially reverts commit 67bafa6302.
---
Co-authored-by: kcze <kpczerwinski@gmail.com>
In a `for item in list` loop, removing items from the list while iterating causes it to skip over the next item. Fix: refactor `interactively_revise_ai_settings` routine to use while loop for iterating over constraints, resources, and best practices.
---------
Co-authored-by: Kripanshu Jindal <polaris@Polaris.local>
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
- Added `task_cumulative_cost` and `task_total_cost` attributes to the `Step.additional_output` in the `AgentProtocolServer.execute_step` endpoint.
- Updated `agbenchmark` dependency in Agent and Forge
- Added a helper function `.app.utils.vcs_state_diverges_from_master()`. This function determines whether the relevant part of the codebase diverges from our `master`.
- Updated `.app.telemetry._setup_sentry()` to determine the default environment name using `vcs_state_diverges_from_master`.
