Commit Graph

5229 Commits

Author SHA1 Message Date
Reinier van der Leer
f734bdb314 fix(forge): Change task_id parameter on AgentDB.get_task from int to str
The `task_id` parameter was incorrectly typed as `int`: the database column is type `String`.
2023-12-08 13:20:20 +01:00
Reinier van der Leer
a17f752705 fix(agent): Fix artifact download method & response
- Update the return type of the `AgentProtocolServer.get_artifact` method to `StreamingResponse`.
- Fix the Content-Disposition header in the response to include quotes around the filename.
2023-12-08 12:36:51 +01:00
Reinier van der Leer
05321c9dce chore(agent): Update forge dependency 2023-12-08 12:17:56 +01:00
Reinier van der Leer
20cf4cce92 fix(forge): Initialize TaskRequestBody.additional_input as None
The `additional_input` field was being initialized as `{}`, which points to the same object for every instance of `TaskRequestBody`.
2023-12-08 12:14:40 +01:00
Reinier van der Leer
55fee0471f fix(agent): Fix setup script
- Remove `poetry install` that caused uninstall+reinstall of benchmark packages on every run
2023-12-08 12:10:07 +01:00
Reinier van der Leer
e2f760aef4 chore: Update forge dependency
- Update forge.sdk.schema import to forge.sdk.model in agent_protocol_server.py
- Update poetry.lock with new forge dependency reference
2023-12-08 01:06:06 +01:00
Reinier van der Leer
8b27f2c47b fix(forge/db): Fix persistence of Step.additional_input and Step.additional_output
- Fixed the persistence issue of additional_input and additional_output in the Step class in `forge.sdk`. The additional_input and additional_output attributes were not typed and initialized properly.
2023-12-08 00:54:38 +01:00
Reinier van der Leer
8985b7b01b fix(agent/serve): Fix artifact creation
- Link agent-created artifacts to the corresponding step
- Mark agent-created artifacts as such
2023-12-07 16:24:14 +01:00
Reinier van der Leer
1f40d72081 feat(agent/workspace): Add GCS and S3 FileWorkspace providers (#6485)
* refactor: Rename FileWorkspace to LocalFileWorkspace and create FileWorkspace abstract class
  - Rename `FileWorkspace` to `LocalFileWorkspace` to provide a more descriptive name for the class that represents a file workspace that works with local files.
  - Create a new base class `FileWorkspace` to serve as the parent class for `LocalFileWorkspace`. This allows for easier extension and customization of file workspaces in the future.
  - Update import statements and references to `FileWorkspace` throughout the codebase to use the new naming conventions.

* feat: Add S3FileWorkspace + tests + test setups for CI and Docker
  - Added S3FileWorkspace class to provide an interface for interacting with a file workspace and storing files in an S3 bucket.
  - Updated pyproject.toml to include dependencies for boto3 and boto3-stubs.
  - Implemented unit tests for S3FileWorkspace.
  - Added MinIO service to Docker CI to allow testing S3 features in CI.
  - Added autogpt-test service config to docker-compose.yml for local testing with MinIO.

* ci(docker): tee test output instead of capturing

* fix: Improve error handling in S3FileWorkspace.initialize()
  - Do not tolerate all `botocore.exceptions.ClientError`s
  - Raise the exception anyways if the error is not "NoSuchBucket"

* feat: Add S3 workspace backend support and S3Credentials
  - Added support for S3 workspace backend in the Autogpt configuration
  - Added a new sub-config `S3Credentials` to store S3 credentials
  - Modified the `.env.template` file to include variables related to S3 credentials
  - Added a new `s3_credentials` attribute on the `Config` class to store S3 credentials
  - Moved the `unmasked` method from `ModelProviderCredentials` to the parent `ProviderCredentials` class to handle unmasking for S3 credentials

* fix(agent/tests): Fix S3FileWorkspace initialization in test_s3_file_workspace.py
  - Update the S3FileWorkspace initialization in the test_s3_file_workspace.py file to include the required S3 Credentials.

* refactor: Remove S3Credentials and add get_workspace function
  - Remove `S3Credentials` as boto3 will fetch the config from the environment by itself
  - Add `get_workspace` function in `autogpt.file_workspace` module
  - Update `.env.template` and tests to reflect the changes

* feat(agent/workspace): Make agent workspace backend configurable
  - Modified `autogpt.file_workspace.get_workspace` function to either take a workspace `id` or `root_path`.
  - Modified `FileWorkspaceMixin` to use the `get_workspace` function to set up the workspace.
  - Updated the type hints and imports accordingly.

* feat(agent/workspace): Add GCSFileWorkspace for Google Cloud Storage
  - Added support for Google Cloud Storage as a storage backend option in the workspace.
  - Created the `GCSFileWorkspace` class to interface with a file workspace stored in a Google Cloud Storage bucket.
  - Implemented the `GCSFileWorkspaceConfiguration` class to handle the configuration for Google Cloud Storage workspaces.
  - Updated the `get_workspace` function to include the option to use Google Cloud Storage as a workspace backend.
  - Added unit tests for the new `GCSFileWorkspace` class.

* fix: Unbreak use of non-local workspaces in AgentProtocolServer
  - Modify the `_get_task_agent_file_workspace` method to handle both local and non-local workspaces correctly
2023-12-07 14:46:08 +01:00
Thomas Runyon
fdd7f8e5f9 Update import path in agent_protocol.py (#6512)
Update agent_protocol.py

Fixed import location of models
2023-12-07 08:44:07 +01:00
SwiftyOS
2c4184b81e Forge: Moved memory and llm out of the sdk 2023-12-06 15:34:08 +01:00
SwiftyOS
ccb700d00a deleted extra files 2023-12-06 15:31:20 +01:00
Swifty
78f5ff17e6 Add GCSWorkspace Implementation to Forge (#6510)
* Added gcs workspace implementation

* Renamed abilites to actions (#6511)

* Renamed abilites to actions

* formatting

* schema to model
2023-12-06 15:30:21 +01:00
SwiftyOS
4c495ce1b0 Remove unnecessary code in cli.py 2023-12-06 14:25:38 +01:00
Reinier van der Leer
c47f81a3e0 chore: Replace deprecated PyPDF2 with pypdf
- Replace deprecated PyPDF2 with successor pypdf to get rid of deprecation warnings and keep receiving updates.
2023-12-05 18:41:52 +01:00
Reinier van der Leer
ffeb45eda3 refactor: Replace deprecated logger.warn calls with logger.warning
- Replaced all instances of logger.warn with logger.warning to get rid of deprecation warnings.
2023-12-05 18:41:12 +01:00
SwiftyOS
0d91006e0b removed entering the arena from the getting started tutorial 2023-12-05 18:00:05 +01:00
Auto-GPT-Bot
a38ddc395b Update cassette submodule 2023-12-05 15:30:16 +00:00
Reinier van der Leer
7b05245286 refactor(agent/config): Modularize Config and revive Azure support (#6497)
* feat: Refactor config loading and initialization to be modular and decentralized

   - Refactored the `ConfigBuilder` class to support modular loading and initialization of the configuration from environment variables.
   - Implemented recursive loading and initialization of nested config objects.
   - Introduced the `SystemConfiguration` base class to provide common functionality for all system settings.
   - Added the `from_env` attribute to the `UserConfigurable` decorator to provide environment variable mappings.
   - Updated the `Config` class and its related classes to inherit from `SystemConfiguration` and use the `UserConfigurable` decorator.
   - Updated `LoggingConfig` and `TTSConfig` to use the `UserConfigurable` decorator for their fields.
   - Modified the implementation of the `build_config_from_env` method in `ConfigBuilder` to utilize the new modular and recursive loading and initialization logic.
   - Updated applicable test cases to reflect the changes in the config loading and initialization logic.

   This refactor improves the flexibility and maintainability of the configuration loading process by introducing modular and recursive behavior, allowing for easier extension and customization through environment variables.

* refactor: Move OpenAI credentials into `OpenAICredentials` sub-config

   - Move OpenAI API key and other OpenAI credentials from the global config to a new sub-config called OpenAICredentials.
   - Update the necessary code to use the new OpenAICredentials sub-config instead of the global config when accessing OpenAI credentials.
   - (Hopefully) unbreak Azure support.
      - Update azure.yaml.template.
   - Enable validation of assignment operations on SystemConfiguration and SystemSettings objects.

* feat: Update AutoGPT configuration options and setup instructions

   - Added new configuration options for logging and OpenAI usage to .env.template
   - Removed deprecated configuration options in config/config.py
   - Updated setup instructions in Docker and general setup documentation to include information on using Azure's OpenAI services

* fix: Fix image generation with Dall-E

   - Fix issue with image generation with Dall-E API

Additional user context: This commit fixes an issue with image generation using the Dall-E API. The code now correctly retrieves the API key from the agent's legacy configuration.

* refactor(agent/core): Refactor `autogpt.core.configuration.schema` and update docstrings

   - Refactor the `schema.py` file in the `autogpt.core.configuration` module.
   - Added docstring to `SystemConfiguration.from_env()`
   - Updated docstrings for functions `_get_user_config_values`, `_get_non_default_user_config_values`, `_recursive_init_model`, `_recurse_user_config_fields`, and `_recurse_user_config_values`.
2023-12-05 16:28:23 +01:00
Reinier van der Leer
03eb921ca6 Roll back cassette updates 2023-12-05 16:25:36 +01:00
Reinier van der Leer
47dc5ad220 fix(test): Fix test that was being skipped because of missing async marker
- Fix a test case in test_web_selenium.py that was failing due to a missing `pytest.mark.asyncio`
2023-12-05 15:34:57 +01:00
Reinier van der Leer
b73fa47089 fix: Exclude Cookie header from VCR matching
- Exclude Cookie header from VCR matching to prevent cache misses caused by CloudFlare cookies.
2023-12-05 15:33:47 +01:00
Reinier van der Leer
775a27eea0 refactor: Modify instruction in prompt for increased JSON output reliability
- Update the instruction in the prompt strategy to ensure the response is pure JSON.
- Remove unnecessary text and make the instruction clearer.
- Also update the error logging to include the received JSON content.

This commit refactors the code in the `one_shot.py` file and the `utilities.py` file.
2023-12-05 15:21:04 +01:00
Reinier van der Leer
b8b792e844 ci: Fix issue in Docker CI
* Stop Docker CI pushing images from PR workflow runs
2023-12-03 17:35:03 +01:00
Auto-GPT-Bot
b1419e856c Update cassette submodule 2023-12-02 14:41:59 +00:00
Reinier van der Leer
760e0a0816 fix: Fix agent poetry env usage in pre-commit hook
- Update the pytest command in the .pre-commit-config.yaml file to use Poetry run instead of directly running pytest in the autogpts/autogpt directory.
2023-12-02 15:39:22 +01:00
Reinier van der Leer
ef35702c4b refactor: Reduce breakage in vector memory module and split MemoryItem class
- Refactored the `MemoryItem` class in the `autogpt.memory.vector.memory_item` module to improve code organization and readability.
- Split the `MemoryItem` class into two separate classes: `MemoryItem` and `MemoryItemFactory`.
- Modified the `get_embedding` function in the `autogpt.memory.vector.utils` module to accept an `EmbeddingModelProvider` for creating embeddings.
- Updated the usage of the `get_embedding` function in the `MemoryItem` class to pass the `embedding_provider` parameter.
- Updated the imports in the affected modules.
2023-12-02 15:38:19 +01:00
Reinier van der Leer
6d439f4f63 fix: Fix check for optional dependencies in check_requirements.py
- Modify check_requirements.py to correctly handle optional dependencies
- Skip optional dependencies when iterating through dependence group dependencies in check_requirements.py
2023-12-02 14:20:11 +01:00
Auto-GPT-Bot
dedab125e1 Update cassette submodule 2023-12-02 13:18:45 +00:00
Reinier van der Leer
6743636996 fix: Fix poetry env issues with the agent's entrypoint scripts
- Update autogpt.bat to use `poetry install` instead of `%PYTHON_CMD% -m poetry install`
- Update autogpt.sh to use `poetry install` instead of `$PYTHON_CMD -m poetry install`
- Use `poetry run` to execute the `autogpt` command in both scripts
2023-12-02 14:15:18 +01:00
Reinier van der Leer
4bcfe72485 fix: fix VCR submodule reference
- Update the reference to the VCR submodule in the autogpt tests
- Previous reference: 1896d8ac12ff1d27b7e9e5db6549abc38b260b40
- New reference: 9996f1d104a1e4f33c1e10aa664d01ea78db2a06
2023-12-02 13:57:50 +01:00
Reinier van der Leer
2c70059c2d fix(agent): Fix setup and run scripts
- Updated the `run` script to also check if `$OPENAI_API_KEY` is empty before copying `.env.example` and prompting the user to set API keys.
- Modified the `setup` script to install `--extras benchmark` separately from the initial `poetry install` command.
- Added `POETRY_INSTALLER_PARALLEL=false` flag to prevent conflicts between `forge` and `agbenchmark` during installation.
2023-12-02 13:48:45 +01:00
Reinier van der Leer
5ca48de07f fix(agent/tests): Fix code execution tests
- Add unique agent ID to mock agent in `conftest.py`
2023-12-02 13:27:36 +01:00
Reinier van der Leer
38f436aa6d test: fix test_huggingface_fail_request_with_delay
- Fix failing test_huggingface_fail_request_with_delay by adding mock API key to agent configuration.
2023-12-02 13:15:23 +01:00
Reinier van der Leer
ba8f83d09d fix: Fix get_current_git_branch test
- Use `InvalidGitRepositoryError` instead of generic `Exception` in `test_get_current_git_branch_failure` to accurately simulate a failure scenario.
2023-12-02 07:02:14 +01:00
Reinier van der Leer
fc429408a3 fix: Update isort config 2023-12-02 06:54:14 +01:00
Reinier van der Leer
e44d70d6d3 fix: Fix pre-commit config for pytest
- Modify the entry command to change directories before running pytest
2023-12-02 06:23:25 +01:00
Reinier van der Leer
fe96f6d783 refactor: Disable mypy and autoflake in CI and pre-commit hook
- Commented out the mypy and autoflake checks in the CI workflow and pre-commit config files.
- The intention is to enable the mypy check again later.
2023-12-02 06:11:51 +01:00
Reinier van der Leer
c072c7d008 refactor: Move isort and mypy configuration into pyproject.toml
- Deleted .isort.cfg and mypy.ini
- Updated pyproject.toml to include isort and mypy configuration
2023-12-02 06:10:45 +01:00
Reinier van der Leer
d938c2595e refactor(agent): Fix all trivial linting errors
* Fix all but one flake8 linting errors
  * Remove unused imports
  * Wrap strings that are too long
  * Add basic autogpts/autogpt/.flake8
* Delete planning_agent.py
* Delete default_prompts.py
* Delete _test_json_parser.py
* Refactor the example function call in AgentProfileGeneratorConfiguration from a string to an object
* Rewrite/update docstrings here and there while I'm at it
* Minor change to the description of the `open_file` command
* Use `user-agent` from config in web_selenium.py
* Delete hardcoded ABILITIES from core/planning/templates.py
* Delete duplicate and superseded test from test_image_gen.py
* Fix parameter definitions in mock_commands.py
* Delete code analysis blocks from test_spinner.py, test_url_validation.py
2023-12-02 05:42:10 +01:00
Reinier van der Leer
604217a957 fix(agent/docker): Fix venv issue fix for Docker build on Google Cloud 2023-12-01 10:39:43 +01:00
Reinier van der Leer
207201d907 fix: Fix venv issue with Docker build on Google Cloud
See also https://stackoverflow.com/a/70340377
2023-11-30 18:39:42 +01:00
Reinier van der Leer
3ad7a2cfaa fix: Fix URL validation test
- Modify the test_url_validation_fails_local_path function to remove the specific match parameter and raise the ValueError without any match requirement.
2023-11-30 18:03:32 +01:00
Reinier van der Leer
6c69e16f31 fix: Fix CI failing tests
- Update `test_config.py` to check if `config.smart_llm` starts with "gpt-4"
- Delete `test_retry_provider_openai.py` as it is no longer needed
- Update `test_url_validation.py` to properly test local file URLs
- Update `test_web_search.py` to assert against expected parts of output
2023-11-30 17:56:05 +01:00
Reinier van der Leer
a771b2c6c1 ci: Push dev build in Docker CI if tests succeed
- Added Docker Hub authentication in the CI workflow
- Added a new step to push the dev build to Docker Hub
2023-11-30 17:45:09 +01:00
Reinier van der Leer
99fe114502 refactor: Make Agent Protocol server DB URL configurable
- Refactor the `run_auto_gpt_server` function to make the Agent Protocol server database URL configurable.
- Use the `os.getenv` method to retrieve the database URL from the environment variable `AP_SERVER_DB_URL`.
2023-11-30 17:26:16 +01:00
Reinier van der Leer
ca6e9e5e34 lint: Address linting issues from #5458
- Fixed linting issues in `json_utils/utilities.py` and `test_json_utils.py`
2023-11-30 17:20:42 +01:00
Reinier van der Leer
ae1e030824 feat: Implement LoggingConfig and structured logging
- Created a new `LoggingConfig` class to represent the logging configuration in the `Config` class.
- Created a new `LogFormatName` enum to represent the available log formats: 'simple', 'debug', and 'structured_google_cloud'.
- Modified the `configure_logging` function to also accept an unpacked `LoggingConfig` object for arguments.
- Updated the `configure_logging` function to use the appropriate log format based on the log level.
- Added a `StructuredLoggingFormatter` class to handle formatting for structured logs.
- Updated the import statements and usages of `configure_logging` etc. in relevant modules to reflect the changes.
- Updated the `config` fixture in the unit tests to include the new logging configuration attributes.
- Updated the CLI with new parameters for log level and format.
- Reordered the parameters of the CLI.
- Removed memory related parameter from CLI.
2023-11-30 17:12:58 +01:00
Reinier van der Leer
c65d6f7ff8 refactor: Remove unused debug argument from plugin setup helpers
- Remove the unused debug argument from the functions `inspect_zip_for_modules`, `initialize_openai_plugins`, `instantiate_openai_plugin_clients`, and `scan_plugins`.
- The debug argument was not being used in these functions and was unnecessary.
2023-11-30 16:17:31 +01:00
Jordan Ellis
0e332c0c12 feat: add evo.ninja (setup only) (#6379) 2023-11-27 17:29:32 +01:00