From 1b2831ab852bd5cab87d36eaab24704e66a3a1f1 Mon Sep 17 00:00:00 2001 From: Plamen Ivanov <65338982+k1lgor@users.noreply.github.com> Date: Fri, 23 Jun 2023 13:01:29 +0300 Subject: [PATCH] Refactoring and suggestions (#334) --- .github/CODE_OF_CONDUCT.md | 4 +- .github/CONTRIBUTING.md | 10 ++-- .github/ISSUE_TEMPLATE/issue-template.md | 3 +- .../PULL_REQUEST_TEMPLATE.md | 1 + .github/workflows/ci.yaml | 16 +++--- .github/workflows/pre-commit.yaml | 14 ++++-- .pre-commit-config.yaml | 50 +++++++++---------- Makefile | 1 - README.md | 16 +++--- ROADMAP.md | 6 +-- benchmark/RESULTS.md | 16 +++--- benchmark/currency_converter/main_prompt | 2 +- benchmark/file_explorer/main_prompt | 1 - benchmark/file_organizer/main_prompt | 2 +- benchmark/image_resizer/main_prompt | 1 - benchmark/markdown_editor/main_prompt | 2 +- benchmark/password_generator/main_prompt | 2 +- benchmark/pomodoro_timer/main_prompt | 2 +- benchmark/timer_app/main_prompt | 1 - benchmark/todo_list/main_prompt | 1 - benchmark/url_shortener/main_prompt | 2 +- benchmark/weather_app/main_prompt | 2 +- gpt_engineer/preprompts/fix_code | 2 +- gpt_engineer/preprompts/qa | 2 +- gpt_engineer/preprompts/respec | 2 +- gpt_engineer/preprompts/spec | 2 +- requirements.txt | 1 - scripts/benchmark.py | 5 +- 28 files changed, 82 insertions(+), 87 deletions(-) diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md index 337a651..7713a6b 100644 --- a/.github/CODE_OF_CONDUCT.md +++ b/.github/CODE_OF_CONDUCT.md @@ -60,7 +60,7 @@ representative at an online or offline event. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at -anton.osika@gmail.com. +. All complaints will be reviewed and investigated promptly and fairly. All community leaders are obligated to respect the privacy and security of reporters of incidents. @@ -128,4 +128,4 @@ For answers to common questions about this code of conduct, see the FAQ at [v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html [Mozilla CoC]: https://github.com/mozilla/diversity [FAQ]: https://www.contributor-covenant.org/faq -[translations]: https://www.contributor-covenant.org/translations \ No newline at end of file +[translations]: https://www.contributor-covenant.org/translations diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 9c43e5d..242ef38 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -15,7 +15,6 @@ To get started with contributing, please follow these steps: 7. Push to the branch: `git push origin my-branch-name`. 8. Submit a pull request to the `main` branch of the original repository. - ## Code Style Please make sure to follow the established code style guidelines for this project. Consistent code style helps maintain readability and makes it easier for others to contribute to the project. @@ -37,13 +36,13 @@ pre-commit installed at .git/hooks/pre-commit Or you could just run `make dev-install` to install the dependencies and the hooks! - If you are not familiar with the concept of [git hooks](https://git-scm.com/docs/githooks) and/or [`pre-commit`](https://pre-commit.com/) please read the documentation to understand how they work. As an introduction of the actual workflow, here is an example of the process you will encounter when you make a commit: Let's add a file we have modified with some errors, see how the pre-commit hooks run `black` and fails. `black` is set to automatically fix the issues it finds: + ```bash $ git add chat_to_files.py $ git commit -m "commit message" @@ -58,6 +57,7 @@ All done! ✨ 🍰 ✨ ``` You can see that `chat_to_files.py` is both staged and not staged for commit. This is because `black` has formatted it and now it is different from the version you have in your working directory. To fix this you can simply run `git add chat_to_files.py` again and now you can commit your changes. + ```bash $ git status On branch pre-commit-setup @@ -71,8 +71,8 @@ Changes not staged for commit: modified: chat_to_files.py ``` - Now let's add the file again to include the latest commits and see how `ruff` fails. + ```bash $ git add chat_to_files.py $ git commit -m "commit message" @@ -86,6 +86,7 @@ Found 2 errors (2 fixed, 0 remaining). ``` Same as before, you can see that `chat_to_files.py` is both staged and not staged for commit. This is because `ruff` has formatted it and now it is different from the version you have in your working directory. To fix this you can simply run `git add chat_to_files.py` again and now you can commit your changes. + ```bash $ git add chat_to_files.py $ git commit -m "commit message" @@ -100,13 +101,10 @@ Now your file has been committed and you can push your changes. At the beginning this might seem like a tedious process (having to add the file again after `black` and `ruff` have modified it) but it is actually very useful. It allows you to see what changes `black` and `ruff` have made to your files and make sure that they are correct before you commit them. - - ## Issue Tracker If you encounter any bugs, issues, or have feature requests, please [create a new issue](https://github.com/AntonOsika/gpt-engineer/issues/new) on the project's GitHub repository. Provide a clear and descriptive title along with relevant details to help us address the problem or understand your request. - ## Licensing By contributing to GPT Engineer, you agree that your contributions will be licensed under the [LICENSE](../LICENSE) file of the project. diff --git a/.github/ISSUE_TEMPLATE/issue-template.md b/.github/ISSUE_TEMPLATE/issue-template.md index 720132d..e6b670a 100644 --- a/.github/ISSUE_TEMPLATE/issue-template.md +++ b/.github/ISSUE_TEMPLATE/issue-template.md @@ -4,12 +4,11 @@ about: All templates should use this format unless there is a reason not to title: '' labels: '' assignees: '' - --- **YOU MAY DELETE THE ENTIRE TEMPLATE BELOW.** -# Issue Template +## Issue Template ## Expected Behavior diff --git a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md index 8a9cae2..8343ddb 100644 --- a/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE/PULL_REQUEST_TEMPLATE.md @@ -3,6 +3,7 @@ ## How Has This Been Tested? Please describe if you have either: + - Generated the "example" project - Ran the entire benchmark suite - Something else diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 5b018df..97dde53 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -1,25 +1,23 @@ name: Pytest Execution on: pull_request: - branches: - - main + branches: [main] + paths: + - "**.py" push: - branches: - - main + branches: [main] + paths: + - "**.py" jobs: test: runs-on: ubuntu-latest - strategy: - matrix: - python-version: - - "3.10" steps: - uses: actions/checkout@v3 - uses: actions/setup-python@v4 with: - python-version: ${{ matrix.python-version }} + python-version: "3.10" cache: pip - name: Install package diff --git a/.github/workflows/pre-commit.yaml b/.github/workflows/pre-commit.yaml index ad71791..1ef45cc 100644 --- a/.github/workflows/pre-commit.yaml +++ b/.github/workflows/pre-commit.yaml @@ -8,7 +8,15 @@ on: jobs: pre-commit: runs-on: ubuntu-latest + + permissions: + contents: write + steps: - - uses: actions/checkout@v3 - - uses: actions/setup-python@v4 - - uses: pre-commit/action@v3.0.0 + - uses: actions/checkout@v3 + + - uses: actions/setup-python@v4 + + - uses: pre-commit/action@v3.0.0 + with: + extra_args: --all-files diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index f532737..1d94807 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,34 +1,32 @@ # See https://pre-commit.com for more information # See https://pre-commit.com/hooks.html for more hooks fail_fast: true - +default_stages: [commit] repos: + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.3.0 + hooks: + - id: mypy -- repo: https://github.com/pre-commit/mirrors-mypy - rev: v1.3.0 - hooks: - - id: mypy + - repo: https://github.com/psf/black + rev: 23.3.0 + hooks: + - id: black + args: [--config, pyproject.toml] + types: [python] -- repo: https://github.com/psf/black - rev: 23.3.0 - hooks: - - id: black - args: [--config, pyproject.toml] - types: [python] - -- repo: https://github.com/charliermarsh/ruff-pre-commit - rev: "v0.0.272" - hooks: - - id: ruff - args: [--fix, --exit-non-zero-on-fix] - -- repo: https://github.com/pre-commit/pre-commit-hooks - rev: v4.4.0 - hooks: - - id: check-toml - id: check-yaml - id: detect-private-key - id: end-of-file-fixer - id: trailing-whitespace + - repo: https://github.com/charliermarsh/ruff-pre-commit + rev: "v0.0.272" + hooks: + - id: ruff + args: [--fix, --exit-non-zero-on-fix] + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.4.0 + hooks: + - id: check-toml + - id: check-yaml + - id: detect-private-key + - id: end-of-file-fixer + - id: trailing-whitespace diff --git a/Makefile b/Makefile index c99a5cf..98d9213 100644 --- a/Makefile +++ b/Makefile @@ -50,4 +50,3 @@ run: @echo -e "$(COLOR_CYAN)Running GPT Engineer on $(COLOR_GREEN)$(name)$(COLOR_CYAN) folder...$(COLOR_RESET)" && \ source venv/bin/activate && \ gpt-engineer projects/$(name) - diff --git a/README.md b/README.md index 0c624f0..9a3620f 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,9 @@ # GPT Engineer + [![Discord Follow](https://dcbadge.vercel.app/api/server/4t5vXHhu?style=flat)](https://discord.gg/4t5vXHhu) [![GitHub Repo stars](https://img.shields.io/github/stars/AntonOsika/gpt-engineer?style=social)](https://github.com/AntonOsika/gpt-engineer) [![Twitter Follow](https://img.shields.io/twitter/follow/antonosika?style=social)](https://twitter.com/AntonOsika) - **Specify what you want it to build, the AI asks for clarification, and then builds it.** GPT Engineer is made to be easy to adapt, extend, and make your agent learn how you want your code to look. It generates an entire codebase based on a prompt. @@ -11,6 +11,7 @@ GPT Engineer is made to be easy to adapt, extend, and make your agent learn how [Demo](https://twitter.com/antonosika/status/1667641038104674306) πŸ‘ΆπŸ€– ## Project philosophy + - Simple to get value - Flexible and easy to add new own "AI steps". See `steps.py`. - Incrementally build towards a user experience of: @@ -39,19 +40,20 @@ With an api key that has GPT4 access run: - `export OPENAI_API_KEY=[your api key]` - **Run**: + - Create an empty folder. If inside the repo, you can run: - `cp -r projects/example/ projects/my-new-project` - Fill in the `main_prompt` file in your new folder - `gpt-engineer projects/my-new-project` - (Note, `gpt-engineer --help` lets you see all available options. For example `--steps use_feedback` lets you improve/fix code in a project) -**Results** +**Results**: + - Check the generated files in `projects/my-new-project/workspace` - ## Features + You can specify the "identity" of the AI agent by editing the files in the `identity` folder. Editing the identity, and evolving the `main_prompt`, is currently how you make the agent remember things between projects. @@ -59,13 +61,13 @@ Editing the identity, and evolving the `main_prompt`, is currently how you make Each step in `steps.py` will have its communication history with GPT4 stored in the logs folder, and can be rerun with `scripts/rerun_edited_message_logs.py`. ## Contributing + We are building the open platform for devs to tinker with and build their personal code-generation toolbox. If you want to contribute, please check out the [roadmap](https://github.com/AntonOsika/gpt-engineer/blob/main/ROADMAP.md), [projects](https://github.com/AntonOsika/gpt-engineer/projects?query=is%3Aopen) or [issues tab](https://github.com/AntonOsika/gpt-engineer/issues) in the GitHub repo. You are welcome to read the [contributing document](.github/CONTRIBUTING.md) and join our [Discord πŸ’¬](https://discord.gg/4t5vXHhu). -We are currently looking for more maintainers and community organisers. Email anton.osika@gmail.com if you are interested in an official role. - +We are currently looking for more maintainers and community organisers. Email if you are interested in an official role. ## Example -https://github.com/AntonOsika/gpt-engineer/assets/4467025/6e362e45-4a94-4b0d-973d-393a31d92d9b + diff --git a/ROADMAP.md b/ROADMAP.md index f735afe..245bd72 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -7,7 +7,6 @@ There are three main milestones we believe will 2x gpt-engineer's reliability an - Make code generation become small, verifiable steps - Run tests and fix errors with GPT4 - ## Steps to achieve our roadmap - [ ] Continuous evaluation of our progress @@ -32,10 +31,10 @@ performance over time task, by giving it few shot example of what are usually "the right-sized steps" to carry out for other projects - - # How you can help out + You can: + - Submit your first PR to address an [issue](https://github.com/AntonOsika/gpt-engineer/issues) - Submit PRs to address one of the items in the roadmap - Review your first PR/issue and propose next steps (further review, merge, close) @@ -48,4 +47,3 @@ Volunteer work in any of these gets acknowledged. ### Ad hoc experiments - [ ] Try Microsoft guidance, and benchmark if this helps improve performance - diff --git a/benchmark/RESULTS.md b/benchmark/RESULTS.md index ce3f72b..200cfc2 100644 --- a/benchmark/RESULTS.md +++ b/benchmark/RESULTS.md @@ -1,10 +1,10 @@ # Benchmarks ```bash -$ python scripts/benchmark.py +python scripts/benchmark.py ``` -# 2023-06-21 +## 2023-06-21 | Benchmark | Ran | Works | Perfect | |--------------------|-----|-------|---------| @@ -20,8 +20,8 @@ $ python scripts/benchmark.py | password_generator | βœ… | βœ… | βœ… | | todo_list | βœ… | βœ… | βœ… | +## Notes on the errors -# Notes on the errors Most errors come from that the "generate entrypoint" are incorrect. Ignoring those, we get 8/11 fully correct. @@ -31,8 +31,7 @@ One error was trying to modify a constant. One error was that the html template was not fully filled in. One error is that a dependency was used incorrectly and easy to fix - -# 2023-06-19 +## 2023-06-19 | Benchmark | Ran | Works | Perfect | |--------------------|-----|-------|---------| @@ -48,9 +47,10 @@ One error is that a dependency was used incorrectly and easy to fix | password_generator | βœ… | βœ… | βœ… | | todo_list | βœ… | ❌ | ❌ | +## Notes on the errors -# Notes on the errors **timer_app** almost works with unit tests config + - failure mode: undefined import/conflicting names **file_explorer** works @@ -58,17 +58,17 @@ One error is that a dependency was used incorrectly and easy to fix **file organiser**: works **image_resizer** almost works with unit tests config + - failure mode: undefined import **todo_list** runs. doesn't really work with unit tests config Uncaught ReferenceError: module is not defined -- failure mode: placeholder text +- failure mode: placeholder text url_shortner starts but gets the error: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 8636125824 and this is thread id 13021003776. - markdown_editor: failing tests, 'WebDriver' object has no attribute 'find_element_by_id' diff --git a/benchmark/currency_converter/main_prompt b/benchmark/currency_converter/main_prompt index 25b9156..61d785b 100644 --- a/benchmark/currency_converter/main_prompt +++ b/benchmark/currency_converter/main_prompt @@ -1 +1 @@ -Build a currency converter app using an API for exchange rates. Use HTML, CSS, and JavaScript for the frontend and Node.js for the backend. Allow users to convert between different currencies. \ No newline at end of file +Build a currency converter app using an API for exchange rates. Use HTML, CSS, and JavaScript for the frontend and Node.js for the backend. Allow users to convert between different currencies. diff --git a/benchmark/file_explorer/main_prompt b/benchmark/file_explorer/main_prompt index 20b3002..eed5cdf 100644 --- a/benchmark/file_explorer/main_prompt +++ b/benchmark/file_explorer/main_prompt @@ -1,2 +1 @@ Create a basic file explorer CLI tool in Python that allows users to navigate through directories, view file contents, and perform basic file operations (copy, move, delete). - diff --git a/benchmark/file_organizer/main_prompt b/benchmark/file_organizer/main_prompt index 00f5045..8319ce1 100644 --- a/benchmark/file_organizer/main_prompt +++ b/benchmark/file_organizer/main_prompt @@ -1 +1 @@ -Create a file organizer CLI tool in Python that sorts files in a directory based on their file types (e.g., images, documents, audio) and moves them into corresponding folders. \ No newline at end of file +Create a file organizer CLI tool in Python that sorts files in a directory based on their file types (e.g., images, documents, audio) and moves them into corresponding folders. diff --git a/benchmark/image_resizer/main_prompt b/benchmark/image_resizer/main_prompt index dc1821f..49a36a7 100644 --- a/benchmark/image_resizer/main_prompt +++ b/benchmark/image_resizer/main_prompt @@ -1,2 +1 @@ Create a CLI tool in Python that allows users to resize images by specifying the desired width and height. Use the Pillow library for image manipulation. - diff --git a/benchmark/markdown_editor/main_prompt b/benchmark/markdown_editor/main_prompt index 1058c0d..9cfb42c 100644 --- a/benchmark/markdown_editor/main_prompt +++ b/benchmark/markdown_editor/main_prompt @@ -1 +1 @@ -Build a simple markdown editor using HTML, CSS, and JavaScript. Allow users to input markdown text and display the formatted output in real-time. \ No newline at end of file +Build a simple markdown editor using HTML, CSS, and JavaScript. Allow users to input markdown text and display the formatted output in real-time. diff --git a/benchmark/password_generator/main_prompt b/benchmark/password_generator/main_prompt index cd4c5b7..f29b890 100644 --- a/benchmark/password_generator/main_prompt +++ b/benchmark/password_generator/main_prompt @@ -1 +1 @@ -Create a password generator CLI tool in Python that generates strong, random passwords based on user-specified criteria, such as length and character types (letters, numbers, symbols). \ No newline at end of file +Create a password generator CLI tool in Python that generates strong, random passwords based on user-specified criteria, such as length and character types (letters, numbers, symbols). diff --git a/benchmark/pomodoro_timer/main_prompt b/benchmark/pomodoro_timer/main_prompt index 4e713de..8527372 100644 --- a/benchmark/pomodoro_timer/main_prompt +++ b/benchmark/pomodoro_timer/main_prompt @@ -1 +1 @@ -Develop a Pomodoro timer app using HTML, CSS, and JavaScript. Allow users to set work and break intervals and receive notifications when it's time to switch. \ No newline at end of file +Develop a Pomodoro timer app using HTML, CSS, and JavaScript. Allow users to set work and break intervals and receive notifications when it's time to switch. diff --git a/benchmark/timer_app/main_prompt b/benchmark/timer_app/main_prompt index a30ea11..102ae6f 100644 --- a/benchmark/timer_app/main_prompt +++ b/benchmark/timer_app/main_prompt @@ -1,2 +1 @@ Create a simple timer app using HTML, CSS, and JavaScript that allows users to set a countdown timer and receive an alert when the time is up. - diff --git a/benchmark/todo_list/main_prompt b/benchmark/todo_list/main_prompt index 13af386..57308b1 100644 --- a/benchmark/todo_list/main_prompt +++ b/benchmark/todo_list/main_prompt @@ -1,2 +1 @@ Create a simple to-do list app using HTML, CSS, and JavaScript. Store tasks in local storage and allow users to add, edit, and delete tasks. - diff --git a/benchmark/url_shortener/main_prompt b/benchmark/url_shortener/main_prompt index b08c15e..6bd2143 100644 --- a/benchmark/url_shortener/main_prompt +++ b/benchmark/url_shortener/main_prompt @@ -1 +1 @@ -Create a URL shortener app using HTML, CSS, JavaScript, and a backend language like Python or Node.js. Allow users to input a long URL and generate a shortened version that redirects to the original URL. Store the shortened URLs in a database. \ No newline at end of file +Create a URL shortener app using HTML, CSS, JavaScript, and a backend language like Python or Node.js. Allow users to input a long URL and generate a shortened version that redirects to the original URL. Store the shortened URLs in a database. diff --git a/benchmark/weather_app/main_prompt b/benchmark/weather_app/main_prompt index 8294c57..e58cab3 100644 --- a/benchmark/weather_app/main_prompt +++ b/benchmark/weather_app/main_prompt @@ -1 +1 @@ -Develop a weather app using Python and a weather API. Display current weather conditions for a given location, including temperature, humidity, and weather description. \ No newline at end of file +Develop a weather app using Python and a weather API. Display current weather conditions for a given location, including temperature, humidity, and weather description. diff --git a/gpt_engineer/preprompts/fix_code b/gpt_engineer/preprompts/fix_code index b71f636..884046a 100644 --- a/gpt_engineer/preprompts/fix_code +++ b/gpt_engineer/preprompts/fix_code @@ -1,3 +1,3 @@ You are a super smart developer. You have been tasked with fixing a program and making it work according to the best of your knowledge. There might be placeholders in the code you have to fill in. You provide fully functioning, well formatted code with few comments, that works and has no bugs. -Please return the full new code in the same format. \ No newline at end of file +Please return the full new code in the same format. diff --git a/gpt_engineer/preprompts/qa b/gpt_engineer/preprompts/qa index 58c3508..f4f9934 100644 --- a/gpt_engineer/preprompts/qa +++ b/gpt_engineer/preprompts/qa @@ -1,3 +1,3 @@ You will read instructions and not carry them out, only seek to clarify them. Specifically you will first summarise a list of super short bullets of areas that need clarification. -Then you will pick one clarifying question, and wait for an answer from the user. \ No newline at end of file +Then you will pick one clarifying question, and wait for an answer from the user. diff --git a/gpt_engineer/preprompts/respec b/gpt_engineer/preprompts/respec index ac907d7..a8994fc 100644 --- a/gpt_engineer/preprompts/respec +++ b/gpt_engineer/preprompts/respec @@ -9,4 +9,4 @@ You have been asked to give feedback on the following: You are asked to make educated assumptions for each unclear item. For each of these, communicate which assumptions you'll make when implementing the feature. -Think step by step to make sure we don't miss anything. \ No newline at end of file +Think step by step to make sure we don't miss anything. diff --git a/gpt_engineer/preprompts/spec b/gpt_engineer/preprompts/spec index 690eb53..7be78bd 100644 --- a/gpt_engineer/preprompts/spec +++ b/gpt_engineer/preprompts/spec @@ -7,4 +7,4 @@ and give details about anything that might be unclear. **Don't leave anything un Second, lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose. -This specification will be used later as the basis for the implementation. \ No newline at end of file +This specification will be used later as the basis for the implementation. diff --git a/requirements.txt b/requirements.txt index ffc5c53..1704ced 100644 --- a/requirements.txt +++ b/requirements.txt @@ -6,4 +6,3 @@ pytest==7.3.1 ruff==0.0.272 termcolor==2.3.0 typer==0.9.0 - diff --git a/scripts/benchmark.py b/scripts/benchmark.py index ac130bb..66ebb71 100644 --- a/scripts/benchmark.py +++ b/scripts/benchmark.py @@ -1,6 +1,7 @@ # list all folders in benchmark folder # for each folder, run the benchmark +import contextlib import os import subprocess @@ -59,7 +60,7 @@ def main( print(f.read()) print() - try: + with contextlib.suppress(KeyboardInterrupt): subprocess.run( [ "python", @@ -70,8 +71,6 @@ def main( "execute_only", ], ) - except KeyboardInterrupt: - pass if __name__ == "__main__":