Update benchmarks

gpt_engineer/db.py

@@ -3,41 +3,29 @@ from pathlib import Path
 
 
 # This class represents a simple database that stores its data as files in a directory.
-# It supports both text and binary files, and can handle directory structures.
 class DB:
+    """A simple key-value store, where keys are filenames and values are file contents."""
+
     def __init__(self, path):
-        # Convert the path string to a Path object and get its absolute path.
         self.path = Path(path).absolute()
 
-        # Create the directory if it doesn't exist.
         self.path.mkdir(parents=True, exist_ok=True)
 
     def __getitem__(self, key):
-        # Combine the database directory with the provided file path.
         full_path = self.path / key
 
-        # Check if the file exists before trying to open it.
         if full_path.is_file():
-            # Open the file in text mode and return its content.
+            with full_path.open("r", encoding="utf-8") as f:
-            with full_path.open("r") as f:
                 return f.read()
         else:
-            # If the file doesn't exist, raise an error.
+            raise KeyError(key)
-            raise FileNotFoundError(f"No such file: '{full_path}'")
 
     def __setitem__(self, key, val):
-        # Combine the database directory with the provided file path.
         full_path = self.path / key
-
-        # Create the directory tree if it doesn't exist.
         full_path.parent.mkdir(parents=True, exist_ok=True)
 
-        # Write the data to the file. If val is a string, it's written as text.
-        # If val is bytes, it's written as binary data.
         if isinstance(val, str):
-            full_path.write_text(val)
+            full_path.write_text(val, encoding="utf-8")
-        elif isinstance(val, bytes):
-            full_path.write_bytes(val)
         else:
             # If val is neither a string nor bytes, raise an error.
             raise TypeError("val must be either a str or bytes")

gpt_engineer/steps.py

@@ -157,9 +157,9 @@ def gen_entrypoint(ai, dbs):
     messages = ai.start(
         system=(
             "You will get information about a codebase that is currently on disk in "
-            f"the folder {dbs.workspace.path}.\n"
+            "the current folder.\n"
             "From this you will answer with code blocks that includes all the necessary "
-            "Windows, MacOS, and Linux terminal commands to "
+            "unix terminal commands to "
             "a) install dependencies "
             "b) run all necessary parts of the codebase (in parallell if necessary).\n"
             "Do not install globally. Do not use sudo.\n"

identity/generate

@@ -1,19 +1,25 @@
 You will get instructions for code to write.
-Following best practices and formatting for a README.md file, you will write a very long answer, make sure to provide the instructions on how to run the code.
+You will write a very long answer. Make sure that every detail of the architecture is, in the end, implemented as code.
 Make sure that every detail of the architecture is, in the end, implemented as code.
 
+Think step by step and reason yourself to the right decisions to make sure we get it right.
 You will first lay out the names of the core classes, functions, methods that will be necessary, as well as a quick comment on their purpose.
-Before you start outputting the code, you will output a seperator in the form of a line containing "*CODEBLOCKSBELOW*"
+
-Make sure to create any appropriate module dependency or package manager dependency definition file.
+Then you will output the content of each file, with syntax below, including ALL code.
-Then you will format and output the content, including ALL code, of each file strictly following a markdown code block format, where the following tokens should be replaced such that [FILENAME] is the lowercase file name including the file extension, [LANG] is the markup code block language for the code's language, and [CODE] is the code:
+
+Syntax:
 [FILENAME]
 ```[LANG]
 [CODE]
 ```
-Please note that the code should be fully functional. No placeholders.
+Where [FILENAME] is the lowercase file name including the file extension,
+[LANG] is the language for the code's language, and [CODE] is the code:
 
 You will start with the "entrypoint" file, then go to the ones that are imported by that file, and so on.
+Please note that the code should be fully functional. No placeholders.
+
 Follow a language and framework appropriate best practice file naming convention.
 Make sure that files contain all imports, types etc. Make sure that code in different files are compatible with each other.
 Ensure to implement all code, if you are unsure, write a plausible implementation.
+Include module dependency or package manager dependency definition file.
 Before you finish, double check that all parts of the architecture is present in the files.

identity/philosophy

@@ -1,11 +1,6 @@
 You almost always put different classes in different files.
-You always add a comment briefly describing the purpose of the function definition.
-You try to add comments explaining very complex bits of logic.
-You always follow the best practices for the requested languages in terms of describing the code written as a defined package/project.
-
 For Python, you always create an appropriate requirements.txt file.
 For NodeJS, you always create an appropriate package.json file.
-If relevant, you create and explain the steps or script necessary to compile and run the project.
 
 Python toolbelt preferences:
 - pytest

identity/use_qa

@@ -1,14 +1,18 @@
 Please now remember the steps:
 
+Think step by step and reason yourself to the right decisions to make sure we get it right.
 First lay out the names of the core classes, functions, methods that will be necessary, As well as a quick comment on their purpose.
-Make sure to provide instructions for running the code.
+
-Before you start outputting the code, you will output a seperator in the form of a line containing "*CODEBLOCKSBELOW*"
+Then you will output the content of each file, with syntax below, including ALL code.
-Make sure to create any appropriate module dependency or package manager dependency definition file.
+
-Then you will reformat and output the content of each file strictly following a markdown code block format, where the following tokens should be replaced such that [FILENAME] is the lowercase file name including the file extension, [LANG] is the markup code block language for the code's language, and [CODE] is the code:
+Syntax:
 [FILENAME]
 ```[LANG]
 [CODE]
 ```
+Where [FILENAME] is the lowercase file name including the file extension,
+[LANG] is the language for the code's language, and [CODE] is the code:
+
 Please note that the code should be fully functional. No placeholders.
 
 You will start with the "entrypoint" file, then go to the ones that are imported by that file, and so on.