🧪 test: level 2

This commit is contained in:
Florian Hönicke
2023-05-01 02:06:09 +02:00
parent 36ccd79bc8
commit 30e0b8cf1e
3 changed files with 13 additions and 6 deletions

View File

@@ -380,6 +380,8 @@ metas:
description=self.microservice_specification.task
)['strategies.json']
packages_list = [[pkg.strip().lower() for pkg in packages] for packages in json.loads(packages_json_string)]
packages_list = [[self.replace_with_gpt_3_5_turbo_if_possible(pkg) for pkg in packages] for packages in packages_list]
packages_list = [
packages for packages in packages_list if len(set(packages).intersection(set(PROBLEMATIC_PACKAGES))) == 0
]
@@ -512,3 +514,8 @@ Test scenario:
while not val:
val = input('you: ')
return val
@staticmethod
def replace_with_gpt_3_5_turbo_if_possible(pkg):
if pkg in ['nltk', 'textblob', 'spacy', 'transformers']:
return 'gpt_3_5_turbo_api'

View File

@@ -120,7 +120,7 @@ contains example: yes (<insert example here>)
**final.txt**
```text
input: "<input here>"
assertion: "<weak assertion of output here>"
assertion: the output is of type <type here>
```
If you did b, you must not do a.
@@ -132,7 +132,7 @@ contains example: yes (Berlin)
**final.txt**
```text
input: "Berlin"
assertion: "contains weather report for the next 5 days"
assertion: the output is of type string
```
Example for "The user inserts a png and gets an svg as response.":
@@ -151,7 +151,7 @@ contains example: yes (https://aquasecurity.github.io/kube-bench/v0.6.5/images/k
**final.txt**
```text
input: "https://aquasecurity.github.io/kube-bench/v0.6.5/images/kube-bench-logo-only.png"
assertion: "is an svg"
assertion: the output is of type svg
```
Example for "The microservice takes nothing as input and returns the current time.":
@@ -160,6 +160,6 @@ contains example: n/a
**final.txt**
```text
input: "nothing"
assertion: "is a string"
assertion: the output is of type string
```
'''

View File

@@ -405,8 +405,8 @@ Note that you must not ask for an example input in case the example input is alr
Example for the case where the example is already mentioned in the refined description or the original description:
**final.txt**
```text
input: "<input here>"
assertion: "the output must be of type <type here>"
input: <input here>
assertion: the output is of type <type here>
```
Note that your response must be either prompt.txt or final.txt. You must not write both.
Note that you must obey the double asterisk and tripple backtick syntax from above.