Auto-GPT Benchmark

A repo built for the purpose of benchmarking the performance of agents far and wide, regardless of how they are set up and how they work

Scores:

Radio chart for each agent coming soon !

⚠️ These results are constantly evolving at the moment. We will publish an official benchmark result very soon.

Interface

Code

Task	Results
Debug Simple Typo With Guidance	❌
Debug Simple Typo Without Guidance	❌
Basic Code Generation	✅
Create Simple Web Server	❌

Memory

Task	Results
Basic Memory	✅
Remember Multiple Ids	❌
Remember Multiple Ids With Noise	❌
Remember Multiple Phrases With Noise	❌

Interface

Code

Task	Results
Debug Simple Typo With Guidance	❌
Debug Simple Typo Without Guidance	❌
Basic Code Generation	✅
Create Simple Web Server	❌

Coming Soon!

Interface

Code

Task	Results
Debug Simple Typo With Guidance	❌
Debug Simple Typo Without Guidance	❌
Basic Code Generation	✅
Create Simple Web Server	❌