Commit Graph

40 Commits

Author SHA1 Message Date
sherif-med
31525dfef7 Text file loaders (#3031)
* adding requiered packages for loading pdf, docx, md, tex files (preferably pure python packages)

* adding text file utils providing function to load file based on extension && adding read_text_file command

* adding test cases for text file loading (pdf file creation is hardcoded due to external package requierment for creation (a sample file can be added))

* formatting

* changing command name from 'read_text_file' to 'parse_text_document'

* fallback to txtParser if file extension is not known to read script and code files

* adding extension respective parsers

* adding binary file check function

* adding file existance check && raising valueError for unsupported binary file formats

* adding check file type (binary) in test_parsers for specific extensions && fixing mock pdf generation to include null bytes

* adding .yml extension parser

* removal of .doc parser

* updating file loading commands names

* updating test (removing .doc mock function)

* fix: import sort

* new cassette for mem A

* feat: update Cassettes

* feat: consolidate commands

* feat: linting

* feat: updates to cassettes

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-21 14:48:40 -05:00
k-boikov
bcc32cc441 Fix split_file when overlap = 0, add test (#3599)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-05-20 20:20:03 -05:00
sidewaysthought
a5f856328d Fix multi-byte character handling in read_file (#3173)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-05-01 19:50:50 +02:00
Bob
94ec4a4ea5 Fix file operations logger (#3489)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-05-01 17:37:30 +02:00
Richard Beales
c1329c92fd rename search_files to list_files (#3595) 2023-04-30 16:14:53 -05:00
Richard Beales
06ae4684c8 replace 50+ occurrences of print() with logger (#3056)
Co-authored-by: James Collins <collijk@uw.edu>
Co-authored-by: Luke Kyohere <lkyohere@mfsafrica.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Media <12145726+rihp@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
2023-04-29 23:40:57 -05:00
k-boikov
2d058feaf8 Extend & improve file operations tests (#3404)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-29 16:55:47 +02:00
Montana Flynn
7cd76b8d8e Add makedirs to file operations (#3289)
* Add makedirs to file operations

* Add new directory tests for file operations

* Fix wrong setUp test error

* Simplify makedirs and use correct nested path

* Fix linter error

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: James Collins <collijk@uw.edu>
2023-04-27 09:12:24 -07:00
James Collins
dcd6aa912b Add workspace abstraction (#2982)
* Add workspace abstraction

* Remove old workspace implementation

* Extract path resolution to a helper function

* Add api key requirements to new tests
2023-04-23 14:36:04 -05:00
ZHAOKAI WANG
019ac37d49 Merge branch 'master' into master 2023-04-22 10:40:22 +08:00
Steve
3ab67e746d Add file op tests (#2205)
Co-authored-by: Steven Byerly <stevenbyerly@microsoft.com>
2023-04-22 04:17:38 +02:00
ZHAOKAI WANG
e7c2a4068e Update file_operations.py 2023-04-21 13:06:44 +08:00
ZHAOKAI WANG
45a9ff6e74 Update file_operations.py 2023-04-21 13:03:52 +08:00
Nick Foster
ec27d5729c Fix label of download_file command (#2753)
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-21 04:55:20 +02:00
gklab
a2e75aabdd adjust file_operations.py code format 2023-04-21 10:19:28 +08:00
BillSchumacher
16f0e22ffa linting 2023-04-19 18:21:03 -05:00
BillSchumacher
d7679d755f Fix all commands and cleanup 2023-04-19 18:17:04 -05:00
BillSchumacher
23c650ca10 Merge branch 'master' of https://github.com/BillSchumacher/Auto-GPT into plugin-support 2023-04-19 17:28:17 -05:00
BillSchumacher
86d3444fb8 isort, add proper skips. 2023-04-18 18:59:23 -05:00
Josh XT
9514919d37 Option to disable working directory restrictions (#1875)
Remove restriction on working directory if RESTRICT_TO_WORKSPACE != True

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-04-19 01:54:38 +02:00
BillSchumacher
085842d43c Merge branch 'master' of https://github.com/BillSchumacher/Auto-GPT into plugin-support 2023-04-18 18:44:40 -05:00
Merwane Hamadi
cf9a94a8b6 isort implemented 2023-04-17 13:42:01 -07:00
Steve Byerly
8637b8b61b whitespace 2023-04-17 17:12:23 +01:00
Steve Byerly
6ac9ce614a whitespace 2023-04-17 17:12:23 +01:00
Steve Byerly
bd670b4db3 whitespace 2023-04-17 17:12:23 +01:00
Steve Byerly
def96ffe2f fix split file 2023-04-17 17:12:23 +01:00
BillSchumacher
81c65af560 blacked 2023-04-16 22:51:39 -05:00
BillSchumacher
03c137741a Merge branch 'master' of https://github.com/BillSchumacher/Auto-GPT into plugin-support 2023-04-16 22:13:37 -05:00
BillSchumacher
c110f3489d Finish integrating command registry 2023-04-16 21:51:36 -05:00
EH
9589334a30 Add File Downloading Capabilities (#1680)
* Added 'download_file' command

* Added util and fixed spinner

* Fixed comma and added autogpt/auto_gpt_workspace to .gitignore

* Fix linter issues

* Fix more linter issues

* Fix Lint Issues

* Added 'download_file' command

* Added util and fixed spinner

* Fixed comma and added autogpt/auto_gpt_workspace to .gitignore

* Fix linter issues

* Fix more linter issues

* Conditionally add the 'download_file' prompt

* Update args.py

* Removed Duplicate Prompt

* Switched to using path_in_workspace function
2023-04-17 03:34:02 +01:00
BillSchumacher
ec8ff0fcde Merge branch 'command_registry' of https://github.com/kreneskyp/Auto-GPT into plugin-support 2023-04-16 15:25:21 -05:00
BillSchumacher
3fadf2c90b Blacked 2023-04-16 14:15:38 -05:00
Jayce Slesar
713e4c1822 Merge branch 'master' into fix/type-annotations 2023-04-16 14:05:13 -04:00
jayceslesar
8990911522 unify annotations to future syntax 2023-04-16 14:02:48 -04:00
Reinier van der Leer
11620cc571 Fix and consolidate command workspace resolution 2023-04-16 18:27:39 +01:00
Pi
ccf3c7b89e Update file_operations.py 2023-04-16 17:25:58 +01:00
Steve
5b428f509b fix file logging issue 2023-04-16 17:25:58 +01:00
BillSchumacher
0d1fd4fcf0 Merge branch 'master' of https://github.com/Significant-Gravitas/Auto-GPT into pr/1436 2023-04-16 01:32:18 -05:00
yueliu1999
58eb0b37b4 Update file_operations.py
Error notation. In the split_file function, line 43, text->content.
2023-04-16 00:52:54 +08:00
BillSchumacher
1073954fb7 Reorg (#1537)
* Pi's message.

* Fix most everything.

* Blacked

* Add Typing, Docstrings everywhere, organize the code a bit.

* Black

* fix import

* Update message, dedupe.

* Increase backoff time.

* bump up retries
2023-04-15 13:56:23 +01:00