Add the following functions to ingest data into memory before Auto-GPT run.
- split_file: given a content, split it in chunks of max_length with (or without) a specified overlap
- ingest_file: read a file, use split_file to split it in chunks and load each chunk in memory
- ingest_directory: ingest all files in a directory in memory