aljaz/Auto-GPT
1
0
Fork 0
mirror of https://github.com/aljazceru/Auto-GPT.git synced 2025-12-17 22:14:28 +01:00
Code Issues Packages Projects Releases Wiki Activity

Support secure and authenticated Milvus memory backends (#2127)

Browse Source 
Co-authored-by: Reinier van der Leer (Pwuts) <github@pwuts.nl>
This commit is contained in:
chyezh
2023-04-24 00:11:04 +08:00
committed by GitHub
parent 6e588bb2ed
commit cac1ea27e2
5 changed files with 108 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/configuration/memory.md
View File

@@ -15,7 +15,7 @@ To switch to either, change the `MEMORY_BACKEND` env variable to the value that
Links to memory backends
- [Pinecone](https://www.pinecone.io/)
- [Milvus](https://milvus.io/)
- [Milvus](https://milvus.io/) &ndash; [self-hosted](https://milvus.io/docs), or managed with [Zilliz Cloud](https://zilliz.com/)
- [Redis](https://redis.io)
- [Weaviate](https://weaviate.io)
@@ -80,17 +80,37 @@ export MEMORY_BACKEND="pinecone"
### Milvus Setup
[Milvus](https://milvus.io/) is an open-source, highly scalable vector database to store huge amounts of vector-based memory and provide fast relevant search.
[Milvus](https://milvus.io/) is an open-source, highly scalable vector database to store huge amounts of vector-based memory and provide fast relevant search. And it can be quickly deployed by docker locally or as a cloud service provided by [Zilliz Cloud](https://zilliz.com/).
- setup milvus database, keep your pymilvus version and milvus version same to avoid compatible issues.
- setup by open source [Install Milvus](https://milvus.io/docs/install_standalone-operator.md)
- or setup by [Zilliz Cloud](https://zilliz.com/cloud)
- set `MILVUS_ADDR` in `.env` to your milvus address `host:ip`.
- set `MEMORY_BACKEND` in `.env` to `milvus` to enable milvus as backend.
1. Deploy your Milvus service, either locally using docker or with a managed Zilliz Cloud database.
- [Install and deploy Milvus locally](https://milvus.io/docs/install_standalone-operator.md)
**Optional:**
- set `MILVUS_COLLECTION` in `.env` to change milvus collection name as you want, `autogpt` is the default name.
- <details><summary>Set up a managed Zilliz Cloud database <i>(click to expand)</i></summary>
1. Go to [Zilliz Cloud](https://zilliz.com/) and sign up if you don't already have account.
2. In the *Databases* tab, create a new database.
- Remember your username and password
- Wait until the database status is changed to RUNNING.
3. In the *Database detail* tab of the database you have created, the public cloud endpoint, such as:
`https://xxx-xxxx.xxxx.xxxx.zillizcloud.com:443`.
</details>
2. Run `pip3 install pymilvus` to install the required client library.
Make sure your PyMilvus version and Milvus version are [compatible](https://github.com/milvus-io/pymilvus#compatibility) to avoid issues.
See also the [PyMilvus installation instructions](https://github.com/milvus-io/pymilvus#installation).
3. Update `.env`
- `MEMORY_BACKEND=milvus`
- One of:
- `MILVUS_ADDR=host:ip` (for local instance)
- `MILVUS_ADDR=https://xxx-xxxx.xxxx.xxxx.zillizcloud.com:443` (for Zilliz Cloud)
*The following settings are **optional**:*
- Set `MILVUS_USERNAME='username-of-your-milvus-instance'`
- Set `MILVUS_PASSWORD='password-of-your-milvus-instance'`
- Set `MILVUS_SECURE=True` to use a secure connection. Only use if your Milvus instance has TLS enabled.
Setting `MILVUS_ADDR` to a `https://` URL will override this setting.
- Set `MILVUS_COLLECTION` if you want to change the collection name to use in Milvus. Defaults to `autogpt`.
### Weaviate Setup
[Weaviate](https://weaviate.io/) is an open-source vector database. It allows to store data objects and vector embeddings from ML-models and scales seamlessly to billion of data objects. [An instance of Weaviate can be created locally (using Docker), on Kubernetes or using Weaviate Cloud Services](https://weaviate.io/developers/weaviate/quickstart).