mirror of
https://github.com/aljazceru/ollama-free-model-proxy.git
synced 2025-12-17 05:04:20 +01:00
docs, default to free mode
This commit is contained in:
2
main.go
2
main.go
@@ -58,7 +58,7 @@ func main() {
|
||||
}
|
||||
}
|
||||
|
||||
freeMode = strings.ToLower(os.Getenv("FREE_MODE")) == "true"
|
||||
freeMode = strings.ToLower(os.Getenv("FREE_MODE")) != "false"
|
||||
|
||||
if freeMode {
|
||||
var err error
|
||||
|
||||
22
readme.md
22
readme.md
@@ -7,6 +7,7 @@ This repository provides a proxy server that emulates [Ollama's REST API](https:
|
||||
Currently, it is enough for usage with [Jetbrains AI assistant](https://blog.jetbrains.com/ai/2024/11/jetbrains-ai-assistant-2024-3/#more-control-over-your-chat-experience-choose-between-gemini,-openai,-and-local-models).
|
||||
|
||||
## Features
|
||||
- **Free Mode (Default)**: Automatically selects and uses free models from OpenRouter with intelligent fallback. Enabled by default unless `FREE_MODE=false` is set.
|
||||
- **Model Filtering**: You can provide a `models-filter` file in the same directory as the proxy. Each line in this file should contain a single model name. The proxy will only show models that match these entries. If the file doesn’t exist or is empty, no filtering is applied.
|
||||
|
||||
**Note**: OpenRouter model names may sometimes include a vendor prefix, for example `deepseek/deepseek-chat-v3-0324:free`. To make sure filtering works correctly, remove the vendor part when adding the name to your `models-filter` file, e.g. `deepseek-chat-v3-0324:free`.
|
||||
@@ -28,6 +29,27 @@ You can provide your **OpenRouter** (OpenAI-compatible) API key through an envir
|
||||
|
||||
./ollama-proxy "your-openrouter-api-key"
|
||||
|
||||
### 3. Free Mode (Default Behavior)
|
||||
|
||||
The proxy operates in **free mode** by default, automatically selecting from available free models on OpenRouter. This provides cost-effective usage without requiring manual model selection.
|
||||
|
||||
# Free mode is enabled by default - no configuration needed
|
||||
export OPENAI_API_KEY="your-openrouter-api-key"
|
||||
./ollama-proxy
|
||||
|
||||
# To disable free mode and use all available models
|
||||
export FREE_MODE=false
|
||||
export OPENAI_API_KEY="your-openrouter-api-key"
|
||||
./ollama-proxy
|
||||
|
||||
#### How Free Mode Works
|
||||
|
||||
- **Automatic Model Discovery**: Fetches and caches available free models from OpenRouter
|
||||
- **Intelligent Fallback**: If a requested model fails, automatically tries other available free models
|
||||
- **Failure Tracking**: Temporarily skips models that have recently failed (15-minute cooldown)
|
||||
- **Model Prioritization**: Tries models in order of context length (largest first)
|
||||
- **Cache Management**: Maintains a `free-models` file for quick startup and a `failures.db` SQLite database for failure tracking
|
||||
|
||||
Once running, the proxy listens on port `11434`. You can make requests to `http://localhost:11434` with your Ollama-compatible tooling.
|
||||
|
||||
## Installation
|
||||
|
||||
Reference in New Issue
Block a user