docs, default to free mode

This commit is contained in:
2025-06-23 11:18:15 +02:00
parent e018dfa8e5
commit 19ad866186
2 changed files with 23 additions and 1 deletions

View File

@@ -58,7 +58,7 @@ func main() {
}
}
freeMode = strings.ToLower(os.Getenv("FREE_MODE")) == "true"
freeMode = strings.ToLower(os.Getenv("FREE_MODE")) != "false"
if freeMode {
var err error

View File

@@ -7,6 +7,7 @@ This repository provides a proxy server that emulates [Ollama's REST API](https:
Currently, it is enough for usage with [Jetbrains AI assistant](https://blog.jetbrains.com/ai/2024/11/jetbrains-ai-assistant-2024-3/#more-control-over-your-chat-experience-choose-between-gemini,-openai,-and-local-models).
## Features
- **Free Mode (Default)**: Automatically selects and uses free models from OpenRouter with intelligent fallback. Enabled by default unless `FREE_MODE=false` is set.
- **Model Filtering**: You can provide a `models-filter` file in the same directory as the proxy. Each line in this file should contain a single model name. The proxy will only show models that match these entries. If the file doesnt exist or is empty, no filtering is applied.
**Note**: OpenRouter model names may sometimes include a vendor prefix, for example `deepseek/deepseek-chat-v3-0324:free`. To make sure filtering works correctly, remove the vendor part when adding the name to your `models-filter` file, e.g. `deepseek-chat-v3-0324:free`.
@@ -28,6 +29,27 @@ You can provide your **OpenRouter** (OpenAI-compatible) API key through an envir
./ollama-proxy "your-openrouter-api-key"
### 3. Free Mode (Default Behavior)
The proxy operates in **free mode** by default, automatically selecting from available free models on OpenRouter. This provides cost-effective usage without requiring manual model selection.
# Free mode is enabled by default - no configuration needed
export OPENAI_API_KEY="your-openrouter-api-key"
./ollama-proxy
# To disable free mode and use all available models
export FREE_MODE=false
export OPENAI_API_KEY="your-openrouter-api-key"
./ollama-proxy
#### How Free Mode Works
- **Automatic Model Discovery**: Fetches and caches available free models from OpenRouter
- **Intelligent Fallback**: If a requested model fails, automatically tries other available free models
- **Failure Tracking**: Temporarily skips models that have recently failed (15-minute cooldown)
- **Model Prioritization**: Tries models in order of context length (largest first)
- **Cache Management**: Maintains a `free-models` file for quick startup and a `failures.db` SQLite database for failure tracking
Once running, the proxy listens on port `11434`. You can make requests to `http://localhost:11434` with your Ollama-compatible tooling.
## Installation