Ollama 与企业系统集成 · 运维手册
适用场景：在生产环境部署 Ollama 服务，对接企业内部系统（OA/工单/知识库），实现安全可控的 AI 能力。
1. 企业集成架构
1.1 典型架构图
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ 企业应用 │ │ API Gateway │ │ Ollama │
│ (Java/Go) │ ──▶ │ (Nginx) │ ──▶ │ (K8s) │
└─────────────┘ └──────┬──────┘ └──────┬──────┘
│ │
┌──────▼──────┐ ┌──────▼──────┐
│ Auth │ │ GPU │
│ (JWT) │ │ (A100) │
└─────────────┘ └─────────────┘
1.2 集成层次
| 层次 | 组件 | 职责 |
|---|---|---|
| 接入层 | Nginx/Caddy | 路由、限流、TLS 终止 |
| 认证层 | OAuth2/JWT | 身份验证、权限控制 |
| 业务层 | 微服务 | 业务逻辑、Prompt 组装 |
| AI 层 | Ollama | 模型推理、Embedding |
| 数据层 | Redis/PG | 缓存、会话、审计日志 |
2. API Gateway 配置
2.1 Nginx 反向代理
# /etc/nginx/conf.d/ollama.conf
upstream ollama_backend {
server ollama-1:11434;
server ollama-2:11434;
keepalive 32;
}
server {
listen 443 ssl;
server_name ai.example.com;
ssl_certificate /etc/nginx/ssl/cert.pem;
ssl_certificate_key /etc/nginx/ssl/key.pem;
# 限流
limit_req_zone $binary_remote_addr zone=ollama:10m rate=60r/s;
location / {
limit_req zone=ollama burst=20 nodelay;
proxy_pass http://ollama_backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
# 超时
proxy_connect_timeout 30s;
proxy_read_timeout 300s;
# 流式响应支持
proxy_buffering off;
proxy_cache off;
}
# 健康检查
location /health {
access_log off;
return 200 'OK';
}
}
2.2 Caddy 配置（更简洁）
ai.example.com {
reverse_proxy ollama-1:11434 ollama-2:11434
header {
Access-Control-Allow-Origin "*"
Access-Control-Allow-Methods "GET, POST, OPTIONS"
Access-Control-Allow-Headers "Content-Type, Authorization"
}
@rate limit rate {
period 1s
requests 100
}
handle_path @rate /rate-limited {
respond "Rate limited" 429
}
}
3. 认证授权
3.1 JWT 验证中间件
# auth_middleware.py
from fastapi import FastAPI, Depends, HTTPException
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from jose import jwt, JWTError
app = FastAPI()
security = HTTPBearer()
SECRET_KEY = os.environ["JWT_SECRET"]
def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
try:
payload = jwt.decode(credentials.credentials, SECRET_KEY, algorithms=["HS256"])
return payload
except JWTError:
raise HTTPException(status_code=401, detail="Invalid token")
@app.post("/v1/chat/completions")
async def chat(
request: ChatRequest,
user: dict = Depends(verify_token)
):
# user 包含 sub, role, exp 等字段
return await process_chat(request, user)
3.2 API Key 管理
# 静态 API Key 验证
VALID_KEYS = {
"sk-app-001": {"app": "web-frontend", "rate_limit": 100},
"sk-app-002": {"app": "mobile-app", "rate_limit": 50}
}
@app.middleware("http")
async def api_key_middleware(request, call_next):
api_key = request.headers.get("X-API-Key")
if api_key not in VALID_KEYS:
return JSONResponse(
status_code=401,
content={"error": "Invalid API key"}
)
# 注入用户信息到请求
request.state.user = VALID_KEYS[api_key]
response = await call_next(request)
return response
4. 高可用部署
4.1 Kubernetes 部署
# ollama-ha.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ollama
spec:
replicas: 3
selector:
matchLabels:
app: ollama
template:
metadata:
labels:
app: ollama
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- ollama
topologyKey: kubernetes.io/hostname
containers:
- name: ollama
image: ollama/ollama:latest
resources:
limits:
nvidia.com/gpu: 1
volumeMounts:
- name: model-store
mountPath: /root/.ollama
volumes:
- name: model-store
persistentVolumeClaim:
claimName: ollama-models
---
apiVersion: v1
kind: Service
metadata:
name: ollama-svc
spec:
selector:
app: ollama
ports:
- port: 11434
targetPort: 11434
type: ClusterIP
4.2 健康检查
# K8s readiness probe
readinessProbe:
httpGet:
path: /api/tags
port: 11434
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
5. 性能监控
5.1 Prometheus 指标采集
# metrics.py
from prometheus_client import Counter, Histogram, generate_latest
REQUEST_COUNT = Counter('ollama_requests_total', 'Request count', ['model', 'endpoint'])
REQUEST_LATENCY = Histogram('ollama_request_latency_seconds', 'Request latency')
@app.middleware("http")
async def metrics_middleware(request, call_next):
start = time.time()
response = await call_next(request)
elapsed = time.time() - start
REQUEST_COUNT.labels(
model=request.state.model,
endpoint=request.url.path
).inc()
REQUEST_LATENCY.observe(elapsed)
if request.url.path == "/metrics":
return Response(generate_latest(), media_type="text/plain")
return response
5.2 Grafana Dashboard
{
"dashboard": {
"panels": [
{
"title": "QPS",
"targets": [{"expr": "rate(ollama_requests_total[5m])"}]
},
{
"title": "P99 Latency",
"targets": [{"expr": "histogram_quantile(0.99, rate(ollama_request_latency_seconds_bucket[5m]))"}]
},
{
"title": "GPU Utilization",
"targets": [{"expr": "nvidia_gpu_utilization"}]
}
]
}
}
6. 安全加固
6.1 网络隔离
# K8s NetworkPolicy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ollama-netpol
spec:
podSelector:
matchLabels:
app: ollama
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
name: api-gateway
ports:
- port: 11434
6.2 审计日志
import logging
audit_logger = logging.getLogger("audit")
@app.middleware("http")
async def audit_middleware(request, call_next):
user_id = request.state.user.get("sub", "anonymous")
audit_logger.info({
"event": "api_call",
"user": user_id,
"method": request.method,
"path": request.url.path,
"status": 200,
"timestamp": datetime.now().isoformat()
})
response = await call_next(request)
return response
7. 常见问题排查
问题 1：跨域请求失败
解决：
add_header Access-Control-Allow-Origin "*";
add_header Access-Control-Allow-Methods "GET, POST, OPTIONS";
add_header Access-Control-Allow-Headers "Content-Type, Authorization";
问题 2：模型加载慢
解决： - 预加载常用模型到本地缓存 - 使用 SSD/NVMe 存储 - 预热 Pod（提前加载模型）
问题 3：GPU 显存泄漏
解决：
- 设置 OLLAMA_MAX_LOADED_MODELS=1
- 定期重启 Pod
- 监控显存使用趋势