Server-side proxy that holds the OpenAI API key and serves a Flutter chat app (Android + iOS). Goal: the key never leaves the server, only the real app can use the proxy, and spend is bounded.
Key insight: a client-generated ID is not an identity. Anyone who pulls the proxy URL from the APK/IPA can mint unlimited IDs. All per-client limits are bypassable until the proxy can verify the caller is the genuine app on a genuine device. App attestation is the foundation everything else rests on.
1. System Overview
2. Request Lifecycle
3. Layered Defense Model
4. Requirements
4.1 Identity & Anti-abuse
ID
Requirement
Priority
ID-1
Verify Play Integrity (Android) and App Attest / DeviceCheck (iOS) server-side before issuing any session credential.
P0
ID-2
Issue short-lived JWTs (e.g. 15–60 min) bound to the attested app instance; refresh requires re-attestation or a refresh token.
P0
ID-3
Never trust a client-chosen ID as identity; derive user ID server-side from the attested credential.
P0
ID-4
Rate limit on multiple axes: per user, per IP, per app version, and a global cap.
P0
ID-5
Limit tokens (input + output) per window, not just request count.
P0
ID-6
Hard daily/monthly spend cap in OpenAI dashboard plus a proxy-side kill switch.
P0
ID-7
Ban list / shadow-ban for abusive users; reject expired or revoked JWTs.
P1
4.2 Request Hygiene
ID
Requirement
Priority
RQ-1
Client sends only user messages. Proxy owns model, system prompt, max_tokens, temperature, tools. Raw bodies are never forwarded.
P0
RQ-2
Cap input message length and total history tokens; truncate or summarize old turns server-side.
P0
RQ-3
Run OpenAI moderation on user input; reject or flag policy violations.
P1
RQ-4
Stream via SSE; enforce request timeouts and max concurrent streams per user.
P1
RQ-5
Strict schema validation on every endpoint; reject unknown fields.
P1
4.3 Infrastructure & Operations
ID
Requirement
Priority
OP-1
OpenAI key in a secrets manager; never in config files, logs, or error messages.
P0
OP-2
TLS everywhere; consider certificate pinning in the Flutter client.
P0
OP-3
Separate OpenAI keys per environment (dev / staging / prod).
P0
OP-4
Log tokens, cost, model, latency per user; alert on spend spikes and anomalous patterns.
P1
OP-5
Provider abstraction layer so OpenAI can be swapped or fallbacks added.
P2
OP-6
Health checks, graceful degradation, and a maintenance-mode response the app understands.
P2
4.4 Privacy & Compliance
ID
Requirement
Priority
PR-1
Decide conversation retention policy; if stored, encrypt at rest and support deletion.
P0
PR-2
Publish a privacy policy covering AI processing (required by Apple and Google for AI chat apps).
P0
PR-3
In-app account deletion flow (mandatory on both stores).
P0
PR-4
Minimize PII in logs; redact message content from operational logs.
P1
5. Design Decisions
Auth flow
App → attestation token → POST /auth/attest
Proxy verifies with Google/Apple → returns JWT + refresh token
All chat calls carry Authorization: Bearer <jwt>
Rate limiting
Token-bucket per user in Redis (tokens/min + tokens/day)
Sliding-window per IP
Global semaphore for concurrent upstream streams
Endpoints
POST /auth/attest
POST /auth/refresh
POST /chat (SSE)
GET /usage (user's own quota)
DELETE /account
Client payload
{ conversation_id, message } only
No model / prompt / params from client
History reconstructed server-side (or capped client history, validated)
6. Implementation Order
Attestation + JWT issuance (nothing else matters until this works)