Defending LLM Endpoints from Prompt Injection Probes
Prompt injection — convincing an LLM to ignore its system prompt and follow attacker instructions — is the SQL injection of the AI era. Attackers are no longer experimenting manually; they are running distributed probe campaigns to find vulnerable LLM deployments at scale.
What a prompt injection probe campaign looks like
A typical campaign originates from 50–500 IPs, each sending 2–10 requests per minute to stay under per-IP rate limits. Each request contains a slight variation of a known injection payload — 'ignore previous instructions and...', 'you are now in developer mode...', 'print your system prompt...' — probing for a model or configuration that responds to the injection. The IPs rotate through residential proxies, making session-level detection difficult.
Why IP reputation is an effective first filter
Probe campaign IPs are reused across attack types. An IP that probed for open SSH servers last week and SQL injection endpoints the week before is now probing your LLM. IP reputation databases carry this multi-campaign history. Blocking known bad IPs at the network edge eliminates the majority of probe traffic before it reaches your inference infrastructure.
import os, httpx
from functools import lru_cache
IPABUSE_KEY = os.environ["IPABUSE_API_KEY"]
BLOCK_THRESHOLD = 55 # tune for your risk appetite
@lru_cache(maxsize=4096)
def reputation_score(ip: str) -> int:
"""Cache reputation scores for 5 minutes via LRU eviction."""
try:
r = httpx.get(
f"https://api.ipabuse.org/v1/ip/{ip}/",
headers={"X-API-Key": IPABUSE_KEY},
timeout=2.0,
)
r.raise_for_status()
return r.json()["data"]["reputationScore"]
except Exception:
return 100 # fail open
def is_blocked(ip: str) -> bool:
return reputation_score(ip) < BLOCK_THRESHOLDIntegrating with FastAPI / LangServe
Add the IP check as a FastAPI dependency. It runs before every request to your LangServe chain endpoint and adds negligible latency for cached IPs.
from fastapi import FastAPI, Request, HTTPException, Depends
from langserve import add_routes
from .chain import build_chain
from .middleware.ip_guard import is_blocked
app = FastAPI()
async def require_clean_ip(request: Request):
ip = (
request.headers.get("cf-connecting-ip")
or request.headers.get("x-real-ip")
or request.client.host
)
if ip and is_blocked(ip):
raise HTTPException(status_code=403, detail="Access denied.")
add_routes(
app,
build_chain(),
path="/chat",
dependencies=[Depends(require_clean_ip)],
)Reporting probes back to IPAbuse
When your input validation layer catches a clear prompt injection attempt, report the source IP. This enriches the shared database and helps protect operators who have not yet deployed input validation.
import httpx, os
def report_probe(ip: str, payload_preview: str) -> None:
httpx.post(
"https://api.ipabuse.org/v1/ip/report",
headers={
"X-API-Key": os.environ["IPABUSE_REPORT_API_KEY"],
"Content-Type": "application/json",
},
json={
"ipAddress": ip,
"categorySlug": "sql-injection",
"description": f"Prompt injection probe: {payload_preview[:120]}",
"severity": 4,
},
timeout=5.0,
)Defence in depth: beyond IP filtering
IP reputation is the cheapest layer and should be deployed first, but it is not the last. Complement it with: a prompt firewall that scans inputs for known injection patterns, output monitoring that alerts when the model response diverges from expected format, and system prompt leakage detection that flags responses containing your instruction text. Each layer catches attacks the others miss.