Endpoint verification
Only public HTTPS addresses on port 443 are allowed. The DNS address is resolved and pinned for the connection; redirects, private, link-local, reserved and metadata networks are blocked.
GetVibeAPITheme
Language
Safe audit of model–API consistency
Model check via API: signs of LLM substitution
Check how consistent an OpenAI-compatible API's behavior is with the model it claims to be. The service runs safe synthetic probes and measures context, tool calls, streaming, metadata and latency. The result is not cryptographic proof and not an accusation against the provider.
Use a temporary key with minimal permissions and a limit. The key is sent only to the endpoint you specify and is not stored by GetVibe.
What the model audit checks
The check compares the endpoint's observed behavior against its documented capabilities and an available reference profile. Neither the model's self-identification, nor the model field, nor response speed counts as proof. Only repeatable signals from independent groups carry weight. Network errors, unsupported features and insufficient budget reduce coverage rather than automatically worsening the score.
Capabilities and instruction following
Small deterministic tasks are repeated across three rounds. A safe instruction conflict uses only harmless nonces and formats.
Context and canary markers
Random non-secret markers are placed in conservative token-aware tiers. Acceptance, rejection, possible truncation, marker skip and timeout are recorded separately.
Tool calls and JSON Schema
A dummy function verifies the name, required fields, types, enum and rejection of extra properties. The resulting tool call is validated but never executed.
Streaming, metadata and latency
SSE framing, stream completion and delta assembly help diagnose transport compatibility. model, /models and latency are shown as weak or operational signals.
How the safe check works
Synthetic requests without user data
The service does not ask you to reveal your system prompt, does not run jailbreak scenarios and does not execute received tool calls. User prompts and documents are not required.
Summary report and repeatability
Requests, time, bytes and budget are bounded. The report shows aggregated statuses, probe durations and data sufficiency without raw responses or upstream errors.
How to read the result
Signals and the four report states
The report uses the phrases “Signals agree”, “Mixed signals”, “Signals disagree” and “Insufficient data”. A single weak signal, timeout or unsupported feature cannot yield an inconsistency conclusion.
How score differs from data sufficiency
Consistency score aggregates only the evaluated signals. Data sufficiency separately accounts for coverage, repeatability, baseline quality and the number of independent groups. No metric is a probability of authenticity.
Method limitations and causes of false positives
A gateway can change aliases, parameters, context limits, tool semantics, metadata and streaming. Results are also affected by system wrappers, safety middleware, the tokenizer, quantization, fine-tuning, load, geography, cold start and buffering.
Without hardware or cryptographic attestation of the inference infrastructure it is impossible to confirm specific weights or a checkpoint. The report shows the consistency of available signals and helps surface mismatches, but it does not deliver a legal or cryptographic verdict.
Key security and privacy
The API key stays in memory of the current request, is never returned in the report and is not written to the database, logs or analytics. URLs, model strings, prompts, responses, headers and raw errors are not stored. A minimal technical log keeps only a daily pseudonym, aggregated statuses and resources, and is deleted on its retention schedule.
Frequently asked questions
Can you definitively prove which model is answering?+
No. Without attestation of the inference server you cannot cryptographically confirm specific model weights. The report evaluates the consistency of several observed signals.
Which APIs are supported?+
OpenAI-compatible HTTPS endpoints for chat completions. Tools, streaming and /models are only checked where the endpoint advertises those capabilities.
Is the API key stored?+
No. The key is used only during the check for requests to the chosen hostname and is not written to the database, logs, analytics or the report.
Are prompts and responses stored?+
No. The check uses synthetic requests without user data; raw requests, responses and upstream errors are not stored.
Why is the result sometimes “Insufficient data”?+
Possible causes are timeouts, a limited budget, unsupported features, a small number of repeats or the absence of a quality reference profile.
Can a proxy hide a substitution?+
Yes. A relay can rewrite the model, metadata and event format. That's why these signals carry little weight and absolute guarantees are not promised.
Why measure latency?+
To show endpoint stability and diagnose buffering, cold start or overload. Speed by itself does not determine the model family.
Are jailbreak tests used?+
No. The service applies only safe synthetic checks and does not try to extract system instructions or bypass protections.