Quantify LLM Brittleness.

A free, web-based platform to evaluate, compare, and visualize the robustness of Large Language Models against real-world prompt variations.

How It Works

Our platform systematically challenges LLMs to reveal their true stability.

Dynamic Prompt Perturbation

Generates lexical, syntactic, semantic, and noise-based prompt variations to test model limits.

Advanced Brittleness Metrics

Computes lexical and semantic similarity scores to quantify model output consistency.

Interactive Visualizations

Side-by-side comparisons with rich charts to clearly visualize model robustness and degradation.

Why Brittleness Matters

An LLM might seem intelligent, but slight changes in a prompt can cause wildly different outputs. This "brittleness" is a major hurdle for reliable AI. Our analyzer helps you measure this instability to build more trustworthy applications.

  • Benchmark LLM reliability for production use cases.
  • Accelerate research on model robustness and safety.
  • Gain insights into model behavior without writing code.
  • Make data-driven decisions when choosing an LLM.

Base Prompt: "Summarize the article."

Model A Output: "The study found a strong correlation..."

Model B Output: "A significant link was discovered..."

Perturbed Prompt: "Could you please summarize the article?"

Model A Output: "Certainly! The research indicates a strong link..." (Stable)

Model B Output: "The article is about science." (Brittle)