We built API-Clarity to answer a simple question: how good are your API docs, really? Not “do they exist” but “can a developer go from zero to first API call in under 5 minutes?”
To prove the engine works, we pointed it at three APIs that millions of developers use daily: Stripe, GitHub, and Twilio. We analyzed their public OpenAPI specs using our automated DX scoring methodology across 5 categories plus a fintech compliance bonus.
The results surprised us.
The Scores
| API | Score | Grade | Endpoints | Verdict |
|---|---|---|---|---|
| Stripe | 51/100 | D | 587 | Strong compliance, weak spec-level docs |
| GitHub | 45/100 | F | 1,079 | Massive API, minimal spec metadata |
| Twilio | 36/100 | F | 197 | Solid product, spec needs serious work |
Before anyone gets defensive: these scores reflect what’s in the OpenAPI spec file itself, not the entire developer portal. Stripe’s docs at stripe.com/docs are legendary. But their raw spec? That’s a different story.
And that’s exactly the point. If your spec doesn’t carry the documentation, every tool that consumes it (SDK generators, API explorers, AI coding assistants) gets a degraded experience.
Category Breakdown
Structural Analysis (max: 40 points)
This checks the basics: does every endpoint have a description? Are response examples provided? Are parameters documented?
| API | Score | Key Finding |
|---|---|---|
| Stripe | 29/40 | Good descriptions, zero response examples in spec |
| GitHub | 27/40 | Massive endpoint count, many missing error code docs |
| Twilio | 26/40 | Similar pattern — descriptions exist, error codes don’t |
The pattern: All three document what endpoints do, but none include response examples directly in the spec. This means any tool consuming the spec can’t show developers what a successful response looks like.
Authentication (max: 20 points)
How well does the spec explain auth? Can a developer figure out how to get credentials?
| API | Score | Key Finding |
|---|---|---|
| Stripe | 8/20 | Security schemes defined, no credential acquisition guide |
| GitHub | 3/20 | Minimal security scheme definitions |
| Twilio | 3/20 | Same — auth info lives outside the spec |
The gap: All three have excellent auth documentation on their websites. None of that makes it into the OpenAPI spec. This is a missed opportunity — modern API tools can render auth flows directly from spec metadata.
Code Samples (max: 15 points)
Does the spec include x-code-samples for Python, Node.js, and cURL?
| API | Score |
|---|---|
| Stripe | 0/15 |
| GitHub | 0/15 |
| Twilio | 0/15 |
Zero across the board. None of the three use the x-code-samples OpenAPI extension. Their code samples live in separate documentation systems. This is the single biggest DX improvement opportunity for all three.
Error Handling (max: 15 points)
Are error responses documented with status codes, descriptions, and consistent formatting?
| API | Score | Key Finding |
|---|---|---|
| Stripe | 7/15 | Some error docs, inconsistent coverage |
| GitHub | 8/15 | Consistent error format, but many endpoints lack codes |
| Twilio | 7/15 | Consistent format, sparse error code documentation |
Rate Limiting (max: 10 points)
Is rate limiting documented in the spec?
| API | Score |
|---|---|
| Stripe | 7/10 |
| GitHub | 7/10 |
| Twilio | 0/10 |
Stripe and GitHub document rate limiting. Twilio’s spec has no rate limit information.
Fintech Compliance (bonus)
This category doesn’t affect the base 100-point score. It checks for PCI DSS awareness, SOC 2 controls, data privacy docs, transport security, idempotency, and webhook security references.
| API | Compliance | Highlights |
|---|---|---|
| Stripe | 88% | PCI, SOC 2, privacy, transport security, idempotency — all present |
| GitHub | 70% | PCI awareness, SOC 2, transport security, webhook security |
| Twilio | 56% | PCI, SOC 2, privacy, transport security — no idempotency docs |
Stripe wins the compliance category by a wide margin. Not surprising for a payments company.
What This Actually Means
These scores don’t mean these are bad APIs. They’re among the best in the industry. What the scores reveal is a documentation architecture problem that affects the entire ecosystem:
The spec is treated as a build artifact, not a documentation artifact.
Most companies generate their OpenAPI spec from code annotations. The spec gets endpoint paths, methods, and basic parameter types. But the rich documentation — the auth guides, code samples, error explanations, rate limit details — lives in a separate system (Readme, Mintlify, custom portals).
This creates a split-brain problem:
- SDK generators only see the spec → they produce SDKs with no usage guidance
- AI coding assistants only see the spec → they can’t write correct integration code
- API explorers only see the spec → they show endpoints with no context
The companies with the best DX scores in our engine are the ones that treat their OpenAPI spec as the single source of truth — not just for paths and types, but for examples, auth flows, and code samples.
What Would Fix These Scores
If Stripe, GitHub, or Twilio enriched their specs with three additions, they’d jump 20-30 points:
- Add
x-code-samplesto each endpoint with Python, Node.js, and cURL examples (+15 points) - Add response examples to every endpoint showing successful and error responses (+10 points)
- Add auth acquisition instructions in the security scheme description (+5 points)
That’s it. Three changes. The information already exists on their websites — it just needs to be embedded in the spec.
Try It Yourself
API-Clarity is open source. You can run it against your own API spec:
git clone https://github.com/user/api-clarity
cd api-clarity/backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Analyze your spec
python -m app.cli analyze your-spec.yaml
# Or from a URL
python -m app.cli analyze --url https://your-api.com/openapi.jsonOr use the web interface at clarifyintel.com — upload your spec and get an instant DX score with a prioritized fix list.
What’s Next
We’re planning a series of API DX reports. Next up: payment APIs (Adyen, Square, Mollie, Razorpay) and cloud APIs (AWS, GCP, Azure). If you want your API included or want a private DX audit, get in touch.
Methodology: All specs were fetched from official public repositories on March 8, 2026. Scoring uses API-Clarity v0.1.0 with 5 categories (Structural 40p, Authentication 20p, Code Samples 15p, Error Handling 15p, Rate Limiting 10p) plus a fintech compliance bonus. Full scoring methodology is documented in the API-Clarity README.