Imagine this.
You’re running an API-based service. Your users authenticate via OAuth2, and your gateway (we’re using Kong) is out front managing all that traffic. Things look good. But then your traffic spikes. You’re at 1000+ requests per second, and every single request needs to be authenticated.
So you start wondering: How much overhead is authentication adding? Is introspection slowing things down? Are expired JWTs being caught properly? What about invalid tokens?
I set up a test to find out. I used JMeter for load testing, Kong Gateway, Keycloak as my OAuth2 provider, and a basic Express.js API. The results? Pretty interesting.
Setup Overview
I kept it simple:
- API Gateway: Kong
- OAuth2 Provider: Keycloak
- Backend API: Express.js returning JSON
- Token Strategy: JWTs from Keycloak, validated using Kong’s JWT plugin
- Load Testing Tool: Apache JMeter
- Environment: Dockerised containers for Kong and Keycloak
Deployment Note
Both Kong Gateway and Keycloak were containerised using Docker for quick setup and isolation. This allowed to spin up clean, reproducible environments across test runs without worrying about system-level conflicts or manual config drift.
Test Scenarios
- Requests with valid JWTs
- Requests with expired JWTs
- Requests with malformed/invalid JWTs
- Real-time introspection of tokens
- Load levels at 100, 500, and 1000 RPS
A quick note here:
The security gap where expired JWTs get through is actually deliberate. I could’ve easily used the OAuth2 Introspection plugin, or configured JMeter to simulate
"active": false"responses for expired tokens. But I intentionally stuck with the JWT plugin to demonstrate the implications when expiry validation is left to assumption. Consider this a teaching moment.
Test Overview
| Test Area | Scope |
|---|---|
| Gateway Auth Layer | JWT Plugin |
| Load Levels | 100, 500, and 1000 RPS |
| Token Types | Valid, Expired, Invalid |
| APIs Tested | Introspect Token, Auth API Req. |
| Traffic Pattern | Constant Throughput Timer |
Token Validation vs Introspection
Quick clarification:
- Token Validation is local. Kong uses the JWT plugin to check the token signature and expiry. It’s super fast. No external call needed.
- Token Introspection asks Keycloak, “Hey, is this token still valid?” It’s more secure, especially if you need to revoke tokens. But it does introduce a network round-trip.
Validation = lightweight, introspection = real-time truth.
Industry Awareness: What We Know vs What We Do
Everyone’s heard of JWT and OAuth2. But when you get into the weeds, many setups are half-baked.
- Some teams skip introspection entirely.
- Others don’t configure expiration checks properly.
- Many think JWT = secure, full stop.
We’ve seen it happen across industries – ecommerce, travel, finance. The standards are there, but implementation varies wildly.
Why does this matter? Because in high-throughput systems, one bad assumption about token handling can become a gaping hole at scale.
The Real Load Testing Data
Test Data Summary
| Load | Scenario | Avg Resp (ms) | Error % | Throughput (RPS) | Notes |
|---|---|---|---|---|---|
| 100 | Generate Token | 162.0 | 0.00% | 6.17 | One-time generation |
| 100 | Introspect Valid JWT | 2.81 | 0.00% | 51.13 | All good |
| 100 | API Request with Valid JWT | 3.17 | 0.00% | 50.55 | Local validation smooth |
| 100 | Introspect Expired JWT | 2.37 | 100.00% | 51.13 | Expected assertion failure due to missing “active” |
| 100 | API Request with Expired JWT | 2.95 | 0.00% | 50.55 | 200 OK despite token expiry |
| 100 | Introspect Invalid JWT | 2.13 | 100.00% | 51.10 | Expected assertion failure due to missing “active” |
| 100 | API Request with Invalid JWT | 1.33 | 100.00% | 50.56 | Returns expected 401 |
| 500 | Generate Token | 162.0 | 0.00% | 6.17 | Same baseline |
| 500 | Introspect Valid JWT | 1.83 | 0.00% | 251.18 | Holding steady |
| 500 | API Request with Valid JWT | 2.07 | 0.00% | 250.39 | Fast even at 500 RPS |
| 500 | Introspect Expired JWT | 1.46 | 100.00% | 251.19 | Expected assertion failure due to missing “active” |
| 500 | API Request with Expired JWT | 2.01 | 0.00% | 250.39 | Same 200 OK despite token expiry as before |
| 500 | Introspect Invalid JWT | 1.36 | 100.00% | 251.20 | Expected assertion failure due to missing “active” |
| 500 | API Request with Invalid JWT | 0.88 | 100.00% | 250.44 | Returns expected 401 |
| 1000 | Generate Token | 199.0 | 0.00% | 5.03 | Slight bump in token gen time |
| 1000 | Introspect Valid JWT | 1.49 | 0.00% | 501.06 | Excellent even at 1K RPS |
| 1000 | API Request with Valid JWT | 1.70 | 0.00% | 500.35 | Efficient |
| 1000 | Introspect Expired JWT | 1.34 | 100.00% | 501.12 | Expected assertion failure due to missing “active” |
| 1000 | API Request with Expired JWT | 1.87 | 0.00% | 500.32 | Same 200 OK despite token expiry as before |
| 1000 | Introspect Invalid JWT | 1.21 | 100.00% | 501.12 | Expected assertion failure due to missing “active” |
| 1000 | API Request with Invalid JWT | 0.81 | 100.00% | 500.35 | Returns expected 401 |
Key Observations
Performance Summary
Average Response Times (ms)
| Operation | 100 RPS | 500 RPS | 1000 RPS | Trend |
|---|---|---|---|---|
| Generate Token | 162 | 162 | 199 | Acceptable |
| Introspect Valid JWT | 2.81 | 1.83 | 1.49 | Improves w/ load |
| API Request with Valid JWT | 3.17 | 2.07 | 1.70 | Improves w/ load |
| Introspect Expired JWT | 2.37 | 1.46 | 1.34 | Stable |
| API Request with Expired JWT | 2.95 | 2.01 | 1.87 | Stable (but wrong) |
| Introspect Invalid JWT | 2.13 | 1.36 | 1.21 | Stable |
| API Request with Invalid JWT | 1.33 | 0.88 | 0.81 | Stable |
Throughput (Transactions/sec)
| Scenario | 100 RPS | 500 RPS | 1000 RPS | Observation |
|---|---|---|---|---|
| Valid JWT APIs | ~50 | ~250 | ~500 | ✅ Linear Scaling |
| Expired/Invalid | ~50 | ~250 | ~500 | ✅ Linear Scaling |
System handles scaling well with consistent throughput across all RPS levels.
Network Bandwidth (KB/sec)
| Load | Received | Sent | Total |
|---|---|---|---|
| 100 | 18.24 | 67.08 | 85.32 KB/s |
| 500 | 89.86 | 330.94 | 420.80 KB/s |
| 1000 | 179.36 | 660.72 | 840.08 KB/s |
Bandwidth usage increases linearly with throughput, confirming efficient scaling.
Error Rate
| Scenario | Total Requests | Failed Requests | Error Rate |
|---|---|---|---|
| All Valid Token Flows | 27657 | 0 | 0.00% ✅ |
| All Expired/Invalid Introspect | 92262 | 92262 | 100% ✅ |
| API Request with Invalid JWT | 54094 | 54094 | 100% ✅ |
| API Request with Expired JWT | 48090 | 0 | ❌ 0.00% (unexpected) |
Expired JWTs not rejected by Gateway, indicating a critical logic gap in the JWT plugin.
Observation Summary
Valid JWTs Scale Exceptionally Well
- Linear throughput at ~50/250/500 TPS
- Response times improved with higher load due to connection reuse
Expired JWTs Are Not Rejected by the Gateway
Despite being clearly flagged as inactive during introspection, expired JWTs were still accepted when routed through Kong’s JWT plugin.
| Operation | Expected Outcome | Actual Outcome |
| Introspect Expired JWT | 401 / “active”: false | ✅ Correct |
| API Request with Expired JWT | 401 or 403 (reject) | ❌ 200 OK |
This wasn’t an accident. I deliberately didn’t configure expiry validation into the plugin to simulate the security gap that happens in many real-world systems. The aim was to show how dangerous that can be.
Invalid JWTs Behave As Expected
- Properly rejected with 401 Unauthorized
- Works fine across all RPS levels
Why This Matters
Under realistic traffic (100–1000 RPS), we saw that invalid and expired tokens were rejected consistently only when tested via introspection. That means:
- JWT validation is performant but blind to token revocation.
- Introspection catches revoked/expired tokens but needs smarter assertions.
If you don’t know what your API gateway is allowing through, you may already have a security gap in production.
Key Takeaways
- Token validation is fast, but it’s not enough. Local JWT plugins may skip expiry checks unless explicitly configured – leading to security blind spots.
- Introspection brings accuracy, especially for expired or revoked tokens – but at the cost of extra latency and backend pressure.
- Security ≠ Speed. You need both. The safest setup balances lightweight validation with real-time introspection when needed.
- Gateway behaviour must be verified under load. Some bugs or config misses only surface at scale – like our expired token acceptance.
- Teaching through failure is powerful. This test intentionally let expired JWTs through to highlight what happens when expiry enforcement is assumed rather than verified.
Final Thoughts
Authentication layers often get treated as a black box – something that “just works.” But as this test showed, what you don’t verify might come back to bite you – especially at scale.
I deliberately built a misconfigured setup using the JWT plugin to simulate a real-world mistake many teams make: assuming the plugin checks everything. It doesn’t. You have to wire it up properly or switch to introspection for accuracy.
In production, that 200 OK from an expired token isn’t just a number but a potential breach.
So, next time you’re load testing your gateway, don’t just watch the RPS and latency graphs. Throw in a few expired tokens. A few malformed ones. You’ll be surprised what falls through.
References
- Keycloak Docker Guide: https://www.keycloak.org/getting-started/getting-started-docker
- Kong Docker Guide: https://developer.konghq.com/gateway/install/docker
- Kong JWT Plugin Guide: https://developer.konghq.com/plugins/jwt/
- Kong OAuth2-Introspection Plugin Guide: https://developer.konghq.com/plugins/oauth2-introspection/
- Case Study 1: https://www.trendmicro.com/vinfo/us/security/news/vulnerabilities-and-exploits/kong-api-gateway-misconfigurations-an-api-gateway-security-case-study
- Case Study 2: https://medium.com/%40nandakishorep/jwt-security-nightmare-why-my-bulletproof-authentication-system-left-users-exposed-to-token-404a100dfc6e