In this document you will find benchmark results for different endpoints of ORY Hydra. All benchmarks are executed using rakyll/hey. Please note that these benchmarks run against the in-memory storage adapter of ORY Hydra. These benchmarks represent what performance you would get with a zero-overhead database implementation.
We do not include benchmarks against databases (e.g. MySQL or PostgreSQL) as the performance greatly differs between deployments (e.g. request latency, database configuration) and tweaking individual things may greatly improve performance. We believe, for that reason, that benchmark results for these database adapters are difficult to generalize and potentially deceiving. They are thus not included.
This file is updated on every push to master. It thus represents the benchmark data for the latest version.
All benchmarks run 10.000 requests in total, with 100 concurrent requests. All benchmarks run on Circle-CI with a "2 CPU cores and 4GB RAM" configuration.
ORY Hydra uses BCrypt to obfuscate secrets of OAuth 2.0 Clients. When using flows such as the OAuth 2.0 Client Credentials
Grant, ORY Hydra validates the client credentials using BCrypt which causes (by design) CPU load. CPU load and performance
depend on the BCrypt cost which can be set using the environment variable BCRYPT_COST. For these benchmarks,
we have set BCRYPT_COST=8.
This section contains various benchmarks against OAuth 2.0 endpoints
Summary:
Total: 1.1567 secs
Slowest: 0.0522 secs
Fastest: 0.0002 secs
Average: 0.0111 secs
Requests/sec: 8645.4305
Total data: 1550000 bytes
Size/request: 155 bytes
Response time histogram:
0.000 [1] |
0.005 [2272] |■■■■■■■■■■■■■■■■■■■■■■■■■
0.011 [2080] |■■■■■■■■■■■■■■■■■■■■■■■
0.016 [3615] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.021 [1493] |■■■■■■■■■■■■■■■■■
0.026 [373] |■■■■
0.031 [94] |■
0.037 [37] |
0.042 [21] |
0.047 [11] |
0.052 [3] |
Latency distribution:
10% in 0.0015 secs
25% in 0.0061 secs
50% in 0.0119 secs
75% in 0.0150 secs
90% in 0.0189 secs
95% in 0.0213 secs
99% in 0.0293 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0001 secs, 0.0002 secs, 0.0522 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0087 secs
req write: 0.0001 secs, 0.0000 secs, 0.0085 secs
resp wait: 0.0107 secs, 0.0002 secs, 0.0438 secs
resp read: 0.0002 secs, 0.0000 secs, 0.0109 secs
Status code distribution:
[200] 10000 responses
This endpoint uses BCrypt.
Summary:
Total: 24.7073 secs
Slowest: 0.8872 secs
Fastest: 0.0206 secs
Average: 0.2403 secs
Requests/sec: 404.7391
Total data: 1570000 bytes
Size/request: 157 bytes
Response time histogram:
0.021 [1] |
0.107 [1165] |■■■■■■■■■■■■■■
0.194 [2017] |■■■■■■■■■■■■■■■■■■■■■■■■
0.281 [3433] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.367 [2008] |■■■■■■■■■■■■■■■■■■■■■■■
0.454 [835] |■■■■■■■■■■
0.541 [400] |■■■■■
0.627 [107] |■
0.714 [29] |
0.801 [4] |
0.887 [1] |
Latency distribution:
10% in 0.1031 secs
25% in 0.1831 secs
50% in 0.2135 secs
75% in 0.2988 secs
90% in 0.3930 secs
95% in 0.4701 secs
99% in 0.5878 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0001 secs, 0.0206 secs, 0.8872 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0134 secs
req write: 0.0001 secs, 0.0000 secs, 0.0708 secs
resp wait: 0.2395 secs, 0.0205 secs, 0.8871 secs
resp read: 0.0004 secs, 0.0000 secs, 0.0873 secs
Status code distribution:
[200] 10000 responses
This endpoint uses BCrypt and generates IDs and secrets by reading from which negatively impacts performance. Performance will be better if IDs and secrets are set in the request as opposed to generated by ORY Hydra.
This test is currently disabled due to issues with /dev/urandom being inaccessible in the CI.
Summary:
Total: 0.7636 secs
Slowest: 0.0313 secs
Fastest: 0.0002 secs
Average: 0.0073 secs
Requests/sec: 13095.4930
Total data: 2670000 bytes
Size/request: 267 bytes
Response time histogram:
0.000 [1] |
0.003 [1990] |■■■■■■■■■■■■■■■■■■■■
0.006 [1573] |■■■■■■■■■■■■■■■■
0.010 [4022] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.013 [1925] |■■■■■■■■■■■■■■■■■■■
0.016 [256] |■■■
0.019 [94] |■
0.022 [37] |
0.025 [55] |■
0.028 [36] |
0.031 [11] |
Latency distribution:
10% in 0.0007 secs
25% in 0.0046 secs
50% in 0.0085 secs
75% in 0.0095 secs
90% in 0.0108 secs
95% in 0.0125 secs
99% in 0.0221 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0000 secs, 0.0002 secs, 0.0313 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0099 secs
req write: 0.0001 secs, 0.0000 secs, 0.0128 secs
resp wait: 0.0063 secs, 0.0001 secs, 0.0281 secs
resp read: 0.0005 secs, 0.0000 secs, 0.0164 secs
Status code distribution:
[200] 10000 responses
Summary:
Total: 0.6049 secs
Slowest: 0.0284 secs
Fastest: 0.0002 secs
Average: 0.0058 secs
Requests/sec: 16531.9945
Total data: 2650000 bytes
Size/request: 265 bytes
Response time histogram:
0.000 [1] |
0.003 [2716] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.006 [2860] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.009 [1873] |■■■■■■■■■■■■■■■■■■■■■■■■■■
0.011 [2048] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.014 [316] |■■■■
0.017 [78] |■
0.020 [28] |
0.023 [30] |
0.026 [44] |■
0.028 [6] |
Latency distribution:
10% in 0.0006 secs
25% in 0.0028 secs
50% in 0.0054 secs
75% in 0.0087 secs
90% in 0.0098 secs
95% in 0.0115 secs
99% in 0.0186 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0001 secs, 0.0002 secs, 0.0284 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0128 secs
req write: 0.0001 secs, 0.0000 secs, 0.0123 secs
resp wait: 0.0034 secs, 0.0001 secs, 0.0259 secs
resp read: 0.0012 secs, 0.0000 secs, 0.0147 secs
Status code distribution:
[200] 10000 responses