Which Metrics Matter in CDN Performance Comparisons? Evaluating Latency, Availability, and Throughput

When comparing two CDNs, the easiest metric to obtain is usually average latency.

Vendor A shows 45 ms and Vendor B 52 ms, so A looks faster. But on closer inspection, A's P95 latency may be noticeably higher, with frequent timeouts in some regions; B may be slightly slower typically but more stable in core user regions.

For software downloads or video delivery, a difference of tens of milliseconds in latency may matter far less than throughput. For login, payment, and API workloads, availability, error rate, and tail latency are more critical than static image download speed.

So a CDN performance comparison should not only ask “which vendor has the lowest latency?” It should answer four questions at the same time:

1. How quickly most users can access content;

2. How the slowest segment of users experiences the service;

3. Whether requests complete successfully and consistently;

4. Whether the CDN is suited to the type of content the business delivers.

Under identical test conditions, you should compare at least:Region and carrier, P50 latency, P95 latency, availability, failure rate, throughput, and sample size. After moving to a real-business POC, also add cache hits, origin performance, error status codes, and peak-period behavior.

Key CDN performance comparison metrics at a glance

These metrics cannot be used independently of the test object. A latency ranking based on a 5KB static file does not directly represent the download experience for a 10MB software package; testing a warm-cache asset also says nothing about dynamic APIs or origin performance on a cache MISS.

P50 latency: the typical experience for most requests

P50, also called the median, is the value in the middle when all request durations are sorted from lowest to highest.

Suppose five requests to a CDN take:

30ms、35ms、40ms、45ms、800ms

Its P50 is 40 ms. This value represents the typical level for most requests and, unlike an average, is not noticeably inflated by a single 800 ms outlier.

P50 is useful for:

• 
  Estimating the approximate access speed for ordinary users;
  

• 
  Comparing typical performance across CDNs in the same region;
  

• 
  Observing overall changes before and after migration;
  

• 
  Assessing basic delivery capability for static files and standard pages.
  

But P50 also hides tail issues. In the test above, P50 was only 40 ms, yet some users still experienced 800 ms requests. If the business looks only at P50, it may conclude that performance is good while support keeps receiving complaints about slow access.

Therefore, P50 should be used together with P95, failure rate, and regional distribution.

P95 latency: what slower users experience

P95 means 95% of requests are at or below this value, while the remaining roughly 5% are slower. It reflects tail latency better than P50.

Google SRE guidance on service level indicators also emphasizes that monitoring only average latency may miss tail performance changes; P50 is closer to the typical experience, while higher percentiles help show what slower users experience.

P95 is especially suitable for:

• 
  Login and identity verification;
  

• 
  Payments and order submission;
  

• 
  Dynamic APIs;
  

• 
  SaaS consoles;
  

• 
  Game event APIs;
  

• 
  Frequently refreshed pages;
  

• 
  Enterprise applications sensitive to response consistency.
  

Three patterns may emerge in a comparison:

If Vendor A has a P50 of 40 ms and P95 of 300 ms, while Vendor B has a P50 of 50 ms and P95 of 90 ms, B may actually be the safer choice for critical APIs.

P95 is not the lower the better in isolation from the sample. With only a few dozen samples, a small number of requests can noticeably change the result, so sample count and coverage must be checked at the same time.

Can average latency still be used?

Yes, but it should not be used on its own.

Average latency is useful for observing overall trends and for capacity and long-term change analysis. However, it is highly sensitive to extreme values. A single timeout, node failure, or temporary network jitter can noticeably raise the average.

For CDN performance comparisons, observe the following combination:

• 
  P50: typical users;
  

• 
  P95: slower users;
  

• 
  Maximum: extreme outliers;
  

• 
  Average: overall trend;
  

• 
  Standard deviation or range: stability;
  

• 
  Failure rate: requests that did not complete successfully.
  

If a platform provides only an “average response time” without node-level detail, percentiles, and failure information, it is difficult to tell whether the gap comes from most users or a small number of abnormal nodes.

Availability: first confirm whether requests can succeed

No matter how fast it is, frequent errors or timeouts still do not count as good performance.

At the test level, availability can generally be understood as:

成功完成的有效请求数 ÷ 符合统计条件的请求总数 × 100%

But the definition of “success” must be determined in advance.

Treating any completed HTTP request as success may count 404s, 403s, or even custom error pages; accepting only 200 status codes may incorrectly exclude normal 206 partial responses or 304 cache responses.

When comparing CDNs, record separately:

• 
  Connection success rate;
  

• 
  HTTP request completion rate;
  

• 
  Proportions of 2xx, 3xx, 4xx, and 5xx;
  

• 
  DNS failures;
  

• 
  Connection timeouts;
  

• 
  TLS failures;
  

• 
  Download interruptions;
  

• 
  Test-side cancellations;
  

• 
  Errors returned by WAF or rate limiting.
  

It is important to distinguish “third-party measured availability” from “contractual SLA.” The former reflects whether probes successfully reached the target during the statistical period; the latter is the service commitment, calculation formula, exclusions, and compensation mechanism defined in the vendor contract.

For example, Amazon CloudFront's official SLA determines whether its service commitment is met based on its defined error rate and statistical conditions. Third-party probes do not use exactly the same contractual methodology, so measured availability cannot be used directly as an SLA conclusion.

Failure rate: easier to diagnose than “99.9% availability”

Availability is suitable for overall comparison; failure-rate details are better for troubleshooting.

If a CDN has 99.8% availability, you still need to ask:

• 
  Are failures concentrated in a particular country?
  

• 
  Do only China Mobile nodes fail?
  

• 
  Are they DNS errors, connection timeouts, or HTTP 5xx?
  

• 
  Do they occur mainly during evening peak hours?
  

• 
  Do they affect only IPv6?
  

• 
  Is the test target rate-limiting the probes?
  

• 
  Do all failures point to the same node IP?
  

Two CDNs may have the same availability rate, yet the nature of the failures can be completely different.

Vendor A's failures may be randomly scattered, occurring occasionally in each region; Vendor B's failures may all be concentrated in the country where its main paying users are located. Even with the same number, the business impact is not the same.

Therefore, CDN performance comparisons should retain raw error types, not just success-rate percentages.

Throughput: video and download workloads cannot be judged on latency alone

Latency measures how quickly a request starts responding; throughput measures how fast content can be transferred continuously.

For a 5KB icon, a latency difference of 40 ms versus 60 ms may be noticeable; for a 2GB video file, completion time is often determined by sustained throughput, congestion control, and transfer stability.

Throughput is usually expressed in Mbps:

吞吐量 = 已传输字节数 × 8 ÷ 传输时间

At a minimum, control the following during testing:

• 
  Same file size;
  

• 
  Same URL and cache state;
  

• 
  Same test nodes;
  

• 
  Same concurrency;
  

• 
  Whether compression is enabled;
  

• 
  Whether Range requests are supported;
  

• 
  Sufficient test duration;
  

• 
  No vendor-side or business-side rate limiting triggered.
  

If files are too small, DNS, connection, and TLS time account for a large share, making it difficult to reflect sustained transfer capability. For large-file tests, avoid consuming bandwidth repeatedly without purpose.

Google Cloud positions its Media CDN for high-throughput scenarios such as video streaming and large-file downloads, while standard Cloud CDN focuses more on web pages and general web assets. This also shows that different workloads place different weights on CDN performance metrics.

TTFB: important, but not simply equivalent to origin processing time

TTFB is the time from request start to receipt of the first byte of the response. It is often used to judge CDN or origin responsiveness, but this metric covers more stages than “server processing time.”

According to web.dev's definition, TTFB may include redirects, DNS lookup, connection and TLS negotiation, and waiting for the server or CDN to begin returning content.

Therefore, a high TTFB may come from:

• 
  Slow DNS resolution;
  

• 
  High TCP or QUIC connection time;
  

• 
  Slow TLS handshake;
  

• 
  CDN cache MISS;
  

• 
  Origin fetch distance too far;
  

• 
  Slow origin application or database;
  

• 
  Time spent on WAF inspection;
  

• 
  Request queuing;
  

• 
  Multiple redirects.
  

When comparing two CDNs, break TTFB down further instead of concluding that Vendor A's edge processing is poor simply because its TTFB is higher.

A more reasonable breakdown is:

DNS
→ 建立连接
→ TLS协商
→ 请求与等待首字节
→ 内容下载

TTFB may be completely different for static cache HITs and dynamic origin requests, so the two scenarios should be tested separately.

Regional performance: global averages may obscure core markets

A CDN that performs well in global average latency is not necessarily the best performer in your core market.

If your users are mainly from mainland China, observe separately:

• 
  East China, South China, North China, Northeast China, Southwest China, Northwest China, and Central China;
  

• 
  China Telecom, China Unicom, and China Mobile;
  

• 
  Residential broadband and data center networks;
  

• 
  IPv4 and IPv6;
  

• 
  Core cities and long-tail regions.
  

If the business targets overseas markets, break results down by country or market, such as North America, Europe, Japan, Southeast Asia, Latin America, and the Middle East. Do not combine the United States, Singapore, Germany, and Brazil into one “overseas average.”

When comparing regional performance, prioritize answering:

1. 
   Can core users hit nodes locally or in a reasonable region?
   

2. 
   Is the gap too large between different carriers?
   

3. 
   Are slow nodes concentrated in a particular country or ASN?
   

4. 
   Is the global average dominated by regions with more samples?
   

5. 
   Does the sample distribution closely match the real user distribution?
   

If 70% of paying users are in Japan, Japan's regional performance should carry more weight, rather than being diluted by averaging samples from other regions worldwide.

Sample size: precise numbers without sufficient sample support are meaningless

Seeing P50 values of 42.1 ms and 43.7 ms for two CDNs makes it easy to assume the first is faster. But if one side has tens of thousands of samples and the other only a few dozen, this small gap may not be stable.

Sample size should be considered together with the following:

• 
  Statistical period;
  

• 
  Test regions;
  

• 
  Number of samples per region;
  

• 
  Completion rate;
  

• 
  Node type;
  

• 
  Whether a uniform test object was used;
  

• 
  How outliers are handled;
  

• 
  Whether it is real-time measurement or a reference baseline.
  

A large total sample count does not necessarily mean reasonable coverage. If 100,000 samples are mainly from North America, they cannot adequately demonstrate performance in mainland China or Southeast Asia.

CdnChart's currenttesting methodology and scoring criteriastipulate that inclusion in actual coverage calculations requires at least 3 qualifying regions, each with at least 24 samples in the last 24 hours and a completion rate of 95%; dimensions that do not meet the real-metric threshold may continue to display the reference baseline. The page also publicly notes that real probe coverage is still being built, and that target-site fluctuations, rate limiting, and temporary network jitter can all affect results.

This type of explanation is important because it tells users which numbers come from current real samples and which still need to be verified through subsequent testing.

Cache hit ratio: determines whether the CDN is actually delivering content at the edge

CDN performance depends not only on how close nodes are to users, but also on whether content can be returned directly from those nodes.

Cache hit ratio generally refers to the proportion of cacheable requests served directly from edge cache. When the hit ratio is low, even if edge node connections are fast, many requests still need to go to the origin, which can increase TTFB and origin load.

When comparing cache performance, distinguish:

• 
  Request hit ratio;
  

• 
  Byte hit ratio;
  

• 
  Static asset hits;
  

• 
  HTML or dynamic content hits;
  

• 
  Popular content versus long-tail content;
  

• 
  Hit ratios by region;
  

• 
  HIT, MISS, EXPIRED, and REVALIDATED;
  

• 
  Share of origin-bound traffic.
  

A high request hit ratio does not necessarily mean much traffic is saved. If the hits are all small files of a few KB while large videos keep going to the origin, the byte hit ratio may still be low.

This data usually needs to be obtained from each CDN vendor's console or logs; third-party public tests can hardly fully reproduce an enterprise's own cache rules. Therefore, cache hit ratio is better compared during trials and POCs rather than judged solely from public rankings.

Variability: consistent stability matters more than occasional speed

Performance comparisons cannot look only at the “fastest single result.”

The same CDN may perform well at dawn but slow down noticeably during evening peak hours; P50 may be stable over time while a few nodes frequently experience connection timeouts.

Recommended observations:

• 
  Hourly P50 and P95 trends;
  

• 
  Minimum, maximum, and median;
  

• 
  Differences between peak and off-peak periods;
  

• 
  Differences between weekdays and weekends;
  

• 
  Whether failing nodes recur;
  

• 
  Jitter range for the same node;
  

• 
  CDN switchover or failure recovery time.
  

For API, login, and transaction workloads, low variability is usually more important than occasionally achieving extremely low latency. Users are often more tolerant of “slightly slower every time” than “fast most of the time but occasionally stuck for seconds.”

Can composite scores be used directly?

Composite scores are useful for quickly narrowing the field, but they are not a substitute for business judgment.

Any composite score implies weights. For example:

综合评分
= 延迟权重
+ 可用性权重
+ 吞吐量权重
+ 安全或协议权重

If a platform places more weight on latency, the fastest vendor will rank higher; if availability weight is increased, results may change; adding throughput weight for video workloads will change the ranking again.

Before using a composite score, check:

• 
  Which metrics are included;
  

• 
  What each weight is;
  

• 
  How missing data is handled;
  

• 
  Whether reference baselines are included;
  

• 
  How global and regional metrics are combined;
  

• 
  Whether samples meet the public threshold.
  

When composite scores are close, do not agonize over a one- or two-point difference. A better approach is to keep several candidates and move on to regional testing and real-business POCs.

How should different workloads weight their metrics?

Corporate websites and content sites

Prioritize:

• 
  P50 in core regions;
  

• 
  P95;
  

• 
  Availability;
  

• 
  Status codes and failure rate;
  

• 
  Static cache hits;
  

• 
  Image and script download time.
  

These sites can place appropriate emphasis on typical access speed, but they must not ignore mobile users, the three major carriers, and page assets.

API and SaaS platforms

Prioritize:

• 
  P95 and even higher percentiles;
  

• 
  Dynamic request TTFB;
  

• 
  Connection and TLS time;
  

• 
  5xx and timeout rates;
  

• 
  Peak-period stability;
  

• 
  Actual compatibility with WebSocket, HTTP/2, or HTTP/3.
  

API requests usually cannot be cached long term, so static-file speed test results have limited reference value.

Images and software downloads

Prioritize:

• 
  Throughput;
  

• 
  Download success rate;
  

• 
  Range requests;
  

• 
  Large-file caching;
  

• 
  Peak-period speed;
  

• 
  Byte hit ratio;
  

• 
  Traffic and origin costs.
  

A latency difference of tens of milliseconds is usually less important than sustained download speed.

Video on demand and live streaming

Prioritize:

• 
  Throughput;
  

• 
  First-frame or startup time;
  

• 
  Buffering/stall rate;
  

• 
  Segment request success rate;
  

• 
  Hits for popular and long-tail content;
  

• 
  Peak concurrency;
  

• 
  Stability across regions.
  

Public CDN rankings can only provide an initial screen. Startup, stalling, and segment errors need to be tested further with real players, actual video encoding, and production traffic.

Websites mainly serving mainland China users

Prioritize:

• 
  China Telecom, China Unicom, and China Mobile;
  

• 
  Core provinces and cities;
  

• 
  P50 and P95;
  

• 
  Failure rate;
  

• 
  Node scheduling;
  

• 
  IPv4 and IPv6;
  

• 
  ICP filing, onboarding requirements, and technical support.
  

Global average latency should not obscure the actual differences across mainland China's three major carriers.

How to use CdnChart for CDN performance comparisons

CdnChart'sCDN vendor comparisonpage currently supports selecting up to 5 CDNs for side-by-side comparison, showing fields such as composite score, P50 latency, P95 latency, availability, P50 throughput, measurable security items, HTTP/3, and billing model.

As of 00:00:14 on September 26, 2026 shown on the page, the comparison page candidate pool contains 61 vendors, including 9 from mainland China and 52 global vendors, with 227,522 page samples in 24 hours.

However, at the same time, the methodology page shows 6/350 healthy probes and 0 coverage-ready vendors, and notes that real metrics that have not met the threshold may continue to use the reference baseline.

Therefore, the current page is better suited for:

• 
  Understanding which metrics can be compared side by side;
  

• 
  Narrowing the vendor list from the candidate pool;
  

• 
  Observing preliminary differences under a consistent methodology;
  

• 
  Identifying metrics that need focused validation in a POC.
  

It is not suitable to treat a current ranking or a difference of a few milliseconds as the final procurement conclusion. When using it, also check the data update time, sample size, and methodology notes to confirm whether a given metric has sufficient real coverage.

If you are still unsure how to narrow the list from the rankings, see “Global CDN Rankings: How to Shortlist Vendors That Fit Your Business.”

How should a real-business POC be designed?

Public data is for initial screening; the final comparison should use your own business content.

It is recommended to prepare the same test objects for each candidate CDN:

Control the following variables during testing:

• 
  Use the same URL and response content;
  

• 
  Use the same nodes, regions, and carriers;
  

• 
  Record response size;
  

• 
  Distinguish cache HIT from MISS;
  

• 
  Test both IPv4 and IPv6;
  

• 
  Cover business peak periods;
  

• 
  Run continuously for multiple days;
  

• 
  Retain status codes and failure reasons;
  

• 
  Calculate P50 and P95 separately;
  

• 
  Do not make decisions based on a single fastest result.
  

If candidate vendors require different access domains, prepare the same origin, cache rules, and files for the test domains to avoid configuration differences contaminating the results.

What else should be compared beyond performance?

When performance is close, what really affects long-term use is often not a difference of a few milliseconds, but product capabilities and operating costs.

Also independently verify:

• 
  Billing for traffic, requests, and origin fetches;
  

• 
  Cache purge and preload limits;
  

• 
  Log availability and latency;
  

• 
  WAF, DDoS, and bot management;
  

• 
  Origin protection;
  

• 
  Certificate management;
  

• 
  API and automation capabilities;
  

• 
  Configuration deployment speed;
  

• 
  Technical support;
  

• 
  SLA and compensation terms;
  

• 
  Compliance and data regions;
  

• 
  Ease of exit and migration.
  

Performance testing cannot replace contract review, security testing, and price negotiation. Vendor-advertised node counts, mitigation capacity, and SLAs also cannot be directly mixed into third-party measured scores.

Frequently asked questions

What are the three most important metrics in a CDN performance comparison?

If you can choose only three to start with, look at P50, P95, and availability in core regions. For video and download workloads, throughput should also be a core metric.

What is the difference between P50 and average latency?

P50 is the median and is closer to the typical request; the average is affected by a small number of very slow requests. Both are worth looking at, but P50 is usually better for describing most users' access experience.

Is a CDN necessarily better the lower its P95?

Not necessarily. You also need to consider sample size, regional distribution, test object, and failure rate. A low P95 from a small sample may not be stable, and testing small static files does not represent dynamic APIs.

Does 99.9% CDN availability mean the 99.9% SLA has been met?

No. Third-party availability comes from probe tests, while SLAs are calculated according to the time window, error definitions, and exclusions in the vendor contract; the two methodologies may differ.

What workloads suit a CDN with high throughput but average latency?

They are usually better suited to video, large-file downloads, software distribution, and large-image delivery. For small API and login requests, connection and response latency may be more important.

Why do CDN rankings differ from my own test results?

Rankings use uniform targets and sample methodologies, while your business has different origins, cache rules, user regions, and content sizes. Rankings are for initial screening; real-business POCs are for final validation.

Should CDN tests use the homepage or static files?

Test both. Static files show edge caching and transfer capability; homepages or APIs show dynamic requests, origin fetches, and origin processing. Testing only one type of content can lead to one-sided conclusions.

How should you choose when two CDNs perform very similarly?

Continue comparing P95 in core regions, failure rate, peak-period variability, cache hits, cost, logs, technical support, and security capabilities. A difference of a few milliseconds usually should not outweigh business fit and operational cost.