Active probing vs. RUM: Which data should you trust when choosing a CDN?

Active probing shows one CDN's latency is only 40 ms, while real user monitoring shows P95 exceeding 500 ms; another CDN performs unremarkably on test nodes, yet once live, users actually access it faster.

When results like this occur, which one should you trust?

The answer isn't either/or:

During CDN selection, prioritize reproducible active probing for apples-to-apples comparison; after adopting a CDN, use RUM to verify whether real users genuinely benefit.

Active probing answers “which CDN performs better on the network when variables are controlled”; RUM answers “what my actual visitors experience.” They observe different things, and inconsistent data doesn't mean one is unreliable.

The real danger is treating active probing as the whole user experience, or using uncontrolled RUM data to directly rank two CDNs.

What Is Active Probing

Active probing, also called synthetic monitoring, uses probe nodes deployed in different regions and carrier networks to visit specified domains or files at preset intervals and record the time taken at each stage.

A relatively complete probe typically records:

• DNS resolution time

• TCP connection time

• TLS handshake time

• TTFB, the time from request sent to first byte received

• Content download time and transfer speed

• HTTP status code

• Resolved IP, node location, and ASN

• HTTP protocol version

• Cache hit status

• P50, P95, and failure rate

The greatest value of active probing is not how realistically it simulates a human, but its ability to control variables.

For example, when testing two CDNs, you can have nodes in the same city and on the same carrier request files of the same size at roughly the same time, with cache, compression, protocol, and origin fetch conditions aligned. Only differences measured this way come closer to reflecting differences caused by the CDNs themselves.

For specific settings for probe nodes, test rounds, and percentiles, seeCDN speed testing methodology: active probing, P50/P95, and reproducible performance rankings.

What Questions Active Probing Is Good At Answering

Active probing is particularly suited to answering the following questions:

1. Can a domain be resolved and accessed reliably in a given region?

2. Which nodes do different carriers route to?

3. On a first connection, how long do DNS, TCP, and TLS each take?

4. After a CDN cache hit, does TTFB drop significantly?

5. What are the sustained download speed and throughput for large files?

6. After a configuration change, has latency or failure rate regressed?

7. Does a candidate CDN that hasn't yet carried production traffic meet expectations?

The last point is especially important for CDN selection. Before user traffic is switched to a candidate CDN, there is naturally no real user data for that CDN, but active probing can compare its basic network quality in advance.

Limitations of Active Probing

Probe nodes are typically deployed on cloud servers, in data centers, or on fixed networks, and they may not represent home broadband, campus networks, mobile networks, or weak-signal environments.

As a result, active probing may miss:

• Last-mile congestion on home broadband

• Signal fluctuations and handovers on mobile networks

• CPU and rendering bottlenecks on older devices

• Routing anomalies in specific small carriers or regions

• The effects of browser cache, connection reuse, and Service Workers

• Page sluggishness caused by third-party scripts, images, and front-end code

So good active probing results only prove good performance under the chosen test conditions; they do not directly imply that all users will experience fast access.

What Is RUM (Real User Monitoring)?

RUM stands for Real User Monitoring. It uses data such as the Performance API, Navigation Timing, Resource Timing, and Web Vitals provided by browsers to collect the performance experienced by real visitors during actual visits.

RUM data comes from real users, so it naturally includes:

• The user's region and network carrier

• Access conditions such as home broadband and mobile networks

• Differences across devices such as phones, tablets, and computers

• Browser version and device performance

• DNS cache, browser cache, and connection reuse

• Page type, number of resources, and third-party scripts

• The time periods when users actually visit

• Page experience metrics such as LCP, INP, and CLS

• Long-tail latency and intermittent errors in real traffic

web.devNotes on performance measurement toolsLab data describes performance under hypothetical conditions, while field data or RUM describes the performance real users actually experience. The two should complement each other, not replace each other.

What RUM Is Best At Uncovering

RUM's greatest value is uncovering real problems that averages hide.

For example, overall TTFB may appear to be only 150 ms, but breaking down the data may reveal:

• P95 in a certain province reaches 1.2 seconds;

• Mobile network users are significantly slower than home broadband users;

• LCP is notably high on low-end Android devices;

• The homepage is normal, but product detail pages are delayed because APIs are slow;

• Weekdays are normal, but long-tail latency appears on a certain carrier during evening peak;

• CDN cache hits are normal, but third-party scripts slow down the page.

These kinds of issues are hard to fully cover with a small number of fixed probe nodes.

RUM Isn't Inherently Fair Either

Although RUM data comes from real users, “real” doesn't mean “directly comparable across groups.”

Suppose CDN A carries mobile traffic during a promotion, while CDN B carries desktop traffic on an ordinary weekday. Even if B's data looks better, that doesn't prove B's CDN performance is stronger.

Variables that affect RUM comparability include:

• The two user groups have different proportions of regions and carriers

• Page version or number of resources differs

• One group is mostly new visitors, while the other heavily reuses cache

• Requested objects and cache hit ratios differ

• Marketing campaigns change traffic composition

• Public internet congestion varies by time period

• User device performance differs significantly

Therefore, when comparing CDNs with RUM, it's best to use stable traffic splitting so both CDNs carry similarly structured traffic at the same time, and observe it segmented by region, carrier, device, and page type.

Key Differences Between Active Probing and RUM

Simply put:

• Active probing is more like a standardized health check;

• RUM is more like continuously recording each real user's actual experience.

Which One Should You Trust When Choosing a CDN?

Initial Selection and RFP Stage: Active Probing Carries More Weight

Before a candidate CDN is brought into production, there isn't enough real user data. At this stage, rely mainly on active probing to compare:

1. Availability in target regions and carriers;

2. DNS, TCP, TLS, and TTFB;

3. Response speed after cache hits;

4. Static asset download speed;

5. P50, P95, and failure rate;

6. Differences between IPv4 and IPv6;

7. Differences between first connection and repeat visits;

8. Stability during peak and off-peak periods.

Test nodes should not be limited to Beijing, Shanghai, Guangzhou, and major carriers; they should be weighted according to the site's actual user distribution. If 30% of users come from Southeast Asia, a ranking based only on tests in mainland China has little decision-making value.

Before choosing a third-party speed testing platform, also confirm whether it discloses node locations, carriers, measurement stages, and raw results. For details, seeHow to Choose a CDN Speed Test Site: First Check Nodes, Metrics, and Result Records.

Small-Traffic Trial Stage: Use Active Probing and RUM Together

When a candidate CDN can carry part of the real traffic, the most valuable approach is stable traffic splitting.

For example, use consistent hashing so the same user always enters the same CDN group during the test period, and ensure both groups use as much of the following as possible:

• The same page version;

• The same cache policy;

• The same origin server;

• The same compression method;

• The same certificate and HTTP protocol;

• The same resource paths and file sizes;

• Similar region, carrier, and device composition.

Active probing continues to monitor the underlying network and availability, while RUM verifies page experience and real-world long-tail behavior. Only when both sets of data improve can you confidently conclude that the CDN is delivering benefits.

After Full Launch: RUM Carries More Weight, but Active Probing Should Not Stop

After full launch, the ultimate goal is not to make probe nodes produce pretty scores, but to give real users a more stable experience. Therefore, RUM should carry more weight in day-to-day optimization.

But active probing should still continue, because it can:

• Continuously check availability in low-traffic regions;

• Proactively detect DNS, certificate, and node anomalies;

• Detect regional failures before real users complain;

• Use fixed conditions to determine whether a configuration change caused a regression;

• Distinguish CDN issues, origin issues, and page issues.

A reasonable combination is: active probing handles continuous inspection and diagnosis, while RUM verifies real-world impact.

How to Judge When the Two Data Sources Conflict

Case 1: Active Probing Is Fast, RUM Is Slow

Common causes include:

• Probe nodes are in data centers, while real users are on mobile networks or home broadband;

• Real users are concentrated in regions with insufficient CDN coverage;

• The probe tests a single static file, while users visit a full page;

• The page contains slow APIs, large images, or third-party scripts;

• Real requests miss cache, while the probed object consistently hits cache;

• User devices have low performance, making the rendering stage take longer.

At this point, don't immediately conclude that “CDN speed testing is inaccurate.” Instead, first break down the RUM data by region, carrier, page type, cache status, and device tier.

If network requests themselves are fast but LCP is still slow, the problem is likely not the CDN network, but image dimensions, render-blocking resources, front-end code, or third-party scripts.

Case 2: Active Probing Is Slow, RUM Is Fast

Possible causes include:

• Each probe forces DNS resolution and a TLS handshake;

• Real users reuse existing connections;

• The browser cache or a Service Worker returns resources directly;

• The network where the probe node resides is not representative;

• The CDN or WAF rate-limits, challenges, or blocks probe traffic;

• Real traffic is concentrated in a few advantageous regions.

In this case, distinguish first visits from repeat visits, and check the probe node's resolved IP, carrier, and HTTP status code.

Case 3: Both Active Probing and RUM Are Slow

When both slow down at the same time, it usually means the issue deserves higher priority.

You can follow the order inDNS, connection, TLS, TTFB, and download time: which stage is most worth optimizing first?to first determine which stage the latency is concentrated in:

• Slow DNS: check authoritative DNS, recursive resolution paths, and TTL;

• Slow TCP: check node distance, routing, and packet loss;

• Slow TLS: check connection reuse, protocol version, and certificate chain;

• Slow TTFB: check cache hits, edge processing, and origin response;

• Slow download: check throughput, file size, compression, and congestion.

Don't focus only on total time, because total time only shows that something is slow, not where the slowness is.

Case 4: Active Probing Is Stable, but RUM P95 Suddenly Rises

This usually means the issue affects only some real users, for example:

• A certain region or small carrier experiences circuitous routing;

• An IPv6 path anomaly;

• Evening peak congestion on mobile networks;

• Compatibility issues on a certain type of device or browser;

• Resources or APIs on specific pages become slower.

At this point, averages often show no obvious change; focus on P75, P95, P99, and segmented dimensions.

Case 5: RUM Is Normal, but Some Probe Nodes Keep Failing

First rule out the possibility that the probe IP is being blocked by a WAF, rate-limiting policy, or bot rules. If request status codes, response headers, or challenge pages are abnormal, the probe environment may simply be restricted.

But if multiple independent nodes in the same region all fail, treat it as an early warning of a regional outage rather than dismissing it just because current RUM samples are few.

How to Build a Trustworthy CDN Evaluation Process

Step 1: First Ensure Consistent Test Conditions

When comparing multiple CDNs, at minimum align the following conditions:

• Test file content and size;

• Origin location and origin performance;

• Cache rules and cache TTL;

• Whether compression is enabled;

• HTTP/2 or HTTP/3 configuration;

• TLS certificate and certificate chain;

• IPv4 or IPv6;

• First visit or repeat visit;

• Cache hit or origin fetch;

• Test time and probe frequency.

If CDN A uses cache-hit results while CDN B goes back to origin every time, the comparison is meaningless.

Step 2: Test Cold Paths and Hot Paths Separately

It's recommended to set up at least two sets of active probes:

• Cold path: use different query parameters or a controlled refresh method to observe cache misses and origin fetches;

• Hot path: repeatedly request a fixed resource to observe performance after edge cache hits.

Note whether query parameters participate in the cache key. If you simply add random parameters arbitrarily, you may be testing not a normal user path but a special path that can never hit cache.

Step 3: Record Percentiles, Not Just Averages

When a site is “sometimes fast, sometimes slow,” averages easily hide problems.

For example, if 99 requests take 100 ms and 1 request takes 5 seconds, the average may still look acceptable, but the user experience for that one slow request is already poor. A more reasonable approach is to record all of the following:

• P50: typical experience;

• P75: experience for most users;

• P95: long-tail stability;

• P99: extreme outliers;

• Error rate and timeout rate.

For a more complete measurement method, seeWhen a Site Is Sometimes Fast and Sometimes Slow, How Should You Test CDN Speed?.

Step 4: Have RUM Record Enough Context

If RUM only reports “how many milliseconds the page load took,” its diagnostic value is very limited. While protecting privacy, it's recommended to also record:

• Page type;

• Country or region;

• Network type;
  - Browser and device tier;

• CDN identifier or experiment group;

• Page version;

• DNS, connection, TLS, TTFB, and download time;

• LCP, INP, and CLS;

• Whether the request reused a connection;

• Cache status or edge node identifier.

If you need the browser to obtain complete Resource Timing data for cross-origin CDN resources, the resource response must be correctly configured withTiming-Allow-Origin. The W3C'sResource Timing specificationexplicitly limits the detailed timing information that cross-origin resources can expose.

Step 5: Weight by Actual Business Traffic

A CDN may rank first on global average results but perform only average in the regions where a site's main users are located.

The final score should be calculated based on actual business weights, for example:

• East China Telecom users account for 30%;

• South China Mobile users account for 20%;

• Southeast Asia mobile users account for 15%;

• European users account for 10%;

• Other regions account for 25%.

You can also increase the weight of key pages and paid-user paths. CDN selection is not a contest over the number of global nodes; it's about choosing a more suitable network for your own real users.

A Basic RUM Collection Example

The browser's Navigation Timing interface can record some stage timings for the main document:

<script>
window.addEventListener('load', () => {
  const nav = performance.getEntriesByType('navigation')[0];
  if (!nav) return;

  const secureStart = nav.secureConnectionStart || nav.connectEnd;

  const sample = {
    dns_ms: nav.domainLookupEnd - nav.domainLookupStart,
    tcp_ms: Math.max(0, secureStart - nav.connectStart),
    tls_ms: nav.secureConnectionStart > 0
      ? nav.connectEnd - nav.secureConnectionStart
      : 0,
    request_to_first_byte_ms: nav.responseStart - nav.requestStart,
    download_ms: nav.responseEnd - nav.responseStart,
    total_response_ms: nav.responseEnd - nav.startTime
  };

  navigator.sendBeacon(
    '/rum',
    new Blob(
      [JSON.stringify(sample)],
      { type: 'application/json' }
    )
  );
});
</script>

Note here: DNS, TCP, or TLS being 0 does not necessarily mean those steps completed instantly; it may mean the browser used a DNS cache, reused an existing connection, or the page did not perform a new TLS handshake.

In actual deployment, you also need to add sampling, outlier handling, and version identification, and avoid uploading sensitive information such as account names, tokens, full query parameters, and raw IPs. RUM data should have a clear retention period and access permissions, and privacy and compliance assessments should be completed for applicable regions.

Is CrUX Data the Same as Self-Deployed RUM?

Not exactly.

The Chrome UX Report, or CrUX, uses real experience data generated by eligible Chrome users and aggregates it at the page or site level.CrUX official documentationclearly states that these metrics come from actual visits by real Chrome users worldwide.

Therefore, CrUX is real user data, but it still differs from RUM deployed by a site itself:

• CrUX is aggregated data and cannot provide details for individual visits;

• Low-traffic pages or sites may not have enough data;

• It's hard to break down by custom business dimensions;

• It cannot precisely mark which CDN a visit used;

• Data updates are periodic, making it unsuitable for minute-level failure alerts;

• Self-hosted RUM can record page version, experiment group, and business path.

CrUX is suitable for observing a site's overall user experience and long-term trends, while self-hosted RUM is better suited for CDN experiments, troubleshooting, and business-dimension analysis. CrUX'sdata methodology documentationalso indicates that individual user experiences are aggregated into distribution data for a page or site.

Recommended Decision Order When Choosing a CDN

Whether you use active probing or RUM, it's recommended to make decisions in the following order:

1. Availability and error rate
   For a CDN that frequently times out or returns errors, even the lowest average latency is meaningless.

2. Performance in target regions and carriers
   Look at the core user base first, not an unweighted global average.

3. P95 and P99 stability
   A CDN's value is not just how fast the fastest case is, but also whether the slowest case spirals out of control.

4. TTFB for the main document and APIs
   They usually directly affect when the page starts rendering.

5. Static asset throughput
   Businesses involving images, video, and installers especially need to focus on sustained download performance.

6. Core Web Vitals and business metrics
   Finally, check whether LCP, INP, bounce rate, conversion rate, and other metrics have actually improved.

7. Troubleshooting and rollback capabilities
   Whether CDN switching, configuration releases, and failure recovery are controllable enough is also part of vendor selection.

Common Misconceptions

Misconception 1: RUM comes from real people, so it must be more accurate than active probing

RUM is closer to real experience, but it isn't necessarily suitable for fair comparison between CDNs that haven't been brought into production. If the two traffic groups differ in region, device, and page version, the results can be significantly biased.

Misconception 2: Active probing isn't real users, so it has no reference value

The point of active probing is precisely to control variables. As long as node coverage, request objects, and test methods are reasonable, it is a very important data source for selection, regression testing, and troubleshooting.

Misconception 3: The fastest CDN in speed tests is the most suitable CDN

The fastest result from a single probe point cannot represent overall performance. You also need to check target user coverage, P95, failure rate, origin fetch capability, cache policy, and experience under production traffic.

Misconception 4: If the average hasn't changed, there's no problem

A regional failure or anomaly on a few carriers may affect only 5% of users, leaving the average almost unchanged while P95 rises significantly.

Misconception 5: TLS being 0 in RUM means TLS performance is especially good

The more common reason is that the connection was reused and the current page did not establish a new TLS connection. First visits and repeat visits must be analyzed separately.

Final Selection Principles

If you can remember only one sentence, remember this:

When choosing a CDN, first trust reproducible active probing for apples-to-apples comparison; after adoption, trust RUM to verify whether real users are actually benefiting.

When the two data sets conflict, don't simply vote on who is right or wrong; instead, keep breaking things down:

• Whether regions and carriers are consistent;

• Whether DNS, TCP, and TLS connections were reused;

• Whether cache was hit;

• Whether the same pages and resources were tested;

• Whether user devices and networks differ;

• Whether the slowness is in the CDN network, origin response, or page rendering;

• Whether you're comparing P50 or P95/P99.

Active probing establishes a repeatable baseline, while RUM tests whether that baseline holds in the real world. Only when the two data sources can explain each other is the CDN selection conclusion truly credible.

Frequently Asked Questions

1. Is active probing data always less accurate than RUM?

No. Active probing and RUM measure different environments. Active probing is suited to controlled comparisons and troubleshooting, while RUM is suited to evaluating real user experience. Accuracy depends on the question itself and the test design.

2. If RUM is already deployed, do you still need a CDN speed testing platform?

Yes. RUM can only cover real traffic that has already occurred, it is hard to continuously check low-traffic regions, and it cannot directly test a candidate CDN that hasn't been onboarded. Active probing also provides a fixed baseline and proactive alerts.

3. If a site has very little traffic, which data should take priority?

RUM samples from low-traffic sites tend to fluctuate, and after breaking them down by region and carrier they may lack statistical significance. In this case, rely mainly on active probing while gradually accumulating RUM data.

4. How do you compare a candidate CDN that has no real user data?

First use a common domain, files, origin, cache policy, and probe nodes for active testing. When conditions permit, allocate a small amount of real traffic for stable A/B validation.

5. Why are DNS, TCP, or TLS times collected by RUM often 0?

Usually because the browser used a cache or reused an existing connection, or because the relevant timing information was unavailable. 0 should not be interpreted directly as no time being spent in the network stages.

6. Can CrUX be used directly as a site's own RUM?

It is not recommended. CrUX is suitable for observing aggregated real user experience and long-term trends, but it lacks dimensions such as custom experiment groups, CDN identifiers, and page versions, so it cannot fully replace self-hosted RUM.

7. How long should data be collected during a CDN trial run?

There is no fixed number of days that applies to all websites. At minimum, it should cover major traffic periods, weekdays, and weekends, and ensure enough samples for core regions and carriers. Campaign-driven sites should also avoid directly comparing promotional periods with ordinary dates.

8. When choosing a CDN, should you look most at the average or P95?

Look first at availability and error rate, then at P50, P95, and P99 for the target user base. Averages can describe overall trends, but they cannot fully reflect the severe long-tail problems experienced by a minority of users.