Real user experiences can tell us much more about our website's performance than a single test in time. If you’re not tracking real user data, then you’re missing out on a wealth of information.
In this article we will look at the Chrome User Experience Report (CrUX) - what it is, how to access it, and how to interpret the data. We will see how this free dataset can give you the insights you need to make better decisions while not costing a penny.
Synthetic vs real user monitoring Jump to heading
There are two popular ways to measure website performance:
- Use a lab/synthetic testing tool to give you a point in time benchmark
- Collect data from real users that visit your site every day, known as real user monitoring (RUM)
Lab testing is used for debugging and initial testing of optimizations. It is also used as a continuous-integration (CI) testing tool.
Quote from “Fix web performance for the long term”
Lab, or synthetic, testing has a place. However, it does not tell us how a real user experiences our website.
To better understand real-world experiences, we need to look at real user data. In the real world, many variables can change how your website performs. These include the device, browser, internet connection, and location. Sometimes we make changes that we think will improve speed, but those changes will not be reflected in real user data because of the variability beyond our control.
Additionally, RUM data is the basis for the impact of performance on your Google search and ad rankings. Don’t let real user metrics be a blind spot.
What is CRUX? Jump to heading
The Chrome User Experience Report (also known as the Chrome UX Report, or CrUX for short) is a dataset that reflects how real-world Chrome users experience popular websites. Chrome collects the data from users that opt in to share data and sends it to central stores for aggregation.
CrUX records a broad set of metrics important to measuring how a user experiences your site. If you don't know which ones to focus on, we recommend starting with the 3 Core Web Vitals. Learn more about each of the metrics in our article, Key web performance metrics in 2022.
Chrome data is also the basis of your Core Web Vitals performance data that is used in Google rankings.
How to access CrUX data Jump to heading
The CrUX documentation lists the different ways to access the data through tools provided by Google. Some are more simple than others and it’s worth reading about the differences. Sometimes the results differ across views, and you'll want to know why.
Page Speed Insights and WebPageTest will include it at the top of your results, if you have enough data.
A handy feature of WebPageTest is that it shows how the performance of the current run (black triangles) compares to the median CrUX performance (colored triangles). Learn how to use WebPageTest
The Google Search Console can also show you how many of your URLs currently have good, needs improvement, or poor Core Web Vital performance. Find it under the “Experience” section in the left-hand menu:
My favorite tool to view CrUX data is the Treo Site Speed tool. It has a very low barrier to access the data, and provides most of the information you need. Enter your domain (e.g., shopify.com), and if there is enough data the graphs will display on the next screen.
How to use the data Jump to heading
The Chrome User Experience Report provides 2 main types of data:
- Your traffic’s 75th percentile for each metric
- Your traffic’s distribution between good, needs improvement, and poor experiences
The 75th percentile translates to 75% of your users had a better experience than that number, but 25% of users had a worse experience. In the following chart, we can see that this site had 75% of users experience a 2.6s LCP or better.
CrUX classifies user experience into categories of Good, Needs Improvement (Moderate) and Poor. For example, a good LCP is considered to be below 2.5 seconds while a poor experience is anything over 4 seconds. Using the same example chart above, we can see that 71.7% of users had a good Largest Contentful Paint experience.
When making changes, aim to increase the number of users getting a good experience. This approach is a move away from single scores provided in Page Speed Insights and Lighthouse. These scores lack context and don't always reflect how users experience your website.
Treo is great because it shows data over time. You can look back and see what impact major changes had on your real users. Sometimes you may see performance changes at times when you made no updates. These can be a result of browser, Shopify or other 3rd Party updates. This view is a great way of tracking regressions or improvements over time.
The distribution view in Treo helps me visualise those changes more easily. You can switch between Percentiles and Distributions in the top right. All of our examples here are shown using the distributions view.
You can also filter the data by the following demographics:
- Device
- Connection type
- Country
Filtering helps you identify if there are any major bottlenecks that need improving. Comparing desktop against mobile is a great way to see how these different groups experience your site. For most websites, mobile traffic is slower than desktop traffic. However, we often run tests based on desktop/laptop devices on fast networks. When optimising your site, make sure you focus on mobile devices and mobile networks.
Things to remember Jump to heading
The Chrome User Experience report contains data that can help you understand real user experiences on your site. It's both amazing and free. However, keep in mind some important points.
First and most importantly, this data only comes from Chrome users. If a large part of your audience uses Apple devices (iPhones, iPads etc.), their experiences are NOT included.
Second, viewing the immediate impact of changes is not possible. You can see your rolling 28-day average for your origin in WebPageTest, in Page Speed Insights, BigQuery, and with the API. Thus, it will gradually shift over a 28-day period after a change. If you use Treo or the CrUX dashboard, monthly data is published about 2 weeks after the end of the month. That means if you make a change you may have to wait 6 weeks to see the full impact.
If you want to see the impact of changes quickly after making them, then a RUM tool will be needed. You could either try building your own, or add the data to an existing analytics solution. Commercial tools like SpeedCurve provide both the analytics as well as handy features like email notifications.
Conclusion Jump to heading
- Real user data is important and should be included in your performance data. It helps add context that other tools cannot provide.
- If you don’t have a RUM solution use CrUX. It is only a subset of your data but it is free to use.
- There are many ways to access the CrUX data, but our favourite is Treo.
- CrUX is presented using percentiles and banded experiences.
- When optimising sites, aim to move groups of users to more positive experiences.
- Remember CrUX data only covers Chrome users.
- If your resources permit, investigate a full RUM solution.