Don’t get scammed by fake performance experts and apps

We uncovered a set of apps and fake experts cheating the performance metrics. We found those practices in almost 15% of extensions that promised one-click optimizations. This means that a large number of merchant sites are affected.

The main trick is to add a script that detects if the page is loaded using a speed testing tool. It then prevents the browser from loading most resources. These lighter pages achieve better scores, which help convince merchants that their money is well spent. However, no real improvement is achieved with real users meaning that visitors are still impacted by poor performance.

In this article, we explain why some people feel incentivised to cheat the metrics. We also walk through common techniques the bad actors use so you can identify and remove them. Finally, we outline the steps we took to make Shopify's performance ecosystem as scam-free as possible.

Why are people cheating the performance Jump to heading #

Tools like PageSpeed Insights and GTmetrix let people quickly assess their websites for performance. In the past, they helped spread awareness about site speed in general. I still remember people sharing their near-perfect Lighthouse scores on social media. Without this step, the web performance industry would not be where it is today.

As Goodhart's law says: "A measure that becomes a target, ceases to be a good measure". When given a quantifiable score, it’s easy to forget the true reason behind the work that we’re doing. It creates space for bad actors to game the system for their profit. It is also not a Shopify-specific issue – the whole web is filled with such practices.

The real goal of improving the performance should always be to make the actual user experience better. Neither your users nor Google care about your scores if they don’t make the website more pleasant to use. Cheating the metrics is especially dangerous because it can hide any underlying problems. You can't fix an issue you don’t know about.

How to know if your store is affected Jump to heading #

Your store may be affected if you:

• Installed one or more apps promising performance improvements (with an emphasis on a better “speed score”).
• Used a relatively cheap and quick performance optimization service where the developer edited theme files.

If one of the above is true, you can check for some high-level symptoms. One of the most common cheating techniques is to detect if the page is being tested by PageSpeed Insights. Fortunately, the same underlying tool that powers it is also built into every Chrome browser. What you can do is to:

1. Run the PageSpeed Insights test against one of the pages on your store and check the screenshots. If the page looks broken, you are probably affected.
2. Run Lighthouse in Chrome’s DevTools on the same page. Use the incognito mode to make sure no extensions or user data can influence the metrics.
3. Compare the scores of the two. If your local test is considerably (at least 20 points) slower, you’re most likely impacted as well.

You can also scroll down to the "Largest Contentful Paint element" diagnostic. If the element seems artificial and not a part of the content, the metric is likely being cheated.

Also, every Shopify merchant has access to the real user measurements (RUM) dashboard. Usually, your clients should be experiencing faster performance than what testing tools suggest. This is because they usually check the worst-case scenario (slow device and network). If your RUM dashboard shows slower speeds than the testing tool, you might have a performance hack on your site. Getting complaints from actual visitors would be another signal that something is going on.

If you have strong suspicions, the next step is to identify and remove the problematic logic. Depending on your case, it either requires uninstalling the app or modifying the theme code.

In the future, do some research before using a particular service. Make sure it focuses on improving the user experience over some abstract scores. After it’s done, browse the page yourself to verify you can actually feel the difference.

Common hacks to look for Jump to heading #

First, we need to start this section with a disclaimer. To let everyone identify the offending code on their websites, we need to provide it all here in one place. It creates a risk of more bad actors learning about those practices and applying them.

However, we believe that the gains outweigh the costs. Preventing performance cheating is a whack-a-mole game. We (and the browsers) may try to adapt, but there will always be new, “clever” ways to outsmart the checks. It is why we decided it’s better to educate. People will stop offering shady solutions if there is no one to sell them to.

You can search the page for the offending code in multiple ways. We recommend using the search function in Chrome’s dev tools. You can also inspect the theme files that were affected the most:

• snippets/social-meta-tags.liquid
• snippets/meta-tags.liquid
• snippets/preconnect.liquid
• layout/theme.liquid

When it comes to Liquid snippets, the harmful code is usually appended at the end of the file.

Largest Contentful Paint hijack Jump to heading #

One hack aims to understate the real time of the Largest Contentful Paint. To achieve this, a fullscreen, transparent element is added to the HTML response. Its role is to hijack the metric from any other candidate in the page content. It is the most harmful method by far. It impacts both synthetic tests and real user measurements.

Fullscreen transparent image Jump to heading #

The first version of this technique involved a base64-encoded, transparent image (PNG or SVG) like this one:

<img width="99999" height="99999" style="pointer-events: none; position: absolute; top: 0; left: 0; width: 99vw; height: 99vh; max-width: 99vw; max-height: 99vh;" src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI5OTk5OSIgaGVpZ2h0PSI5OTk5OSIgdmlld0JveD0iMCAwIDk5OTk5IDk5OTk5IiAvPg=="/>

The main things to look for are:

• Large width and height attribute values.
• Large width and height properties set in CSS using viewport units (usually between 96 and 99).
• The CSS pointer-events: none rule which makes sure the actual content can still be interacted with.

Chrome now ignores these types of low-entropy images as LCP candidates. Thus, this hack is less effective in Chromium-based browsers but still affects Firefox.

Fullscreen transparent text Jump to heading #

This technique emerged after the previous one became unreliable. It is similar, but uses text instead of an image:

<div style="position:absolute; font-size:1200px; max-width:99vw; max-height:99vh; pointer-events:none; z-index: 99999999999; color:transparent; overflow:hidden;">-----</div>

It works because blocks of text are also considered LCP candidates, but there is no entropy requirement for them. Here, a huge font-size makes sure the element is big enough to hijack the metric.

Performance testing tools detection Jump to heading #

Before the new RUM dashboard launched, the Shopify speed score used Lighthouse to report web performance. This made some people realize they could sell fake optimizations as long as the score improved even if real user experience did not.

This technique involves adding JavaScript code that does two steps:

1. Checks if the page is opened by the performance testing tool.
2. Modifies the content to force better results if it is a testing tool.

When it comes to augmenting the content during the test, the most popular practices include:

• Interrupting the HTML parsing by calling: document.open(), document.write(), and document.close() to empty the page.
• Dynamically adding the LCP hijack that was mentioned above.
• Hijacking any lazy-loading mechanisms to make sure they never fire.
• Trying to prevent the loading of subsequent resources using the Yett library or MutationObserver directly.

The most notable aspect of this method is that it has a near-zero impact on the actual users. It only aims to fool owners into thinking their website is faster than it really is.

Let’s dive deeper into how these hacks check if the page is being opened by a performance testing tool.

System platform detection Jump to heading #

The navigator.platform property contains a string that identifies the platform on which the browser is running. At some point, performance cheaters decided that looking for the Linux x86_64 value would be a good enough detection technique. Linux is often used for servers and has a small user base compared to other operating systems. This way, they can detect all tools running in data centers, not only the most popular ones. Of course this means that any human using Linux would also be affected.

Here’s an example of one:

var _cs=["Li","nu","64","x x","86_","ose","rm","cl","te","pla","wri","tfo"]; if(navigator[_cs[9]+_cs[11]+_cs[6]] !=_cs[0]+_cs[1]+_cs[3]+_cs[4]+_cs[2]){ document[_cs[10]+_cs[8]](''); } else { } document[_cs[7]+_cs[5]]();

Usually the code is obfuscated like this – notice how the words are broken up in the first line. This lowers the probability of being detected by both humans and automatic checks. You can still look out for 2 things:

• Any mentions of Linux x86_64.
• References to the \x4c\x69\x6e\x75\x78\x20\x78\x38\x36\x5f\x36\x34 string, which is the hexadecimal escape sequence representation of the above.

User agent detection Jump to heading #

The user agent is a string that browsers and tools send to identify themselves in the network. It allows servers, proxies, and websites to adjust their behavior based on this information. Sadly, it can also be abused, like in this example:

var e = navigator,
  t = e.userAgent;
(t[l]("power") > -1 || t[l]("rix") > -1) &&
  new MutationObserver((e) => {
    e[n](({ addedNodes: e }) => {
      e[n]((e) => {
        1 === e.nodeType &&
          ("IFRAME" === e[d] &&
            (a(e, "loading", "lazy"), a(e, "data-src", e.src), o(e, "src")),
          "IMG" === e[d] && r++ > 20 && a(e, "loading", "lazy"),
          "SCRIPT" === e[d] &&
            (a(e, "data-src", e.src), o(e, "src"), (e.type = "text/lazyload")));
      });
    });
  }).observe(document.documentElement, {
    childList: !0,
    subtree: !0,
  });

As before, the code is usually way less readable than the one above. The most common patterns include looking for a subset of:

• Moto G power – a device that PageSpeed Insights emulates.
• Chrome-Lighthouse – a string that Lighthouse appends to the user agent.
• GTmetrix – used by the tool of the same name.

How to fix the offending code Jump to heading #

First of all, not all occurrences of checking the platform or user agent are malicious. They are often used by analytics to better identify users for example. Make sure to double-check this is not the case.

Once confirmed, you can generally remove the performance-cheating code. We strongly recommend doing any code changes in a copy of the live theme and testing it before publishing.

Sadly, there may be other modifications that make the process more complicated. In some cases, problematic code was also handling all of the lazyloading. Deleting it would mean breaking the page for all of the visitors. This is why you need to verify that the store looks and functions properly before deploying.

Addressing the issue long term Jump to heading #

When we learned of this issue, our team analyzed and addressed all of the speed optimization apps in the Shopify App Store. We also reached out to all of the partners offering this kind of “improvement” to make sure they stop. Stores used for hosting the malicious files were taken over, sanitized, and closed as well.

Still, we can’t cover everything internally. Mass-removal of all of the patterns that we found is too unsafe. Also, some of these scam services are conducted through completely external channels such as Fiverr.

That’s why we’re also aiming to solve the root cause by sharing our findings with the performance ecosystem:

• The Largest Contentful Paint hacks are already known to the Chromium browser engine. We hope this will allow them to improve the algorithm to ignore incorrect candidates.
• We asked for an improvement to the navigator.platform value to match the emulated device.
• Google Lighthouse already anonymized its user agent after our input. The change should propagate to PageSpeed Insights at some point.
• The team at GTmetrix was informed as well. They committed to let the free users disable the custom user agent in the future.

Making ecommerce faster for everyone Jump to heading #

The ultimate goal of this post is to create a healthier performance ecosystem, both inside and outside of the Shopify platform. We hope that shedding more light on the issue will let website owners identify those shady practices that put their businesses at risk. You can also help by spreading awareness and sharing the article with others.

Speed optimization apps and services can still be a great investment. Just verify they focus on what really matters – improving the actual experience. Do some research before deciding and test the impact in multiple ways afterward. The real user measurement report in the admin panel should also confirm your visitors get better performance.