How to make sure Monitoring can access your website

Consent Studio Monitoring opens your pages in a real browser to detect tracking technologies and check your implementation. Use the checks below to make sure our scanner can reach your website and read your pages. If our scanner is blocked, you get the critical recommendation Site access blocked.

Allow our scanner IP addresses

Firewalls, security plugins and bot protection services sometimes block Monitoring because it is an automated visitor. The fix is always to allowlist all of the addresses listed in Scanner IP addresses.

Cloudflare

1. Log in to Cloudflare and select the domain you use with Consent Studio.

2. Add a rule that allows requests from each of our scanner IP addresses, so they are not blocked or challenged by Cloudflare's firewall or bot protection. [TODO Thierry: exact Cloudflare menu names and rule type to use.]

3. Save the rule.

4. Get a new scan, or wait for the next weekly one (see the Monitoring scanning FAQ). Once a scan succeeds, the Site access blocked recommendation closes by itself.

Other firewalls and bot protection

The same approach works for any other protection in front of your website, such as a WAF, a security plugin, a bot protection app or your hosting provider's firewall: add our scanner IP addresses to its allowlist.

• Using more than one layer, for example Cloudflare and a security plugin? Allowlist our IP addresses in each of them.

• Don't manage the firewall yourself? Ask your hosting provider or developer to allow our scanner IP addresses.

• No firewall of your own, but still blocked? Some hosting platforms, such as Webflow, block automated visitors on their side. See Site access blocked.

• Still blocked after allowlisting? Contact us with your domain and we'll dig deeper on our side.

Password-protected sites (Basic Authentication)

Is your website protected with a username and password, for example a staging or test site? Then Monitoring cannot see your pages until you add the username and password under Basic Authentication.

1. In Consent Studio, open Domain Settings via the gear/cog icon in the top bar.

2. Open the Access tab.

3. Under Basic Authentication, enter the username and password of your website and save.

4. Get a new scan, or wait for the next weekly one (see the Monitoring scanning FAQ). Once a scan succeeds, the Site access blocked recommendation closes by itself.

Pages timing out

If a page loads too slowly, our scanner cannot process it in time and the page times out. Your consent banner still works fine for your visitors; only the scan of that page is affected.

• Our scanner bypasses caches. We use cache busting when we scan, so a page that loads quickly from cache for you can still be slow for our scanner.

• Check mobile speed too. A page can load fast enough on desktop but too slowly on mobile.

• A few timeouts are not always a problem. If most pages scan fine, Monitoring still finds the tracking technologies on your site.

Once you have improved your page speed, get a new scan (see the Monitoring scanning FAQ).

SSL and certificate problems

Make sure your domain has a valid SSL certificate. If it doesn't, our scanner cannot load your pages.

www, non-www and redirects

Make sure the non-www variant of your domain is reachable, as well as the www variant if you use it.

Choose the right pages to scan

Monitoring checks your home page plus the pages on your Pages to Scan list, so add the pages that load something different, such as a video or a form. See How to tell Consent Studio which pages to scan? On Professional and up, a sample of your sitemap is added too: see How sitemap-based scanning works.

FAQ

Does Basic Authentication work with a login form, such as a WordPress login page?

No. Only HTTP Basic Authentication (the browser's own username/password prompt) is supported. Monitoring cannot fill in login forms on a web page, such as a WordPress login page.

My Shopify store is password-protected. Does Basic Authentication work?

No. A password-protected Shopify store can't be scanned until the store password is removed. Basic Authentication does not support Shopify stores with password protection (the storefront password page). To let Monitoring past Shopify's bot protection, use a crawler signature: see Shopify Monitoring access.

My staging site is added as a domain alias. Is it scanned?

No, only your primary domain is scanned. See How Domain Aliases (and subdomains) work in Consent Studio.

My staging site only allows certain IP addresses. What do I do?

Allow our scanner IP addresses, as described under Allow our scanner IP addresses above.

Where can I find more answers about scanning?

See the Monitoring scanning FAQ, or contact us with your domain.