How is the cache preloaded?


Sitemap preloading

Sitemap-based preloading in a nutshell:

  • When you create/edit/update posts or pages on your website:
    • No sitemap-based preload.
  • When cache lifespan has expired and the entire cache has been cleared:
    • All URLs listed on your sitemap(s) get preloaded.
  • When you click the toolbar link “Preload Cache”:
    • All URLs listed on your sitemap(s) get preloaded.
  • Preload doesn't work for separate mobile or cookie-specific cache files.

The advantage of using an XML sitemap to “warm” (preload) the cache is that it will preload all the URLs on your sitemap file(s)—other than bot-based preloading which will only preload the front page and the page links found on it.

Sitemap-based preloading does not depend on our preload bot. So if you had any trouble with too much load on your website and had to disable the preload bot, sitemap-based preloading will enable you to still benefit from a preloaded cache.

Sitemaps for preloading

Sitemaps generated from Yoast SEO, All in One SEO, and Jetpack are detected and displayed as options to base your preload process on.

Custom sitemap URIs can be added in a settings field:

Sitemap crawl interval

WP Rocket’s default interval for crawling a sitemap is 500 ms. That’s the time it will wait before moving from one URL on the sitemap to the next.

In versions prior to 3.0, that value could be changed in the user interface. If you had set a different value than 500 ms, your value would still be the one WP Rocket uses after you updated to version 3—even though the visual option has been removed in 3.0.

If you need to change the crawl interval in version 3 and above, there’s a little extra plugin you can use for that purpose:

📥  Download (.zip):  WP Rocket | Custom Sitemap Preload Intervals
Developers: You can find the code for this plugin on GitHub.

Heads up! Remember to edit the value for the WPROCKETHELPERS_PRELOAD_INTERVAL_IN_SECONDS constant; the default is 5 seconds.

Preload bot

Bot-based preloading in a nutshell:

  • When you create/edit/update posts or pages on your website:
    • That specific post/page and adjacent posts/pages get preloaded.
  • When cache lifespan has expired and the entire cache has been cleared:
    • The front page of your website and all posts/pages linked on it get preloaded.
  • When you click the toolbar link “Preload Cache”:
    • The front page of your website and all posts/pages linked on it get preloaded.
  • Preload doesn't work for separate mobile or cookie-specific cache files.

When you write or update a new post or page, WP Rocket automatically clears the cache for that specific content and any other content related to it (i.e. home, category, tag, pagination etc.). The preload bot will crawl these URLs to re-generate the cache immediately—other than sitemap-based preloading which will run automatically no sooner than when cache lifespan expires. (Although you can always trigger it from the toolbar.)

Heads up! Visits from the preload bot can cause problems on weaker server environments. If you notice server issues that seem related to our bot wprocketbot, switch to sitemap-based preloading which uses cURL.

Preloading at a specific time

If you want to make sure that a preload happens at a certain time every day, you can create a custom cron job through your webhosting control panel and launch one of these function at whatever time you want:

  • To trigger bot-based preloading: run_rocket_bot()
  • To trigger sitemap-based preloading: run_rocket_sitemap_preload()

For further instructions, follow this guide on clearing the cache via cron job. Remember to change the function names to the ones above!

Prefetch DNS requests

If you have third-party content on your website (e.g fonts loaded from Google, or a video from YouTube), you may want to add its origin domain to the  Prefetch DNS Requests option. This can give your loading time a minor boost, since DNS resolution will already have been processed when the external resources get requested.

Add the origin host with relative protocol (double slash, no http:) like so: