How is the cache preloaded?


Sitemap preloading

Sitemap-based preloading in a nutshell:

  • When you create/edit/update posts or pages on your website:
    • No sitemap-based preload.
  • When cache lifespan has expired and the entire cache has been cleared:
    • All URLs listed on your sitemap(s) get preloaded.
  • When you click the toolbar link “Preload Cache”:
    • All URLs listed on your sitemap(s) get preloaded.

Since WP Rocket 2.8, there is the possibility to preload your content using an XML sitemap. The advantage of this feature is that it will preload all the URLs in your sitemap file(s)—other than bot-based preloading which will only preload the front page and the page links found on it.

Sitemap-based preloading does not depend on our preload bot. So if you had any trouble with too much load on your website and had to disable the preload bot, sitemap-based preloading will enable you to still benefit from a preloaded cache.

Sitemap crawl interval

You can control the crawl interval to prevent server overload:

Preload crawl interval selection

If you want to add custom intervals to the selection, you can do that by using this helper plugin:

📥  Download (.zip): WP Rocket | Custom Sitemap Preload Intervals

Developers: You can find the code for this plugin on GitHub.

Sitemaps for preloading

Sitemaps generated from Yoast SEO, All in One SEO, and Jetpack are detected and displayed as options to base your preload process on.

Custom sitemap URIs can be added in a settings field.

Preload bot

Bot-based preloading in a nutshell:

  • When you create/edit/update posts or pages on your website:
    • That specific post/page and adjacent posts/pages get preloaded.
  • When cache lifespan has expired and the entire cache has been cleared:
    • The front page of your website and all posts/pages linked on it get preloaded.
  • When you click the toolbar link “Preload Cache”:
    • The front page of your website and all posts/pages linked on it get preloaded.

When you write or update a new post or page, WP Rocket automatically clears the cache for that specific content and any other content related to it (i.e. home, category, tag, pagination etc.). The preload bot will crawl these URLs to re-generate the cache immediately—other than sitemap-based preloading which will run automatically no sooner than when cache lifespan expires. (Although you can always trigger it from the toolbar.)

Heads up! Visits from the preload bot can cause problems on weaker server environments. If you notice server issues that seem related to our bot wprocketbot, switch to sitemap-based preloading which uses cURL.

Preloading at a specific time

If you want to make sure that a preload happens at a certain time every day, you can create a custom cron job through your webhosting control panel and launch one of these function at whatever time you want:

  • To trigger bot-based preloading: run_rocket_bot()
  • To trigger sitemap-based preloading: run_rocket_sitemap_preload()

For further instructions, follow this guide on clearing the cache via cron job. Remember to change the function names to the ones above!

Prefetch DNS requests

If you have third-party content on your website (e.g fonts loaded from Google, or a video from YouTube), you may want to add its origin domain to the Prefetch DNS requests option.

This can give your loading time a minor boost, since DNS resolution will already have been processed when the external resources get requested.

Watch this video for further guidance on this setting: