Google’s web crawler is the bot that goes around and collects URLs of pages to show in search. However, not every URL on every website is crawled or indexed by the massive search engine. Sometimes you might need to recrawl WordPress URLs in Google.
This is when you ask Google to crawl the site or a specific page to find new updates or changes. Sometimes the page will be indexed quickly while other times it may take a while before your content is accessible through Google search.
In this tutorial, I’ll show you a couple of ways to recrawl your WordPress URLs in Google. You’ll need a Google account in order to continue, which is a free service and worth the time setting up.
For both of these methods, you’ll use the Google Search Console after you submit the site to Google.
Just click the “Add property” link and follow the instructions. It’s fairly self-explanatory and doesn’t require a lot of effort.
What is the Purpose of Recrawling the Site with Google?
Let’s say you have a page of important information that is not being seen by visitors. Are you sure that it’s indexed? It may simply be a case of Google not knowing there is a page available with content.
Another instance is if you update content on older pages. Because you want the most current information available in search, you want to make sure Google has crawled the new material.
This is especially helpful if you want to update evergreen content from several years ago. It will help keep the webpage relevant for today’s searches.
The bottom line is that you can’t expect your pages to be in search results if Google doesn’t know it exists.
Submit Your Sitemap
Perhaps the easiest way to avoid problems of not having pages indexed is by submitting a sitemap to Google. This file will inform the search engine of the location of all URLs on your website.
And using automated systems in WordPress, such as Yoast SEO or Google XML Sitemaps, the file is updated any time you add new content.
For instance, you can turn on the sitemap in the features tab of General settings in Yoast SEO.
To get the sitemap file, click the question icon and then the sitemap link.
Then, simply plug the sitemap’s URL into Google Search Console.
Now, having a sitemap saved in Serch Console doesn’t necessarily mean all of your pages will be indexed immediately. It may take a bit of time depending on how active your website is in terms of generating content.
But if you have pages that are still not showing in search, you can move on to the URL inspection tool.
Using Google URL Inspection Tool
In the new version of Google’s Search Console, you have access to the URL Inspection Tool. This is a quick and easy way to find out if URLs are indexed by Google.
If they are not, Google will show an error message pertaining to why and give you possible solutions regarding how to make fixes.
Once you fix the error, run the Google crawler tool again to find changes. It may quickly index the page if you’ve addressed the errors.
From Search Console, switch to the website you want to crawl. If you only have one site listed in Google, it will be your default.
Click the “URL Inspection” tool on the left.
Add the URL of the page you want to inspect.
If the page is indexed and Google found no errors, you’ll see a screen with green check marks across the board.
Otherwise, the system will tell you why the page isn’t indexed including possible fixes.
If you have new content on that particular page or have made changes, click the “Request Indexing” link. Google will then schedule the page for a recrawl.
Keep an Eye on Search Console
Search Console is an excellent tool to manage how Google views your website. It has more than just the ability to recrawl WordPress URLs. You can get insights from page performance as well as making sure the content is mobile friendly.
Make it a habit to keep an eye on the system. It’ll save you a lot of aggravation later on by giving you warnings and errors regarding your website.
Do you find the information in Search Console to be valuable; why or why not? What’s your favorite plugin for submitting a sitemap to Google?