If you've ever received an email from Google about an issue with your website, it's normal to feel concerned. Google dominates when it comes to ranking sites in an online search. So, if there's an issue or discrepancy between your site and their algorithm, you may not show up in a search result and potentially lose opportunities for business.
To top it off, the message from Google may have talked about page indexing issues, using tech jargon you don't understand, which only makes matters feel worse. Not to fear. We're here to explain what might be happening and help you rest easy!
Some of the most common notifications our clients receive are about page indexing, so let's walk through what that means. Page indexing is simply the process by which search engines like Google crawl and store web pages in their index. When a search engine crawls a website, it reads the content on each page and stores it in a massive database. Once a page has been indexed, it can potentially appear in search results when users search for relevant keywords or phrases.
Please note that just because a page has been indexed does not guarantee it will rank highly in search results. The ranking of a page depends on many factors, including the relevance of the content to the search query, the authority of the website, the overall quality of the content, the relevance of any links pointing to the page, and, ultimately, Core Web Vitals.
If you've received a notification indicating there's an issue with page indexing, it simply means Google has encountered difficulties in indexing some or all of the pages on your website.
While it's important to carefully review any emails from Google about your website, it can be reassuring to know that most of the time, the ‘issue' really isn't an issue at all. For example, sometimes things are blocked on purpose, so Google does not index a page or piece of content.
With that in mind, we've compiled a list of the top five most common notifications our clients receive from Google, along with simple explanations about why we tell them not to worry.
1. The message includes: "Indexed, though blocked by robots.txt"
If you're our client, we control the robots.txt file for you. So, if we ever need to block anything intentionally, like a password-protected page, we can do that. This has no effect on the total site regarding indexing or ranking for SEO. It just impacts a page or piece of content we don't want Google to index. Reference link for further information.
2. The message includes: "Soft 404"
This happens when Google tries to crawl a page that isn't currently published, but it may have been in the past. For example, if you're a realtor and you listed a page or two for a house that has now sold. This content doesn't need to be indexed any longer. Once it's unpublished, it may trigger a "soft 404", which is nothing to worry about in this case. When Google crawls the site, it will pick up the new site map, see that it's not there, and won't try to hit that page again the next time it crawls.
3. The message includes: "Page with redirect"
This is just Google saying we tried to go here, and it redirected us. This is what you want to happen. For example, if someone tries to hit a non-secure version of your site, we redirect to the secure one. This is the same if someone goes to "www." We don't serve on that subdomain. We intentionally direct traffic where we want users to go. And Google doesn't care or score your site in any way because of this. They're just letting you know, "Hey, this happened."
4. The message includes: "Blocked due to access forbidden (403)"
While this language sounds a bit strong, it's truly nothing to worry about! This is Google attempting to access a folder that has been intentionally locked down. For security purposes, we don't allow users or bots to access folders directly. This typically occurs when Google tries to read entire folders of files rather than publicly accessible web pages. If you try to go to this folder in your browser, you will also see the ‘access forbidden' message. This keeps our file system secure and away from prying eyes. It does not affect SEO in any way since we are intentionally not allowing access to folders, only content that we want users and bots to see. Reference link for further information.
5. The message includes: "Alternate page with proper canonical tag'
This is not an issue, and we don't know why Google sends it out as such. Even their own documentation says there is no action to take if this notification pops up. You can give a big eye roll on this one and move about your day with joy and ease. Reference link for further information.
Other Potential Messages
Google sometimes sends an email if we tell them not to index a page. Their response can be, "When Google tried to index the page, it encountered a 'noindex' directive and therefore did not index it." They're simply responding to a command we've provided. Sometimes clients get an indexing error, which is just a straight-up error on Google's side. The bot failed, and they will eventually clear it up on their own.
If you've received a different notification, here's a list of all indexing issues that you can reference directly in Google's Support Console.
Perhaps the most reassuring thing to know is that if you're our client, we get the same alerts. We watch for any critical indexing issues and correct them right away. If you're with another website provider and you're not sure if they're actively checking potential issues for you, definitely pick up the phone and have a conversation. We always encourage clients to take ownership of their website and understand what's going on behind the scenes in addition to on-page optimization. We wish you the best and are always happy to help if you have further questions or need a good web partner.
Kate is a creative leader with a talent for writing and passion for helping companies grow. She has spent more than 20 years in advertising helping some of the world's best-known brands. She has an in-depth understanding of how to connect with customers and build loyalty in today's ever-changing digital landscape.