John Mueller, a Google Search Relations Advocate, confirmed this morning that the search company changed how it handles soft 404 detection and classifications. The change is that now Google looks at each page by device and potentially can assign the soft 404 classification differently across the same URL on mobile versus desktop.
What are soft 404s. Soft 404s is when a page returns an HTTP status code of 200, which says the page is okay and all is good. But in reality, the page should likely return a 404 status because the page does not really load content or the page cannot be found. When this happens, Google will flag those pages as soft 404s and thus treat the URL as a real 404 and not index the page.
What changed. Google said it changed how it does soft 404 classifications and now looks at a URL by device type. So if Google sees a URL and accesses that URL on desktop and then accesses that URL on mobile, it can potentially return a soft 404 for the desktop and not the mobile. In short, Google is detecting the soft 404 status on a URL by URL basis but also by device type. This change happened about a month ago, according to the complaints I’ve tracked.
Why this can be an issue. Issues have been arising where SEOs are noticing Google no indexing pages when doing queries in Google Search or these SEOs are seeing spikes in soft 404 errors in Search Console but do not see an issue in Google Search.
What is happening is that a page can be returning fine for the mobile device type and Google won’t show soft 404 errors in Search Console. Google only shows soft 404s errors based on the mobile crawl, so if the page works fine on mobile, Search Console won’t show an error. But Google may assign soft 404 errors to your desktop version and in that case, when you search Google on desktop, you might not see those pages indexed and coming up in Google Search. But at the same time, Search Console shows everything is fine, when it is only fine on mobile and not fine on desktop.
John Mueller said “in Search Console we do show soft 404s but we show it for the mobile version. So if on the mobile version everything is okay on your side, then in Search Console it will look like its indexed normally. But for desktop, you won’t be able to see that directly in Search Console.”
Google working on a solution. John Mueller said the team is working on improving its soft 404 classifications and if you have examples you want to share, share it with Google in the support forums or maybe with @johnmu on Twitter.
The video: Here is the video embed, again, it starts at about 22 minutes into the video:
Why we care. If you are noticing indexing issues in Google Search starting around a month ago, or if you see a spike in soft 404s around that time frame, this change may be related to that. Make sure to send Google examples of URLs that are having this problem and Google may be able to address it in the future. This may cause traffic issues for your site if you are impacted by this change.
About The Author
Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry’s personal blog is named Cartoon Barry and he can be followed on Twitter here.