Google Will Slow Crawling For 429/503 HTTP Status Codes But…

Gary Illyes from Google came on to Twitter for a brief moment to share a PSA about 429/503 HTTP status codes. He said while you probably know 429/503 HTTP status codes will slow GoogleBot’s crawling activity, what you might not know is that if those codes are left in place for too long, it can remove the pages completely from the Google search index.

He wrote “Fun fact: serving 429/503 http status codes will slow down crawling, yes, but if you serve them for a longer period of time, they will remove the pages served with said codes from the index.”

Here is the tweet with Stan, the dinosaur:

Fun fact: serving 429/503 http status codes will slow down crawling, yes, but if you serve them for a longer period of time, they will remove the pages served with said codes from the index. pic.twitter.com/p4Wf6YD6SX

— Gary “鯨理／경리” Illyes (@methode) February 20, 2020

Forum discussion at Twitter.