Reconsideration Requests
Show Video

Google+ Hangouts - Office Hours - 16 December 2014

Direct link to this YouTube Video »

Key Questions Below

All questions have Show Video links that will fast forward to the appropriate place in the video.
Transcript Of The Office Hours Hangout
Click on any line of text to go to that point in the video

JOHN MUELLER: OK, welcome, everyone, to today's Google Webmaster Central Office Hours Hangout. We have a bunch of people here already. I bet some of them already have questions as well. We have a bunch of questions submitted with the Q&A feature. But as always, maybe we can get started with a question from you guys.

AUDIENCE: Hi, John. Can I step in with the first question?

JOHN MUELLER: All right.

AUDIENCE: I'd like to ask, is rel canonical treated like a 301 redirect by Google in terms of page rank? I mean, I have a page which is gaining page rank from the links pointing to it. This page has a rel canonical towards a different page URL. Is this page ever gaining that page rank or the page rank flow through the rel canonical toward the destination URL?

JOHN MUELLER: It's very similar, but it's not really equivalent to a 301 redirect. With a 301 redirect, we can forward everything directly. With a rel canonical, first, we have to index that page anyway to be able to see the rel canonical. And then we have to make a decision on how we should forward the signals that are collected there with that rel canonical, assuming we can trust that rel canonical. So that's something where, with the rel canonical, it takes some extra work on our side. In the end, if everything aligns properly, then we can treat it similar to a 301 redirect. But a few more things that can take more time or that can go wrong in between.

AUDIENCE: Hi, John. Sorry, can I ask you a quick question?


AUDIENCE: Regarding Freebase, if I am submitting something to Freebase, why does it takes so long? I heard it's around two months, but to bring it to the search results within like-- usually like with Wikipedia, it's like three to four days. Can you just explain-- elaborate on that a little bit?

JOHN MUELLER: I don't know how the Freebase system operates, so I can't really help much there. Sorry.


AUDIENCE: Hi, John, can I jump in with my question?


AUDIENCE: We've got links from universities and local councils. But the main problem is, the universities and local councils, they've got a gate element with at least 15, 20 other domains. And they use the exact same database and same layer. So at the end, we're supposed to have only one link from various domains at those councils and universities. But it's ended up having 20 links from 20 different domains which use the same database.

JOHN MUELLER: I wouldn't really worry about that. That's more of a technical problem that we should be able to solve on our side to understand better, but I wouldn't worry about that as something that could be even seen as problematic.

AUDIENCE: So I don't have to add those domains [INAUDIBLE]?

JOHN MUELLER: No. No. I think that's perfectly fine.

AUDIENCE: Hey, John, can I jump in for a quick second?


AUDIENCE: How important is domain age in Google's algorithms? Are webmasters at a disadvantage registering a new domain and launching on it right away? Is there any benefit in using an older domain, either one that they've parked for some time or acquiring an expired domain?

JOHN MUELLER: I imagine we do take a look at that, but I wouldn't really see that as a primary factor. I wouldn't see that as something I'd really worry about there. If you have a new website that you want to put up, I'd just put it up on whatever domain you think is best fitting for that. I wouldn't artificially try to find an older-looking one and use that. Usually the effects around the age of a domain are more indirect in that if you've had a website running for 10 years ago, then everybody knows your website, and they've linked to your site. It's something that's kind of known among the community of the people who are going to your site already. And that's something that you can even build up over years. So you have trust from these people. They come to your site. They link to your site. They know what to expect there. And that's something that kind of comes with time, and that's something you can build on if you do it right. So that's the primary advantage I'd see there.

AUDIENCE: So is it safe to say you're kind of playing with fire if you were to purchase an expired domain?

JOHN MUELLER: We have systems that try to recognize that and to treat that appropriately. So if we can see that this is something completely different on this old domain, then we'll say, well, these old signals don't apply to this website. So I wouldn't necessarily say that you're playing with fire, but I wouldn't assume that you have any kind of artificial advantage over starting with a new domain.

AUDIENCE: May I throw something on top of that?

JOHN MUELLER: All right.

AUDIENCE: It's just that I deal with this on a more than regular basis. And most of the time, if you're thinking about buying an expired domain, I would recommend doing some due diligence just to make sure that you haven't picked up an old churn and burn that's got trouble left right and center, especially from the linked profile in. So just-- if you're thinking about buying something expired or aftermarket, just do half an hour's work to see what state the thing actually is before you hand over the cash.

JOHN MUELLER: I think that definitely makes sense. That makes sense with anything that you're buying, be it a car or domain name, I guess. Make sure that you're buying something that's really what you're trying to get. Make sure it doesn't have any weird history associated with it that you've got to clean up before being able to use that on your own. Let me just mute you guys when you're not speaking. Feel free to unmute if you have any questions. Sometimes it picks up some background noise from that. All right, let me go through some of the questions that were submitted. If you guys have any comments or further or similar questions, feel free to jump on in. 'We've restructured our website by implementing 301 redirects. Now we have a big amount of redirects and significantly less index pages. We've merged many pages. Does a big amount of 301 redirects affect our ranking?" No. That should be absolutely no problem. So this isn't something where I'd say you'd have any problems from a large number of 301 redirects or a large number of 404 pages. If this is something that has been cleaned up and, for technical reasons, you've implemented it like this, that's perfect. That's not something that you need to artificially limit or kind of work around a 301 redirect. I'd just make sure-- if you're using 301 redirects to clean up, make sure you're redirecting directly to the final page to kind of make sure that we can process that in one run instead of having to jump multiple hoops to find the final page. "Does user behavior impact search engine ranking? I mean, like being on the first page and not getting enough visits with a good bounce rate? Does your brand search in AdWords also matter in judging how much of your website is authoritative?" We don't use AdWords at all for web search, so I wouldn't worry about that part. With regards to user behavior, usually that's something we see indirect effects a lot stronger than anything directly. So if people are going to your site and they're bouncing out because they think it's a terrible website, then they're not going to be recommending your site to anyone else. So that's something where you're automatically blocking people from even recommending your site to others. And those are signals that we can definitely pick up on very directly. So instead of focusing on a potential search engine impact, I'd really focus on why people are bouncing from your site and what you can do to prevent that, what you can do to make sure that users are actually seeing something great on your site that they do want to recommend to others.

AUDIENCE: You said to jump on in. Can I ask you about the mobile usability? I've cleaned up mobile usability pages, and so I was just wondering if there's an option that you can ever-- I guess for 2015-- add a feature where, once we've cleaned out the mobile usability error, so the old ones, the old dates are cleaned up, are you going to be doing that?

JOHN MUELLER: So it recrawls those pages regularly. And as we see fewer issues there, that graph will go down. It's not something that happens automatically.

AUDIENCE: So on a regular basis? Like weekly, monthly?

JOHN MUELLER: I mean, as we recrawled over the pages on your site anyway. So it's not something where we have a monthly cycle where we say, oh, we'll recrawl your whole website every month. It's kind of part of the natural recrawling process.

AUDIENCE: OK. Thank you, John.

JOHN MUELLER: All right. "The site has 5 million pages, books. New internal site-wide links show up as 1.5 million in Webmaster Tools. Other site-wide links are 80,000. New internal links, [? tank ?] rankings plus 30"-- ooh, this is a complicated question-- "removing link and waiting 3 to 6 months [INAUDIBLE] issues. [INAUDIBLE] the penalty and what can be done?" This sounds like something that's probably best written up as a forum post somewhere where I could take a look there and can really see what you're running into. I imagine from the space that you have available in the question for the Q&A, it's just not enough to really bring across you're seeing there. So I'm not really sure where you're seeing problems or what exactly is happening there. If you want, feel free to post a link to your forum thread in the event, and I can take a look at that. "If a new domain with about 5,000 URLs, quality content, good internal link structure is published, what should be the average crawl rate in Webmaster Tools per day after two weeks or after four weeks?" This is something where you're not going to get any specific answer. So I can't say what the crawl rate would be. Ideally, if this is new content that you published that doesn't change at all, we'd crawl all 5,000 pages on the first day, and we wouldn't have to crawl anything anymore after that. But I guess what will happen in practice is that some of these pages will be updated, and we should be trying to find them and recrawling them. So you'll probably see us recrawl the homepage maybe every day or so. You'll probably see us recrawling lower-level pages a little bit slower. So that's something where there is no absolute number where I'd say, you should have this crawl rate or a different crawl rate. It really depends on your website, the type of content you have there, how quickly we can crawl your website, how fast your server is, what kind of errors your server brings back, what kind of content we see there, if this is something that we really want to pick up on for search, or if this is something where we'd say, well, we've looked at a few of these and they don't really look that awesome, so maybe we won't crawl as quickly for the rest. So it's really hard to say what kind of crawl rate number you'd be seeing there. It really depends, a lot. "We're using alternate and canonical tag on desktop and mobile versions, but the desktop search for keyword are mobile pages coming up in desktop search. Could you suggest how to deal with this problem?" That sounds like we're not much picking up your canonicals or your link between the desktop and mobile page properly. So what I imagine might be happening there is that the canonical isn't working as you expect it to work. Maybe it's the case that the mobile side is significantly different than the desktop side, so we'd say, well, we can't trust this canonical. Maybe the canonical tag is implemented technically wrong. Maybe we're also picking up problems with the alternate markup for the mobile page. So this seems like something that's more of a technical issue where you need to take a sample of the pages on both the desktop and mobile versions and really double check to make sure that all of the signals are aligned properly, that you're using the exact URLs as they're indexed, that you've got the canonical tag set properly, that you don't have any weird no index tags there, that you don't have weird redirects between those pages so that we can really confirm the link between the desk top pages and the mobile pages so that we can trust the data that you're bringing us there.

AUDIENCE: John, can I quickly ask you a malware question?


AUDIENCE: You know, I sent you that site the other day, and one of my friend's websites got hit by a WordPress malware attack. And so all the pages have been reindexed. Everything seems to be coming back to normal, but I'm still not able to do a Fetch as Google in Webmaster Tools. So I'm not quite sure why that is and whether or not-- what's the process when somebody goes through a malware attack? Has anybody else who's watching this as well? Because you get a security message that gives you an example, and then you can submit for a review. Are you supposed to see those examples removed? Do they stay there for years to come or-- can you go through the process for us, please?

AUDIENCE: Did you check the htaccess?

AUDIENCE: The ht-- yes. Yeah, I've cleared up what I believe to be everything. The site was having a 500 response, and now everything seems to be back to normal. All the problems are fixed, and the site's been reindexed. There's 58 pages back in there. The site only has about 60. I was just wondering why the Fetch wouldn't work.

JOHN MUELLER: There are different types of ways that a site can be had. So it's really tricky to give you one process that does everything. We tried to simplify it a little bit, but there's still some differences there. And so if you got a malware message, then you'd have to go to the Security Issues part of Webmaster Tools, where you see some sample URLs, and you can request a review there. And that process is fully automated. So basically, we recrawl those pages on your site. We double check to make sure that there's no malware on those pages anymore, and then we remove that. In Webmaster Tools, we remove the warning in the search results. We remove the browser warning. We remove it from the safe browsing API list, and that process is pretty automated. So you should see that kind of go through within about a day. Oftentimes, it's just a matter of a few hours.

AUDIENCE: Does the security message get removed? In Webmaster Tools itself?

JOHN MUELLER: Yes, it should be removed as well. So--

AUDIENCE: There's still like a snippet example. And I just wondered whether or not, like in various other things, that it just sort of stays there as a reference.

JOHN MUELLER: I'm not actually sure now. So especially with regard to the snippets, that's something that we might keep a little bit longer, because that's sometimes useful information to figure out what exactly happened. But the malware warning and the ability to request a review, that should disappear if that's all clean now. But usually that's something where your site is completely blocked in search anyway, where you'd have to enter the URL manually. So if that part is resolved, then the malware side is resolved, definitely. The other option is, if your site got hacked with some kind of SEO spam with hidden links, where they added hidden pages to your site that aren't malware, but they're kind of fishing landing pages or kind of spammy landing pages, that's something where you would go through the normal manual actions process where you do a reconsideration request. Someone from the Web Spam team would manually review those pages and double check to make sure that everything's OK. The main reason for that difference is that malware is something that's very black and white. It's either malware or it isn't malware, whereas the kind of SEO spam, hacking, that's something that's a lot harder to judge. Is this something that the webmaster put on their site, or is this something that some hacker put on their site and is just trying to gain from a legitimate website? So that's something that the Web Spam team has to manually review.

AUDIENCE: John, and what would I do about the Fetch as Google? Is that-- why would that not work?

JOHN MUELLER: That's usually more of a technical issue. So the 500 errors, that's something that might play a role there. Usually it kind of fails either if on our side something in the system is broken, which is rare, but I guess it can happen, or if we think that we can't crawl more pages from your website, if we think that this server is so slow or so overloaded that additional requests would put it over the limit. And in cases like that, you'll see that it takes a really long time for the Fetch as Google to be processed. And then in the end it'll say, we couldn't do it, or there was an error, something like that. I think that's what you might be seeing there. Because if your site had a lot of 500 errors, if it had a lot of timeouts, then we'll reduce the amount of crawling that we do on your site to a very small number. And then as we can recrawl again and we see, oh, this is actually pretty good, over a couple of days or a week or so, we'll kind of ramp up that number and say, OK, now we can crawl completely normally again. Now things like Fetch as Google will work normally as well.

AUDIENCE: So technically, in some sort of weird way, I could actually sort of change the crawl rate. And then maybe the crawl rate would then affect whether or not I'd be able to do a Fetch, possibly, if I were to play around with it. I'm not going to.

JOHN MUELLER: That crawl rate setting is actually a limit. So setting it higher than it currently is won't change anything. If you set it lower, it'll limit it, but you can't set it higher and expect us to crawl more. What might be possible is if you use a mobile-friendly test to kind of test to see what the URL looks like with the Google mock user agent. With a smartphone user agent, of course, it's not the same Google bot, but it's like the smartphone Google bot to see if the content is the same.

AUDIENCE: Yeah. OK, thanks, John. Much appreciated.


AUDIENCE: John, a short question related. Even the response from the Webmaster team for the malware is automated?

JOHN MUELLER: For malware it's completely automated. Yes.

AUDIENCE: OK. Because I forgot the answer very fast, so thank you.


AUDIENCE: Hi, John. I had one. I have a question. It's regarding expired content. So we have seen that most of our competitors, they are using a kind of loophole in the algorithm. We have seen that they redirect with a 301, move permanently HTTP status, and they redirect directly to the category when the content is expired. So we believe that it's a kind of gray or black character SEO technique. And we tried to serve a 410 status. But we would like also redirect with a category page, because for the users, usually this page is really valuable. So how do you handle or how do you like this strategy? Is it a soft-- or how do you take it?

JOHN MUELLER: Yeah. So these kinds of situations where, if you remove a number of pages and you redirect to the category or home page of a site, we call those a soft 404. And internally, we treat them the same as a 404. So if you return a 404 directly or if you do something that we treat as a soft 404, then it's kind of the same. It's not that there's any kind of advantage from doing a redirect. You don't pass page rank around, anything like that. So that's something where they wouldn't have any advantage of that, but on the other hand, what would happen is we'd crawl those pages more frequently, because we'd think, oh, these old, expired pages have a redirect now, but maybe they don't have a redirect later. So we'll kind of waste some crawling time trying to confirm those redirects. Whereas if they returned a 404 directly, then we would say, well, these pages don't have any content. We can drop them from index. We don't have to crawl them that quickly. So it's almost a disadvantage if you redirect instead of return a 404. I think overall it depends a little bit on how big your website is. If you have a really big website with a lot of pages that we see as a soft 404, then that's a lot of time that we waste with crawling content that we could spend on crawling new and updated content instead. So that's something where you probably see at least a small effect from those soft 404s, which is negative. Whereas if this is a smaller website and we find 1,000 pages that have a soft 404, then that's not going to change anything.

AUDIENCE: Our competitors are eBay or Gumtree. And if you check them, actually they are doing that.

JOHN MUELLER: Yeah. There are lots of sites doing this.

AUDIENCE: And so we thought that 410 would be better because it says that the content is gone, and then do a redirection, because we didn't want to duplicate the pages, the category pages. So this strategy, you would not advise it? You would really advise only a 410, without redirection?

JOHN MUELLER: Technically, a 410 is the proper solution for that. From a usability point of view, I think that's something that you have to find out for yourself. To some extent, it can make sense to show a category page for an expired page. To some extent, it might make sense to show a related items page. I don't know. Instead of the category or the expired page, it really is something that you can work with your users to try to figure out what the best solution is there. And you can return a 410 for those pages and still return normal content. It doesn't have to be a page that says, this is an error. It can be a page that says, these are related items, and you return a 410. Search engines, they understand, this page is not valid. But users still have some value from actually seeing that page.

AUDIENCE: But when there is a chaining, there is first a 410 then a redirection which is serving 200 or 300.

JOHN MUELLER: That's something we wouldn't see. So if you have a 410, then that's like a final result code for us. We don't crawl that page. That's essentially closed for us. What you can do is, for example, use JavaScript on a page like that to redirect to a related page. So from our point of view, we see the 410, the user sees the related page, and you can kind of satisfy both sides like that. And I'd also shy away from assuming that really big websites like eBay, or even Google, that they always get it technically right. Because I know even on big websites there are a lot of mistakes that are made, and a lot of things that could be done in a lot better way. And we see that at Google as well, where sometimes we'll look at our website and say, oh, gosh, why did they do this? Or sometimes we'll see external posts where people will say, oh, look how crazy Google has implemented this pagination or whatever. So just because a big website does it, doesn't mean that it's a good thing to do.

AUDIENCE: Love the watch, John, by the way.


AUDIENCE: Thank you very much, John.

AUDIENCE: I love the watch, by the way.

JOHN MUELLER: Oh, yeah. They're fun.

AUDIENCE: John, if I can jump in with one quick fundamental question. What role does social media have in Google's [? outgroups? ?]

JOHN MUELLER: What role. That's a very philosophical question almost.

AUDIENCE: It's not an easy one.

JOHN MUELLER: Yeah. So I think there are two aspects there. On the one hand, we don't use social signals in our search algorithms, so that's something that we currently don't use at all. I don't know. Maybe that will change over time. These things always evolve. So I think that's one aspect to kind of keep in mind, that if you buy 1,000 likes or +1s for your site, that's not going to improve anything on search. The other thing to keep in mind is that if this content is publicly hosted, then that's something we can crawl and index. So if you have a page where you post content on Facebook, if you post on Google+ and it's something that's public that can be crawled and indexed, then we can crawl and index that content, and we can show it in search.

AUDIENCE: Does the same thing happen when other personal, normal, everyday people are sharing content from your website on their pages, and if their pages are public, you're able to crawl those as well?

JOHN MUELLER: Sure. Sure. If this is public content, if this is something that we can crawl and index, then we'll try to crawl and index it. If it's something they're sharing privately or if they're sharing it with a closed group, then, of course, we can't crawl and index it. We can't really do anything with it. So on the one hand, all this social media content is something we can pick up and index for search. But on the other hand, the social signals from that kind of content isn't something we'd use in search.

AUDIENCE: Thank you.

JOHN MUELLER: All right. Question from the Q&A. "I've changed my website to HTTPS. What should I have to do in Google Webmaster Tools? Like change of address or maybe adding HTTPS site to Webmaster Tools?" Yeah. The main thing you need to do there is really add the HTTPS version to Webmaster Tools so that we can pick that up and give you the information about that. If you're redirecting from HTTP to HTTPS, then most of the information in Webmaster Tools will be shown in the HTTPS version of your site, because that's the version we have indexed. So that's essentially the main thing you'd have to do in Webmaster Tools. You don't have to do any change of address, anything like that in there. I don't think you can even do that at the moment. So set up your site, add it to Webmaster Tools, and then you should be able to also see the index status information, where the HTTP side will kind of slowly go down as we recrawl or reindex it, and HTTPS will slowly go up.

AUDIENCE: Yes. Actually over the course of the last few days, I've seen a couple-- well, handful of people in the forum going, oh, we can't do a change of address. So maybe just having a little note, as more and more people are going to switch to HTTPS, in Webmaster Tools, go look if you're doing this. Don't worry about the change of address not working, because you're not changing it.

JOHN MUELLER: It's always an interesting discussion around that. Because when we talk to engineers, we're like, well, everyone wants to use this change of address tool for moving to HTTPS as well, because for them it's a site move. And the engineers really strongly object to even making that to look like it would work for HTTPS, because it actually doesn't have any effect on HTTPS. So it's a tricky situation, but I understand it's confusing, so maybe we can solve that by adding a little text. That's probably an easier solution than anything else.

AUDIENCE: Little notes here and there seem to do the trick quite nicely at times.

JOHN MUELLER: Yeah. Good idea. "How does Google extract the article date for the OneBox in the universal search results? We're seeing problems with news articles predated in the OneBox, whereas Google News snippet displays the correct date." Marcus, I'd pass this on to the team here to double check to see what's happening there. I didn't hear anything back before the Hangout, but I know they're looking at it. And I think you pointed to a forum thread, so I'll try to post an answer there if I have anything back from them that you could implement differently. Sometimes there are simple things that you can do to make it easier for us to pick up the date. Sometimes people do everything right and our algorithms just have a higher opinion of themselves and think that they have it right. So that's something that maybe we need to fix. Maybe it's something you can improve. I'll let you know. "We made the switch to a new URL structure and to HTTPS. Our old URLs are going through two 301 redirects until they reach a new format. Is this a major issue? Now our new URLs are indexed, but we're still seeing minus 30%." With two redirects, it's not really a problem. We follow up to five redirects right away when we crawl. And if it's more than five redirects chained, then that's something where we have to recrawl again to kind of pick up the next set of URLs. So with two URLs, you're still in the good area. We recommend redirecting directly if possible, primarily because users also see this redirect time as something that takes a long time for them to see, especially if they're on a mobile phone. So if you can set up a 301 redirect directly, I think that would be a great idea. I don't think that this would have any kind of effect in your search results or your visibility in search. So the changes you're seeing there are probably more the general changes that you would see with any kind of a site move situation, where you just see some fluctuations for a while until things settle down. And sometimes this is something that takes a little bit longer than just from one day to the next, maybe it could even take like a month or so for things to really settle down to a similar situation as before.

AUDIENCE: John, I'm actually on the call now.

JOHN MUELLER: Oh, great.

AUDIENCE: Actually, I'll ask this question. And we've looked at this from multiple points of view. We did have some problems when we made the switch, like we realized we didn't redirect the images. So then we redirected the images. Then we also realized that the canonicals for the main categories were set to something else. So then we switched that again. But it's been actually-- since November 23rd, I think, that we made the switch-- almost a month now and we're still anxious because it's a 30% drop year over year on Google organic traffic. But not all--

JOHN MUELLER: Wow. That's a pretty big drop, yeah. If you have a forum thread or something that you can send me so I can look up the details, that would be really useful. I'd love to take a look at that. I know with images, especially if you have a very image-heavy site, then that automatically takes a little bit longer because we have to make sure that the image landing page and the image itself kind of match. So what will often happen there is, if you move an image-heavy site, then we'll recrawl the landing pages. We'll see the new landing page refers to a different image URL, and then we have to recrawl those images. And we tend to recrawl images at a lower rate than web pages, so that kind of automatically takes at least two or three or four cycles to settle down again. But if this has been the case since a month, then that seems like something--

AUDIENCE: We realized this about one or two weeks after we made the switch. So now my concern is if they've been disappearing, if they've been gone or [INAUDIBLE] just for one or two weeks, or will we ever have a chance to actually get back some rank?

JOHN MUELLER: Yeah, definitely.

AUDIENCE: So even if they're gone for one or two weeks [INAUDIBLE].

JOHN MUELLER: Yeah. So the images themselves were gone, or the landing pages, or which part?

AUDIENCE: Yeah, the images themselves were-- the old image URLs were going to 404 errors.

JOHN MUELLER: OK. I don't see a problem with that. That's definitely not something that would have a long-term effect. But what will happen there is we'll probably recrawl a few cycles before we pick up on that redirect to the new URLs. So that's something which probably wasn't optimal to do, but looking back it's always easier. But I imagine over the course of a couple of weeks that should settle down a little bit, at least visibly with the more popularly seen images that we probably recrawl a little bit faster.

AUDIENCE: And the one other thing that I thought that was wrong or that we-- I thought might be a signal was the fact that the number of error pages was quite high. Because we had a lot of junk pages on the old site which were actually disabled products that were responding with 200 status codes. So obviously that reduced-- we can't fix or do anything about those. We just let them kind of die out from the index.

JOHN MUELLER: That's perfect. Yeah.

AUDIENCE: But I think this could also be maybe a reason, because the ratio between our good pages and our error pages was quite skewed. There were a lot of errors all of a sudden.

JOHN MUELLER: Should be no problem.

AUDIENCE: Should be no problem.

JOHN MUELLER: That should be no problem. So if they're returning a 404 or a 410, that's absolutely no problem. If they were returning a server error, like a 500, 503, then that's something that could result in us crawling a little bit slower, because we think maybe our crawling is causing the server errors. But if they're returning a 404 now, which I imagine is what's happening, then that's perfectly fine. That's not something--

AUDIENCE: They're actually returning-- this is the funny one, because they're actually returning a 302, then a 404.

JOHN MUELLER: That's fine.

AUDIENCE: Still fine? OK.

JOHN MUELLER: That's perfect.

AUDIENCE: Cool. Thank you.

JOHN MUELLER: All right. "Late October, Webmaster guidelines were revised to recommend allowing crawling of CSS and JavaScript in robust text. As an agency managing auto dealer sites, do we need to open up all these sites to ensure our sites are ranked?" Ideally, we'd be able to recrawl the-- or ideally, we'd be able to crawl all of those CSS and JavaScript files so that we can really see what these pages look like. This has somewhat of an effect in normal web search, but it definitely has a really strong effect for mobile pages. Because if we can't see the CSS and JavaScript for mobile pages, then we can't tell that it's actually mobile-friendly. So this is something where you'll see that we won't be able to show the mobile-friendly batch in the search results, and we won't be able to recognize that these pages actually work well on mobile, for example. So that's something where-- especially if you've spent time on making pages that work well on mobile, if you don't let us crawl the CSS and the JavaScript, we can't really tell that they actually work well on mobile, because we can't see that. So that's something where I'd definitely recommend that you do it. If this is a normal desktop-type website where the CSS just kind of switches between different fonts and adds a background color, something simple like that, that's less of a problem. In the end, the easiest solution is really just to allow us to crawl these pages so that we can pick up on both of those options without you having to explicitly make a decision on that. So that's something where-- usually crawling CSS and JavaScript files doesn't take up a lot of bandwidth, doesn't take up a lot of crawling, because we can reuse those in our index for other pages. So that's something where I'd recommend just opening that up so that we can render all of these pages and see what they actually look like and break your pages appropriately based on that.

AUDIENCE: John, I just have a quick question if you get a second. Kind of piggybacking off the social media outgrowth with Google stuff, how about video content? I know that falls into the realm of social media, but what role-- if any role-- does video content play? Either on a website or just through YouTube and things like that.

JOHN MUELLER: We do show it in search. So in that sense, it's something that we'd show to users in search. If you have video content and we can tell a user is searching for videos, we'll try to present that to the user. We also show that in video search. So if someone is explicitly searching for videos and we know that you have videos on your web page and we can associate that appropriately, then we can show that in search. So that's something where you kind of have an additional way from the search results to your site. I know some sites prefer not to be seen with a video snippet because they think, oh, I have 10 pages of informational content and a video, and I don't want people to assume that my site is just a video. Those are pretty rare cases that I've seen, and I think our algorithm does pretty well now in understanding what the primary content of these pages is. If we can tell the video is really primary there, then we'll try to pick that up and show that in video search. If we tell that the video is like something in the sidebar that users can expand if they want to see it but it's not the main thing, then we'll try not to show the video snippet there.

AUDIENCE: And you also give videos a boost, right? Sometimes.

JOHN MUELLER: We don't really give them a boost in the sense that if you have a video you'll rank higher than other sites. But if we can tell the user is searching for a video, then of course we'll try to present that kind of content a little bit higher in search.


AUDIENCE: Hi, John. Would Google actually prioritize the links or the quality of links above the home page or technical aspects of the website? What I mean is, we've [? got a ?] competitors a website that's kind of a disaster in terms of [INAUDIBLE] websites from a technical point of view. But they are really stable in terms of search items. They've got a couple of these links, so that's the reason why I'm asking.

JOHN MUELLER: We do try to take into account all signals that we pick up on and use them appropriately. If a website is really bad from a technical point of view and we find a lot of good other signals, like good links, then it's kind of hard to draw the line and figure out, like, on average is this actually pretty reasonable? I think an extreme case would be, for example, if a website were blocked by robust text but is referred to by a lot of really authoritative sources, maybe a link from CNN or the "New York Times" or something like that, then that's something where we don't have any on-page information at all, but we just have to focus on the links. And that's obviously an extreme case. My general recommendation in a situation like that is not to rely on external signals alone, because if you know that there are things on your site that you can improve, then work to improve those things. And make sure that across the board you have reasonable signals from everything instead of trying to compensate one thing with two much of another thing.

AUDIENCE: Cheers. Thank you.

JOHN MUELLER: OK. Question here from the Q&A. "Google search for a site, example [? ?] minus example [? ?] returns 3,450 results. Webmaster Tools shows 550 results. And search for just the site shows 157 results. Which number can I believe?" Probably none of them, I'd guess. I wouldn't focus too much on this number alone, especially if you're looking at things like the index status number or just a site query number. Site query numbers by themselves are very optimized for speed, not for accuracy. So I wouldn't really use that as any kind of a metric. It's useful to see that we have a bunch of pages indexed, but I wouldn't use it as any kind of actionable metric. What I would use instead of all of these is a create a sitemap file with the links-- with the URLs that you really trust, that you think are really important, and look at the index count for those sitemap URLs, because those are the URLs that you really care about, the URLs that you know that exist on your site, the ones that you want to have indexed. And that's the number that's kind of actionable from your point of view. If we're not picking up the sitemap URLs and you're doing everything right, then that's a sign that maybe something's wrong with your internal linking. Or maybe you're submitting the wrong sitemap URLs. Or maybe we don't like your site that much. Or maybe your site is just completely new, and we haven't had a chance to pick up on things. But in any case, that number is the more actionable number because it compares the number of indexed URLs based on the ones that you actually submit. "Some websites with just picture only, without content, get good rankings in Google. Why, when content is king?" It's similar to the previous question, where if we don't have a lot of signals from a web page's on-page content, from the things that we find on the page, then that's something where we have to pick up signals from externally or take into account other signals as well. So in an extreme case, again, if it's blocked by robust text, sometimes it still makes sense to show it for very competitive queries as well. So that's something where I wouldn't say your website will never show up if it doesn't have any content. But at the same time, if you are trying to rank for competitive queries and you know that your content is low-quality or bad or technically broken in some way, then that's something that you can fix on your side. It's a lot easier to resolve technical issues on your site than to assume that you can compensate that through other means. So try to make sure that your website is reasonable across the board.

AUDIENCE: John, a quick question. Are we allowed once, maybe once a month, to talk about Penguin? Just the developments, of how the developments are going?


AUDIENCE: Not like, hey, when did the update happen? Today or-- you know? Just basically--

JOHN MUELLER: Would you like to talk to me afterwards if-- which time?

AUDIENCE: Well, no, I just wanted to know because I'm still seeing unnormal shifts in the past two months as opposed to-- I've been on Google for a long, long time. It's not like I'm on Google once a week. I see pages all the time, but not pages like that, like in the past three months. I just wanted to know, what's happening right now?

JOHN MUELLER: What's happening right now is really always a hard question when it comes to our algorithms, because there are always so many things that are running at the same time that are kind of running automatically. So similar to how when we recrawl and reprocess the web, we pick up things like hidden text and have reindexed pages based on those kind of things. There are algorithms as well that kind of rerun all the time automatically. So it's not something where I'd say there are people at Google randomly pushing buttons during the week to say, hey, let's do this today, let's do this tomorrow. These are essentially automated algorithms that are running automatically. So it's not something where I'd say there's anyone manually causing these shifts and saying, oh, we should juggle everything around today and tomorrow and then settle things back down again the day afterwards. [INTERPOSING VOICES]

AUDIENCE: Sorry. Was there a rollback of any kind, John, of the search?

JOHN MUELLER: A rollback?

AUDIENCE: We've seen in the last 10 days in the UK, we've seen the results that were previous to the last Penguin refresh have just sort of appeared. And all the spammy sites that were gotten rid of have all just reappeared in exactly where they were previous to that Penguin refresh. So that kind of seemed a bit strange. I just wondered if something had happened there.

JOHN MUELLER: Most of the time, we try to run these things on a global level. So it wouldn't be the case that we'd kind of roll back something like a web spam algorithm just in individual countries. So these are things that should be running globally across the board for all websites in all countries. It's a bit different when it comes to algorithms that look at the quality of pages, where sometimes we look at the page and we say, well, we don't understand this language. And then suddenly we can improve our algorithms and say, oh, well, we understand French or Arabic now as well, and we can process that a little bit better. But with regards to web spam algorithms, usually those are things that we'd run globally that wouldn't be limited to individual countries.

AUDIENCE: I'm only paying attention, obviously, to the UK. So that's why I specifically said that. But it does seem strange that it's exactly as it was sort of 20 days ago when it seemed like the results had cleaned up quite a bit. And there's been quite a lot people chatting about that. I mean, we know it's sick because the traffic to our site gets hit quite significantly. And obviously, since we got hit by Panda and Penguin four years or so ago, we're still 800% down on traffic for our business. So I'm really just trying to get somewhere close back to like 50 or 60%. And it's just annoying to keep seeing those things. And you did mention in a previous Hangout that you don't like to mess around with people too much over Christmastime. So I just wondered maybe you rolled something out, thought, oh, that's not too great, let's flip it back and do a change later on after the holidays. So I just wondered if that was something.

JOHN MUELLER: I don't think so. But I don't have all of the details with regards to those kinds of changes. So these are things that the teams are working on. I know they're trying to avoid causing too much fuss, but a lot of this is really just continuous rollouts. And sometimes within a continuous rollout you'll still see fluctuations.

AUDIENCE: Yeah. No, I-- well, thanks, John, anyway.

JOHN MUELLER: I don't know. Maybe at some point we'll have a little bit more information on those kinds of things. I know Barry is asking the Press team as well. So I'm sure from all sides we might get a little bit more information to you guys that's more actionable for you as well. All right. "A site ranking in Google on a major key word with some bad review of my site, so how can I remove that site from Google ranking?" Essentially, you can't. If there's a site that is talking about your business or your content and it's also showing up in the search results, then that's not something where you'd be able to submit a removal request and say, hey, I don't like what this guy's writing. I don't want it mentioned. I don't want it shown in the search results for my name. So these are things that we essentially try to handle algorithmically, where we try to figure out which ones are the most relevant and show those in the search results. It might be different if there's a site that is copying your content, where maybe you can use something like a DMCA to resolve that. Or maybe there are other legal means that you can use to have this content removed, but it's not the case that you as a webmaster can just come to us and say, hey, I don't like what this other guy is writing. Please remove it from the search results for my name. That's not something that we would do.

AUDIENCE: But actually, with the new European regulation, are not people using it to solve this issue?

JOHN MUELLER: That's more of a legal issue then. That's something where you can use this form so let us know about this, and we can kind of process that, but that's essentially a legal process. That's not something from a search quality point of view where you can just say, I don't like this content, I don't want it shown.

AUDIENCE: Because, for example, we simply-- we had pages, and there was simply some content entered by the user himself, and he sent the request with the new-- I don't know exactly in English the new name for the regulation in Europe. So he went to lawyers or whatever, and then it fall back simply on us. And it looks like some people are using now this regulation to solve this kind of issue.

JOHN MUELLER: It's something where, especially when it comes to legal issues, we essentially have to follow the law. And if the courts come to us and say we have to take action on this like this, then we kind of have to do that. So I think that's something, especially if you're based in Europe, it's worth understanding how that process works and how that might be affecting your website. But for the more general situation where there's just some comment on the web that you don't like, that's probably not something that would apply. But it's definitely worth understanding how that works so that you can work out does it make sense to go down this path? Maybe I should talk to a lawyer and get their advice on this. These are things that could be playing a role as well.

AUDIENCE: Hey, John, I just have one final followup question for you. It's for a personal venture that I'm working on with a couple of partners. I think we've talked about this in the past. We are relentlessly focusing on content marketing, email outreach, social media sharing, I mean, just putting all our best feet forward. And for competitive search terms, we are page 4, page 5, or nowhere to be found at all. Is that something still related to that-- I know we're not going to use the word "sandbox," but the trust factor of just maybe not having enough trust yet to compete with the big boys?

JOHN MUELLER: Probably something similar like that where maybe we just haven't built up enough signals around your site to understand how we should be ranking that site within maybe an existing kind of-- I guess you'd call it kind of like an ecosystem of sites that are competing around those terms. That's very possible. That's something where essentially if you're active or if you're going into an area where there's already a lot of competition, then just having a great website is probably not enough. You really need to make sure that everything is working the way it should. Maybe it makes sense to focus on a smaller niche first and kind of leverage yourself into the bigger area with more competition. Maybe it makes sense to do social media marketing, marketing campaigns, ads, or whatever to draw attention to your website so that you indirectly pick up those search signals as well.



AUDIENCE: We're also spending $7,500 a month with Google just on bringing more attention to the site via-- whether it's a social media channel we're promoting or the website itself, and we just-- if we keep fighting and keep fighting that hard fight, fighting the good fight to bring more attention, more content, more links, more social media, and ultimately more value to the end user, that's something where we should see results in time, right?

JOHN MUELLER: Yeah. But especially if it's a really competitive area, then that's going to take a lot of time sometimes. That's going to take a lot of work. Sometimes what helps is to be a little bit unconventional and do something a little different than the way other people are doing it. It sounds like you're trying to attack this from all sides, so I think you're probably on the right path there. But I wouldn't assume that there is this one trick to get into competitive search results.


JOHN MUELLER: Because that's sometimes really, really hard. And sometimes there are really big businesses that have spent a lot of time figuring out exactly what they need to do to kind of get to that place where they are now. And that's not really easy to circumvent and get around and say, this is the shortcut. But it sounds like you're fighting the right battle there, and a lot of times this also helps you to build up an alternate stream of users that go to your site, that come back, that recommend your site directly so that in the case when search does pick up, it becomes an additional path of getting users to your site, but it's not the primary path. You don't have to rely on it. You're getting enough traffic from other people as well. And that way, if you see fluctuations in search-- things go up, things go down-- then that doesn't cause any significant problems on your side. So that's, I think, always a good strategy to have.

AUDIENCE: Thank you.

JOHN MUELLER: All right. Let me run through some of the questions here before someone grabs the room I'm in. "In Webmaster Tools, we've noticed a recent and significant drop in the number of URLs indexed. We've not changed our sitemap in any way. This is having a negative impact on our traffic. Please advise." If it's just an impact in the number of URLs indexed, that could have a lot of different reasons, but that shouldn't necessarily mean that the traffic to your site will drop. If the traffic to your site is dropping because of the number of URLs indexed, then chances are there's a technical problem on your site where we're just dropping these pages because they're technically-- maybe they have no index on them, something like that. So that's what I tend to point out there. "A site ranking in Google on a major keyword with some bad review"-- oh, we did this one. "Did the UK search result have a rollback?" I think we looked at this one briefly. "In the clients Webmaster Tools under links to your site, it shows an abnormally large number of links from a company with a Singaporean domain. However, Webmaster Tools only shows five different pages or-- and it's like 135,000 links under the domain, but-- or should I be worried?" Sometimes that happens. So I wouldn't necessarily assume that anything is broken. That's probably just the way that we're counting the links. Maybe we're crawling into a lot of duplicate URLs and we're dropping them actually, but we've counted those links. So if there are things like session IDs and net URLs, then maybe we crawled millions of pages and found that link on there, so we'll count that for the links, but that doesn't mean that there's any kind of a technical problem or anything that you need to solve there. "Google ranks my flyer with a PDF with more info and text next to a short product page where the PDF is linked to increase-- improve user experience. Is redirecting the user an OK solution?" Oops. Well, I have to run. So it's been great talking to you guys, and hopefully I'll see you guys next week again. I think we have--

AUDIENCE: Thank you. [INTERPOSING VOICES] | Copyright 2019