All questions have Show Video links that will fast forward to the appropriate place in the video.
Transcript Of The Office Hours Hangout
Click on any line of text to go to that point in the video
JOHN MUELLER: So today is Google
WebMaster Central Office HoursHangouts.My name is John Mueller.I am a WebMaster Trends Analyst
here at Google in Switzerland.And part of what I do is talk
with webmasters and publisherslike the ones here in the
Hangout, the ones thatsubmitted lots of
questions, and tryto help answer anything
that's still kind of open.As always, if there
are any new people herein the Hangout that
have any questions thathave been on their
mind for a while,feel free to jump on
in and ask them now.
EYEPAQ: John, I
have two questions,and then maybe I will
leave the Hangoutand watch the live stream
so the other people join.Basically, I have, like,
two short question,and then I will let
you be with the Q&A,because I saw that there
are plenty of those.My first question
would be, how importantare the arrows in
WebMaster Toolsfor href line return no tag?So basically, I have
the href pointingto a different language,
but there is no return backfrom that one.Is it because it's a
redirect or it's a mismatch?Would that cause any problems?Or it's something
that we can slide?
JOHN MUELLER: So
what happened thereis we would ignore that
set of href lang links.So if one page links to the
other one and the other onedoesn't confirm
that link, then weassume that link isn't correct.And we ignore it.So it's kind of--
EYEPAQ: Oh, both
JOHN MUELLER: Yeah.It really needs to be
confirmed from both sides.
EYEPAQ: OK.And one other question,
somewhat related to this one.There is a lot of
talk online form SEOsand also your colleagues.Is it still OK to put rel
canonical on every single pagepointing to itself?Just in order to avoid, like,
the duplicate parametersand things like that.Is it the--
JOHN MUELLER: Yeah.Sure.
EYEPAQ: So pointing bits
out, that's OK site-wide.One million pages,
it doesn't matter.
JOHN MUELLER: It doesn't
matter how many pages.You just need to make
sure that it pointsto the clean URL version,
that you're not pointingto the parameter
version accidentally,or that you're not always
pointing to the home pageaccidentally, because those
are the kind of mistakesthat we try to catch.
EYEPAQ: And that would include
having www in front or n/?So like really the
canonical versionthat you see up there
in the versions?
JOHN MUELLER: Sure.You can do that across
millions of pages.We'll try to take
that into account.
EYEPAQ: Perfect.Last one and really
short this time.The document that was
released, like, some days ago,with all the raters and all
the information, at some point,there was a
discussion that theremight be some issues with
having affiliate links.But it's an older
problem anyway.The question is,
is there a problemif there are affiliate within
content, so in body somehowhidden with bitly or
any URL shortener,like having the URL short?Can that be seen as
[INAUDIBLE] somehow?I mean, that done from,
like, tracking perspective,not to hide the affiliate.I mean, Google anyway sees
it at the end of the day.But do you think
that it's possiblethat those type of
link, especially if it'sin large numbers, can
be seen as somehowgrayish, or like sneaky
JOHN MUELLER: I wouldn't worry
about it in a case like that.I think there are
definitely better waysto track clicks on a page.So that's something that,
within Analytics, youhave ways of doing that,
so that you kind ofsave that extra redirect
through the external site.So I try to just set up the
links directly and track themin a different way.But if this is the only
way you can do trackingand you kind of
need that trackingfor particular
reasons, then it'snot going to be a big problem.
EYEPAQ: So you're
saying Google won'tsee those as, I don't know, a
mean to deceive someone, eitherthe users or Google by any way?
JOHN MUELLER: No.I mean, that's,
like, a common setupfor affiliate programs anyway.So it's not completely unseen.I just really try to make sure
that the links go directly,so that they're a bit
faster for the users.And there are definitely better
ways to track these things.But if that's the way you
have your site set up,I think that's fine.It's not going to
change anythingto, like, point directly
instead of pointingto a redirecting URL.
EYEPAQ: Perfect.Thanks a lot.Thanks for the invite.I will jump out, so I'll
leave room for someone else.And I will just [INAUDIBLE].Thanks a lot.
JOHN MUELLER: Thanks
for dropping by.All right.Any more questions
from someone who'skind of new to
the Hangouts here?
THIAGO POJDA: Um, I am.Hi, John.Hi, everyone.
JOHN MUELLER: Hi.
THIAGO POJDA: The question
I have is a friend of minehas this forum for the past,
I think, 10 years or so.And well, you
know, it's a forum.People ask stupid questions,
and people answer stupid thing.And since 2012 or something,
his traffic dropped heavily.And I think it's
something relatedto Panda, because of that tons
of low-quality content he has.The question is not
if it's Panda or not.The question is just, how
can a webmaster like him,who he doesn't know much about
SEO word content or whatever,be responsible for that?He also has some
linking problems,because it's all
PH, PBB, and stuff.But that's another issue.
JOHN MUELLER: Yeah.So I guess the
overarching kind of themethere is as a webmaster, you're
responsible for your websiteand how it's presented to other
users and to the search engine.So if you have a forum and
you have a lot of, let's say,low-quality, user-generated
content within the forum,then that's something that we
see as part of your content.So it's not that we would
say, well, this is a forum,this content was
generated by someone else,we'll ignore it more or less.But essentially, that's
a part of the content.Like the webmaster of any
other kind of website,you have to work on making
sure that the content that youprovide is really useful,
compelling, high-quality,unique for the users.And obviously, with
a really large forum,that's not something
that's easily done.That takes a lot of time, either
doing it manually or workingwith a team of admins or
moderators within the form,or setting up some kind
of automated systemto kind of figure out which
content is really greatand which of the content is
kind of, well, more like fillermaterial or maybe just
chatter among friends,those kind of things.So I don't think there is
just, like, one solution thatworks everywhere.
THIAGO POJDA: What
I would recommendhim is to remove
the chatter forums,the chatter groups,
the low-quality contentto being indexed by-- it
doesn't matter to be indexed.Like, just the
good-quality stuff, right?
JOHN MUELLER: Yeah.I think that's
definitely an option.Some forums just put that
content behind, like, a login.So if you're logged in
and you're a regular user,you find all of this content.It's not that it's removed.But it's just not like a part
that's front and center whena random user comes to visit.
THIAGO POJDA: All right.Awesome.Thanks, John.
JOHN MUELLER: Sure.
MARIO: Hi, John.
JOHN MUELLER: Hi.
MARIO: Hi.This is Mario.I would like to make
you two questions.Sorry for my English.One is about structured data.We think we did almost
everything in the WebmasterTool Search Console.But we don't know
how to recognizeif we are indexed properly.How can we?
JOHN MUELLER: So with
regards to structured data,if it's indexed properly, I
guess the main place to lookis the Structured Data Dashboard
within Search Console thattells you which of the
types of structured datawe found on the website,
we were able to pick up,which ones we were
not able to pick up.That's one thing.The other one is the
Indexed Status informationin Search Console,
which tells youhow many of your pages in
general have been indexed.
MARIO: Already did.There is the magic number
with the plus nearby.But I really mean inside
the search engine.How can I recognize?We have bread crumbs
and stars-- OK, rating.But we don't see, in
the search results,how, why-- sorry-- maybe.
JOHN MUELLER: OK.So to see if
they're visible, youcan just search like a
normal user would do.But if they're not shown
in the search results,then we kind of have
three main itemsthat are responsible
there, wherewe have to figure out if a page
should be showing rich snippetsor not.The first one is if
they are technicallyimplemented properly, which you
can test with a testing tool.You're nodding.So you probably have
seen that before.The other one is if they're
implemented correctlyfrom a policy point of view.So you're using recipe
markup on recipe pages,you're using bread crumb markup
for the bread crumb pages.
JOHN MUELLER: Yeah.I mean, those are
the kind of thingswhere if you use the
right type of markup,that's already important.Whereas if you use, like, recipe
markup on a product reviewpage, then that doesn't fit.That's something
we might throw out.And the third one, which is
the trickiest one, I think,is that we have to recognize
that your website isof high enough quality that
we really trust this markupand want to show it
in the search results.And that's something where
there's not a simple test whereyou can say, oh, it reached,
like, this high quality,where it's really hard to tell.So one thing I think you
can do is do a site colonquery for your website.
MARIO: Do what?Sorry?
JOHN MUELLER: A site colon.So in the Search
box in Google, youcan enter a site, colon,
and then your domain name,and then maybe some
of the keywords there.
MARIO: Yeah, I did it.
JOHN MUELLER: And if,
in the site query,we show the structured markup,
the rich snippets, thenfrom a technical point of
view, we have it right.From a kind of policy point
of view, we accepted those.And if they're not shown in a
normal search, then that means,from a quality point of
view, we are not really surehow we should
treat this website.
MARIO: Despite the fact
that I did everything,but not-- I don't know the
high quality, you know?You think you make
the best content,but maybe it's not
enough for Google.How can I know this?
JOHN MUELLER: It's
really hard to tell.Yeah.I don't think there is
a simple solution there.So what I would do is post in
one of the WebMaster forumsand get, like, honest feedback
from other webmasters.And see if there's something,
from a quality point of view,that maybe you're missing, that
you're kind of glancing over,because this is your baby,
and your website is alwaysthe best.
JOHN MUELLER: I know how it is.
JOHN MUELLER: But kind
of get the hard feedbackfrom other peers, the things
that you might kind of knowin the back of your
head, but you don'twant kind of see directly.But that's the
kind of feedback--
MARIO: So asking in--
JOHN MUELLER: Yup.
MARIO: Yeah.Asking into the
WebMaster forum, right?
JOHN MUELLER: Yeah.
MARIO: Thanks.The second question
is about Google News.I don't know if you
can answer to this.
JOHN MUELLER: Maybe.Maybe not.
MARIO: OK.Thanks.Another side.From one day to
another a long timeago, I would say three
years ago, before that day,it reached the first
position normally.Usually.From that day, whatever
we were going to write,whatever we were
going to post, wehave never ever reached
the first position.I think that it
could be one penalty.And this is my first time
that I have the possibilityto ask to you, to
someone, to Google.Is there any-- is something
where I can ask if thereis any penalty or whatever?
JOHN MUELLER: If there
is a manual action,you should see that
in Search Console.
MARIO: There isn't.
JOHN MUELLER: There
isn't one there.Then that's essentially
the algorithmsthat are trying to
do something there.But what you can do, as
a Google News publisher,is contact the Google News team.In the Help Center, they
have a Contact link.And you can send them,
I think, an email,and ask them the
question directly.I don't know if they can
help with a ranking questionlike that.So maybe there's a
technical problem.Maybe there's
something that doesn'tshow us an error in the tools,
but technically makes it hardfor us to rank.So I would just ask them and see
if you can get some help there.
MARIO: OK.Thanks a lot.Bye to everybody.I'll leave.
JOHN MUELLER: Sure.Bye.Thanks.
MARIO: OK, bye.
JOHN MUELLER: All right.Let me run through
some of the questionsthat I picked out from today.And then we should
potentially have timeto go through more and more of
your questions here as well.Let me just try to find
the right questions here.Google Search Console
Fetch and Render,we added a map
where we are locatedto try to improve our SEO,
but Googlebot has it blocked.And it's a high severity."So what should we do?So essentially, what
sometimes happens, dependingon the way that you embed a
map, is that the map itselfwas blocked by robots.txt
maybe on the Google Maps side,maybe on whatever other map
provider side that you'reusing.And if we can't see that map,
then we can't kind of indexthat content from that map.But if you have your address
on the HTML of this page,if you have a link
to Google Mapsor whatever on
those pages, then wecan pick that up, and use that,
and understand your locationbetter.The tricky part is if there's
something within the map thatis only visible within the map,
that can't be found on the HTMLdirectly, then
that's something wewouldn't be able to pick
up if the map is blockedby robots.txt.So if you have something like a
pop-up bubble with the openinghours, or your phone
number's in it,then I would just copy that
content to within your HTMLas well, so that it's
visible directly on the page.And then we can pick
that content up there.And for the users,
the map is obviouslya nice touch that gives them
directions, all of that.Let's see. [INAUDIBLE] about
kind of the indexing question."Google Search isn't
indexing my site.There are No Content keywords.It doesn't provide
structured data.Most pages appear if
you do a site query."So I double-checked this site.And what I noticed in
the URL is that youhave an umlaut in there, which
means that it's essentiallya special character.And what we often find
is that people whohave kind of these special
characters in their domainnames, they sometimes
have an alternate versionof their site as well.So in this case, with
an O with an umlautcan be written as "oe."And I noticed that
some of your contentis indexed with the
"oe" version and somewith the umlaut version.So what I would do there is
pick one of these versionsand really make sure that
it's the canonical versionfor your site, that you
have a rel canonical set up,a redirect set up
to that version.And also double-check that
that's the version that youlook at in Search Console.So it's probably not that
we're not indexing anything.It's just that we're not
looking at the variationthat you have currently
opened in Search Console.All right.Another bunch of site
structure questions.And let me try to find them."Is there an optimum
way of setting upa Blog section on your website,
so it improves overall quality?Or is it just a
simple case of linkingto the content within a relevant
product or a section page?"So essentially, you can set
that up however you want.It's not, from
our point of view,that we say a blog is
completely different contentand needs to be handled
in a special way.It's essentially content
that you have on your site.Maybe these are articles.If you call it a blog, or
if you call it an articlecollection, or product
reviews, or whateveryou want to call
it, is essentiallydoesn't really matter for us.Content is content.So if you link to it, we'll
try to pick that up and crawland index it."Larger sites with unique URLs,
with over 50,000 unique URLs.And a better folder
structure taxonomy.Will that help Googlebot
to crawl and index the pageor the site more efficiently
and faster on a regular basis?"So we essentially don't look
at the taxonomy of a URL.When we crawl content,
we essentiallyjust look to see if
this is a unique URLand if we think that it
leads to unique content.So if you have a website
set up in a way thathas minimum duplicate content
that we don't get sent offinto URL sections of a site
that are completely irrelevantor that are duplicates,
then essentially,we can crawl and index that
content fairly efficiently.So you don't need to
put text into URLsor make it have a semantic
structure within the URLsitself.Here's a similar question."Could it hurt our rankings
if we have missing levelsin the URL structure of a site?For example, we have
/clothing/shirts and then/color/blue, but no single
slash color folder."From our point of view,
that doesn't matter.Again, we don't try to
semantically take apartthe URL itself for
crawling, for indexing.We essentially try to see
which URL it's pointed atand crawl and index that URL
as efficiently as we can.It doesn't really matter
if there are partsof the site that are missing.Sometimes you'll see
users try to edit the URL.But I think that's fairly rare.Mostly, I guess, more advanced
users would be doing that.Let me see where
the other one is."Is it possible that
my internal pagesare more authoritative
than my home page?"So with page rank gone,
domain authority--I guess they check
various tools.From our point of view, sure.Other pages of your
site can be moreauthoritative than
your home page.Sometimes we'll see that a
specific product is really,really popular from a website.And everyone goes to
that product page,and you don't really
go to the home page.And that's completely fine.That's essentially up
to you, essentially upto how you organize
your business,how you organize your website.It's not something
where we would say,this is a bad sign if
everyone loves your products,but nobody likes your home page.Let's see.There's another one kind
of towards authority."What can we do to
our content to makeGoogle believe that we are
an authority on the subject?Should we, for instance, be
including links in our contentto places that have
source information?Is that going to have
a positive SEO effect?"So I guess first of all, I
would take a step back and say,if you want to be an authority
for a specific topic,you should be an authority
for that topic for your users,not for Google.Because essentially,
we try to seewhat comes out in the end, how
users react to this, how theylink to this content,
how kind of theyrecommend it to other people.That's the kind of
stuff we try to pick up.And that's the kind
of stuff that youcan influence by really
being an authorityrather than by trying to fake
being an authority through kindof links on a page and
those kind of things.So specifically, with regards
to links on a page, onereally old-school spam topic is
that people put Wikipedia linkson their pages.And they assume that
Google will look at thisand say, oh, this person
must be really fantastic,because look at all of these
Wikipedia links on there.And that's not really how
our algorithms work there.So just by placing
links on a pagedoesn't necessarily make
it a high-quality page,high-quality content.You really should
aim to actually havehigh-quality content and really
be an authority in that area,not just show that for Google.
BARUCH LABUNSKI: Can I
transition into a press releasewith this question?
JOHN MUELLER: Try it.Sure.
Well, I just wantedto know that if it's a
high-quality press release,I mean, content-wise, like
seven pages, for instance.Seven pages.And I know that's a
long press release.But if it's a nofollow,
it's fine, right,if it goes to a
nofollow to the site?That's OK, right?So you have no issues whatsoever
as long as the press releaseis a No Follow and
you can go nutsif it's really a serious brand
and they're releasing shoesevery single month.And well, I'm just-- yeah.
JOHN MUELLER: Yeah.So for our point of view, the
part about passing page rankis definitely something
that we look ator that we try to recognize.If it's trying to gain page
rank through press releases,that's definitely a problem.So if you have the
No Follow there,I don't see a problem with that.With regards to
the content itself,that's essentially
up to you and whatyou're trying to achieve there.And for some businesses,
it makes senseto do press releases,
because that'show they kind of reach
out to the people thatare talking about
this topic, thatare writing about this topic.
BARUCH LABUNSKI: [INAUDIBLE]
to the shareholders.
JOHN MUELLER: Yeah.I mean, for other businesses, it
might not make that much sense.Or if you're just
like an online shopand nobody really reads or
uses your press releases,then that's probably
time that you couldbe spending somewhere else.But for some businesses,
it definitely makes sense.
BARUCH LABUNSKI: Like
Google Search reviewers,it'll take them a while
to come to this page.But when they rate it, well,
I mean, once they rate it,that press release can
also become authoritivelater on, no?
JOHN MUELLER: Well, the
search quality ratersare more to kind of
help guide the waythat we build the algorithms.So they're not going to crawl
the whole way and then saying,oh, this is a good page,
this is a bad page.But rather, the
algorithm is doingthe right thing for this query
by showing this set of pages.So that's essentially just
content on other people's siteson these press release
sites that we try to pick upand index.
BARUCH LABUNSKI: OK.Thanks.
NIK: I've got a syndicate
question, if that's OK, John.
JOHN MUELLER: Sure.
NIK: On a site with half a
million pieces of content,would a ratio of
eight syndicatedarticles, high-quality
syndicated articles-- letme stress high quality--
to every original pieceof content, would that be
viewed negatively by Panda?
JOHN MUELLER: So eight
syndicated articles to oneoriginal article?Something like that?
NIK: Yeah.Yeah.Yeah.Of things that happen, is it
worth sharing with everybody?
JOHN MUELLER: I don't know.It's really to say, because
these quality algorithmslook at the site overall.And that's something where
syndicated content isn't, perse, like, low quality and bad.But it does need to provide
some value for users.So if people come
there and they say,oh, I've seen this article
five times already today,then I don't know
if that's really,like, the best kind
of content to puton a site front and center.But I don't think there is any
static threshold between thisis the amount of copies
you can have on your siteand this is the
amount of unique,all new content that you
have to have on a site.It's really better algorithms.Try to look at the site
overall and determinehow it should be handled
from a quality point of view.
NIK: Yeah.When you get to big
figures of pages,when there's a lot
of pages-- there'slike a million pages--
it's just difficult to keepthe original stuff to the
levels of this syndicated stuffthat may be great news,
like entertainment news,like Bono falls off stage
and breaks his leg, whichmay have happened
10 minutes ago,and it comes through on a feed.That might be great
information, and the peoplethat are visiting the
site may not know this.But it's just difficult
to keep the levelsof the original stuff up.And do we need to rewrite
"Bono falls of stageand breaks his leg"?Does that need to be
written a million times?To the user, the value
would [AUDIO OUT]
JOHN MUELLER: Oops.Don't hear you anymore.
SANTA CLAUS: John, that's
essentially how press or pressreleases work, isn't it?One company-- let's say
Apple's launching a new iPhone.They issue a press release.And then after a million
sites take that and rewriteit, that's surely the
ones that republish it,they're all citing the original
source in one way or another.So yes, surely you
should rewrite it.But do it in a way that's
compatible with your site.
JOHN MUELLER: Yeah.I think that's
something you probablyneed to test with your own users
and just see how they respondto this kind of content.I imagine for some
things, it makes senseto get more in-depth
information outthere in an article, where
you're not just, like,tweaking the words to
make it look unique,but actually providing
some value of your own.And in other cases, maybe like
the syndicated content directlyalso makes sense.But I think that's something
you probably can easily test.Or easier than, at
least, like, seeingwhat Google does
with it, test itby double-checking how your
users are responding to that.Do AV testing.Take the different types
of content that you haveand see how users
actually respondto that kind of content.Great.Let me move on with the
next bunch of questionshere just so that we can
kind of run through these.And then we'll
hopefully still havetime for more live
questions along the way."Our business has individual
branch pages for our depots.But in order for our services
to rank for each branch,do we need an internal
link between the branchand the service pages?Or is Google clever enough
to figure that out?"So in general, we
do crawl a websiteby looking through links.So if, within your
website, we don'thave links between
the individual pages,then it's really hard for
us to crawl those pages,to understand the context of
the individual pages there.So I'd definitely make sure
that it's kind of crawlableinternally, that we
can go from one pageto the other pages
within a website.I think in this
specific situation,when you're talking about
individual branch pages,you kind of need to
watch out that youdon't head into the doorway
page situation, wherethat you create maybe individual
pages for all the servicesfor each of the locations.And then suddenly, you
have thousands of citiesthat each have 5 or 10 or
20 different pages thatare essentially all the same.Then that looks really
kind of low quality,kind of doorway pagey,
almost spammy type content.So that's kind of what
I'd watch out for there.It's not so much the internal
links between these pages,but really kind of the
general structure of whatyou're trying to put out there.But if these are
normal internal pagesand you otherwise
have them on your sitewithout direct links
between each other,then that makes it a lot
harder for us to crawl."Do impressions or traffic
affect crawl frequency?Pages with high traffic
are crawled more frequentlythan ones with low traffic."So I think this is more of like
correlation, causation typesituation, where
we don't really seehow many people are actually
going through these pages.But if we can tell that
this is an important pagewithin a website, we will try
to crawl it more frequently.So that specifically
happens whenwe recognize, within a
website, that maybe thisis one of the main pages where
all the news articles areposted or this is
kind of like the newsfeed within your website.Then that's
something where we'llrecognize that
things are changingquickly there, that it's
interesting for usersthat we have that content.And we will try to crawl
that more frequently.So that's essentially us trying
to figure out where does itmake sense to crawl more
frequently compared to whichparts of the site kind
of stay more stableand don't need to be
crawled that frequently?
THIAGO POJDA: John, would you
say that links matter on that?
JOHN MUELLER: Links matter
with regards to crawling.Yes.Yeah, yeah.I mean, we need to be able
to find those pages first.So links matter at
least in that regard.But also, if we
can tell that thisis an important page
within the website,that things kind of come
together at this point,then that's something
that also helps us there.It's not so much that
you need to buildexternal links in those pages.But even internally,
within the site,if we can recognize that
everything kind of comestogether there, then
that's a sign for usthat this is something
that maybe weshould crawl more frequently.And I think the
other thing to addthere is that just because
something is being crawledmore frequently
doesn't mean that it'smore important for search.So we do try to separate the
crawling from the indexingand ranking part in the sense
that maybe something needsto be updated very frequently.But that doesn't mean
that it's automaticallymore relevant for people
searching for that topic.It's just, from a
technical point of view,we try to update it frequently.But that doesn't
mean that we're goingto rank it very high
just because it'schanging frequently.
THIAGO POJDA: All right.
JOHN MUELLER: All right.Kind of similar to
crawling questions."Is it possible to
have a robots.txt linedisallow the middle of a query
string, not just the beginning?Sure.You can do that.You could use the asterisk
wild card characterto kind of let us know that
essentially anything upto that point can be ignored.And then there's a specific part
that you're looking at there.
NIK: John, if a site no indexes,
like, 2/3 of its content,would that cause any kind
of a problem or penalty--
JOHN MUELLER: No.
NIK: --in an attempt to
clean out the weaker stuff?No?
JOHN MUELLER: That's fine.Yeah.
NIK: OK.Thank you.
JOHN MUELLER: "We've hit our
size limit for robots.txt,so we're looking
for alternate waysto preserve our crawl
bandwidth by nothaving thousands of variations
indexed for our search resultspages.Is it just as useful
to add noindex,nofollow at a page level?"Sure.You can use noindex,
nofollow on a page level.And that will help us to
not crawl those pages.The tricky part
there is we stillhave to crawl those pages that
have the noindex, nofollowon it, so that we
can see that markup.But I think the
bigger problem that Isee kind of in the
background of this questionis when you're saying
you've hit the sizelimit for the robots.txt file,
then probably your robots.txtfile is way too
complicated and isgoing to be really tricky
to actually maintainin the long run.So that's something where
I'd spend a bit of timeto really significantly
reduce the sizeof the robots.txt file, so that
it's a lot easier to maintain,so that you easily see
which parts of your siteare being blocked by robots.txt,
which parts aren't, andthose kind of things.Also, with regards
to URL variations,I try to avoid using
URL parameters thatare kind of unique or
search path elementsthat are really
unique in the sensethat sometimes it makes
it really hard for usto crawl a site and go
through the unique contentand [INAUDIBLE].So simplifying the
URLs in a way thatmakes it easier for us to
focus on the unique URLsis really important.But I'd really also try to
simplify the robots.txt filein a case like this.
THIAGO POJDA: John.Sorry to jump in again.It's just on that matter, if he
adds the noindex and nofollow,either way, Googlebot would
have to index that page in orderto see the noindex.And that would eat his
crawl budget anyhow, right?
JOHN MUELLER: That would
definitely have an effect.But if, from that
page, for example,you link to thousands
of search variations,then at least we
wouldn't follow thatto those other pages
that are being linked.
THIAGO POJDA: Yeah.But if that would be
like noindex follow,it wouldn't matter
that much, right?
JOHN MUELLER: If
it's just noindex,we would still
follow those links.Exactly.
THIAGO POJDA: OK.
JOHN MUELLER: Let me see.What can we pick up here?"Over 18 months ago, we
301-redirected our sitefrom one TLD to another TLD.That was, we
withdrew the requestand removed the redirects
three weeks ago.It seems like Google's still
processing this as a site move.So in general, you'll
probably still seesome of those old URLs if you
specifically search for them.So if you do a
site query, that'sprobably what you'll
still see therewith regards to withdrawing
the site move requestand, I imagine, putting
content on the old URLs.Again, that's something that
essentially just takes timeonce the content is there.So that's not something
that will immediately work.But as soon as we see that
there's unique content there,that we can index it
separately, then Ithink those 18
months are probablyenough for us to realize that
this content has actuallymoved to the new domain.A site move question.Let's see."If we would want to move the
section of our large websiteto a new domain, would it be OK
to keep a copy of the sectionon the original site
for old customersbut canonical all the
pages to a new domain?"Sure.You can do that.What will just
happen there is we'llcrawl both of these versions.We'll see the canonical.For us, the canonical
is kind of a signal.It's not a directive, like
a 301 redirect would be.So we'll probably pick
up most of the pageson the new location, but we
might still index some of themwith the older location.And if that's not specifically
a problem for you at the moment,that's fine.It's not something that would
be a significant problem for us.
MIHAI APERGHIS: Hey, John, can
I have a quick followup on that?
JOHN MUELLER: Sure.
MIHAI APERGHIS: So
regarding site moves,I sent you a couple of
weeks ago a Google+ message.With the new Google+, I'm not
sure if I sent it correctly.It was about a longer
redirect to a custom domainnot hosted on Blogger that
featured a page asking usersabout the redirect.So I don't know if
that was the best way.In case you didn't get
the URL, here it is again.So that extra page, I'm
not sure if it, for Google,if it's redirecting properly,
the Googlebot, or whether weshould use some other
type of redirect.
JOHN MUELLER: I don't know.I'd have to double-check.I imagine this was right
while I was travelingto the various conferences.But I will double-check.Or if you can ping that
link, drop another commenton that thread, then hopefully
it'll bubble up on my inboxagain.And I can take a look at that.I think that was a
general question,like, how do you move from a
blogger site to your own customdomain, essentially, right?
MIHAI APERGHIS: Yeah.
JOHN MUELLER: Yeah.
MIHAI APERGHIS: OK, will do.Thanks.
JOHN MUELLER: Sure.Here's one. "We get sometimes
a couple of directoriesare building a massive amount
of links to our website,and they want some money for
each link to be removed."So they're essentially
trying to be jerks and chargemoney for something that
they put up, a type of spamto try to, I don't know,
harm your site maybe.But the question goes on. "We
disavow all the bad links,and they keep
building new links.What can we do?"So from our point
of view, I lookedinto a couple of these cases.And we essentially
already [INAUDIBLE]most of these directories.What you can do on
your side is if you'reseeing this happening, I would
just disavow the whole domainand move on.That's not something you
need to have that removed.You definitely don't
need to pay anythingto have those kind
of things removed.
BARUCH LABUNSKI: Well,
there's a lot of criticismout there that it
doesn't work, disavowing.
JOHN MUELLER: It definitely
works on our side,so from that point of
view, I don't reallyknow what to add there.If, essentially,
the criticism is,we don't believe
what Google says,then I don't know
what I could sayto make people believe
that it's actually true.But from our point of view, the
Disavow tool definitely works.It definitely
handles submissionsthat you do there correctly.Let's see.
NIK: Can I just jump in
with a quick quickie?
JOHN MUELLER: Sure.
NIK: I'll just send you a link.Submit.We've had a problem with
we have photo pages,where there may be 10, 20, 50
photos in a gallery, of a gigor something like that.We've had a problem where we
had them all on individual pagesoriginally.And we think that was
causing us issues.So we've merged
everything into one page.So all the similar photos
have merged into one page.We've had a problem
with Google Images,where we were using a slider.And anything that's lazy loading
or out of view of the sliderwas not ranking
anymore, which was-- wewere kind of cutting off our
nose despite our face there.I've just sent you a link to
a page with some disclosurephotos on it from a
gig from the other day.I just wonder if
you could recommenda better way of handling it.We want the images
to individually rank,but we don't want
to be penalizedfor duplicate content.
JOHN MUELLER: Putting
them on separate pagesshouldn't be a
problem in general.So I need to take a look
at the URL afterwards.
NIK: We was using Prev, Next as
well to show that it is a set.
JOHN MUELLER: Yeah.So that shouldn't
really be a problemif you put them on
URLs separately.With lazy loading,
like you mentioned,the bigger problem is
that we might not actuallysee the image link directly,
because it's not reallyshown when we pick up the page.But that's probably
something wherewe could take a
look at your exampleand see if we can
maybe do a blog postabout this topic in general.
NIK: I would very
much appreciate that.
JOHN MUELLER: Because especially
with lazy loading, with images,image galleries, that's
the type of questionwe get every now and then.I think Mihai has already
battled through a bunch of thiswith his site.So it's definitely not
something that's unique to you.I'll definitely
take a look at thatand see what we can do there.
NIK: Thank you very much.Appreciate that.Thank you.
JOHN MUELLER: All right.One more spam type question."Are sites tarnished in any
way after a penalty recovery?And is there any
situation when itwould be easier to rank by
moving to a new domain hostpenalty?"So from our point of view,
at least from a manual actionpoint of view, sites don't have
any kind of bad karma if theyused to be flagged for kind
of spammy things and now therearen't.There is one kind
of exception there,in that when we see
that a site keepsgoing into a manual action,
cleans itself up a little bit,goes out, and goes back
in, and kind of sneakilyswitches from a good
site to a spammy siteand back again a few times, then
it's possible that the ManualAction team will say, well,
we'll just wait and seewhat this site ends up as in
a half year of a little bitlater down the line.So that's one situation where
it might have a kind of a longereffect there.But that's really
a rarer situation,really the type
of situation wherewe think that this is
kind of malicious leavingsswitch back and forth.So it's not something that
would accidentally happen.The other thing to
mention there isthat sometimes our
algorithms alsolook at spammy or bad techniques
that are used on a site,and try to flag that.And in those
situations, sometimes itcan happen that
our algorithms takea bit longer to
process everything,to pick up the problems,
but also to recognizethat things have been resolved.So you might see
some effect there.That's specifically true around
maybe the quality algorithmsthat we have that
take a bit of timeto actually pick up the changes,
those kind of situations.But if we are just looking
at it from a manual actionpoint of view, if
you've cleaned it up,if you've gone through the
reconsideration request,then you're OK.There's nothing manually
holding your site backafter that kind of a cleanup.All right.We still have a bunch
of questions lined up.But let me just open it
up for questions from youguys in the meantime.And we'll see-- pick and choose
from one of the other lists.
SANTA CLAUS: Can
I ask a question?
JOHN MUELLER: Sure.
SANTA CLAUS: We had a link
today from "The Daily Telegraph"because of the type
of company we are.So as a gift article
for Christmas,which is fairly normal.So in normal
circumstances, an articleor a link from "The
Daily Telegraph"would be a good thing, because
I assume Google would treatthat as an authoritive source.But when you look
through the article,it's clear that there are
other people in there as well.And there's maybe 10
companies in there.They're all nofollow
except for one,which means it's clearly some
kind of sponsored articlewithout it mentioning it
unless I'm being paranoid.But does that mean that
it's harder for younow to spot what is-- given that
newspapers have to make moneysomehow, even the good ones?How is Google handling that
sort of fairly obvious linkplacement?Without mentioning sponsorship,
it's clearly a sponsored post.
JOHN MUELLER: Yeah.I don't know.[LAUGHS]It's probably not something
that we would alwaysbe able to pick out
algorithmically, automatically,right away.So I imagine there might
be some of these situationsthat that either slips
through or where we recognizethat a site is, like,
regularly doing sneaky things,we say, well, we can't really
trust the site completely.They have great content, but we
don't really trust their links.Those kind of things
can happen as well.
SANTA CLAUS: I
mean, I assume theyhave to make money these days.So it must be harder and
harder to not sell content.
JOHN MUELLER: Well, I mean, you
can make money otherwise, too.You don't have to
do that in waysthat, like, negatively
affect search enginesor that specifically go
against the guidelinesthat we put together.So from my point of
view, it's not somethingwhere you have to break
our guidelines in orderto survive as a business.That definitely
shouldn't be the case.
SANTA CLAUS: Right.But there should
be, in that case,there should be no real penalty
for the person that has gotthe follow link versus
the 10 others that don't.They should still
get the same benefiteven though it's clearly
a sponsored post.
JOHN MUELLER: I don't know
how, in that specific case,it would be handled.I do know that the
Web Spam team doeslook for patterns like that.And they do regularly
do these kindof roundups, where they
go through sites thatlook like they're doing
these type of games or sitesthat are primarily getting
their links like that.And they do take action on that.And that's not something
that you would directlysee in the search results.Because if the Web Spam
team goes through and says,oh, this site is doing
something sneaky,we're just going to ignore all
of the outbound links on there,then that's not something you'd
see as a flag in the searchresults.
SANTA CLAUS: You
wouldn't, as Santiago justsaid, you wouldn't see
that as buying links,because that is buying links.
JOHN MUELLER: That
would be-- sure.I mean, we would see that as
buying links or selling links.
SANTA CLAUS: Yeah.OK.
BARUCH LABUNSKI: Hi.I sent an email.I'm seeing some stuff
that's happening here.A lot of different areas,
a lot of spammy sitesare reaching to the top.I mean, is that just
one of that situationwhere we have to wait until
the other character comes in?
JOHN MUELLER: The
BARUCH LABUNSKI: The other
character starting with a P,yeah.I'm just tired of
even saying that name.
JOHN MUELLER: Pirate?
No, he's right here.You see?He's right here, waiting.
JOHN MUELLER: OK.
BARUCH LABUNSKI: But no.On a serious note, I'm
just seeing a lot of-- Isent you an email
regarding that.And I'm seeing a lot of
the sister sites back.So it's like, hey,
OK, I can create also.But if you don't mind
going into that one,because it's so strange to see
now that-- I sent you an email.So I don't--
JOHN MUELLER: I mean,
that's the kind of stuffthat we do pass on to
the Search Quality team,also to the Web
Spam team to review.It's not the case that we
can take all of these reportsand say, well, we're going
to take manual actionon this specific set of sites.But especially if we see
a bigger pattern of thingsthat we've missed either
algorithmically or manually,then we will try
to do what it takesto improve our search results.
BARUCH LABUNSKI: So did you pass
my email, the one I sent you?
JOHN MUELLER: Of course.
BARUCH LABUNSKI: OK.It's just it's-- I guess it
feels like a restore somehow.
JOHN MUELLER: Yeah.I know it's frustrating.If you're working
on a site, and yousee someone else
kind of jump aheadby doing all of these
spammy techniques,it's always frustrating.And it's something
we do take seriously.It's something we do pass on to
a team and discuss with them.But we can't promise
to do anything specificfor a lot of these reports,
because sometimes itmakes more sense to say,
well, this is another one thatgoes into this bucket.And once this bucket
is big enough,then we'll, like, maybe
create a new algorithmbased on the feedback
that we have there.Or maybe we'll say, well,
this is a big enough bucketthat we need to
handle it manually.In the meantime until we
do have a better algorithmto pick them out automatically.
BARUCH LABUNSKI: There's
one bucket that has about 27of them.And I think I sent you
that one [INAUDIBLE].
JOHN MUELLER: 27 is
still a small numberfor, like, the general web.But yeah, I continue
sending them.Again, I can't promise
that we actuallydo take action on
these all individually,because otherwise, the team
would be busy just, like,hitting one site after
another and wouldn't have timeto actually find ways to do
it in a more scalable way,so that we can improve
all of the search resultsrather than just this
one specific query.
BARUCH LABUNSKI: OK.
NIK: A real quick one, last
one from me, if you don't mind.
JOHN MUELLER: Sure.
NIK: We pulled users
and found that wehad a very small navigation.So we've expanded it quite a lot
by adding all the sort of bestand new content to it.It seems to have impacted
the search negatively.Could you pass comment on that?Do you know what
the navigation is?Does Google know [AUDIO OUT]
JOHN MUELLER: Yeah.I mean, usually, we're
pretty good at figuring outwhich part of the content
is, like, the main contentand which part is more
like the boilerplate-- sostuff that's repeated
across the website.If you change
something significantlywithin your templates,
then obviously,it's going to take a
while for everythingto settle down again.But once it's
settled down, then weshould be able to pick that up
properly and use that properly.I guess one thing
that sometimes happensis if the navigation is
significantly increased,then we have a lot more
text on these pages, whichmakes it a little bit harder to
pick out which part is actuallythe relevant part.
NIK: It seems to have
finned the relevance.That's the effect that
I believe I'm seeing.Could that be the case?
JOHN MUELLER: Potentially.I don't know.It's hard to say.But these are the type of
things that usually settle downafter a while.So if you've recently
made that change,then maybe give it a couple
of months to settle down.If it's been a year now, then
probably it's something else.
NIK: OK.Thank you, John.
BARRY: I have two quick
questions, if I may.
JOHN MUELLER: Sure.
BARRY: One is a yes or no.Is Penguin still on
track to be launchedby the end of this year?
JOHN MUELLER: As
far as I know, yes.
BARUCH LABUNSKI: All right.So he's riding in the bag.He's right here in my bag.
BARRY: And the second
question is location filterin the search tools.Did it go away?Is it a bug?Do you know anything about that?
JOHN MUELLER: I don't know.I saw a bunch of reports
about that on Google+.But I don't know
what's behind that.So a lot of these search
features come and go.You have experiments that
kind of turn them on, try themin different ways.So I imagine it's
something around there,where maybe we did
some experimentsand we decided, well, it's
not that important anymore.We'll be able to pick up the
location better automaticallynow.I don't know the
BARUCH LABUNSKI: So
basically, the bus driverhas gone-- he's lost.Because it just says "Canada."Like where I am, it
just says "Canada."It doesn't give you an
option to, like, see.
JOHN MUELLER: Maybe we can pick
it out better automaticallynow.I don't know how
that works there.At least in Switzerland,
I don't think we've everhad the Location box there.
BARUCH LABUNSKI: I can
show you if you want.
JOHN MUELLER: I haven't seen
it [INAUDIBLE] in a long time.So these are things that,
from my point of view,in the Search UI, they just
change from time to time.These are changes that we
make for a variety of reasons.It's not to decrease the
quality of the search results.But maybe to make it easier
to make it more kind offit together with other
types of search resultsthat we have from the different
devices, those kind of things.It's definitely not something
where we're saying, oh,what can we do today to
mess with webmasters?Because we don't have
that kind of time either.And we like working
with you guys.
BARUCH LABUNSKI: No.Just the recent thing
is it was so precise.Like, as soon as that feature
came, it was like precise.OK, I'm in Toronto,
I want to-- you know,this is the precise
result. So I guesswe have to-- if it
stays like this,we get accustomized
to this new thing.It's hard.
JOHN MUELLER: Yeah.Some of these you
can still kind oftrigger by just
modifying the URL.So I don't know if
that's with the location.I believe there's, like,
a URL parameter thatused to be pulled out
with the location.Maybe you can try that.If you see that the
search results are reallylower quality because of this
kind of change, where you say,well, I'm searching
for Burger Kingand it's showing me
Burger King in Vancouverand not in Toronto, then that's
the type of thing we would sayis a bug that we
really need to fix.But if we're able to pick out
the location automatically,and you're in
Toronto, and it showsyou results from Toronto,
then from my point of view,that's fine.The less work the searcher has
to do to get usable results,the more likely they will get
usable results by default.
GLEB: Hey, John.Can I jump on with my last
question about numerous pageson the website?
JOHN MUELLER: Sure.
GLEB: So what if a
website has a lotof pages with similar
products on them and content?Can it cause apply in
some Google filtersor influence on the ranking
of the entire website?
JOHN MUELLER: So a lot
of similar productsto other websites or
within the same website?
GLEB: No.For example, it's a
website about car parts.And it's separated on
year, make, models.So basically, pages may
be very, very similar,but some of the products
may actually be different.For example, one product will
be available on the page for '85Pontiac, but it will be missing
only one year for '86 Pontiac.So basically, all content
is kind of the same,but it's really, like,
important for userto see the current year,
current model, and current make.
JOHN MUELLER: Yeah.That sounds fine.That wouldn't be something I'd
kind of artificially suppress.
JOHN MUELLER: Maybe
it makes senseto just make one page with
listing all the models,obviously, for it.That's one variation
that we sometimes see.Like, have all the
variations on separate URLor have one URL that has a list
of the different variationson it.
GLEB: OK.Well, it kind of
doesn't, because theremay be some exceptions,
which will causeyour [INAUDIBLE] to be-- what
if there is about 1 millionof pages on this kind?So does that [INAUDIBLE]?
JOHN MUELLER: I
don't see a problem.
GLEB: No problem.
JOHN MUELLER: I
don't see a problem.I mean, we have to crawl
and index these pages.But we can do that.It shouldn't be a problem.I think some of that probably
can be optimized by grouping itmore into higher level pages.But in general, that shouldn't
be possible like that.
GLEB: What that be possible
if I could send you, like,a couple of examples
of that website?Because it's kind of a
big automotive website.
JOHN MUELLER: Sure.
GLEB: So you could just
take a look more clearly.All right.
BARUCH LABUNSKI: John, are you
promoting Sweden's area code?
JOHN MUELLER: No.
Because I see the 46.Trying to figure it out.
JOHN MUELLER: Yeah.I have to go now.We have people
waiting for this room.So it's been great
talking to you all.Hopefully, I'll see you again
in one of the future Hangouts.Bye, everyone.