Last week, Google officially announced that their new web indexing system “Caffeine” is live on all Google pages. What does this mean for your website and do you have to change anything on your pages?

What is Google Caffeine?

Google Caffeine is the name for the new method that Google uses to index web pages. In contrast to Google’s old method, Caffeine can index new web pages faster:

“Caffeine provides 50 percent fresher results for web searches than our last index, and it’s the largest collection of web content we’ve offered. Whether it’s a news story, a blog or a forum post, you can now find links to relevant content much sooner after it is published than was possible ever before.”

Google changed they way in which they index the web because they want to show new pages more quickly in the search results.

What is the difference between the old system and Google Caffeine?

Google’s previous system updated the search index in batches.

“Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks.

To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you.”

With Google Caffeine, Google’s search index is updated continually:

“With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index.”

That means that new pages will be displayed in Google’s search results sooner if they are relevant to the search query.

What do you have to change on your web pages?

Caffeine is not a ranking algorithm update. It does not change the way Google ranks web pages. Caffeine just means that new pages will be shown much quicker on Google’s result pages.

To get your own website on Google’s first result page, analyze the web pages that currently have top 10 rankings on Google. The pages that now have a top 10 ranking on Google have done everything right to please Google’s latest ranking algorithm.

Analyze the top ranked pages and check how and where they use their keywords. Also check which web pages link to the top ranked pages and how they link to the top ranked pages.

Doing this is a lot of work if you do it manually. For that reason, we developed the Top 10 Optimizer. The Top 10 Optimizer automatically analyzes the web pages that currently have high rankings for your keywords and it compares them to your own website.

You will also get detailed instructions on how to change the content of your web pages and the links to your site so that your site will be listed in Google’s top 10 results.

Last summer Google announced a new project called “Caffeine”, which was described as a re-write of Google’s web search architecture.

comparing caffeine’s changes to the “Big Daddy Update” of 2005, which consisted of changes to the way Google crawls and indexes websites. It appears that more people are now seeing the effects from Caffeine out in the wild.
Back before the holidays, Google made it a point to assure everybody that Caffeine would not be rolled out (except for at one data center) until after the holidays were over – January at the earliest. The reason for this was that Google didn’t want to shake everything up during a key time for businesses, The company let everyone know about its intentions at PubCon in November. In fact, a few days ago, Google’s Matt Cutts posted a video running through his presentation from that event on his blog. He also provided a slideshow. It covers much more than just Caffeine, but if you missed it, you may want to consider watching it anyway (Caffeine discussion starts at about 22:10 in the video and at slide 29 in the presentation).

Google Caffeine Presentation PDF

The Holidays are Over. Is it Caffeine Time?

Last summer Google announced a new project called “Caffeine”, which was described as a re-write of Google’s web search architecture. Around that time, Matt Cutts discussed Caffeine with WebProNews, comparing it to the “Big Daddy Update” of 2005, which consisted of changes to the way Google crawls and indexes websites. It appears that more people are now seeing the effects from Caffeine out in the wild.

Have you seen possible Caffeine effects in use? Tell us about it.

Back before the holidays, Google made it a point to assure everybody that Caffeine would not be rolled out (except for at one data center) until after the holidays were over – January at the earliest. The reason for this was that Google didn’t want to shake everything up during a key time for businesses (they didn’t want a repeat of the Florida update).

The company let everyone know about its intentions at PubCon in November. In fact, a few days ago, Google’s Matt Cutts posted a video running through his presentation from that event on his blog. He also provided the slideshow. It covers much more than just Caffeine, but if you missed it, you may want to consider watching it anyway (Caffeine discussion starts at about 22:10 in the video and at slide 29 in the presentation).

“It’s a re-write of our indexing infrastructure. It’s taking the old way that we used to index things that we’d crawled around the web, and we’re replacing that with new architecture that’s fresh and that had been written to be more scalable, more flexible, [with] the ability to attach different types of data, and in the process of indexing, the ability to do more documents for a more comprehensive version of the web, and the ability to do it faster,” Cutts says of Caffeine.

But enough background. Barry Schwartz at Search Engine Roundtable points to a WebmasterWorld forum thread where administrator Tedster claimed to have seen Caffeine in action at a number of IP addresses. He wrote:

I’m seeing the Caffeine data-set being served via this set of IP addresses: 64.233.169.147, 64.233.169.105, 64.233.169.103, 64.233.169.104, 64.233.169.99,64.233.169.106

It seems to take 5 IP addresses to build the complete SERP, where in the past it often took only 3.

Schwartz also pointed to another member’s post (Whitenight), who said:

Well, just tripled checked with offices/employees in Texas, Colorado, and Indiana. All 5 “control” keywords/sites showed live Caffeine.

That member’s latest post says that the Caffeine Dataset is also on http://66.102.7.99 and http://66.102.7.104.

We don’t know for sure if this is all really Caffeine in action though. Google hasn’t commented on it, and has not made any announcements regarding Caffeine since what Matt said above. Some people don’t believe this is Caffeine at all. As Schwartz notes, we’ll have to wait for Google to say something.

Still, January is almost over, and Google said it would wait until after the holidays, specifically mentioning the month of January. It’s about time for this to be rolling out to some extent. Speed has been emphasized a significant amount in Caffeine discussion, and Cutts told us that page speed would likely become a ranking factor. Regardless of whether or not you are witnessing Caffeine in action yet, rest assured that it will be here sooner or later, and any edge you can give yourself in the meantime is for the good of your own site’s performance. Speed will not only supposedly help you in search going forward, but it just makes for a better user experience.

Speculation in the search industry is rife this week with claims that Google Search Personalization has changed the SEO playing field. But has it really? Or are people freaking out for no good reason? To find out, we’ll look at how it impacts SEO in the negative and positive. But first, let’s have a quick refresher on how Personalized Search works.

What is Personalized Search?

For the past few years, Google has been monitoring what you search for when logged into your Google account and in particular, what sites you click on in the SERPs. If you favor particular sites, Google takes note and customizes future searches to show you more results featuring your favorite sites, more often and in higher positions.

For example, if you like t-shirt shopping online and are a regular visitor to Threadless as a result of logged in Google searches, Google would feature pages from Threadless more in the SERPs you see for t-shirt related search queries than would normally be featured in SERPs shown to others for the same search queries. Likewise, pages from Threadless would be pushed higher up the search results than they would normally be.

Personalized Search has been in place for signed-in users for years, but this month Google rolled out personalized search to users worldwide, whether they are signed into a Google account or not.

Apart from privacy concerns, the announcement has prompted the inevitable “SEO is dead” claims that always seem to surface whenever Google announces a change to their search functionality.

So let’s take a look at how/why personalization might influence search engine optimization.

Why Personalization DOES Impact SEO:
• If everybody sees different SERPs based on their searching patterns, how can you measure a consistent ranking? How can you reach an audience if their search queries are already *rigged* to show your competitor’s brand?

• On page optimization and link building will no longer have as much influence on your site’s rank for competitive search queries.

• Clients who opt-ín to personalization and visit their own sites may have a false impression that their sites are ranking well in the SERPs and cease or refuse SEO services.

• Clients who opt-ín to personalization and visit their competitor’s sites may have a false impression that their sites AREN’T ranking well in the SERPs and blame their SEO.

• Companies / brands with greater traffic have a better chance to gain new business because searchers will see more impressions of snippets to their sites. This creates branding opportunities via snippets.

• Webmasters will start optimizing more for other search engines like Bing where they can have more of an impact on organic results.

• It will become even more difficult to rank for generic keywords and search phrases (as larger brands will tend to dominate based on market search share), meaning long tail search queries will become much more important in an SEO campaign.

• Search spam should start to be filtered out as very few people will be revisiting spammy pages. That should eventually push more relevant, naturally optimized pages higher up the SERPs, particularly those in competitive industries.

• Fresh content will give sites an advantage because new pages are more likely to stand out to searchers in personalized SERPs. Same goes for real-time content generated by Twitter, Facebook etc. Static sites are going to fall to oblivion.

• Audience targeting and snippet relevancy will become more important when optimizing web pages.

• PPC ads will have to try harder to compete with increasingly brand-biased SERPs.

• PPC will become more popular as people find organic SEO too complex and abandon it.

• Personalization should help normally lower ranked sites to get to the top a little faster via loyal customers and visitors.

• Titles, META descriptions and text snippet optimization will become SEO priorities.
• Top SERP performers will fall down the ranks if their snippets and offerings are not competitive enough, allowing lower ranked sites to take over.
• Manually checking your site rankings, or those of your clients with personalization switched on will result in skewed, inaccurate SERPs.

• Rank checking tools like WebPosition will no longer be accurate. Clients will stop asking for ranking reports (hooray!).

• Some think that Google could be using personalization to monitor user-driven search in order to tweak the PageRank algorithm based on what users actually search for.

• Brand new sites targeting competitive search queries have very little chance of appearing in SERPs customized by personalization, even with SEO.
• If you don’t rank well now for your target search queries, you might slip further and further off the radar as searchers refine their SERPs by clicking on the higher ranked sites.

• If clicking on SERPs begins to impact what users see, hackers may develop malware etc. that automates SERP clicking.
Convinced that SEO is dead yet? Hold your horses. Let’s aim for some perspective here.

Why Personalization DOESN’T Impact SEO:

• Personalization has been in place for some time already – since 2005 in fact.

• The main Google PageRank algorithm still applies, it’s just the delivery of the results that has changed.

• Any SERP emphasis is user-driven rather than algorithm driven and personalization changes only relate to search queries closely aligned to your web history.

• Most non-personalized SERPs are not identical these days anyway. There is evidence of changes even based on the same search query on same PC in the same location a few minutes apart. Different datacenters and Everflux between them mean consistently shifting SERPs.

• SEO isn’t just about SERP ranking. Think usability, keyword selection, conversion design, branding, social media, online reputation management etc.

Google Caffeine is the name given to Google’s “Next Generation” search engine, which it will use to rank and index all the pages on the wonderful world wide web. According to all indications, this is not just another one of Google’s infamous Updates, but a major “Overhaul” of its index and algorithm – the complex formula and calculations Google uses to rank all web pages, including yours.

If that doesn’t sound ominous enough, according to Matt Cutts (Google Spokesperson) one database is already showing Google Caffeine, and the full blown version will be released after the holidays. The reasoning behind this – Google doesn’t want to upset webmasters and site owners during the lucrative holiday buying season. In the past, other major Google Updates have come around this time of the year, most notably the “Florida Update” which severely affected many web sites and webmasters.
Recently, Google has been more aware and much more generous to webmasters by being more open and forthcoming in regards to how it indexes its pages. This time around, webmasters were even given access to a beta version of Caffeine which Google released last summer (’09) where webmasters could check to see how well their keywords and site would fare in this new search index. This beta site (www2.sandbox.google.com) has now been taken down by Google.

Like any professional search engine marketer who works online, I was constantly checking my sites and keywords in Google’s new search engine. I have drawn some conclusions from what I have observed, but please be aware it is often very foolish to draw conclusions and make predictions from a small sampling of results. You can end up with egg on your face very quickly, especially when you consider Google is probably still making adjustments and refinements on Caffeine as it analyzes the results.

However, there are certain ranking factors that even Google is telling us about, mainly “Site Speed” or how fast your site loads will play a part in how its ranked. We have also heard a lot about “Broken Links” and if your page or site has them, then it will probably be ranked lower. Of course, linking out to “Bad Neighborhoods” will probably still not be a good practice, if you want higher rankings within Google.

It should not come as a shock or a surprise, that “Over-All Page Quality” will play a greater role in how well your page ranks. Keep in mind, Google is like any other company putting out a product, if that product doesn’t have a high standard of quality, it reflects badly back on everybody concerned. Google’s SERPs (Search Engine Results Pages) are the key to all their online revenue, they must do everything in their power to keep that product fast, relevant, current and above all high quality.

Therefore, expect “OnPage Factors” to play a much greater role in Google Caffeine. Quality unique content, page design, good navigation, title, meta tags, description, keyword density, alt tags, page views, bounce rate, traffic numbers, time spent on page, and the number of social bookmarks may play an increased role in achieving high rankings. A perfectly optimized keyworded page, with the keyword in the title, description, meta tags, alt tags, on the page… will probably get you ranked higher in Caffeine, as well as most search engines on the web. This may be pure speculation on my part, but one of the areas Caffeine will be addressing or incorporating is “Social BookMarking”, that is the number of social bookmarks a page receives will determine how high it is ranked. I also believe one of the major reasons these bookmarks will become much more important has to do with the whole nasty issue of link buying.

Now, the integrity of Google’s index is not in question, but any savvy marketer or webmaster knows any individual or company with deep pockets and huge resources can buy their way into the top spot. Despite Google’s attempt to stop it, link buying and keyword positioning, is a thriving industry on the web. Rightly or wrongly, money and unlimíted resources will get you or your company to the top in organic search, regardless of which search engine you’re targeting.

All moral and ethnical issues aside, the small webmaster and/or online marketer is stuck right in the middle, with Google on one side and these major multinational competitors on the other. Looming on the horizon is Google Caffeine, a new sheriff in town!

What New Rules Will This Sheriff Bring?

The major question here is this: has the importance of backlinking been downplayed in this new index in favor of the keyworded domain and onsite content and optimization? Has there been a major shift to listing more quality content rather than relying on the number of backlinks a site is receiving, even from important related themed sites? The major problem and question to Google is this: if links can be bought, how do you keep your organic results democratic and fair, which was the original intention of Larry Page and Sergey Brin when they started Google in 1998.
One Possible Solution is Social Bookmarking

Will we see an ever growing importance of social bookmarks and links in this new index. It is quite easy to buy 1000 links, but getting 1000 or 10,000 “re-tweets” is a little more difficult. Similarly, getting two or three thousand “diggs” may be a little harder to pull off. Same goes for Del.icio.us bookmarks, Facebook fans… well you get the picture. Will Google’s use of these new social sites make Caffeine faster, more relevant, more current and most importantly of all, can it bring some democracy back into their index?

Of course, nothing in Google’s new index will be that cut and dry, that black and white. Other ranking factors such as age of site, past history and reputation, traffic numbers, authority branding… will all play a role in whether your site gets listed on that all important first page. However, on page factors may play a greater role – title, meta tags, description, keyword density, alt tags, page views, bounce rate, time spent on page, and the number of social bookmarks may play an increased role in achieving high rankings. Website speed or how fast your site loads may also be a new ranking factor.

Underlying this whole issue is the fact which many experienced webmasters/marketers already know, Google’s SERPs are not a one-trick pony anymore. For very lucrative (monetized) keyword phrases, Google’s results are broken up into Five categories… Info listings, Video listings, News Listings, Shopping Listings and Corporate Listings. Forget Caffeine, this is probably the fairest move Google has made in the last few years to make its SERPs more democratic.

Another even more puzzling issue for me concerning Google Caffeine is how much emphasis or ranking power will it place on “Keyworded Domains”, domain names which have your keyword or keyword phrase in them. Will these domains be ranked higher? Webmasters and marketers for years have been telling us we should always pick domain names which have our major keywords in them. Just common sense really, someone searching for “brown widgets” will more likely than not find that item at a domain called brownwidgets(dot)com or brownwidget(dot)com. The major SEO reasoning, all your backlinks will inherently have your searched keyword in the URL, thus bringing it up in the rankings.