Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.
Marcus Miller is an experienced SEO and PPC consultant based in Birmingham, UK. Marcus focuses on strategy, audits, local SEO, technical SEO, PPC and just generally helping businesses dominate search and social. Marcus is managing director of the UK SEO and digital marketing company Bowler Hat and also runs wArmour aka WordPress Armour which focuses on helping WordPress owners get their security, SEO and site maintenance dialled in without breaking the bank.
Thanks Brain, these tips are useful. The key thing with most of the tips that you provided is that it will take time and most people want to have more traffic, but they do not want to do the work and put in the time. However, if you put in the word and you do a quality job then it will work out. I think that is the overall strategies that a lot of SEOs have to do today is just to take the time and figure out quality strategies.

In December 2009, Google announced it would be using the web search history of all its users in order to populate search results.[33] On June 8, 2010 a new web indexing system called Google Caffeine was announced. Designed to allow users to find news results, forum posts and other content much sooner after publishing than before, Google caffeine was a change to the way Google updated its index in order to make things show up quicker on Google than before. According to Carrie Grimes, the software engineer who announced Caffeine for Google, "Caffeine provides 50 percent fresher results for web searches than our last index..."[34] Google Instant, real-time-search, was introduced in late 2010 in an attempt to make search results more timely and relevant. Historically site administrators have spent months or even years optimizing a website to increase search rankings. With the growth in popularity of social media sites and blogs the leading engines made changes to their algorithms to allow fresh content to rank quickly within the search results.[35]
Thanks Brain, these tips are useful. The key thing with most of the tips that you provided is that it will take time and most people want to have more traffic, but they do not want to do the work and put in the time. However, if you put in the word and you do a quality job then it will work out. I think that is the overall strategies that a lot of SEOs have to do today is just to take the time and figure out quality strategies.
Hi, my name is Dimitrios and I am responsible for Crave Culinaire’s digital marketing. I would like to drive more traffic to Crave’s blog. Since Crave Culinaire is the only catering company who provides molecular cuisine, I thought about craving a blog post about that. The influencers in this niche have great success in utilizing recipes on their blogs. I will share some recipes of Brian Roland, owner and head chef of Crave Culinaire.
It’s an awesome post which I like the most and commenting here for the first time. I’m Abhishek founder of CouponMaal want to know more like you’ve said above in the points relaunch your old posts. Here I want to know is there any difference between changing the date, time and year while we’re relaunching old post OR we should relaunch the old post with the previous date, time and year. I mean it matters or not.
Hey Brian, love your site + content. Really awesome stuff! I have a question about dead link building on Wikipedia. I actually got a “user talk” message from someone moderating a Wikipedia page I replaced a dead link on. They claimed that “Wikipedia uses nofollow tags” so “additions of links to Wikipedia will not alter search engine rankings.” Any thoughts here?
Structured data21 is code that you can add to your sites' pages to describe your content to search engines, so they can better understand what's on your pages. Search engines can use this understanding to display your content in useful (and eye-catching!) ways in search results. That, in turn, can help you attract just the right kind of customers for your business.
An influencer is someone who people listen to online. When it comes to search engine optimization, having an influencer in your corner will mean more people link to your website, share your blog posts and trust your content. If possible, have an influencer who is a subject-matter expert head up the content creation on your website. This person can be you, someone from your company or someone you align with.
To prevent users from linking to one version of a URL and others linking to a different version (this could split the reputation of that content between the URLs), focus on using and referring to one URL in the structure and internal linking of your pages. If you do find that people are accessing the same content through multiple URLs, setting up a 301 redirect32 from non-preferred URLs to the dominant URL is a good solution for this. You may also use canonical URL or use the rel="canonical"33 link element if you cannot redirect.

Hi Brian, Awsome content as ever! I’m very interested in your idea of creating an ‘uber’ resource list or expert roundup post i.e. linking out to lots of to other authorities in my niche within one post. But should you always create ‘no-follow’ links to these authority sites to prevent juice from passing to them? And similarly if you sprinkle a few outbound authority links in other posts should they all be ‘no-follow’ or do you think big G ignores ‘no follow’ these days?
Relevancy is the first qualifier of a quality link opportunity. The next qualifying factor is the authority of the opportunity. Since Google doesn’t update PageRank (PR) anymore, you must rely on third party metrics. I recommend you use Domain Authority (DA) from Open Site Explorer, Domain Rate (DR) from Ahrefs, or Trust Flow from Majestic to determine the quality of your link opportunities. You should use all three tools if you can.
The intent behind “SEO agency” is obvious… The searcher is looking for an SEO agency. Most of these searchers aren’t looking for life lessons from an SEO agency owner. Instead, they are just looking for the best SEO agency to get them more traffic and customers from Google. Plain and simple. I knew this when I created that page, but my SEO ego was too big.
Although this is a step-by-step series, everyone's methods will (and should) vary, so it really depends on how much time you think it will take (if you're billing hourly).  What tools do you have at your disposal vs. how much researching for information will you have to do on your own? Will you have to pay for research reports or companies? Do you pay a monthly service for data or research?
Traditionally, defining a target audience involves determining their age, sex, geographic locations, and especially their needs (aka pain points). Check out usability.gov’s description of personas and how to do task analysis & scenarios for more details, or better yet, read Vanessa Fox’s upcoming book about personas related to search and conversion.
Every website should have a content strategy focused around your top keywords. When you create content such as blog posts, videos, whitepapers, research reports and webinars, it gives people something to link to. In addition, the content you create can rank by itself in the search engines. For example, if you write a blog post on “How to Pick an SEO Company,” there is a possibility it will rank for some of the keywords you use in the title and in the body post, especially if the post gets linked to from other websites or shared a lot on social media. It also helps if your website as a whole already has significant high-quality links. 
×