Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.
I am a little confused on your first point. Sorry if it is a simple one to understand and I’m just missing it. What good would finding dead links on Wiki do for my personal website? I thought you would explain how to find dead links faster within my own site… but it seems that your tip is way more valuable than that. I just don’t quite understand what I do to positively affect MY site with this. Any help would be great 🙂 THANKS!
For some reason I had to delete some pages, these pages are using the HTML suffix, so I blocked them in robots.txt use Disallow: /*.html, but it’s been almost a year, I found that google robot often capture these pages, How can I quickly let Google completely remove these pages? And I have removed these URL from google webmaster tool by google index-> remove URLs, but Google still capture these pages.
Think about the words that a user might search for to find a piece of your content. Users who know a lot about the topic might use different keywords in their search queries than someone who is new to the topic. For example, a long-time football fan might search for [fifa], an acronym for the Fédération Internationale de Football Association, while a new fan might use a more general query like [football playoffs]. Anticipating these differences in search behavior and accounting for them while writing your content (using a good mix of keyword phrases) could produce positive results. Google Ads provides a handy Keyword Planner34 that helps you discover new keyword variations and see the approximate search volume for each keyword. Also, Google Search Console provides you with the top search queries your site appears for and the ones that led the most users to your site in the Performance Report35.
It’s rare to come across new SEO tips worth trying. And this post has tons of them. I know that’s true BECAUSE…I actually read it all the way to the end and downloaded the PDF. What makes these great is that so many are a multiple step little strategy, not just the one-off things to do that clients often stumble across and ask if they are truly good for SEO. But there are also some nice one-off tips that I can easily start using without ramping up a new project.
Make it as easy as possible for users to go from general content to the more specific content they want on your site. Add navigation pages when it makes sense and effectively work these into your internal link structure. Make sure all of the pages on your site are reachable through links, and that they don't require an internal "search" functionality to be found. Link to related pages, where appropriate, to allow users to discover similar content.
Another example when the “nofollow" attribute can come handy are widget links. If you are using a third party's widget to enrich the experience of your site and engage users, check if it contains any links that you did not intend to place on your site along with the widget. Some widgets may add links to your site which are not your editorial choice and contain anchor text that you as a webmaster may not control. If removing such unwanted links from the widget is not possible, you can always disable them with “nofollow" attribute. If you create a widget for functionality or content that you provide, make sure to include the nofollow on links in the default code snippet.
Hey Sammy, I would always advise against buying traffic, social followers, or anything else in that area. It mostly ends up being a vanity metric without business benefits. It’s always better to earn the traffic by creating a valuable, high-quality website and marketing it properly. When you do that, you attract the kind of visitors who are interested in what you have to offer, which is usually better for the bottom line.
There are many SEO tactics you can perform on each of your website pages to increase their rank in search engines and get more visitors. This includes producing high-quality content that your audience is searching for, and writing concise meta descriptions for your pages. The meta description appears below your URL in search results. Knowing what a page is about and what will result in a click makes users much more likely to do so. On-page SEO tactics such as these are free, but do take some time. For more help with on-page SEO, check out this blog post: Google Ranking Factors: On-Page vs Off-Page SEO.
Landing pages are another free source of traffic to your website. These are pages specific to your offers, such as for redeeming a discount code, downloading a free guide, or starting a free trial. They contain the details users need in order to move forward and convert, and focus on one specific call to action, making it more likely to happen. Because landing pages are so specific, you can get very targeted in your messaging, increasing the traffic coming to those pages.

I read The Art of War in college, written by the Chinese general Sun Tzu (author of the quote above). While his actual existence is debated, his work is often considered as brilliant military strategy and philosophy. Thus, The Art of War is often co-opted into business for obvious reasons. Throughout the book, you'll realize tactics and strategy are not interchangeable terms.
If you havent see it already, check out the links in shor's comment below - there are some great resources in there. In some cases you can also consider surveying your current audience or customers through email, on-site surveys or SurveyMonkey.  Be sure to ask for some profiling information that you can use for determining specific persona needs like age, sex, location, etc. (Probably best not to make it sound like a creepy text chat like I just did though...)  :)
Search engines attempt to rank results for a given search based on their relevance to the topic, and the quality and reliability a site is judged to have. Google, the world’s most popular search engine, uses an ever-evolving algorithm that aims to evaluate sites in the way that a human reader would. This means that a key part of SEO involves ensuring that the website is a unique and relevant resource for readers.
Not sure exactly why, perhaps I used a number too big and since my page is about classifieds, it probably seemed too much to browse through 1500 ads, I assume? Somewhat like you would post 800 tips for better ranking? Don’t know, will try to change things a bit and see how it goes, but you really gave me some new suggestions to go for with this article. Thanks again 🙂

You mentioned: "many times clients have already done this work.  Ask them for copies of their market research reports when you start a project.  It will save you a ton of time and effort!"  We do this with most of our clients, like you said we have found that around 75% of the have some kind of Market research done, that saves you a lot of time and helps setting up the right SEO Strategy. 
You have also mentioned Quuu for article sharing and driving traffic. I have been using Quuu for quite sometime now and I don’t think they’re worth it. While the content does get shared a lot, there are hardly any clicks to the site. Even the clicks that are there, average time is like 0.02 seconds compared to more than 2 minutes for other sources of traffic on my website. I have heard a few guys having a similar experience with Quuu and so, I thought should let you know.

Smartphone - In this document, "mobile" or “mobile devices" refers to smartphones, such as devices running Android, iPhone, or Windows Phone. Mobile browsers are similar to desktop browsers in that they can render a broad set of the HTML5 specification, although their screen size is smaller and in almost all cases their default orientation is vertical.

A variety of methods can increase the prominence of a webpage within the search results. Cross linking between pages of the same website to provide more links to important pages may improve its visibility.[48] Writing content that includes frequently searched keyword phrase, so as to be relevant to a wide variety of search queries will tend to increase traffic.[48] Updating content so as to keep search engines crawling back frequently can give additional weight to a site. Adding relevant keywords to a web page's metadata, including the title tag and meta description, will tend to improve the relevancy of a site's search listings, thus increasing traffic. URL canonicalization of web pages accessible via multiple URLs, using the canonical link element[49] or via 301 redirects can help make sure links to different versions of the URL all count towards the page's link popularity score.

Hi Brian! Very good and exactly what I was looking for. I have a problem though, we are creating the first video editing software that edits video WHILE FILMING. We are video geeks with a lot of experience, however we are trying to appeal to GoPro users and video tutorial makers but we have little knowledge in that field. Any suggestions on how we write about that if we have no idea about the space?

Wow Brian, You have solved my problem. A few days back I was looking for ways to increase traffic on my tech blog, I found this blog post by you while I was looking out for possible tricks to increase traffic. I must say that few of the tricks mentioned above really worked for me. For example, I updated a few old posts on my blog, I did try the broken link building technique and the last I did was to repost my content on Medium.
Great article as always. My wife is about to start a business about teaching (mainly) Mums how to film and edit little movies of their loved ones for posterity (www.lovethelittlethings.com launching soon). We have always struggled with thinking of and targeting relevant keywords because keywords like ‘videography’ and ‘family movies’ don’t really some up what she is about. Your article ties in with other learnings we have come across where we obviously need to reach out to right people and get them to share to get her product out there because purely focusing on keywords I don’t think will get us anywhere.
To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[47]
I first heard you talk about your techniques in Pat Flynn’s podcast. Must admit, I’ve been lurking a little ever since…not sure if I wanted to jump into these exercises or just dance around the edges. The clever and interesting angles you describe here took me all afternoon to get through and wrap my brain around. It’s a TON of information. I can’t believe this is free for us to devour! Thank you!! Talk about positioning yourself as THE expert! Deep bow.
However I feel that batching all the things influencers share , filter whats relevant from whats not… and ultimately niche it down to identify which exact type of content is hot in order to build our own is a bit fuzzy. Influencers share SO MUCH content on a daily basis – how do you exactly identify the topic base you’ll use build great content that is guaranteed to be shared?
Hi, my name is Dimitrios and I am responsible for Crave Culinaire’s digital marketing. I would like to drive more traffic to Crave’s blog. Since Crave Culinaire is the only catering company who provides molecular cuisine, I thought about craving a blog post about that. The influencers in this niche have great success in utilizing recipes on their blogs. I will share some recipes of Brian Roland, owner and head chef of Crave Culinaire.
In March 2006, KinderStart filed a lawsuit against Google over search engine rankings. KinderStart's website was removed from Google's index prior to the lawsuit, and the amount of traffic to the site dropped by 70%. On March 16, 2007, the United States District Court for the Northern District of California (San Jose Division) dismissed KinderStart's complaint without leave to amend, and partially granted Google's motion for Rule 11 sanctions against KinderStart's attorney, requiring him to pay part of Google's legal expenses.[70][71]
LinkedIn has become much more than a means of finding another job. The world’s largest professional social network is now a valuable publishing platform in its own right, which means you should be posting content to LinkedIn on a regular basis. Doing so can boost traffic to your site, as well as increase your profile within your industry – especially if you have a moderate to large following.
As an Internet marketing strategy, SEO considers how search engines work, the computer-programmed algorithms that dictate search engine behavior, what people search for, the actual search terms or keywords typed into search engines, and which search engines are preferred by their targeted audience. SEO is performed because a website will receive more visitors from a search engine the higher the website ranks in the search engine results page (SERP). These visitors can then be converted into customers.[4]
I definitely learned tons of new things from your post. This post is old, but I didn’t get the chance to read all of it earlier. I’m totally amazed that these things actually exist in the SEO field. What I liked most is Dead Links scenario on wikipedia, Flippa thing, Reddit keyword research, and at last, the facebook ad keyword research. Its like facebook is actually being trolled for providing us keywords thinking they are promoting ads.
Instead, in this instance, we started at wireframe stage, plopping in keywords and meta tags. Of course, the site really needed those things, and although it launched technically “optimized”, it wasn’t enough to provide a better product than our top competitor(s). A product that people want to visit, revisit, email to friends, share on social networks, and link to more than our competitors. It wasn’t even enough to move up in the rankings.
The goal of SEO is to get a web page high search engine ranking. The better a web page's search engine optimization, the higher a ranking it will achieve in search result listings. (Note that SEO is not the only factor that determines search engine page ranks.) This is especially critical because most people who use search engines only look at the first page or two of the search results, so for a page to get high traffic from a search engine, it has to be listed on those first two pages, and the higher the rank, the closer a page is to the number one listing, the better.  And whatever your web page's rank is, you want your website to be listed before your competitor's websites if your business is selling products or services over the internet.
By relying so much on factors such as keyword density which were exclusively within a webmaster's control, early search engines suffered from abuse and ranking manipulation. To provide better results to their users, search engines had to adapt to ensure their results pages showed the most relevant search results, rather than unrelated pages stuffed with numerous keywords by unscrupulous webmasters. This meant moving away from heavy reliance on term density to a more holistic process for scoring semantic signals.[13] Since the success and popularity of a search engine is determined by its ability to produce the most relevant results to any given search, poor quality or irrelevant search results could lead users to find other search sources. Search engines responded by developing more complex ranking algorithms, taking into account additional factors that were more difficult for webmasters to manipulate. In 2005, an annual conference, AIRWeb (Adversarial Information Retrieval on the Web), was created to bring together practitioners and researchers concerned with search engine optimization and related topics.[14]
You have also mentioned Quuu for article sharing and driving traffic. I have been using Quuu for quite sometime now and I don’t think they’re worth it. While the content does get shared a lot, there are hardly any clicks to the site. Even the clicks that are there, average time is like 0.02 seconds compared to more than 2 minutes for other sources of traffic on my website. I have heard a few guys having a similar experience with Quuu and so, I thought should let you know.
In a very crowded, noisy space – entrepreneurs and small business owners with a ton of “experts and influencers.” How do I get “above the noise?” I have built up a great brand and, I think, some great content based on a boatload of practical, real-life experience. I also have some products and services that I’m trying to sell, but I remain, “all dressed up, with no place to go.” Thoughts?
He is the co-founder of NP Digital. The Wall Street Journal calls him a top influencer on the web, Forbes says he is one of the top 10 marketers, and Entrepreneur Magazine says he created one of the 100 most brilliant companies. Neil is a New York Times bestselling author and was recognized as a top 100 entrepreneur under the age of 30 by President Obama and a top 100 entrepreneur under the age of 35 by the United Nations.
SEO techniques can be classified into two broad categories: techniques that search engine companies recommend as part of good design ("white hat"), and those techniques of which search engines do not approve ("black hat"). The search engines attempt to minimize the effect of the latter, among them spamdexing. Industry commentators have classified these methods, and the practitioners who employ them, as either white hat SEO, or black hat SEO.[50] White hats tend to produce results that last a long time, whereas black hats anticipate that their sites may eventually be banned either temporarily or permanently once the search engines discover what they are doing.[51]
To gain more customer engagement, the website must reach its visitors/customers efficiently. Obviously, you want the visitors to read your site content. Check the forms and click through on your Call To Actions (CTA’s) when they arrive on your web page. These features initiate user engagement in action, but it is essential to comprehend the in-depth analysis.
We now have a dedicated SEO strategist who, among other things, develops 90 day plans for our websites. 90 days isn't longterm planning, but at least we have a strategic objective for the quarter. He also works closely with our UX team to identify the target audience - the crew that does the persona research and focus groups prior to the wireframe stage.
There's often a top-down marketing strategy already baked before you get to pitch your SEO work, to which you may find opportunity on a battlefield where access is not granted. It's reckless to assume you can go into any established company and lob a strategy onto their laps, expecting them to follow it with disregard to their existing plans, politics, and red tape. Candidly, this may be the quickest way to get fired and show you're not aligned with the existing business goals.
What are the pain points? What things drive the members of this organization to drink? From the customer support to the higher-ups, there are things that knock the company down. How do they get back up? Why are the pains they're looking to work around? It may not be realistic to interview the whole company, but ideally you can get a representative to answer these.
You should build a website to benefit your users, and any optimization should be geared toward making the user experience better. One of those users is a search engine, which helps other users discover your content. Search Engine Optimization is about helping search engines understand and present content. Your site may be smaller or larger than our example site and offer vastly different content, but the optimization topics we discuss below should apply to sites of all sizes and types. We hope our guide gives you some fresh ideas on how to improve your website, and we'd love to hear your questions, feedback, and success stories in the Google Webmaster Help Forum1.
×