To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually  ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[47]

This philosophy is beautiful in its simplicity, and it serves to correct the “more, more, more” mentality of link building. We only want links from relevant sources. Often, this means that in order to scale our link-building efforts beyond the obvious tactics, we need to create something that deserves links. You have links where it makes sense for you to have links. Simple.
Before you say it – no, true guest blogging isn’t dead, despite what you may have heard. Securing a guest post on a reputable site can increase blog traffic to your website and help build your brand into the bargain. Be warned, though – standards for guest blogging have changed radically during the past eighteen months, and spammy tactics could result in stiff penalties. Proceed with caution.
Black hat SEO attempts to improve rankings in ways that are disapproved of by the search engines, or involve deception. One black hat technique uses hidden text, either as text colored similar to the background, in an invisible div, or positioned off screen. Another method gives a different page depending on whether the page is being requested by a human visitor or a search engine, a technique known as cloaking. Another category sometimes used is grey hat SEO. This is in between black hat and white hat approaches, where the methods employed avoid the site being penalized but do not act in producing the best content for users. Grey hat SEO is entirely focused on improving search engine rankings.

Specifics: Be as specific as you can with your recommendations. For example if you’re suggesting partnering with meal home delivery sites, find out which ones are going to provide the most relevant info, at what cost if possible, and what the ideal partnership would look like for content and SEO purposes. Even provide contact information if you can.


Like the hundreds of people already, I thought this was an amazing post. You have a great way of breaking things down into ways that the average reader will be able to understand and make actionable. I think this is a great resource for our readers, so I included it in my monthly roundup of the best SEO, social media, and content marketing articles. https://www.northcutt.com/blog/2014/02/january-resource-round-up-the-best-of-seo-social-media-and-content-marketing/

Yesterday I was re doing our process for ideas and alltop was a part of it. Now I have also known it was a bit spammy (some of my grey sites are featured ) but now it seems way too bad. You have places like new York times next to random adsense blog x. Guy kawasaki needs to really start giving some sort of influence ranking or at least culling the total crap ones.


Dedicate some time to brainstorm all the different ways you can attract inbound links to your website. Start small –- maybe share your links with other local businesses in exchange for links to their sites. Write a few blog posts and share them on Twitter, Facebook, Google+, and LinkedIn. Consider approaching other bloggers for guest blogging opportunities through which you can link back to your website.


This information hits the mark. “If you want your content to go viral, write content that influencers in your niche will want to share.” I love the information about share triggers too. I’m wondering, though, if you could share your insights on how influencers manage to build such vast followings. At some point, they had to start without the support of other influencers. It would seem that they found a way to take their passion directly to a “ready” world. Excellent insights. Thanks for sharing.
If your social media profiles contain a link to your website, then you’ve turned your engagement into another channel for website traffic. Just be sure to engage moderately and in a sincere way, and avoid including links to your website in your comments—lest you appear spammy and hurt your online and business reputation. Increased traffic should not be the goal of your engagement, but rather a secondary result.
This one is so obvious, we’re going to look at it first. Paid search, social media advertising and display advertising (try our Smart Ads Creator!) are all excellent ways of attracting visitors, building your brand and getting your site in front of people. Adjust your paid strategies to suit your goals – do you just want more traffic, or are you looking to increase conversions, too? Each paid channel has its pros and cons, so think carefully about your objectives before you reach for your credit card.
Just a suggestion, but maybe you could write an article about generating traffic to a brand new blog. As you know, when you start out, you have only a couple posts and very little credibility with other bloggers, also the search engines will take considerable time to be of any benefit initially. Would be interesting to know how Brian Dean approaches that dilemma!

Holy Engagement! This was an awesome post, full of great info… and then I realized that 3/4 of the actual page was comments… which is even better for shares, SEO and overall engagement. I was lucky enough to attend an event where Neil Patel was giving some great blogging training and a lot of what you covered was there. https://www.thatbloggingthing.com/69-blogging-secrets-i-stole-from-neil-patel/ The simple fact that you comment back is awesome.
I first heard you talk about your techniques in Pat Flynn’s podcast. Must admit, I’ve been lurking a little ever since…not sure if I wanted to jump into these exercises or just dance around the edges. The clever and interesting angles you describe here took me all afternoon to get through and wrap my brain around. It’s a TON of information. I can’t believe this is free for us to devour! Thank you!! Talk about positioning yourself as THE expert! Deep bow.
When referring to the homepage, a trailing slash after the hostname is optional since it leads to the same content ("https://example.com/" is the same as "https://example.com"). For the path and filename, a trailing slash would be seen as a different URL (signaling either a file or a directory), for example, "https://example.com/fish" is not the same as "https://example.com/fish/".
Like the hundreds of people already, I thought this was an amazing post. You have a great way of breaking things down into ways that the average reader will be able to understand and make actionable. I think this is a great resource for our readers, so I included it in my monthly roundup of the best SEO, social media, and content marketing articles. https://www.northcutt.com/blog/2014/02/january-resource-round-up-the-best-of-seo-social-media-and-content-marketing/

Hey Mischelle, thanks for the input! It’s true, SEO is definitely a long game. You need to lay the foundation and keep improving your site, publish new content and promote what you already have. However, if you keep at it, it can pay off nicely over time. And you are right, picking the right keywords is one of the foundations for SEO success. Thanks for commenting!
When Googlebot crawls a page, it should see the page the same way an average user does15. For optimal rendering and indexing, always allow Googlebot access to the JavaScript, CSS, and image files used by your website. If your site's robots.txt file disallows crawling of these assets, it directly harms how well our algorithms render and index your content. This can result in suboptimal rankings.
If your social media profiles contain a link to your website, then you’ve turned your engagement into another channel for website traffic. Just be sure to engage moderately and in a sincere way, and avoid including links to your website in your comments—lest you appear spammy and hurt your online and business reputation. Increased traffic should not be the goal of your engagement, but rather a secondary result.
On a dating niche site I took the ‘ego-bait’ post one step further and had sexy girls perform a dance and strip to revel the names of the major bloggers in my niche written on their bodies. As you can imagine it got a lot of attention from the big players in my niche and my audience and is a little more creative for getting links, shares and traffic.
The days when internet browsing was done exclusively on desktop PCs are long gone. Today, more people than ever before are using mobile devices to access the web, and if you force your visitors to pinch and scroll their way around your site, you’re basically telling them to go elsewhere. Ensure that your website is accessible and comfortably viewable across a range of devices, including smaller smartphones.
I’d add one thing to number 5: Writing good copy is crucial not just for your Title/snippet, but for your whole page, especially your landing page. You want people to stay on your page for a while and (hopefully) even navigate to other pages you have. Google looks at bounce rate and where they go after they hit your page. Learning to write good copy can not only increase conversion (if you’re selling something) but make your content more impactful and engaging. There are free books at most libraries or online to help.
Commenting on blog posts written by industry experts with lots of followers can bring your website a lot of traffic. When you post a comment (most) blogs allow you to leave a link back to your site for other readers to check out – as long as you leave an insightful comment you WILL get traffic from your blog comments. Make sure you comment as quickly as possible when new blog posts go up. The higher in the comments you are the more clicks you’ll get. I have Google Reader setup to alert me when new blog posts are made on the industry blogs I follow and I comment immediately to lock in my first place spot.
Keep resources crawlable. Blocking page resources can give Google an incomplete picture of your website. This often happens when your robots.txt file is blocking access to some or all of your page resources. If Googlebot doesn't have access to a page's resources, such as CSS, JavaScript, or images, we may not detect that it's built to display and work well on a mobile browser. In other words, we may not detect that the page is "mobile-friendly," and therefore not properly serve it to mobile searchers.
All sites have a home or "root" page, which is usually the most frequented page on the site and the starting place of navigation for many visitors. Unless your site has only a handful of pages, you should think about how visitors will go from a general page (your root page) to a page containing more specific content. Do you have enough pages around a specific topic area that it would make sense to create a page describing these related pages (for example, root page -> related topic listing -> specific topic)? Do you have hundreds of different products that need to be classified under multiple category and subcategory pages?
So I’m not good at English. Well I’m running a new vacation rental and travel website in French. But it seems that in francophone area people are reluctant to implement backlinks. I do need links to rank because I have strong competitors. So I’ve decided to ask for those links to Anglophone website owner. Since my content is in French, I thought I could ask for links to pages with solely touristic spots photos. What do you thinks of that?
Hack #1: Hook readers in from the beginning. People have low attention spans. If you don’t have a compelling “hook” at the beginning of your blogs, people will click off in seconds. You can hook them in by teasing the benefits of the article (see the intro to this article for example!), telling a story, or stating a common problem that your audience faces.
So, here, in this article, we will give you the best tips following which you can increase visitor’s engagement. But, there is one more option that you can also opt for the best WordPress development companies that could help you to turn your visitors into customers and can enhance more engagement. The rest is up to you. Let’s proceed without wasting any further time!!
The best way to avoid this is proactively asking the right questions. Ask about resource support. Ask about historic roadblocks. Ask to be introduced to other players who otherwise hide behind an email here and there. Ask about the company's temperature regarding a bigger SEO strategy vs. short, quick-hit campaigns. Don't be your own biggest obstacle — I've never heard of anyone getting angry about over-communication unless it paralyzes progress.
Think about the words that a user might search for to find a piece of your content. Users who know a lot about the topic might use different keywords in their search queries than someone who is new to the topic. For example, a long-time football fan might search for [fifa], an acronym for the Fédération Internationale de Football Association, while a new fan might use a more general query like [football playoffs]. Anticipating these differences in search behavior and accounting for them while writing your content (using a good mix of keyword phrases) could produce positive results. Google Ads provides a handy Keyword Planner34 that helps you discover new keyword variations and see the approximate search volume for each keyword. Also, Google Search Console provides you with the top search queries your site appears for and the ones that led the most users to your site in the Performance Report35.
Commenting on blog posts written by industry experts with lots of followers can bring your website a lot of traffic. When you post a comment (most) blogs allow you to leave a link back to your site for other readers to check out – as long as you leave an insightful comment you WILL get traffic from your blog comments. Make sure you comment as quickly as possible when new blog posts go up. The higher in the comments you are the more clicks you’ll get. I have Google Reader setup to alert me when new blog posts are made on the industry blogs I follow and I comment immediately to lock in my first place spot.
Thanks Brian. I’ve had a “a-ha” moment thanks to you! Great advice. I knew that backlinks would improve the organic SEO rankings to our client-targeted landing pages but I never knew it was through getting influencers to backlink blogs. I always just assumed it was great content that users wanted to share with others. It was driving me mad why people love my content but never share enough. Now I know!
×