Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.
Sorry for the long comment, I just am really happy to see that after all those years of struggle you finally made a break through and you definitely deserve it bro. I’ve had my own struggles as well and just reading this got me a little emotional because I know what it feels like to never wanting to give up on your dreams and always having faith that one day your time will come. It’s all a matter of patience and learning from failures until you get enough experience to become someone who can generate traffic and bring value to readers to sustain long term relationships.

Black hat SEO involves techniques such as paying to post links to a website on link farms, stuffing the metadata with nonrelated keywords, and using text that is invisible to readers to attract search engines. These and many other black hat SEO tactics may boost traffic, but search engines frown on the use of such measures. Search engines may punish sites that employ these methods by reducing their page rank or delisting them from search results.
Just a suggestion, but maybe you could write an article about generating traffic to a brand new blog. As you know, when you start out, you have only a couple posts and very little credibility with other bloggers, also the search engines will take considerable time to be of any benefit initially. Would be interesting to know how Brian Dean approaches that dilemma!

Btw, I was always under the impression that digg and delicious were dying but I’m really mistaken. Your(and Jason’s) thinking is foolproof though. If these guys are already curating content, there’s no reason they wouldn’t want to do more of just that! Seo has become a lot of chasing and pestering…it’s good of you to remind us that there are people out there just waiting to share stuff, too.:)
Sending out regular newsletters and promoting offers through email is a great way to stay in touch with your customers and can also help to get traffic to your website. Provide useful information and links to pages on your website where they can learn more, such as through blog posts and landing pages for particular offers. Just make sure that you don`t continually bombard your readers with emails or your customers will either disengage with, delete, or unsubscribe from your emails.
×