Here are 7 things you can do to prevent your web pages from getting de-indexed ever again.
Who controls the Internet? Don't be surprised if we say it's the major search engine (SE) companies. While it gets more baffling to keep up with their ever-changing algorithms or whatever criteria (as and when they like it) and more convenient to label them as notorious for indexing your web pages only to see them suddenly dropped altogether,
there are at least some things you can do to maintain your SE visibility in the long term.
1) Create a sitemap. This sitemap page contains all possible links to sections that make up your site's overall structure. Submit this page and you are submitting the whole site to be spidered, not just the home page.
2) Keep your site flat. As far as spider algorithm goes, it's not easy to register web pages more than 3 levels deep. Pages that are too deep are considered insignificant and submitting them can be detriment to your site's overall ranking status. Focus on building significant sections of your sites, create a sitemap pointing to these sections and let the spiders search for themselves.
3) Submit everywhere except the major SEs. There are hundreds of minor SEs you can submit your site to, but as your site gets more prolific, soon the spiders from Google, Yahoo, MSN, AllTheWeb, Hotbot, AOL, Teoma and Lycos will take notice and see it for themselves. Sites 'naturally' caught by spiders are seen as quality content, will rank highly and less likely to drop.
4) Avoid CGI-generated pages. These type of pages usually contain symbols like a question mark and ampersand to separate parameters. A typical URL would look like: http://www.sitename.com/cgi-bin/pagename.pl?cat=gardens&subcat=japanese
If you observe search results from time to time, most major engines refuse to index these pages. Wherever possible, create non-CGI pages. They give no clues that they are instantly and artificially 'churned' out which is exactly what spiders are looking for to censure. Pages employing dynamic technologies like PHP, XML etc. are just fine.
5) Cultivate your site. As long as you add in fresh content, spiders will take notice and come back just as often. Spiders seem to use a 'trained' schedule to learn that your site always stays up-to-date and come back to take a look. Update less often and the spidering frequency lessens.
6) Linking strategies. More than just one-to-one manual link exchanges, you will find every possible way to submit articles, give testimonials, comment in blogs, participate in forums etc. all for the sake of creating link-backs to your site by leaving your signature behind. To put it bluntly, hijack other people's web space to create backward links. If your pages don't show up, at least your keywords or domain name shown from other sites will.
7) Create a mini-web. Build mini-sites around an overall and similar theme and link them together interdependently. This pattern of incoming links here and there establishes and builds up each site's link popularity, and links from same-subject websites are now ranked higher in the search engines than they do if stand-alone.
