New InSites into the FAST Search Engine . . . an interview with Stephen Baker of Fast, Part 2

(Continued from Part 1)

Next subject . . .

What about free submissions to FAST? Are they a thing of the
past?

According to Baker, "Free submit will continue to operate. There
will be no effect on pages (already) in the index. The inclusion
service is simply a guarantee that your pages will be indexed
regularly and are guaranteed a reservation in the FAST index.
The trick with free submit is the amount of spam we receive
through that channel. It makes it difficult to keep up with all
of the submissions. PartnerSite provides an economic incentive to
NOT submit spam."

How quickly does FAST index free submissions?

Mr. Baker answered, "Usually within 2-6 weeks. However, 95% of
what comes through free submission is spam, and that's what
causes the bottleneck. Also, there's no guarantee on the refresh
rate of those sites that we pick up through free submit.
Unfortunately, it's the 5% of the submitters that are spammers
that ruin it for the rest of us. The extent that people go
through to spam the index is truly amazing."

Is there a penalty for submitting your site through their free
Add URL?

Baker replied, "Not really. Freshness does effect rank, but only
marginally. I have never seen freshness boost ranking more than 1
or 2 spaces."

What does FAST consider spam? As stated earlier, FAST estimates
that an amazing 95% of the submissions through their free Add URL
page are spam.

Baker further explained, "Unfortunately this is the case. We
believe there are approximately 30 million crawlable servers
globally, two-thirds of which have been blacklisted as spam
servers."

Whew! Think about it: 20 MILLION crawlable servers globally are
blacklisted as spam servers. If this is really true, it explains
why the engines have collectively gone to such great lengths in
their efforts to curtail spam.

At the Dallas Search Engine Strategies Conference in November,
FAST introduced their new spam policies. Baker explained that
according to FAST, spam comes in three different categories:

* Page Spam, which consists of any measures to boost ranking,
such as link farms, etc.
* Spam stuffing, such as keyword stuffing, invisible text, etc.
* Offensive content, which is not so much spam, but is something
that we detect and flag as such.

You can access and read Fast's entire spam policy at:
http://www.alltheweb.com/info/spampolicy.html

And, if you're so inclined, you can report spam to FAST by
emailing spam@fastsearch.com.

The ultimate fear of a search engine marketer: your site has been
banned. What recourse is there for a site that gets on Fast's
spam list and becomes banished from the engine? Is this forever?

Baker replied, "This really is handled on a case by case basis.
We have worked with sites that have contacted us and informed us
that they have cleaned up their act. Obviously, this is very time
consuming, so the combination of PartnerSite, the spam
guidelines, and not having your site hosted on a banned server
should enable a site to get in the index through a variety of
means if they think they have cleaned up their act."

Translated: If your site is hosted on one of the 20 million
servers tagged for spamming, you'd best move it to a different
server before you initiate the kiss-and-makeup ritual with FAST.

Baker further explained, "What really matters is the server where
the submission is coming from. So many servers have been
completely blacklisted due to the proliferation of spam. I
suggest 'know thy neighbor.' The crawler will take care of the
rest."

This again underscores the importance of your site having its own
unique IP address to insure against problems caused by an unruly
site sharing the same IP.

How does FAST feel about cloaking? In Dallas, Mr. Baker and I
had a long discussion about cloaking and how the FAST engine
feels about it. We even served on a panel where a question came
up about cloaking and about responsible cloaking guidelines.

As I mentioned to him, from our position as SEO's, we see the
issues from opposite perspectives. Of course, we don't have a
front row seat to the parade of spam but, even so, our viewpoint
is dictated by that of legitimate businesses trying to ethically
compete in arenas that are often very competitive and sometimes
dominated by nefarious position jockeying. Therefore, on the
issue of cloaking, I pointed out instances where it's the only
tool available that prevents our work from being stolen. My
thinking is there's no harm done provided that a person follows
all of the engine's guidelines and does not spam in any way.

Baker's response: "Unfortunately, the 20% of sites that use
redirects maliciously ruin it for the rest. Redirects and cloaks
have become such a hassle that we can't afford to risk indexing
them. We do work through PartnerSite IV customers to index
cloaks. However, they are sent through a rigorous spam-detection
process."

When I asked if we could show we're not trying to hide anything -
- perhaps form a partnership aimed at quality control within the
framework of cloaking? His reply, "I agree with that. That is the
best option."

The rest of the our dialogue went like this...

Robin: Does it have to be PartnerSite IV -- some small companies
couldn't quite afford that option.

Baker: Currently, it would be through PartnerSite IV. However, we
do plan to extend that offering to the other PartnerSite versions
in the future.

Robin: Since redirects are frowned upon, what should you do if
you've moved your entire site, for example, and you need to
direct traffic to the new domain?

Baker: It is always best to use META REFRESH tags rather than
JavaScript to implement redirects if you want search engines to
know about it. Most search engines do not fully parse JavaScripts
to find out what they actually do. Using a META REFRESH tag tells
us in a much better way what you are trying to achieve so that we
can do whatever we think is most sensible with the page.

How to score at the top of FAST . . .

Now let's get down to the nitty gritty. What does it take to
achieve a top ranked page with FAST?

Here are some tips from the FAST rep himself: Baker suggests,

"Good content that is unique, not general, always has the
'authoritative' effect and people begin to link to that site.
This achieves good scores in the two areas we care most about:

1) Static Rank, which is link analysis, and
2) Dynamic Rank, which is keywords and content."

Also . . .

"Get a handful of authoritative sites linking to you. Link
popularity plays a large factor in determining rank. If your
site is diverse, with multiple focuses, make sure the pages are
well written, keywords are appropriately placed, and have a
handful of authoritative sites linking to the site."

Baker added, "The trick, in my experience, is to get a handful of
really good sites to link to a site that is non-thematic."

How important is link popularity with FAST? As with most
engines, link popularity is an extremely important factor in
determining relevancy.

Baker explains as follows, "Be concerned with links in, not links
out. Link score is part of the relevancy calculation and has a
formulaic impact on ranking. However, if a site has no link
score, ranking is negatively impacted."

I asked him, "Is link popularity is based on the entire site, or
on individual pages? In other words, if one page has a high link
popularity, will all the pages of a site be boosted, or only that
one page?"

His reply, "Just that one page."

So, if your site has no links pointing to it, you'll likely rank
poorly but the site can still be indexed provided the FAST spider
can find it.

Robin: Does FAST make allowances for brand new sites?

Baker: "Not currently, but we are beginning to work with some of
the new domains that are popping up."

Robin: Do links from pages on your own site count toward building
link popularity?

Baker: "No . . . that would be too easy." ...he said with a
smile.

Robin: How does FAST deal with asp and database-generated pages?

Baker: "As long as they are not forms and they are linked to, we
can crawl them. Of course, the PartnerSite service has the
ability to index dynamic pages as well."

Conclusive remarks . . .

As with most engines FAST sees spam as public enemy number one.
They've identified two-thirds of the servers in the world as spam
servers. The best way to gain entrance into the FAST index is to
make certain that your site does not fall into any of their spam
categories.

We suggest you take a few minutes to review their spam policy.

For guaranteed indexing and respidering every 48 hours, give
their paid inclusion a test. The primary benefit being that you
can experiment with keyword factors that boost relevancy and see
the results of your tests in 48 hours or less.

Remember, paid inclusion guarantees your space in the directory
and provides you the opportunity to experiment endlessly with
relevancy factors within your page for a whole year.

Focus your attention on gaining high quality incoming links.
Doing so will increase what FAST calls your "link score" (aka,
link popularity). Remember also that sites with no incoming links
are actually negatively impacted.

All other factors being equal, the freshest (most recent) page
wins by an ever-so-slight boost in relevancy. There are many ways
to maintain site "freshness," such as using server side includes.

For shades of things to come, keep an eye on alltheweb.com --
we'll be watching to see how query analysis develops as a
determiner of page relevancy in the near future.

Copyright 2002 Robin Nobles. All rights reserved.