SBI! Help & Strategies - Submit-Spider-List Report

- - - - - Contents - - - - -
Search Engines

Sitemap

Submit-Spider-List Report
ListChecker

Keyword Searches Report
Keyword Ranking Report

SBI!'s WorldSubmitter updates your sitemap file and pings (informs) Google, Yahoo!, Live Search (formerly MSN) and Ask whenever you make a change to your site (add, edit or delete a page).

You can rebuild a page 100 times per day, if you like (i.e., each time you edit the page). WorldSubmitter ignores frequent editing work, only allowing a rebuilt page to be "submitted" once via the sitemap file (it pings the SEs only once per day).

When it adds a page to the sitemap, it enters the date of submission into the Date Submitted column of the Submit-Spider-List Report. If you edit the page, that date is entered as the date of submission. The date will change every time you edit the page.

(You can even use WorldSubmitter, through Priority Build It!, to adjust the sitemap file to inform the Search Engines about which pages are more important. Click here for more information.)

If you add a META NOROBOTS tag to your page, you will see META for the Date Submitted.

Note: If you added this page earlier than 2008, and haven't edited it since then, you'll see the original submission date for Live, not the sitemap submission date.

So much for submitting. What about spiders and listing?

SpiderWatcher checks your site's log files regularly, looking to see which Search Engine (SE) spiders have visited, and which of your pages they "took back to their mother ships." (The spider-watching updates this database every six hours.)

When SpiderWatcher spots a bot in your log files, it enters that date into the Date Spidered column of the Submit-Spider-List Report for that page.

The dates reported for submissions and spidering are always the most recent dates for each. For example, if a spider revisits at a later date, SpiderWatcher will report that new date, overwriting the previous one.

Once the page has been spidered, it's ListChecker's turn...

ListChecker

After a spider visit, the next step is to see if the page has been listed (indexed) by the SEs. You initiate this process by clicking on the Update It! button at the top of the Keyword Ranking Report column in SE HQ, which will also update the Keyword Ranking Report at the same time.

ListChecker then checks with the engines that have spidered the page to verify if the page is listed in their database. Once the process has completed, you'll receive an e-mail to notify you that you can now check your updated Submit-Spider-List report, where the Date Listed has been updated for each page. (You'll also receive an e-mail advising you of the status of the Keyword Ranking Report.)

Important Note

This listing and ranking process is very resource-intensive for SBI!, so you can only run the update once every three weeks.

If you try to run the reports more often than every three weeks, you'll receive an alert telling you when you can return and run the reports again.

There are four possible date options for the Date Listed column. The fourth occurs only if your site has more than 300 pages...

If this page has never been listed, ListChecker reports Not Yet for the Date Listed. If the page has a sitemap priority of at least 0.7, you'll see a Not Yet link for the Date Listed (click this link to review the help now, or click the link in the report when you see it).
If this page was indexed since the last report, ListChecker gives the date of this report for the Date Listed. After that, ListChecker will return whenever you run the report and verify that the page is still listed. If it is, the date remains the original Date Listed.
If this page is no longer listed, Date Listed gives a "Dropped" link (click this link to review the help now, or click the link in the report when you see it).
If your site has more than 300 pages, and if this page is not in the 300 pages returned by the SE, the Date Listed shows ">300." This will tell you that this page may be listed at that engine. It just was not in the first 300 pages returned in the update. This page will also have ">>>" in the Keyword Ranking Report for that engine.

The date reported for listing remains the date the page was first listed, until it's reported as Dropped. The Dropped date will remain the same through subsequent updates. If the page is later listed again, that new date will appear (and continue to appear) as the Date Listed.