nous information technology company solutions referance contact us
 297 Sydney Rd.,
 Brunswick Vic.3056
 ph.+ 61 (0) 3 9388 1265
 fax.+ 61 (0) 3 9387 1262

go top

Northern Light
Open Text

 Nous Information Technology


webcrawler General Info

Webcrawler uses an interface to search through all its databases for sites that contain all the words in the query. It then searches for the individual words in individual pages. The pages Webcrawler likes and considers apt are added to a Yahoo-type hierarchical directory system. Your site can be accessed through a search or through the directories. Webcrawler also has a random URL service, web reviews, as well as a "find similar site" service.


Click on the Add URL button to submit a URL to Webcrawler. The tool allows you to add 9 URLs at a time. It may take up to a fortnight for submissions to be added to the Webcrawler database.

Delete Method

To remove your site from the Webcrawler database, place it in the robots.txt file. The following visit the robot makes to your site, your site will be removed. The process can be accelerated through submitting your URL again, so the robot knows it has to visit your site.


To have your site return on more searches you should include meta tags and not use irrelevant words in your home page. Meta tags are good for graphical pages and for customizing your keywords to what is appropriate to your page. The constant repetition of keywords, or spamming, is not appreciated by Webcrawler. Offending pages are being removed. Once the spamming has been removed from a page, it can be resubmitted and added to the index again.

 Nous Information Technology
 Copyright © 1995-2013.
 All Rights Reserved.