Thread: Loading times
View Single Post
Old April 26th, 2007, 12:45 AM  
Retired Administrator
Kiros's Forum Picture
Name: Ben
Join Date: May 22, 2004
Location: Albany, LA - USA
Age: 28
Gender: Male
Blog Entries: 17
Default Re: Loading times

Originally Posted by orchdork
define spider please
i understand everything else except this term
A spider is a robot like program that "crawls" web pages. It's job is to read and index all the words on a page. Some more complex spiders index the page's Meta data and gives the page keywords based on everything that it finds. The spiders are what allow search engines (like Google and Yahoo!) to show so many results. Without search engine spiders, there would have to be a person that views one page at a time, marking keywords that stick out on that page, and then that person would have to type in all the keywords they found in a large database and have it "linked" in some way to that webpage - meaning when a user searches for keywords, the page may pop up in the search results.

I'm not sure if that makes much sense, but I tried to explain it

Originally Posted by Heather View Post
Is there a way to only get a couple of the yahoo things here, instead of so many?
It's actually not so many. Yes, in the Currently Active section, Yahoo! spiders can take up a good bit of spots, but if you view the times that each spider was last active, you might see how Yahoo! wants their spiders to operate.

From what I've analyzed, Yahoo! has many, many spiders - all with different IP addresses. Instead of using one or two spiders to browse the entire website, Yahoo! uses one spider to track one page, and after all the index is done, the spider calls another spider to go to a link that it's found, and then that spider leaves to go to another site.

So most of the time, there have only been 3-5 Yahoo! spiders active within the past 5 minutes and only 1 active within the past 2 minutes. Google keeps it around that time too (1-2 minutes).

That's just what I've noticed anyway.

The site's load times may be coming from the shared host's server load, which would mean that other websites on the same host would be experiencing the same slowness. In that case, we really cannot help it. Though it might be something to do with vBulletin. We could test this by disabling some features, but like it hences, users would lose convenient functionality and statistics.

Personally, I have not been having a problem with the load times, but it could be that I'm never on when it's really bad.

Kiros || Ben

Happiness is not about being perfect.
It is about seeing beyond the imperfections.
Kiros is offline   Reply With Quote