Log in

View Full Version : sitemap generator


Teenbay
February 23rd, 2011, 10:26 PM
What do you think is the best sitemap generator for large sites?

lengthy_brochure
February 23rd, 2011, 11:45 PM
I have deleted the contents of this post

TheMatrix
February 26th, 2011, 02:15 AM
i could probably write you a perl script - but not now as i'm using my psp.

Teenbay
February 26th, 2011, 01:27 PM
Thanks for the advice. I need one that I can install on my windows machine. I have installed 4 or 5 and they all seem to have problems with bigger sites

TheMatrix
February 26th, 2011, 09:09 PM
Right, so basically it is just a search without any keywords. Are you using a CMS? Loose HTML files? A MySQL database with articles, etc.?

For the latter, I would use this Perl script:

#!C:\Perl\bin\perl
#Change the above path to your perl interpreter. Try ActiveState for a Windows one.

#Use these libraries. If you don't have this library try the url listed next to each "use" statement
use CGI; #http://search.cpan.org/CPAN/authors/id/L/LD/LDS/CGI.pm-3.49.tar.gz
use DBI; #http://search.cpan.org/~tag/POE-Component-Pool-DBI-0.01/DBI.pm
use DBD::mysql; #http://search.cpan.org/src/ADAMK/DBD-SQLite-1.31/lib/DBD/SQLite.pm

my $q = new CGI; #Use the CGI module to print headers, etc.
print $q->header( -type => "text/html" );

# These are all of the MySQL database configuration variables
my $database = "content"; #Change to the database name
my $table = "articles"; #Change to the table name
my $username = "bot"; #Change to the MySQL username
my $password = "bot_pwd"; #Change to the password of the user
my $server = "example.com"; #Change to your server name. DO NOT include the "http://". It won't work.
my $port = 3306; #Change to port name. This is almost always 3306, so try it with that first.
my $field1 = "title"; #The first field in your database

#Now connect to the database!
my $dsn = "dbi:mysql:$database:$server:3306"; #This is the needed information to connect
my $connect = DBI->connect($dsn, $username, $password); #This sends the user data
my $query = "SELECT * FROM $table WHERE $field1 LIKE '%%'"; #The SQL query
my $query_handle = $connect->prepare($query); #Ready the query handle
$query_handle->execute() or die "Error with executing"; #Execute the query
$query_handle->bind_columns(undef, \$title, \$date, \$authour, \$url); #All of the fields in the database. Add or remove these as necessary; but remember to put a \ before each variable here.

#Generate the HTML output
print $q->start_html( -title => "Sitemap of $server", -bgcolor => "#ffffff" ),
$q->h1( "The sitemap of $server"),
$q->hr,
$q->p( "This is the sitemap of $server."),
"<ul>";
#Now print all of the articles
my $num = $query_handle->rows; #Get the number of rows in the database
my $i = 0; #Initialise a counter
while(($i < $num) && ($query_handle->fetch())) {
print "<li><a href=\"$url\">$title</a></li>";
print "<p>By $authour on $date</p>";
$i++;
}
if($i == 0) { #Print an error for no results
print "<li><p>There were no results to generate. Sorry.</p></li>";
}
print "</ul>",
$q->hr,
$q->end_html;
#And we're done!!
#Just 50 lines of code(including comments)

It may differ for you; this is purely an example.
Try it!

lengthy_brochure
February 26th, 2011, 09:32 PM
I have deleted the contents of this post

TheMatrix
February 27th, 2011, 02:01 AM
and you may get a perl interpreter here:
http://strawberryperl.com/ or http://www.activestate.com/activeperl/downloads
(this is what runs the above script)
^^this.

those are probably the best windows perl interpreters available.

Teenbay
February 27th, 2011, 09:57 AM
I really appreciate it. Most of the sitemap generators out there are terrible. Some locked up. I will give this a try.
Thank you again.