DZone Snippets: Lordrich's Code Snippets

securing the /home directory

lordrich () — Tue, 16 May 2006 14:46:58 GMT

I'm still working on getting this one perfect.



chmod 701 /home/*

chmod 705 /home/*/public_html

chmod 604 /home/*/public_html/*.*

slashdot protection

lordrich () — Sun, 23 Apr 2006 14:43:02 GMT

Replace aaa.bbb.ccc.ddd with your own IP (so you always get to your site).
Replace www.yourdomain.com with your actual domain.

Just don’t mess with the user_agent and query_string lines. Those ensure that the Coral servers themselves can retrieve your page when they need to.

(Shamelessly stolen from http://ottodestruct.com/diggprotectionrules.txt)



RewriteEngine on

RewriteCond %{REMOTE_ADDR} !^aaa.bbb.ccc.ddd$

RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx

RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$

RewriteCond %{HTTP_REFERER} ^http://(www\.)?digg\.com [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?slashdot\.org [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?slashdot\.com [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?fark\.com [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?somethingawful\.com [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?kuro5hin\.org [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?engadget\.com [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?boingboing\.net [OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?del\.icio\.us

RewriteRule ^(.*)$ http://www.yourdomain.com.nyud.net:8080$1 [R,L]

Banning bad bots

lordrich () — Tue, 18 Apr 2006 15:49:13 GMT

The following code is the contents of /banme/index.php. This file is linked to from my main website but invisible to web browsers and disallowed in robots.txt. Therefore, only bad bots will ever follow this link and when they do so they will get banned in .htaccess and their ip address will be emailed to webmaster@example.com.



$i = getenv('REMOTE_ADDR');

$handle = fopen("../.htaccess", "a");

fwrite($handle, "Deny from $i\n");

fclose($handle);

echo "You've just got $i banned from this domain.  You are a very bad person.";

mail("webmaster@example.com", "Banned IP", "Deny from $i");

?>

Googleit WordPress Plugin

lordrich () — Thu, 13 Apr 2006 16:20:36 GMT

// description of your code here



/*

Plugin Name: Googleit

Plugin URI: http://lordrich.com/archives/2005/04/02/just-google-it/

Description: Link to google for the current title.  Usage: google_it();

Version: 0.1

Author: Richard Kirkcaldy

Author URI: http://lordrich.com

*/



function google_it(){

	$google = 'Google It';

	echo $google;

	}

?>

Cerberus recent requesters

lordrich () — Tue, 11 Apr 2006 20:52:48 GMT

Display a list of email addresses for people who have filed tickets in the past 30 days.



$database = 'cerberus';

$username = 'cerberus';

$password = 'cerberus';

$hostname = 'localhost';



$sql = "

SELECT distinct address.address_address

FROM ticket, requestor, address

WHERE  DATE_SUB(CURDATE(),INTERVAL 30 DAY) <= last_update_date

AND ticket_status='resolved'

AND ticket.ticket_id=requestor.ticket_id

AND requestor.suppress='0'

AND requestor.address_id=address.address_id";



mysql_connect($hostname,$username,$password) or die('error connecting to the database');

@mysql_select_db($database) or die('unable to select database');

$result = mysql_query($sql) or die(mysql_error());

$num=mysql_numrows($result);

mysql_close();



$i = 0;

while($i < $num) {

	$address_address = mysql_result($result,$i,"address.address_address");

	echo "$address_address
";

	$i++;

}

?>

Random Flickr Photo

lordrich () — Tue, 11 Apr 2006 20:50:20 GMT

// description of your code here





    // Displays a single random photo from recent flickr photos with a given tag.

    // Original code stolen from many sources including http://www.thebishop.net/geodog/archives/2004/09/29/fun_hacking_with_flickr_making_a_homemade_flickr_tag_badge_with_magpierss.html and http://prwdot.org/archives/002468.html





    // USER CONFIGURATION SECTION



    // MagpieRSS Configuration

    // This is an example based on my system;

    // you will need to customize it for your system

    // and your preferences. You can remove it entirely

    // if you have done it elsewhere

    // refer to http://magpierss.sourceforge.net/

    require_once('/var/www/bradford/magpierss-0.61/rss_fetch.inc');

    error_reporting(E_ERROR);

    define(MAGPIE_CACHE_ON, true);

    define(MAGPIE_CACHE_DIR, '/var/www/bradford/magpie_cache');

    define(MAGPIE_CACHE_AGE, 300);

    define(MAGPIE_CACHE_FRESH_ONLY, false);

    define(MAGPIE_DETECT_ENCODING, true);

    define(MAGPIE_DEBUG, 0);

    define(MAGPIE_FETCH_TIME_OUT, 15);

    define(MAGPIE_USE_GZIP, true);    



    // flickr configuration

    // How many photos you want to display

    $num_photos = 2; // for some reason it doesn't like 1

    $tag = 'Bradford';

    // URL for the flickr feed you want to use

    $flickr_feed_url ='http://www.flickr.com/services/feeds/photos_public.gne?tags='.$tag.'&format=rss_200';

?>



    // Fetch the feed

    $flickr = fetch_rss( $flickr_feed_url );

    if ($flickr) {

        $flickr_title = $flickr->channel["title"];

        $flickr_link = $flickr->channel["link"];

?>







    // Pick some random photos

        $random_photos = array_rand($flickr->items,$num_photos);

        foreach ( $random_photos as $random_photo ) {

            $description = explode("\n\n",$flickr->items[$random_photo]["description"]);

?>



    



		  die();?> // ok we've got our first photo - lets exit



        }

      } else {

?>





An error occurred in the MagpieRSS parser:







    } 

?>

CPP SMTP Client

lordrich () — Tue, 11 Apr 2006 20:43:40 GMT

An SMTP client in CPP, apparently for a Linux machine. It was found in my old source code folder and so may not be fully working.





#include 

#include 

#include 

#include 

#include 



#define HELO "HELO\n"

#define DATA "DATA\n"

#define QUIT "QUIT\n"



FILE *fin;

int sock;

struct sockaddr_in server;

struct hostent *hp, *gethostbyname();

char buf[BUFSIZ+1];

int len;

char *host_id;

char *from_id;

char *to_id;

char *file_id;

char wkstr[100];



/*=====Send a string to the socket=====*/



send_socket(char *s)

{

	write(sock,s,strlen(s));

	write(1,s,strlen(s));

}



/*=====Read a string from the socket=====*/



read_socket()

{

	len = read(sock,buf,BUFSIZ);

	write(1,buf,len);

}



/*=====MAIN=====*/

int main(int argc, char* argv[])

{



if(argc != 5)

{

 printf("USAGE: %s    \n\n", argv[0]);

 exit(1);

}



host_id=argv[1];

from_id=argv[2];

to_id=argv[3];

file_id=argv[4];



/*=====Create Socket=====*/

sock = socket(AF_INET, SOCK_STREAM, 0);

if (sock==-1)

{

 perror("opening stream socket");

 exit(1);

}



/*=====Verify host=====*/

server.sin_family = AF_INET;

hp = gethostbyname(host_id);

if (hp==(struct hostent *) 0)

{

 fprintf(stderr, "%s: unknown host\n", host_id);

 exit(2);

}



/*=====Connect to port 25 on remote host=====*/

printf ("hostent %s\n", hp->h_addr_list[0]);

memcpy((char *) &server.sin_addr, (char *) hp->h_addr, hp->h_length);



server.sin_port=htons(25); /* SMTP PORT */



if (connect(sock, (struct sockaddr *) &server, sizeof server)==-1)

{

 perror("connecting stream socket");

 exit(1);

}



/*=====Write some data then read some =====*/



read_socket(); /* SMTP Server logon string */



send_socket(HELO); /* introduce ourselves */

read_socket(); /*Read reply */



send_socket("MAIL from: "); /* Mail from us */

send_socket(from_id);

send_socket("\n");

read_socket(); /* Sender OK */



send_socket("RCPT To: "); /*Mail to*/

send_socket(to_id);

send_socket("\n");

read_socket(); /*Recipient OK*/



send_socket(DATA);/*body to follow*/

read_socket(); /*ok to send */



fin=fopen(file_id, "r"); /* open file */

while(1)

{

 if(fgets(wkstr, 100, fin)==NULL) break; /* exit on EOF */

 send_socket(wkstr);

}

fclose(fin); /* close file */



send_socket(fin); /*send file*/

send_socket(".\n");



read_socket(); /* OK*/

send_socket(QUIT); /* quit */

read_socket(); /* log off */



/*=====Close socket and finish=====*/

close(sock);

exit(0);

}

Another perl crawler

lordrich () — Tue, 11 Apr 2006 20:41:05 GMT

Again found in my old source folder, it may not fully work.

This Perl script reads in the existing links from links.dat into the array @bigarray. It then loops through the array reading in each link and appending the new links it finds to links.dat. If the script were run in a loop it would add every single web address it can find to links.dat.



#!/usr/bin/perl 

use IO::Socket; 

use URI; 

 

open(LINKS, "<< links.dat"); 

@bigarray = (); 

while () { 

        chomp; 

        push(@bigarray, $_); 

} 

close(LINKS); 

 

foreach $uri (@bigarray) { 

        ($domain = URI->new($uri)->authority) =~ s/^www\.//i; 

        $socket = IO::Socket::INET->new(PeerAddr 

                                => $domain, 

                                PeerPort => 80, 

                                Proto => 'tcp', 

                                Type => SOCK_STREAM) 

        or die "Couldn't connect"; 

        print $socket "GET / HTTP/1.0\n\n"; 

        #$page = <$socket>; 

        open(LINKS, ">> links.dat"); 

        while (defined($line = <$socket>)) { 

                $line =~ m{href="(.*?)"}ig; 

                print LINKS "$1"; 

            } 

        close(LINKS); 

        close($socket); 

}

PHP Web Crawler

lordrich () — Tue, 11 Apr 2006 20:39:36 GMT

Example output:

-bash-2.05b$ php asp.php
http://www.example.com
http://www.rfc-editor.org/rfc/rfc2606.txt
No links.

-bash-2.05b$ cat links.dat
http://www.example.com
http://www.rfc-editor.org/rfc/rfc2606.txt



$datafile = "links.dat"; // file to keep the list of links in

$regex = "/<\s*a\s+[^>]*href\s*=\s*[\"']?([^\"' >]+)[\"' >]/isU";  // regex to search for hrefs



$handle = fopen($datafile, "r"); // open the data file

$buffer = fgets($handle, 4096);

$oldlinks[] = $buffer; // read the first link into an array

while (!feof($handle)) {

	$buffer = fgets($handle, 4096);

	array_push($oldlinks,$buffer); // read the rest of the links into an array

}

fclose($handle); // close the data file



foreach($oldlinks as $value) { // for every link in the array

	print $value; // print it out

	$remote = fopen(trim($value), "r") or die(); //open it or fail nicely

	while (!feof($remote)) {

		$html = fread($remote, 8192); // read in the remote page

	}

	fclose($remote); // close it

	if (preg_match_all($regex, $html, $links)) { // if we find new links

		$local = fopen($datafile, "a+"); // open the data file

		foreach($links[1] as $value) { // for every new link

			$value.="\n"; // append a new line

			if(!in_array($value,$oldlinks)) { // if we haven't seen it before (nb - case sensitive)

				print($value); // print it out

				fwrite($local, $value); // and write it to file

			}

		}

		fclose($local); // close the data file

	}

	else {

		print("No links."); // we didn't find any links in the new file

	}

}

?>

Ruby web crawler

lordrich () — Tue, 11 Apr 2006 20:36:29 GMT

NB. Again, this script was found in my old source code folder, it may not be fully working.

This Ruby script reads in a list of links from links.dat, it then picks out the ones it can easily spider and gets a list of URLs from each page listed in links.dat. Every new URL it finds will be added to newlinks.dat for later spidering by another bot running along side this one.



require 'socket' 

links = File.open("links.dat") 

while links.gets do 

        #domain = ($_ =~ /http:\/\/.*\.([0-9a-zA-Z\-]+\.com|net|org)/); 

        if %r{http://([^/]+)/([^/]+)}i =~ $_ 

                domain,path = $1, $2 

        end 

        if proto="http" 

                begin 

                        t = TCPSocket.new(domain, 'www') 

                rescue 

                        puts "error: #{$!}" 

                else 

                        t.print "GET /"+path+" HTTP/1.0\n\n" 

                        answer = t.gets(nil) 

                        t.close 

                end 

 

                if %r{]*>([^<]*)}i =~ answer 

                        proto, url, text = $1, $2, $3 

                end 

 

                print proto+"://"+url+"\n" 

                old = File.open("newlinks.dat") 

                new = File.open("links.dat.tmp", File::WRONLY|File::TRUNC|File::CREAT) 

                while old.gets do 

                        if $_ != proto+"://"+url 

                                new.print $_ 

                        end 

                end 

                new.print proto+"://"+url 

                old.close 

                new.close 

                File.rename("newlinks.dat", "links.dat.orig") 

                File.rename("links.dat.tmp", "newlinks.dat") 

        end 

end 

links.close