DZone Snippets: http code

Seamlessly return a string from http or https feeds

m0wfo (Chris Mowforth) — Mon, 09 Jun 2008 20:15:15 GMT

// This class uses URI module to detect regular or secure links, and returns the response as a string, in my case to pass onto a feed parser like simple-rss. I'm working on a simple http authentication addon.



#beef.

require 'net/http'

require 'net/https'

require 'simple-rss'



class SeamlessFeed

  def initialize(url,user=nil,password=nil)

    @url = URI.parse(url)

  end



  def output

    if self.is_secure?

      http = Net::HTTP.new(@url.host, 443)

      http.use_ssl = true

      http.start do |http|

        request = Net::HTTP::Get.new(@url.path)

        @response = http.request(request)

        @response.value

      end

    else

      @response = Net::HTTP.get_response(@url) #Why can't they all be like this, eh?

    end

    return @response.body

  end



  def is_readable?

    feed = SimpleRSS.parse(self.output)

    return true unless feed.channel.items.size < 1

  rescue SimpleRSSError

    return false

  end



protected

  def is_secure?

    @url.scheme == 'https'

  end

end

Creating a bucket in Amazon S3 through an irb session

jrobertson (James Robertson) — Tue, 29 Apr 2008 17:20:25 GMT

1) Log into an irb session, and enter your S3 login details.



require 'rubygems'

require 'aws/s3'



  AWS::S3::Base.establish_connection!(

    :access_key_id     => 'REPLACE_ME',

    :secret_access_key => 'REPLACE_ME'

  )

output:
=> #, @secret_access_key="", @options={:server=>"s3.amazonaws.com", :access_key_id=>"", :port=>80, :secret_access_key=>"", :persistent=>true}, @access_key_id="19S45GYAGWK8DC2B8VG2">

2) Browse the existing buckets.
AWS::S3::Service.buckets
output:
=> [#"ogg.twitteraudio.com", "creation_date"=>Sat Apr 26 10:40:16 UTC 2008}>, #"t1000", "creation_date"=>Fri Apr 25 21:35:21 UTC 2008}>, #"t2000", "creation_date"=>Fri Apr 25 21:53:15 UTC 2008}>]

3) Browse the buckets in a programmatical way.
AWS::S3::Service.buckets.each {|b| puts b.name}
output:
ogg.twitteraudio.com
t1000
t2000

4) Add a new bucket called t3000.
AWS::S3::Bucket.create('t3000')
output:
=> true

5) Observe adding the bucket again doesn't cause an error.
AWS::S3::Bucket.create('t3000')
output:
=> true

6) View the buckets again.
AWS::S3::Service.buckets
output:
=> [#"ogg.twitteraudio.com", "creation_date"=>Sat Apr 26 10:40:16 UTC 2008}>, #"t1000", "creation_date"=>Fri Apr 25 21:35:21 UTC 2008}>, #"t2000", "creation_date"=>Fri Apr 25 21:53:15 UTC 2008}>]

Note: You would expect t3000 to be in there however it didn't appear possibly because of the bucket permissions.

7) Let's then look for bucket t3000.
t3000 = AWS::S3::Bucket.find('t3000')
output:
=> #nil, "name"=>"t3000", "marker"=>nil, "max_keys"=>1000, "is_truncated"=>false, "xmlns"=>"http://s3.amazonaws.com/doc/2006-03-01/"}>

8) Now that we've found the bucket let's upload a text file called works.txt.
file = "works.txt"
output:
=> "works.txt"
AWS::S3::S3Object.store(file, open(file), 't3000', :access => :public_read)
output:
=> #

9) Setting the file access to :public_read allows us to view the file from the http location http://t3000.s3.amazonaws.com/works.txt

References:
http://amazon.rubyforge.org/
upload_to_s3 - Ruby S3 upload client [dzone.com]

*update: 14:30 30 April 2008 *
I didn't use Bucket.objects(:reload) which is the reason why the bucket t3000 didn't show up with the statement Service.buckets

Reference: spatten design - Amazon S3, Ruby and Rails slides [spattendesign.com]

Redirect a URL with Ruby CGI

jrobertson (James Robertson) — Mon, 21 Apr 2008 12:25:30 GMT



#!/usr/bin/ruby



require 'cgi'



cgi = CGI.new

print cgi.header({'Status' => '302 Moved', 'location' =>  'http://www.wired.com'})



url = 'http://www.wired.com/'

print cgi.header({'status'=>'REDIRECT', 'Location'=>url})

References:
RE: cgi redirect [nagaokaut.ac.jp]
Ruby/CGI - assari [mokehehe.com]
HTTP/1.1: Status Code Definitions [w3.org]

https://hosted67.renlearn.com/77571/HomeConnect/Login.aspx

benthere (Patricia) — Tue, 11 Mar 2008 01:09:27 GMT

// description of your code here



// insert code here..

ProjectX client-side code

jrobertson (James Robertson) — Mon, 18 Feb 2008 18:49:42 GMT

This Ruby code uses a unified XML format to create a password record on a web server. It's intended to be run as a batch file which gets called from another Ruby application called maintain_projectx which gets called from a cronjob.

In this example the password is stored on the server not for authentication but simply to provide a reminder service in the event the user forgets it.



require 'net/http'

require 'rexml/document'

include REXML



class ProjectXClient

  attr :doc

  def initialize(raw_url)

    url = URI.escape(raw_url)

    xml_data = Net::HTTP.get_response(URI.parse(url)).body

    @doc = Document.new(xml_data)

  end

  

end



if __FILE__ == $0



  xml_project = <  

    

      

        

        

      

    

  

PROJECT

  

  pxc = ProjectXClient.new("http://yourdomain.com/p/projectx.cgi?xml_project=" + xml_project)

  doc = pxc.doc

  puts doc

    

end

output -what's returned from the server is an XML response containing a result. The result echos the method executed and the output from that method, which in this instance is the xml record node 'entry'.





  

    

      p6789c

      hotmail

Getting Started With WWW::Mechanize

jrobertson (James Robertson) — Tue, 12 Feb 2008 17:53:54 GMT

This Ruby code uses WWW:mechanize to act like a web browser.



 require 'rubygems'

 require 'mechanize'



 agent = WWW::Mechanize.new

 page = agent.get('http://google.com/')

Refer to the documentation at http://mechanize.rubyforge.org/mechanize/. Then gem install mechanize, and try running the code in an irb session.

output (extract):



=> # {url #}

 {meta}

 {title "Google"}

 {iframes}

 {frames}

 {links

  #   "Images"

   "http://images.google.com/imghp?hl=en&tab=wi">

  #   "Maps"

   "http://maps.google.com/maps?hl=en&tab=wl">

  #   "News"

   "http://news.google.com/nwshp?hl=en&tab=wn">

  #   "Shopping"

   "http://www.google.com/prdhp?hl=en&tab=wf">

  #   "Gmail"

   "http://mail.google.com/mail/?hl=en&tab=wm">

bScan - Simple Web Aplications Scanner

Black_H (Black_H) — Sun, 03 Feb 2008 11:51:05 GMT

// Web application scanner (ex: phpBB, myCMS, myBlog, mySite etc..) - Only in PHP !
// Find XSS, sql injection, remote file inclusion



#####################################################################################

#	Black_H  / Nooz -- 30:01:07 

#	Bl4ck.H<>gmail<>com

#



class BScan



#####################################################################################

#	Regex

#



@@space    = '([[:space:]]*)'



@@userdat  = '('

@@userdat += '(\$_SERVER\[([\'\"]*)HTTP_)|'

@@userdat += '(\$_GET)|'

@@userdat += '(\$_POST)|'

@@userdat += '(\$_COOKIE)|'

@@userdat += '(\$_REQUEST)|'

@@userdat += '(\$_FILES)|'

@@userdat += '(\$_ENV)|'

@@userdat += '(\$_HTTP_COOKIE_VARS)|'

@@userdat += '(\$_HTTP_ENV_VARS)|'

@@userdat += '(\$_HTTP_GET_VARS)|'

@@userdat += '(\$_HTTP_POST_FILES)|'

@@userdat += '(\$_HTTP_POST_VARS)|'

@@userdat += '(\$_HTTP_SERVER_VARS\[([\'\"]*)HTTP_)'

@@userdat += ')'



@@regex = Hash.new

@@regex = 

	{'TYPE' => 'vars overwrite','LEVEL' => '2','REGEX' => /extract#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'vars overwrite','LEVEL' => '2','REGEX' => /import_request_variables#{@@space}\((.*)\)/i},

	{'TYPE' => 'fopen vuln','LEVEL' => '3','REGEX' => /fopen#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'copy vuln','LEVEL' => '3','REGEX' => /copy#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'fwrite vuln','LEVEL' => '3','REGEX' => /fwrite#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'sql injection','LEVEL' => '2','REGEX' => /(mysql_query|mssql_query|mysqli_query)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'crlf injection','LEVEL' => '1','REGEX' => /mail#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'cross site scripting','LEVEL' => '1','REGEX' => /\<\?\=#{@@space}(.*)#{@@userdat}/i},

	{'TYPE' => 'cross site scripting','LEVEL' => '1','REGEX' => /(print|echo|print_r|var_dump)#{@@space}(|\(|\")(.*)#{@@userdat}/i},

	{'TYPE' => 'php code execution','LEVEL' => '3','REGEX' => /eval#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'php code execution','LEVEL' => '3','REGEX' => /file_put_contents#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'variable attribution', 'LEVEL' => '2','REGEX' => /(.*)\$#{@@userdat}(.*)/i},

	{'TYPE' => 'chmod affectation','LEVEL' => '1','REGEX' => /chmod#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'file disclosure','LEVEL' => '2','REGEX' => /(readfile|file_get_contents|file)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'file disclosure','LEVEL' => '2','REGEX' => /(show_source|highlight_file)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'bzopen vuln','LEVEL' => '2','REGEX' => /bzopen#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'file deletion','LEVEL' => '2','REGEX' => /(rmdir|unlink|delete)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'command execution','LEVEL' => '3','REGEX' => /(exec|system|passthru|shell_exec|proc_open|pcntl_exec)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'buffer overflow','LEVEL' => '3','REGEX' => /(confirm_phpdoc_compiled|mssql_pconnect|mssql_connect|crack_opendict|snmpget|ibase_connect)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'ip falsification','LEVEL' => '1','REGEX' => /(.*)(HTTP_CLIENT_IP|HTTP_X_FORWARDED_FOR|HTTP_PC_REMOTE_ADDR)(.*)/i},

	{'TYPE' => 'putenv vuln','LEVEL' => '2','REGEX' => /putenv#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'full path disclosure','LEVEL' => '1','REGEX' => /(htmlentities|htmlspecialchars)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'magic_quotes_gpc bypass','LEVEL' => '1','REGEX' => /(stripslashes|urldecode)#{@@space}\((.*)#{@@userdat}(.*)\)/i},

	{'TYPE' => 'file inclusion','LEVEL' => '3','REGEX' => /(include|include_once|require|require_once)#{@@space}(|\(|\")(.*)#{@@userdat}/i}



#####################################################################################

#	Main

#



  def initialize()



	################

	#	Usage



if (ARGV.length < 4)

puts  '

 ---------------------------------------------------------------------

|             Credits: Black_H                     |

|                 URL: Lemon-Inside.sup.fr                            |

|                Note: Premier code Ruby                              |

 ---------------------------------------------------------------------



 ---------------------------------------------------------------------

|   Usage:  scan.rb -d  -i                        |

|   Ex:  scan.rb -d ./ -i output.html                                 |

 ---------------------------------------------------------------------		

 '

 end

 

	################

	#	Options & Vars

	

	@@scan_alldir =  self.options('d')

	@@out_file =  self.options('i')

	

	@@ban = [".", "..", "scan.rb", @@out_file.to_s]



	@@scan_buffer = Array.new

	

	################

	#	Options Error ?

	

	if (@@scan_alldir != false and @@scan_alldir.empty? == false)

	self.dscan(@@scan_alldir)

	self.output(@@scan_buffer)

	@@scan_buffer = ''

	end





	



  end



#####################################################################################

#	Dir Scan 

#

  

  def dscan(dir)

      

	d = Dir.open(dir.to_s)

	d = d.sort - @@ban

	

      d.each { |fichier|



      case File.ftype(dir+fichier)

        when "directory"

          self.dscan(dir + fichier + "/")

        when "file"

		  puts  'Scan => ' + dir + fichier 

          self.fscan(dir + fichier)

      end



	  }

  end



#####################################################################################

#	File Scan 

#

  

  def fscan(file)



	fichier = File.readlines(file)

	i = 1



	fichier.each { |line|

						

		@@regex.each  { |info|

			

			test = (line  =~ info['REGEX']) 

		

				if (test) 

			

				@@scan_buffer += ['FILE' => file, 'LINE' => i.to_s, 'MATCH' => line, 'LEVEL' => info['LEVEL'], 'TYPE' => info['TYPE']]

				#	5 , 1 , 3 , 4 , 2

				next @@scan_buffer

				end

		}



	i += 1

  	} 

	

  end



#####################################################################################

#	Output buffer

#

  

  def output(buffer)

  

	@html_hmodel = ''

	@html_hmodel += 'BScan v1.0
'



	code = @html_hmodel

	

	buffer.each { |infos|

	

	keys = infos.keys

	code += "" + keys[1].to_s + ' : ' + infos["TYPE"] + '
'

	code += "" + keys[3].to_s + ' : ' + infos["LEVEL"] + '
'

	code += "" + keys[4].to_s + ' : ' + infos["FILE"] + '
'

	code += "" + keys[0].to_s + ' : ' + infos["LINE"] + '
'

	code += "" + keys[2].to_s + ' : ' + infos["MATCH"] + '
'

	



	}

		code += ""

		fhtml = File.open(@@out_file.to_s, "w")

		fhtml.write code

		code = ''



	

  end

#####################################################################################

#	Parse & Get Options

#

 

  def options(param)

  

	i = 0

		ARGV.each  { |valeur|

		

    		if (valeur == '-' + param.to_s)

				return ARGV[i+1]

			elseif (valeur != '-' + param.to_s)

				return false

			end

		i += 1

		}

		

	end

  

end



scan = BScan.new

Ruby equivalent of a simple Wget

jrobertson (James Robertson) — Mon, 15 Oct 2007 11:20:43 GMT

Reads the file contents from a URL. I used this code to work around the problem with the Ruby RSS reader which couldn't read the RSS file from digg.com. The reason being that the website would not allow files to be downloaded without supplying the User-Agent string.



require 'open-uri'

puts open('http://digg.com/rss/index.xml',

     'User-Agent' => 'Ruby-Wget').read

Ruby remote file checker

sikelianos (Zeke Sikelianos) — Fri, 12 Oct 2007 00:17:27 GMT



require 'rubygems'

require 'open-uri'

require 'net/http'



def remote_file_exists?(url)

  url = URI.parse(url)

  Net::HTTP.start(url.host, url.port) do |http|

    return http.head(url.request_uri).code == "200"

  end

end

Resolving a tinyURL to destination URL

mixdev (Arun Vijayan) — Sun, 30 Sep 2007 08:05:34 GMT

Resolving a tinyURL to its original destination URL in a fastest way.



    < ?php



    // request like this http:///tinyurl.php?c=



    $num = $_GET['c'];



    if($fp = fsockopen ("tinyurl.com", 80, $errno, $errstr, 30))

    {

    if ($fp) {

    fputs ($fp, "HEAD /$num HTTP/1.0\r\nHost: tinyurl.com\r\n\r\n");

    while (!feof($fp)) {$headers .= fgets ($fp,128);}

    fclose ($fp);

    }

    $arr1=explode("Location:",$headers);

    $arr=explode("\n",trim($arr1[1]));

    echo trim($arr[0]);

    }

    ?>

http://www.webforth.com/2007/07/resolving-tinyurls-to-the-desination-url