Mechanize / Hpricot / Scraping setup
require 'rubygems' require 'cgi' require 'open-uri' require 'hpricot' require 'mechanize' agent = WWW::Mechanize.new doc = Hpricot(agent.get(the_url).parser.to_s)
Never been to DZone Snippets before?
Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world
Scraping Google Search Results with Hpricot
// snagged from http://g-module.rubyforge.org/
require 'rubygems' require 'cgi' require 'open-uri' require 'hpricot' q = %w{meine kleine suchanfrage}.map { |w| CGI.escape(w) }.join("+") url = "http://www.google.com/search?q=#{q}" doc = Hpricot(open(url).read) lucky_url = (doc/"div[@class='g'] a").first["href"] system 'open #{lucky_url}'
