Mechanize / Hpricot / Scraping setup
1 2 require 'rubygems' 3 require 'cgi' 4 require 'open-uri' 5 require 'hpricot' 6 require 'mechanize' 7 8 agent = WWW::Mechanize.new 9 doc = Hpricot(agent.get(the_url).parser.to_s)
Never been to DZone Snippets before?
Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world
Ruby: Strip html tags from a string
1 2 str = "<html>This and <b>that</b> and <br />and <span class='something'>the other</span>?<html>" 3 puts str.gsub(/<\/?[^>]*>/, "") 4
Ruby: Count number of words in a string
1 2 class String 3 def count_words 4 n = 0 5 scan(/\b\S+\b/) { n += 1} 6 n 7 end 8 end
Scraping Google Search Results with Hpricot
// snagged from http://g-module.rubyforge.org/
1 2 require 'rubygems' 3 require 'cgi' 4 require 'open-uri' 5 require 'hpricot' 6 7 q = %w{meine kleine suchanfrage}.map { |w| CGI.escape(w) }.join("+") 8 url = "http://www.google.com/search?q=#{q}" 9 doc = Hpricot(open(url).read) 10 lucky_url = (doc/"div[@class='g'] a").first["href"] 11 system 'open #{lucky_url}'
Snazzy url fixer using Ruby's super method
Within the body of a method, a call to super acts just like a call to that original method, except that the search for a method body starts in the superclass of the object that was found to contain the original method.
1 2 def url=(addr) 3 super (addr.blank? || addr.starts_with?('http')) ? addr : "http://#{addr}" 4 end
