http://snippets.dzone.com/posts Sun, 18 May 2008 12:17:34 GMT DZone Snippets: mechanize code http://snippets.dzone.com/posts/show/5134 This Ruby code uses WWW:mechanize to act like a web browser. <br /> <br /><code> <br /> require 'rubygems' <br /> require 'mechanize' <br /> <br /> agent = WWW::Mechanize.new <br /> page = agent.get('http://google.com/') <br /></code> <br /> <br />Refer to the documentation at http://mechanize.rubyforge.org/mechanize/. Then gem install mechanize, and try running the code in an irb session. <br /> <br />output (extract): <br /><code> <br />=> #<WWW::Mechanize::Page <br /> {url #<URI::HTTP:0xfdbbbb286 URL:http://www.google.com/>} <br /> {meta} <br /> {title "Google"} <br /> {iframes} <br /> {frames} <br /> {links <br /> #<WWW::Mechanize::Page::Link <br /> "Images" <br /> "http://images.google.com/imghp?hl=en&tab=wi"> <br /> #<WWW::Mechanize::Page::Link <br /> "Maps" <br /> "http://maps.google.com/maps?hl=en&tab=wl"> <br /> #<WWW::Mechanize::Page::Link <br /> "News" <br /> "http://news.google.com/nwshp?hl=en&tab=wn"> <br /> #<WWW::Mechanize::Page::Link <br /> "Shopping" <br /> "http://www.google.com/prdhp?hl=en&tab=wf"> <br /> #<WWW::Mechanize::Page::Link <br /> "Gmail" <br /> "http://mail.google.com/mail/?hl=en&tab=wm"> <br /></code> Tue, 12 Feb 2008 17:53:54 GMT http://snippets.dzone.com/posts/show/5134 jrobertson (James Robertson) http://snippets.dzone.com/posts/show/4548 I bank at Washington Mutual, and I wanted to see when I was spending too much, or get automated updates of what I was spending money on each day and what my balance was. However, it's really difficult to do and WaMu won't let you use an API or something helpful like that, so I wrote my own little screen-scraper to do it. <br /> <br />This requires WWW::Mechanize, but it works quite well. <br /> <br /><code> <br />require 'rubygems' <br />require 'mechanize' <br /> <br />agent = WWW::Mechanize.new <br /> <br /># login to the account <br />puts "Login" <br />page = agent.get('https://online.wamu.com/IdentityManagement/Logon.aspx') <br />login_form = page.forms.with.name("frmLogin").first <br />login_form.txtUserID = '<UNAME>' <br />login_form.password = '<PWD>' <br />page = agent.submit(login_form, login_form.buttons.first) <br /> <br /># click on the download button <br />puts "Download" <br />dl_link = page.links.with.text(/WAMU FREE CHECKING/) <br />page = agent.click(dl_link) <br /> <br />line_items = [] <br /> <br />trs = (page/"table#_ctl0_depositTransactionsGrid/tr") <br />trs.shift <br />trs.each do |tr| <br /> tds = (tr/:td) <br /> <br /> dtd = tds[1].inner_html <br /> js_call = dtd.match(/showDDATransactionDetails\('(.*)'\);/)[1] <br /> js_fields = js_call.split("','") <br /> <br /> item = {} <br /> item['type'] = js_fields[1] <br /> item['descr'] = js_fields[3] <br /> item['amount'] = js_fields[4] <br /> item['tranid'] = js_fields[6] <br /> # if !tranid = ovedraft charge / bank fee <br /> <br /> item['date'] = tds[0].inner_html <br /> item['debit'] = tds[3].inner_html <br /> item['credit'] = tds[4].inner_html <br /> item['balance'] = tds[5].inner_html <br /> <br /> line_items << item <br />end <br /> <br />pp line_items.first <br /></code> <br /> <br /> Sun, 16 Sep 2007 21:18:32 GMT http://snippets.dzone.com/posts/show/4548 schacon (Scott Chacon)