DZone Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

Michael has posted 2 posts at DZone. View Full User Profile

Get web file with cache

02.05.2014
| 1688 views |
  • submit to reddit
require 'open-uri'
require 'tmpdir'
require 'fileutils'
require 'digest'

def wget(url)
  cachedir = File.join( Dir.tmpdir, 'httpcache' )
  filename = File.join( cachedir, Digest::MD5.hexdigest(url) )
  
  FileUtils.mkdir_p(cachedir) if !File.exists?(cachedir)
  if File.exist?(filename)
    body = File.binread(filename)
  else
    body = open(url).read
    File.open(filename, "wb") do |saved_file|
      saved_file.write(body)
    end
  end

  body
end

Gets a file from the web over http.  A cache is maintained in the tmp directory.  Cached entries are permanent, so a file will never be downloaded twice even if it has changed on the server.

If you want a dynamic cache, you might try open-uri-cache .