Never been to DZone Snippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

« Newer Snippets
Older Snippets »
Showing 1-10 of 10 total  RSS 

Retrieve your Gmail messages as an XML feed

This Ruby example shows how to retrieve your most recent email as an atom XML feed from the Google Apps website.

require 'rubygems'
require 'httpclient'

url = "https://mail.google.com/a/yourwebsite.com/feed/atom"
client = HTTPClient.new
client.debug_dev = STDOUT if $DEBUG
client.set_auth(url, 'yourname@yourwebsite.com', 'yourpassword')
resp = client.get(url)


Note: You might require to gem install httpclient.

Download all xkcd.com comics

This goes through all the first 329 (you might want to change this) pages, downloading the comic strips.

#!/bin/bash

for i in `seq 1 329`
do
	wget http://xkcd.com/$i/
	wget `grep http://imgs.xkcd.com/comics/ index.html | head -1 | cut -d\" -f2`
	rm index.html
done

Example file download servlet

// description of your code here

  /**
     *  Sends a file to the ServletResponse output stream.  Typically
     *  you want the browser to receive a different name than the
     *  name the file has been saved in your local database, since
     *  your local names need to be unique.
     *
     *  @param req The request
     *  @param resp The response
     *  @param filename The name of the file you want to download.
     *  @param original_filename The name the browser should receive.
     */
    private void doDownload( HttpServletRequest req, HttpServletResponse resp,
                             String filename, String original_filename )
        throws IOException
    {
        File                f        = new File(filename);
        int                 length   = 0;
        ServletOutputStream op       = resp.getOutputStream();
        ServletContext      context  = getServletConfig().getServletContext();
        String              mimetype = context.getMimeType( filename );

        //
        //  Set the response and go!
        //
        //
        resp.setContentType( (mimetype != null) ? mimetype : "application/octet-stream" );
        resp.setContentLength( (int)f.length() );
        resp.setHeader( "Content-Disposition", "attachment; filename=\"" + original_filename + "\"" );

        //
        //  Stream to the requester.
        //
        byte[] bbuf = new byte[BUFSIZE];
        DataInputStream in = new DataInputStream(new FileInputStream(f));

        while ((in != null) && ((length = in.read(bbuf)) != -1))
        {
            op.write(bbuf,0,length);
        }

        in.close();
        op.flush();
        op.close();
    }

cURL Download

// function to dowload a $remote file and store as $local file, using curl

function curl_download($remote, $local)	{
	$cp = curl_init($remote);
	$fp = fopen($local, "w");
	
	curl_setopt($cp, CURLOPT_FILE, $fp);
	curl_setopt($cp, CURLOPT_HEADER, 0);
	
	curl_exec($cp);
	curl_close($cp);
	fclose($fp);	

}

downloader.py

#!/usr/bin/env python

__author__="Andrew Pennebaker (andrew.pennebaker@gmail.com)"
__date__="3 Nov 2005 - 14 Feb 2007"
__copyright__="Copyright 2006 2007 Andrew Pennebaker"
__license__="GPL"
__version__="0.5"
__URL__="http://snippets.dzone.com/posts/show/2887"

from urllib import urlopen

import os

import sys
from getopt import getopt

def getURLName(url):
	directory=os.curdir

	name="%s%s%s" % (
		directory,
		os.sep,
		url.split("/")[-1]
	)

	return name

def createDownload(url, proxy=None):
	instream=urlopen(url, None, proxy)

	filename=instream.info().getheader("Content-Length")
	if filename==None:
		filename="temp"

	return (instream, filename)

def download(instream, outstream):
	outstream.write(instream.read())

	outstream.close()

def usage():
	print "Usage: %s [options] <url1 url2 url3 ...>" % (sys.argv[0])
	print "\n--httpproxy <proxy>"
	print "--ftpproxy <proxy>"
	print "--gopherproxy <proxy>"
	print "\n--help (usage)"

	sys.exit()

def main():
	systemArgs=sys.argv[1:] # ignore program name

	urls=[]
	proxies={}

	optlist=[]
	args=[]

	try:
		optlist, args=getopt(systemArgs, None, ["url=", "httpproxy=", "ftpproxy=", "gopherproxy=", "help"])
	except Exception, e:
		usage()

	if len(args)<1:
		usage()

	for option, value in optlist:
		if option=="--help":
			usage()

		elif option=="--httpproxy":
			proxies["http"]=value
		elif option=="--ftpproxy":
			proxies["ftp"]=value
		elif options=="--gopherproxy":
			proxies["gopher"]=value

	urls=args

	for url in urls:
		try:
			outfile=open(getURLName(url), "wb")
			fileName=outfile.name.split(os.sep)[-1]

			url, length=createDownload(url, proxies)
			if not length:
				length="?"

			print "Downloading %s (%s bytes) ..." % (url.url, length)
			if length!="?":
				length=float(length)
			bytesRead=0.0

			for line in url:
				bytesRead+=len(line)

				if length!="?":
					print "%s: %.02f/%.02f kb (%d%%)" % (
						fileName,
						bytesRead/1024.0,
						length/1024.0,
						100*bytesRead/length
					)

				outfile.write(line)

			url.close()
			outfile.close()
			print "Done"

		except Exception, e:
			print "Error downloading %s: %s" % (url, e)

if __name__=="__main__":
	main()

How to download files with Ruby

// Note: the "b" in "wb" in the open method may not be needed in
// non-Windows environments. In Windows it indicates that you're writing
// binary information. You probably won't need it for downloading straight text
// or html either.

require 'net/http'

Net::HTTP.start("static.flickr.com") { |http|
  resp = http.get("/92/218926700_ecedc5fef7_o.jpg")
  open("fun.jpg", "wb") { |file|
    file.write(resp.body)
   }
}
puts "Yay!!"

Download recent flickr pictures with ruby and the flickr api

// To make this work, you need to get your own flickr api key.
// Get one here: http://www.flickr.com/services/api/misc.api_keys.html
// Other than that, just plug and chug and have fun!
// The "b" in "wb" in the second open method may not be necessary in
// non-windows environments.

require 'open-uri'
require 'rexml/document'

open('http://www.flickr.com/services/rest/?method=flickr.photos.getRecent&api_key=YOUR_KEY_HERE') { |f|
    doc = REXML::Document.new f.read
    i = 0
    doc.elements.each("rsp/photos/photo") { |element|
        if i < 3
            open("images/file" << i.to_s << ".jpg", "wb").
                write(open("http://static.flickr.com/" << \
                element.attributes["server"] << "/" << \
                element.attributes["id"] << "_" << \
                element.attributes["secret"] << "_o.jpg").read)
        else
            break
        end
        i = i + 1
    }
}

puts "Done!"

Force output to download as a file in Rails

response.headers['Content-Type'] = 'text/csv' # I've also seen this for CSV files: 'text/csv; charset=iso-8859-1; header=present'
response.headers['Content-Disposition'] = 'attachment; filename=thefile.csv'

Download the pys60 tutorial

Many wonderful examples by Jürgen Scheible.
I omit the zip,exe,pdf to save site's bandwidth.
C:\>wget -kmnp -l3 -Rzip,exe,pdf http://www.mobilenin.com/pys60/menu.htm

You can just browse the site.

Mirror a site with wget

I create a batch file (w.bat) containing just
wget -krmnp %1


Then I call
C:\>w [what-ever-url]
« Newer Snippets
Older Snippets »
Showing 1-10 of 10 total  RSS