DZone Snippets: rss code

Seamlessly return a string from http or https feeds

m0wfo (Chris Mowforth) — Mon, 09 Jun 2008 20:15:15 GMT

// This class uses URI module to detect regular or secure links, and returns the response as a string, in my case to pass onto a feed parser like simple-rss. I'm working on a simple http authentication addon.



#beef.

require 'net/http'

require 'net/https'

require 'simple-rss'



class SeamlessFeed

  def initialize(url,user=nil,password=nil)

    @url = URI.parse(url)

  end



  def output

    if self.is_secure?

      http = Net::HTTP.new(@url.host, 443)

      http.use_ssl = true

      http.start do |http|

        request = Net::HTTP::Get.new(@url.path)

        @response = http.request(request)

        @response.value

      end

    else

      @response = Net::HTTP.get_response(@url) #Why can't they all be like this, eh?

    end

    return @response.body

  end



  def is_readable?

    feed = SimpleRSS.parse(self.output)

    return true unless feed.channel.items.size < 1

  rescue SimpleRSSError

    return false

  end



protected

  def is_secure?

    @url.scheme == 'https'

  end

end

PHPアップローダー RSS機能

roytam1 (Roy Tam) — Fri, 15 Feb 2008 12:53:23 GMT

PHPアップローダー RSS機能



/**** PHPアップローダー RSS機能 ***

 * by ◆RTphpfqies

 ****/



  $title	= 'PHPアップローダー';

  $logfile	= "./log/upup.log";	//ログファイル名（変更する事）

  $updir 	= "./log/";	//アップ用ディレクトリ(変更する場合は、35.48.50行も変更）

  $prefix	= '';		//接頭語（up001.txt,up002.jpgならup）

  $page_def	= 20;		//RSSの表示行数





function FormatByte($size){//バイトのフォーマット（B→kB）

  if($size == 0)			$format = "";

  else if($size <= 1024)		$format = $size."B";

  else if($size <= (1024*1024))		$format = sprintf ("%dKB",($size/1024));

  else if($size <= (10*1024*1024))	$format = sprintf ("%.2fMB",($size/(1024*1024)));

  else					$format = $size."B";



  return $format;

}



error_reporting(0);

$base_dir = 'http://'.$_SERVER['HTTP_HOST'].str_replace('\\','/', dirname($_SERVER['SCRIPT_NAME'])); if(substr($base_dir,-1) != '/') $base_dir .= '/'; // URL

	header('Content-Type: text/xml;charset=shift-jis');

echo'





'.$title.'

'.$base_dir.'

'.$title.'

ja

PHPアップローダー RSS機能';



$lines = file($logfile);

for($i = 0; $i < $page_def; $i++){

  if($lines[$i]=="") continue;

  list($id,$ext,$com,$host,$now,$size,$mtype,$pas,$orig,)=explode("\t",$lines[$i]);

  $fsize = FormatByte($size);



  $filename = $prefix.$id.".$ext";

  $target = $updir.$filename;



  echo '

	'.$filename.' - '.$orig.' ('.$fsize.')

	'.$base_dir.$target.'

	

	'.$com.'

	]]>

	

	'.$now.'



';

  }

echo '

';

Simple Ruby Snarl RSS alerter

zuwiki (Mike Zink) — Sat, 02 Feb 2008 23:02:17 GMT

This simple Ruby program will send a Snarl alert when a new post is added to a feed. Assumes your Snarl command line tool is named "snarl.exe" and in your PATH.



require 'rss/1.0'

require 'rss/2.0'

require 'open-uri'



def snarl(title, msg)

  # This part actually sends the notification.

  # Change "snarl" to your snarl command line tool if you have problems

  system "snarl /M \"#{title}\" \"#{msg}\""

end



source = ARGV[0]

content = ''

open(source) { |s| content = s.read }

rss = RSS::Parser.parse(content, false)

puts rss.channel.title

puts rss.channel.link

puts rss.channel.description



last_item = nil



while true

  puts "Checking for updates..."

  open(source) { |s| content = s.read }

  rss = RSS::Parser.parse(content, false)

  if last_item == rss.items[0]

    sleep 5*60*1000

  else

    last_item = rss.items[0]

    snarl last_item.title + " (#{rss.channel.title})", last_item.description.gsub(/<\/?[^>]*>/, "")

    sleep 10*60*1000

  end

end

RSS output of a Prologue post

als (alex) — Wed, 30 Jan 2008 00:25:57 GMT

// This is how Prologue outputs a post





		

		http://localhost/wordpress/?p=3

		http://localhost/wordpress/?p=3#comments

		Tue, 29 Jan 2008 23:54:15 +0000



		admin

		

		



		



		http://localhost/wordpress/?p=3

		

			

                Pretty cool, here’s the Twitta!
]]>

			http://localhost/wordpress/?feed=rss2&p=3

Transforming XML into RSS

jrobertson (James Robertson) — Sun, 27 Jan 2008 16:21:37 GMT

Using the previous code snippet which prepared an XML file it can now be transformed to RSS using the XSL below.

file: gang2rss.xsl







	



	



			

		

		The Gang

		http://newsgang.net/audio/

		The Gang podcast



  	en



	



		

		



	



	



	

		<xsl:value-of select="title"/>

		http://newsgang.net

		

		http://newsgang.net

To transform the xsl file from the command-line you would type:

xsltproc gang2rss.xsl thegang_rss.xml

output







  

    The Gang

    http://newsgang.net/audio/

    The Gang podcast

    en

    

      TheGangXII-II

      http://newsgang.net/gangitem/id=6501

      Jan 25

      http://newsgang.net/gangitem/id=6501&from=audio

    

    

      TheGangXII-I

      http://newsgang.net/gangitem/id=6499

      Jan 25

      http://newsgang.net/gangitem/id=6499&from=audio

    

    

      NewsGangLive01.24.08

      http://newsgang.net/gangitem/id=6445

      Jan 24

      http://newsgang.net/gangitem/id=6445&from=audio

    

    

      NewsGangLiveII

      http://newsgang.net/gangitem/id=6377

      Jan 23

      http://newsgang.net/gangitem/id=6377&from=audio

Note: The enclosure url in this example does not reference the media file directly.

see also: http://en.wikipedia.org/wiki/RSS_(file_format)

Scrape an XHTML document using Ruby

jrobertson (James Robertson) — Sun, 27 Jan 2008 14:09:24 GMT

A simple Ruby script to scrape an XHTML file with the selected content being saved to an xml file ready for transformation into an RSS feed. This example uses the XHTML file from http://newsgang.net/audio/ which is then saved locally as 'thegang.xml'.



#!/usr/bin/ruby

# file: thegang.rb



require 'rexml/document'

include REXML



class TheGang

  def initialize()

  end

  

  def rssify()

    file = File.new('thegang.xml','r')

    doc = Document.new(file)

    rss_doc = Document.new

    root = Element.new('rss')

    rss_doc.add_element(root)

    

    doc.root.elements.each("body/div/ul/li/h2/a") do |node|    

      o_rssitem = Element.new('item')

      o_li = node.parent.parent

      

      o_rsstitle = Element.new('title')

      o_rsstitle.text = node.text.gsub(/[\n,' ']/,'')

      o_rssitem.add_element(o_rsstitle)

      

      o_rsshref_audio = Element.new('href_audio')

      o_rsshref_audio.text = node.attributes.get_attribute('href').to_s.gsub('amp;&','')      

      o_rssitem.add_element(o_rsshref_audio)

      

      o_rsshref = Element.new('href')

      o_rsshref.text = o_rsshref_audio.text.gsub('&from=audio','')      

      o_rssitem.add_element(o_rsshref)

      

      o_rssdate = Element.new('date')

      o_rssdate.text = "#{o_li.elements["p/span[1]"].text} #{o_li.elements["p/span[2]"].text}"

      o_rssitem.add_element(o_rssdate)

      rss_doc.root.add_element(o_rssitem)

      

    end



    file = File.new('thegang_rss.xml','w')

    file.puts rss_doc

    file.close

  end

end





if __FILE__ == $0

  gang = TheGang.new

  gang.rssify

end

see also: www.dapper.net

output (extract)





  TheGangXII-II/gangitem/id=6501&from=audio/gangitem/id=6501Jan 25

  TheGangXII-I/gangitem/id=6499&from=audio/gangitem/id=6499Jan 25

  NewsGangLive01.24.08/gangitem/id=6445&from=audio/gangitem/id=6445Jan 24

  NewsGangLiveII/gangitem/id=6377&from=audio/gangitem/id=6377Jan 23

  ...

RSS Feed for DZone Snippets Comments

timmorgan (Tim Morgan) — Wed, 23 Jan 2008 14:23:09 GMT

http://feed43.com/8055673348816228.xml

Archive Flagged Items from NetNewsWire into Yojimbo

justinwr (Justin R) — Fri, 18 Jan 2008 04:02:07 GMT

This lil Ruby-OSA script will allow you to import your "Flagged Items" in NetNewsWire as Web Archive Items in Yojimbo. Thus allowing you to save RSS articles for off-line viewing/storage.



#!/usr/local/bin/ruby

['rubygems', 'rbosa'].each {|lib| require lib}

nnw = OSA.app('NetNewsWire')

yojimbo = OSA.app('Yojimbo')



nnw.subscriptions.find { |s| s if s.display_name == 'Flagged Items' }.headlines.each do |article|

  unless yojimbo.web_archive_items.map { |f| f.source_url }.include?(article.url)

    archived = yojimbo.make(OSA::Yojimbo::WebArchiveItem, 

                            article.url, 

                            :name => article.title)

    # Adjust this for slower/faster bandwidth connections (or your feeling lucky, punk)

    sleep(5)

    # Uncomment below to remove the flagged items upon successfully archiving

    # if archived.name == 'untitled' && archived.source_url.empty?

    #   puts "!!! #{article.title} does not look to be imported !!!"

    # else

    #   article.delete

    # end

  end

end

A simple RSS Reader and Podcatcher

jrobertson (James Robertson) — Mon, 29 Oct 2007 23:28:18 GMT

Written in Ruby this class reads an RSS feed and downloads the latest enclosure if it exists.



require 'rss/1.0'

require 'rss/2.0'

require 'open-uri'

require 'open-uri'



class Rssreader

  def initialize(url)

    source = url # url or local file

    content = "" # raw content of rss feed will be loaded here

    open(source) do |s| content = s.read end

    @rss = RSS::Parser.parse(content, false)

  end



  # returns the first 3 titles from the rss feed

  def get_summary()

    buffer = '['

    for i in 0..2

      buffer += @rss.items[i.to_i].title + ' | '

    end      

    buffer.slice(0,buffer.length-3) + ']'

  end



  def enclosure?

    @rss.items.to_s.scan(' 0

  end



  def get_enclosure_url

    enclosure = @rss.items[0].enclosure

    enclosure.url

  end



  def rwget(url, filename)

    file = File.new(filename, 'w')

    file.puts open(url, 'User-Agent' => 'Ruby-wget').read

  end



  def download_enclosure()

    if self.enclosure? then

      enclosure_url = self.get_enclosure_url()

      local_filename = File.basename(enclosure_url)

      #puts local_filename

      if not File.exist?(local_filename) then

        puts 'downloading enclosure ...'

        self.rwget(enclosure_url, local_filename)

        puts 'download completed'

      else

        puts 'enclosure downloaded already'

      end

    end

  end    

end



if __FILE__ == $0

  url = "http://mysite.com/gwd/feed/lugradio.rss"

  rss = Rssreader.new(url)

  puts rss.get_summary()

  rss.download_enclosure()

end

Twitter bot weatherlisbon

griflet (guillaume riflet) — Mon, 18 Jun 2007 21:37:40 GMT

This little bash script shows how to use curl, grep, tail, sed and perl one-liners in order to compose a bleeding-edge twitter bot.
This one returns daily weather forecasts for Lisbon city based on the BBC weather forecast rss feed.



#! /bin/sh



#Goto here

here=/home/guillaume/Personal

cd $here



#BBC Lisbon weather id

id=0048



#BBC weather RSS feed address

feed="http://feeds.bbc.co.uk/weather/feeds/rss/5day/world/${id}.xml"



#City

city=lisbon



#temporary file

file="weather${city}"



#Weather twitter bot

twitbot=weatherlisbon:*******



#Timestamp the log file

echo .>> $file.log

date >> $file.log



#Read the RSS feed and filter it

curl $feed | grep 'title' | tail -n 1 | perl -wlne'm/title>(.*)<\/title/i && print $1' | sed -e "s/°//g" > $file.txt



#Read the forecast into a weather variable

read weather < $file.txt



#Twit the weather variable away

curl --basic --user $twitbot --data status="$weather" http://twitter.com/statuses/update.xml >> $file.log