Never been to DZone Snippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

« Newer Snippets
Older Snippets »
Showing 21-30 of 39 total

RSS Twitter Bot

Republish an RSS feed on a twitter account. This was the source I used to run the Woot Twitter Bot before they took it over.

   1  
   2  require 'rubygems'
   3  require 'active_record'
   4  require 'simple-rss'
   5  require 'open-uri'
   6  require 'twitter'
   7  
   8  #twitter account to post to
   9  twitter_email = "yourtwitteremail@bla.com"
  10  twitter_password = "secret"
  11  
  12  #rss feed to post
  13  rss_url = "http://yoursite.com/index.xml"
  14  rss_user_agent = "http://twitter.com/yourbot"
  15  
  16  #sqlite db
  17  path_to_sqlite_db = "/PATH/TO/db.sqlite"
  18  
  19  
  20  ActiveRecord::Base.logger = Logger.new(STDERR)
  21  ActiveRecord::Base.colorize_logging = false
  22  
  23  ActiveRecord::Base.establish_connection(
  24      :adapter => "sqlite3",
  25      :dbfile  => path_to_sqlite_db
  26  )
  27  
  28  #uncomment this section the first time to create the table
  29  #
  30  #ActiveRecord::Schema.define do
  31  #    create_table :item do |table|
  32  #        table.column :title, :string
  33  #        table.column :link, :string
  34  #    end
  35  #end
  36  
  37  class Item < ActiveRecord::Base
  38    def to_s
  39      "#{self.title[0..(130-self.link.length)]} - #{self.link}"
  40    end
  41  end
  42  
  43  #run the beast
  44  rss_items = SimpleRSS.parse open(rss_url ,"User-Agent" => rss_user_agent)
  45  
  46  for item in rss_items.items
  47    Item.transaction do
  48      unless existing_item = Item.find(:all, :conditions => ["link=?", item.link]).first
  49        twitter ||= Twitter::Base.new(twitter_email, twitter_password)
  50        new_item = Item.create(:title => item.title, :link => item.link) 
  51        twitter.post(new_item.to_s)
  52      end
  53    end
  54  end


Run this once with the lines uncommented to create the DB, then slap it in your crontab.

Set::extract - parsing an RSS feed for all post titles

// description of your code here
http://www.thinkingphp.org/2007/02/24/cake-12s-set-class-eats-arrays-for-breakfast/

The following code will produce the result:
Array
(
[0] => How-to: Use Html 4.01 in CakePHP 1.2
[1] => Looking up foreign key values using Model::displayField
[2] => Bug-fix update for SVN/FTP Deployment Task
[3] => Access your config files rapidly (Win32 only)
[4] => Making error handling for Model::save more beautiful in CakePHP
[5] => Full content RSS feed
[6] => Visual Sorting - Some Javascript fun I had last night
)

   1  
   2  uses('Xml');
   3   
   4  $feed = xmltoArray(new XML('http://feeds.feedburner.com/thinkingphp'));
   5  $postTitles = Set::extract($feed, 'rss.channel.item.{n}.title'); 
   6  

parsing an RSS feed for all post titles

// description of your code here
http://www.thinkingphp.org/2007/02/24/cake-12s-set-class-eats-arrays-for-breakfast/

   1  
   2  PHP:
   3  
   4  function xmltoArray($node)
   5  {
   6      $array = array();
   7      
   8      foreach ($node->children as $child)
   9      {
  10          if (empty($child->children))
  11          {
  12              $value = $child->value;
  13          }
  14          else
  15          {
  16              $value = xmltoArray($child);
  17          }
  18          
  19          $key = $child->name;
  20          
  21          if (!isset($array[$key]))
  22          {
  23              $array[$key] = $value;
  24          }
  25          else 
  26          {
  27              if (!is_array($array[$key]) || !isset($array[$key][0]))
  28              {
  29                  $array[$key] = array($array[$key]);
  30              }
  31              
  32              $array[$key][] = $value;
  33          }
  34      }
  35      
  36      return $array;
  37  } 

Importing XML into a database with Scriptella ETL

The following Scriptella ETL simple usage example imports RSS file into a database table.
   1  
   2  <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3  <etl>
   4      <connection id="in" driver="xpath" url="http://snippets.dzone.com/rss"/>
   5      <connection id="db" driver="hsqldb" url="jdbc:hsqldb:db/rss" user="sa" classpath="hsqldb.jar"/>
   6  classpath="hsqldb.jar"/>
   7      <query connection-id="in">
   8          /rss/channel/item
   9          <script connection-id="db">
  10              INSERT INTO Rss (ID, Title, Description, Link) 
  11              VALUES (?rownum, ?title, ?description, ?link);
  12          </script>
  13      </query>
  14  </etl>


Here is the full version of the example described above. It creates an RSS table, downloads rss file, inserts rss records into a database, converts rss.xml to a plain text file and saves it to rss.txt.
   1  
   2  <!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
   3  <etl>
   4      <connection id="in" driver="xpath" url="http://snippets.dzone.com/rss"/>
   5      <connection id="out" driver="text" url="rss.txt"/>
   6      <connection id="db" driver="hsqldb" url="jdbc:hsqldb:db/rss" user="sa" classpath="hsqldb.jar"/>
   7      <script connection-id="db">
   8         CREATE TABLE Rss (
   9             ID Integer,
  10             Title VARCHAR(255),
  11             Description VARCHAR(255),   
  12             Link VARCHAR(255)
  13  
  14         )
  15      </script>
  16      <query connection-id="in">
  17          /rss/channel/item
  18          <script connection-id="out">
  19              Title: $title
  20              Description: [
  21              ${description.substring(0, 20)}...
  22              ]
  23              Link: $link
  24              ----------------------------------
  25          </script>
  26          <script connection-id="db">
  27              INSERT INTO Rss (ID, Title, Description, Link) 
  28              VALUES (?rownum, ?title, ?description, ?link);
  29          </script>
  30      </query>
  31  </etl>

RSS Reader - Reads Name and URL into HashMap

// description of your code here
// RSS reader for web reads them into HashMap

   1  
   2  
   3  
   4  /**
   5   * Created by IntelliJ IDEA.
   6   * User: Rapid
   7   * Date: Oct 9, 2006
   8   * Time: 3:18:23 PM
   9   * To change this template use File | Settings | File Templates.
  10   */
  11  import java.net.URL;
  12  import java.util.Iterator;
  13  import java.util.HashMap;
  14  
  15  import com.sun.syndication.feed.module.Module;
  16  import com.sun.syndication.feed.synd.SyndEntry;
  17  import com.sun.syndication.feed.synd.SyndFeed;
  18  import com.sun.syndication.io.SyndFeedInput;
  19  import com.sun.syndication.io.XmlReader;
  20  
  21  /**
  22   * Reads and prints any RSS/Atom feed type. Adopted from the example by the
  23   * same name at http://wiki.java.net/bin/view/Javawsxml/Rome05TutorialFeedReader
  24   *
  25   */
  26  public class FeedReader {
  27  
  28  
  29      HashMap hm = null;
  30      String[][] rss = null ;
  31      SyndFeedInput input ;
  32      URL feedUrl;
  33      SyndFeed feed ;
  34      int count =-1;
  35  
  36  
  37      public HashMap readRSS(String url) {
  38          boolean readOk = false;
  39  
  40              try {
  41  
  42                  hm = new HashMap();
  43  
  44                 feedUrl  = new URL(url);
  45  
  46                  input  = new SyndFeedInput();
  47                   feed = input.build(new XmlReader(feedUrl));
  48  
  49                  System.out.println("Title: " + feed.getTitle());
  50                  System.out.println("Author: " + feed.getAuthor());
  51                  System.out.println("Description: " + feed.getDescription());
  52                  System.out.println("Pub date: " + feed.getPublishedDate());
  53                  System.out.println("Copyright: " + feed.getCopyright());
  54                  System.out.println("Modules used:");
  55  
  56  
  57  
  58                  String metaRSS = "Title: " + feed.getTitle() + "\n" +
  59                  "Author: " + feed.getAuthor()  + "\n" +
  60                   "Description: " + feed.getDescription()  + "\n" +
  61                   "Pub date: " + feed.getPublishedDate()  + "\n" +
  62                   "Copyright: " + feed.getCopyright() ;
  63  
  64  
  65  
  66  
  67                  rss = new String[ feed.getEntries().size()][2];
  68  
  69  
  70                  System.out.println("Titles of the " + feed.getEntries().size() +
  71                                     " entries:");
  72                  for (final Iterator iter = feed.getEntries().iterator();
  73                       iter.hasNext();)
  74                  {
  75  
  76  
  77  
  78  
  79  
  80  
  81                      rss[++count][0] =      ((SyndEntry)iter.next()).getTitle().toString();
  82  
  83  
  84  
  85  
  86                  }
  87                  count = -1 ;
  88                  for (final Iterator iter = feed.getEntries().iterator();
  89                       iter.hasNext();)
  90                  {
  91  
  92                     rss[++count][1] =      ((SyndEntry)iter.next()).getUri().toString();
  93                  }
  94  
  95  
  96                  if (feed.getImage() != null)
  97                  {
  98                      System.out.println("Feed image URL: " +
  99                                         feed.getImage().getUrl());
 100                  }
 101  
 102                  readOk = true;
 103                  hm.put( feed.getTitle(), rss);
 104              }
 105              catch (Exception ex) {
 106                  ex.printStackTrace();
 107                  System.out.println("ERROR: " + ex.getMessage());
 108              }
 109  
 110  
 111          String[][] rs = (String[][])hm.get("LinuxInsider");
 112  
 113            System.out.println("************************");
 114          for( int i=0; i<rs.length; i++){
 115  
 116              System.out.println( rs[i][0]);
 117               System.out.println( rs[i][1]);
 118  //             System.out.println( rs[i][2]);
 119          }
 120          if (! readOk) {
 121              System.out.println();
 122              System.out.println("FeedReader reads and prints info on any RSS/Atom feed.");
 123              System.out.println("The first parameter must be the URL of the feed to read.");
 124              System.out.println();
 125          }
 126  
 127          return hm;
 128      }
 129  
 130  }

RSS Reader - simple with Main

// description of your code here
//Reads rss from given url

   1  
   2  
   3  
   4  /**
   5   * Created by IntelliJ IDEA.
   6   * User: Rapid
   7   * Date: Oct 9, 2006
   8   * Time: 3:18:23 PM
   9   * To change this template use File | Settings | File Templates.
  10   */
  11  import java.net.URL;
  12  import java.util.Iterator;
  13  
  14  import com.sun.syndication.feed.module.Module;
  15  import com.sun.syndication.feed.synd.SyndEntry;
  16  import com.sun.syndication.feed.synd.SyndFeed;
  17  import com.sun.syndication.io.SyndFeedInput;
  18  import com.sun.syndication.io.XmlReader;
  19  
  20  /**
  21   * Reads and prints any RSS/Atom feed type. Adopted from the example by the
  22   * same name at http://wiki.java.net/bin/view/Javawsxml/Rome05TutorialFeedReader
  23   *
  24   */
  25  public class FeedReader1 {
  26  
  27      public static void main(final String[] args) {
  28          boolean readOk = false;
  29          if (args.length == 1) {
  30              try {
  31                  final URL feedUrl = new URL(args[0]);
  32  
  33                  final SyndFeedInput input = new SyndFeedInput();
  34                  final SyndFeed feed = input.build(new XmlReader(feedUrl));
  35  
  36                  System.out.println("Title: " + feed.getTitle());
  37                  System.out.println("Author: " + feed.getAuthor());
  38                  System.out.println("Description: " + feed.getDescription());
  39                  System.out.println("Pub date: " + feed.getPublishedDate());
  40                  System.out.println("Copyright: " + feed.getCopyright());
  41                  System.out.println("Modules used:");
  42                  for (final Iterator iter = feed.getModules().iterator();
  43                       iter.hasNext();)
  44                  {
  45                      System.out.println("\t" + ((Module)iter.next()).getUri());
  46                  }
  47                  System.out.println("Titles of the " + feed.getEntries().size() +
  48                                     " entries:");
  49                  for (final Iterator iter = feed.getEntries().iterator();
  50                       iter.hasNext();)
  51                  {
  52                      System.out.println("\t" +
  53                                         ((SyndEntry)iter.next()).getTitle());
  54                      
  55                  }
  56                  if (feed.getImage() != null)
  57                  {
  58                      System.out.println("Feed image URL: " +
  59                                         feed.getImage().getUrl());
  60                  }
  61  
  62                  readOk = true;
  63              }
  64              catch (Exception ex) {
  65                  ex.printStackTrace();
  66                  System.out.println("ERROR: " + ex.getMessage());
  67              }
  68          }
  69  
  70          if (! readOk) {
  71              System.out.println();
  72              System.out.println("FeedReader reads and prints info on any RSS/Atom feed.");
  73              System.out.println("The first parameter must be the URL of the feed to read.");
  74              System.out.println();
  75          }
  76      }
  77  }

2ch RSS Icon

   1  
   2  // ==UserScript==
   3  // @name  2ch RSS Icon
   4  // @namespace  http://d.hatena.ne.jp/youpy/
   5  // @include  http://*.2ch.net/test/read.cgi/*
   6  // @exclude  http://b.hatena.ne.jp/*
   7  // ==/UserScript==
   8  
   9  (function () {
  10   var a = document.createElement('a');
  11   a.href = location.href.replace(/^http:\/\/(\w+\.2ch\.net\/)test\/read\.cgi\/(.+)$/, "http://rss.s2ch.net/test/-/$1$2");
  12   var image = document.createElement('img'); 
  13   image.src = 'data:image/png;base64,'+
  14      'iVBORw0KGgoAAAANSUhEUgAAACQAAAAOCAMAAABw6U76AAAAA3NCSVQICAjb4U/gAAADAFBMVEX/'+
  15      '///18e346N3/5tXv5Nz33czw2svo18z20rvw0Lv/yKTmzLrwxqrgyrreybrowqrwvJnfv6nvsYj0'+
  16      'qXflrYjxqHfTsJnlo3fVp4f/mlfgoXfunGbeoHbWnXbjmGbFoIfflma7nIfJmHffjFXsh0TmhUS6'+
  17      'kXe5kXf0gDPUiFXufjPfgkTNhVX2dyLHglXxdSLSfUTkdy6zhWXedzO+f1WqgWa3fFT/ZgDScjPd'+
  18      'bSLraBH3YwCseFXFbTPvYACkdFTsXgDmXADiXQW4aDPGYyLUXxGzZjPeWQDNXBGhaUTZVwDGWRGm'+
  19      'YTPTVADPUwDHUACyURGUWjPFSgC7SwCcUyK1SACsRQCbSBGoQwCNQxGcPgCZPQCTOwB9MwI/GgEh'+
  20      'DgEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  21      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  22      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  23      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  24      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  25      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  26      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  27      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'+
  28      'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABJnr9zAAABHUlEQVR4nIWRWVODMBSFL5XWJVbc60K1'+
  29      '7gvuVYNGUuIWRQMtSNG6/f9/YQLTaRkfel7Omcw3uWfuhaGxgTqFRaxElHAWKaVp7Dr+zqDdum3b'+
  30      'T4IRTLZM8+BeUHK9YladgJEeBKnKbX5cSNOdmE4dRW4/dPG4p8FZWICdTmAZzgboH/FDzQhyUCti'+
  31      'o7AdAawlgcfZHOjPseDcJfmfTIBEjMgZ+nxT7KthE+fRv05aK3bJpKZSIjZL6mk1zndaB2hIiPGr'+
  32      'mibncs4tA+CL5zuNg/YWFJ3Q80pwUzVDn1ugf+agV58WQW/LRkiH4feKXEgZ4DK87UEzSy8MH1bM'+
  33      'k/osQlNHEafLBkILTZ/2QTi7AFVijFGVpbvds3R+Bur3DxzBWgFMxGSIAAAAAElFTkSuQmCC';
  34   image.width = 36;
  35   image.height = 14;
  36   a.appendChild(image);
  37   document.body.appendChild(a);
  38  })();
  39  

Digg/RSS Reader

Grabs the RSS-feed from digg.com and reads it out
loud using festival.

   1  
   2  #!/bin/sh
   3  # Copyright (c) 2005 Davor Babic <davorb@gmail.com>
   4  # All rights reserved.
   5  # Usage of the works is permitted provided that this
   6  # instrument is retained with the works, so that any
   7  # entity that uses the works is notified of this
   8  # instrument.
   9  # DISCLAIMER: THE WORKS ARE WITHOUT WARRANTY.
  10  
  11  url="http://digg.com/rss/index.xml"
  12  
  13  echo "Parsing RSS..."
  14  curl --silent "$url" | grep -E '(title>|description>)' | \
  15          sed -n '4,$p' | \
  16  	sed -e 's/<title>//' -e 's/<\/title>//' -e 's/<description>/  /' \
  17  	    -e 's/<\/description>//' | head -5 > digg 
  18  echo "Reading..."
  19  festival --tts digg
  20  rm -rf ./digg
  21  echo "Done."

Random Flickr Photo

// description of your code here

   1  
   2  <?php
   3  
   4      // Displays a single random photo from recent flickr photos with a given tag.
   5      // Original code stolen from many sources including http://www.thebishop.net/geodog/archives/2004/09/29/fun_hacking_with_flickr_making_a_homemade_flickr_tag_badge_with_magpierss.html and http://prwdot.org/archives/002468.html
   6  
   7  
   8      // USER CONFIGURATION SECTION
   9  
  10      // MagpieRSS Configuration
  11      // This is an example based on my system;
  12      // you will need to customize it for your system
  13      // and your preferences. You can remove it entirely
  14      // if you have done it elsewhere
  15      // refer to http://magpierss.sourceforge.net/
  16      require_once('/var/www/bradford/magpierss-0.61/rss_fetch.inc');
  17      error_reporting(E_ERROR);
  18      define(MAGPIE_CACHE_ON, true);
  19      define(MAGPIE_CACHE_DIR, '/var/www/bradford/magpie_cache');
  20      define(MAGPIE_CACHE_AGE, 300);
  21      define(MAGPIE_CACHE_FRESH_ONLY, false);
  22      define(MAGPIE_DETECT_ENCODING, true);
  23      define(MAGPIE_DEBUG, 0);
  24      define(MAGPIE_FETCH_TIME_OUT, 15);
  25      define(MAGPIE_USE_GZIP, true);    
  26  
  27      // flickr configuration
  28      // How many photos you want to display
  29      $num_photos = 2; // for some reason it doesn't like 1
  30      $tag = 'Bradford';
  31      // URL for the flickr feed you want to use
  32      $flickr_feed_url ='http://www.flickr.com/services/feeds/photos_public.gne?tags='.$tag.'&format=rss_200';
  33  ?>
  34  
  35  <?php
  36      // Fetch the feed
  37      $flickr = fetch_rss( $flickr_feed_url );
  38      if ($flickr) {
  39          $flickr_title = $flickr->channel["title"];
  40          $flickr_link = $flickr->channel["link"];
  41  ?>
  42  
  43  <!-- Display the title and link to the feed -->
  44  
  45  <?php
  46      // Pick some random photos
  47          $random_photos = array_rand($flickr->items,$num_photos);
  48          foreach ( $random_photos as $random_photo ) {
  49              $description = explode("\n\n",$flickr->items[$random_photo]["description"]);
  50  ?>
  51  
  52      <!-- Display the given photo -->
  53  
  54  	<?php echo ereg_replace('<img src=(.*) width=(.*)>', '<img src=\\1 width="150px"/>', $description[1]);
  55  	  die();?> // ok we've got our first photo - lets exit
  56  
  57  <?php
  58          }
  59        } else {
  60  ?>
  61  
  62  <!-- Display an error message if things didn't work -->
  63  <p>An error occurred in the MagpieRSS parser:</p>
  64  
  65  <p><?php echo magpie_error(); ?></p>
  66  
  67  <?php
  68      } 
  69  ?>
  70  

RSS to Html conversion

This file is intended to transform a RSS feed (version 0.9x or 2.0) into html code fragment to be included in a complete page.

   1  
   2  <?xml version="1.0"?>
   3  <xsl:stylesheet version="1.0"
   4   xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
   5   xmlns:wfw="http://wellformedweb.org/CommentAPI/">
   6      <xsl:output method="html"/>
   7      <xsl:template match="/">
   8          <xsl:apply-templates select="/rss/channel"/>
   9      </xsl:template>
  10      <xsl:template match="/rss/channel">
  11          <h3><xsl:value-of select="title"/></h3>
  12          <p><xsl:value-of select="description"/></p>
  13          <ul>
  14              <xsl:apply-templates select="item"/>
  15          </ul>
  16      </xsl:template>
  17      <xsl:template match="/rss/channel/item">
  18          <li>
  19              <a href="{link}" title="{substring(pubDate, 0, 11)}"><xsl:value-of select="title"/></a>
  20              <p><xsl:value-of select="description" disable-output-escaping="yes" /></p>
  21          </li>
  22      </xsl:template>
  23  </xsl:stylesheet>
« Newer Snippets
Older Snippets »
Showing 21-30 of 39 total