Never been to DZone Snippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

About this user

Peter Cooperx http://www.petercooper.co.uk/

« Newer Snippets
Older Snippets »
Showing 1-10 of 134 total  RSS 

runas - Run a program under a specific user / group (poor man's suexec)

#!/usr/bin/env ruby

# runas - Run another program under the privileges of a specified user and group.
# This is necessary because sudo demands a password, as we need it to be hands off.
# A poor man's suexec basically.

require 'etc'

user, group, cmd = ARGV

begin
  uid = Etc.getpwnam(user).uid
  gid = Etc.getgrnam(group).gid

  unless Process.euid == uid && Process.egid == gid
    Process.initgroups(user, gid)
    Process::GID.change_privilege(gid)
    Process::UID.change_privilege(uid)
  end

  exec cmd
rescue
  puts "Could not run as #{user}:#{group}"
  exit 1
end


Usage example: ./runas username groupname "sleep 10"

Patchwork quilt pattern in Ruby (from Knuth TAoCP 4 - 7.1.3)

In Donald Knuth's TAoCP Pre-Fascicle 1a: Bitwise Tricks and Techniques, he shows a patchwork quilt defined by f(x, y) = ((x ^ y) & ((y - 350) >> 3)) ** 2, designed by D. Sleator in 1976. I wanted to recreate this quilt myself using Ruby. The following program generates a PNG file of the "quilt."

require 'rubygems'
require 'png'

def f(x, y)
  ( (x ^ y) & ((y - 350) >> 3) ) ** 2
end

canvas = PNG::Canvas.new(500, 500)

0.upto(499) do |y|
  0.upto(499) do |x|
    canvas[x, 499 - y] = ((f(x, y) >> 12) & 1) == 1 ? PNG::Color::Black : PNG::Color::White
  end
end

png = PNG.new(canvas)
png.save 'pattern.png'

Getting a client's IP address with EventMachine

This is from, and by: http://nhw.pl/wp/2007/12/07/eventmachine-how-to-get-clients-ip-address/ .. I just wanted a more permanent reference as it's cool code.

EventMachine::run {
   EventMachine::open_datagram_socket $conf[:address],
                                 $conf[:udp_port], CustomServer
}

module CustomServer
   def receive_data d
      pp get_peername[2,6].unpack "nC4"
   end
end

Logic gates in Ruby

NAND = lambda { |i, j| !(i && j) }
NOT = lambda { |i| NAND[i, i] }
AND = lambda { |i, j| NOT[NAND[i, j]] }
OR = lambda { |i, j| NAND[NAND[i, i], NAND[j, j]] }
NOR = lambda { |i, j| NOT[OR[i, j]] }
XOR = lambda { |i, j| NAND[NAND[i, NAND[i, j]], NAND[j, NAND[i, j]]] }
XNOR = lambda { |i, j| NOT[XOR[i, j]] }

XOR[true, true] => false
XOR[true, false] => true
XOR[false, true] => true
XOR[false, false] => false

See your most used shell commands

Found here.

history | awk '{print $2}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -n | tail | sort -nr

Fast stop word detection in Ruby

Requires BloominSimple (a pure Ruby Bloom filter class).

List of stop words obtained from http://www.dcs.gla.ac.uk/idom/ir_resources/linguistic_utils/stop_words

# Detect stop words QUICKLY
# Uses a bloom filter instead of searching literally through a list of stopwords
# for > 3x speed increase
# 
#    using bloom filter: 2.580000   0.030000   2.610000 (  2.698829)
#  using literal search: 7.850000   0.120000   7.970000 (  8.181684)


require 'bloominsimple'
require 'digest/sha1'
require 'pp'

# Create a simple bloom filter that uses a SHA1 hash (more effective than BloominSimple's default hashing)
b = BloominSimple.new(50000) do |word|
  Digest::SHA1.digest(word.downcase.strip).unpack("VVV")
end

# Add stopwords to the bloom filter!
stopwords = []
File.open('stopwords').each { |a| b.add(a); stopwords << a.downcase.strip }

# Read in a whole dictionary of regular words
words = File.open('/usr/share/dict/words').read.split.collect{|a| a.downcase.strip }

# Define two ways to detect stopwords for comparison..
using_filter = lambda { |word| b.includes?(word) }
using_array = lambda { |word| stopwords.include?(word.downcase.strip) }
techniques = [using_filter, using_array]

# Run stopword comparisons with both techniques
t = techniques.collect { |l| words.collect { |a| l[a] } }

# See how effective the bloom filter has been compared to the literal search
if t[0] == t[1]
  puts "GOOD"
else
  words.zip(t[0],t[1]).each do |x|
    puts x.first if x[1] != x[2]
  end
end

# Now do speed benchmarks..
techniques.each { |l| puts Benchmark.measure { words.each { |a| l[a] } } }

BloominSimple: Ultra-easy, pure Ruby Bloom filter library

Requires BitField.

#        NAME: BloominSimple
#      AUTHOR: Peter Cooper
#     LICENSE: MIT ( http://www.opensource.org/licenses/mit-license.php )
#   COPYRIGHT: (c) 2007 Peter Cooper
# DESCRIPTION: Very basic, pure Ruby Bloom filter. Uses my BitField, pure Ruby
#              bit field library (http://snippets.dzone.com/posts/show/4234).
#              Supports custom hashing (default is 3).
#
#              Create a Bloom filter that uses default hashing with 1Mbit wide bitfield
#                bf = BloominSimple.new(1_000_000)
#              
#              Add items to it
#                File.open('/usr/share/dict/words').each { |a| bf.add(a) }
#
#              Check for existence of items in the filter
#                bf.includes?("people")     # => true
#                bf.includes?("kwyjibo")    # => false
#
#              Add better hashing (c'est easy!)
#                require 'digest/sha1'
#                b = BloominSimple.new(1_000_000) do |item|
#                  Digest::SHA1.digest(item.downcase.strip).unpack("VVVV")
#                end
#
#              More
#                %w{wonderful ball stereo jester flag shshshshsh nooooooo newyorkcity}.each do |a|
#                  puts "#{sprintf("%15s", a)}: #{b.includes?(a)}"
#                end
#
#                 #  =>   wonderful: true
#                 #  =>        ball: true
#                 #  =>      stereo: true
#                 #  =>      jester: true
#                 #  =>        flag: true
#                 #  =>  shshshshsh: false
#                 #  =>    nooooooo: false
#                 #  => newyorkcity: false

require 'benchmark'
require 'bitfield'

class BloominSimple
  attr_reader :bitfield, :hasher
  
  def initialize(bitsize, &block)
    @bitfield = BitField.new(bitsize)
    @size = bitsize
    @hasher = block || lambda do |word|
      word = word.downcase.strip
      [h1 = word.sum, h2 = word.hash, h2 + h1 ** 3]
    end
  end
  
  def add(item)
    @hasher[item].each { |hi| @bitfield[hi % @size] = 1 }
  end
  
  def includes?(item)
    @hasher[item].each { |hi| return false unless @bitfield[hi % @size] == 1 } and true
  end
end

BitField: A fast(ish), pure Ruby bit field "type"

#        NAME: BitField
#      AUTHOR: Peter Cooper
#     LICENSE: MIT ( http://www.opensource.org/licenses/mit-license.php )
#   COPYRIGHT: (c) 2007 Peter Cooper (http://www.petercooper.co.uk/)
#     VERSION: v4
#     HISTORY: v4 (fixed bug where setting 0 bits to 0 caused a set to 1)
#              v3 (supports dynamic bitwidths for array elements.. now doing 32 bit widths default)
#              v2 (now uses 1 << y, rather than 2 ** y .. it's 21.8 times faster!)
#              v1 (first release)
#
# DESCRIPTION: Basic, pure Ruby bit field. Pretty fast (for what it is) and memory efficient.
#              I've written a pretty intensive test suite for it and it passes great. 
#              Works well for Bloom filters (the reason I wrote it).
#
#              Create a bit field 1000 bits wide
#                bf = BitField.new(1000)
#
#              Setting and reading bits
#                bf[100] = 1
#                bf[100]    .. => 1
#                bf[100] = 0
#
#              More
#                bf.to_s = "10101000101010101"  (example)
#                bf.total_set         .. => 10  (example - 10 bits are set to "1")

class BitField
  attr_reader :size
  include Enumerable
  
  ELEMENT_WIDTH = 32
  
  def initialize(size)
    @size = size
    @field = Array.new(((size - 1) / ELEMENT_WIDTH) + 1, 0)
  end
  
  # Set a bit (1/0)
  def []=(position, value)
    if value == 1
      @field[position / ELEMENT_WIDTH] |= 1 << (position % ELEMENT_WIDTH)
    elsif (@field[position / ELEMENT_WIDTH]) & (1 << (position % ELEMENT_WIDTH)) != 0
      @field[position / ELEMENT_WIDTH] ^= 1 << (position % ELEMENT_WIDTH)
    end
  end
  
  # Read a bit (1/0)
  def [](position)
    @field[position / ELEMENT_WIDTH] & 1 << (position % ELEMENT_WIDTH) > 0 ? 1 : 0
  end
  
  # Iterate over each bit
  def each(&block)
    @size.times { |position| yield self[position] }
  end
  
  # Returns the field as a string like "0101010100111100," etc.
  def to_s
    inject("") { |a, b| a + b.to_s }
  end
  
  # Returns the total number of bits that are set
  # (The technique used here is about 6 times faster than using each or inject direct on the bitfield)
  def total_set
    @field.inject(0) { |a, byte| a += byte & 1 and byte >>= 1 until byte == 0; a }
  end
end


Here's the tests if you want to run:

require "test/unit"
require "bitfield"

class TestLibraryFileName < Test::Unit::TestCase
  def setup
    @public_bf = BitField.new(1000)
  end
  
  def test_basic
    assert_equal 0, BitField.new(100)[0]
    assert_equal 0, BitField.new(100)[1]
  end
  
  def test_setting_and_unsetting
    @public_bf[100] = 1
    assert_equal 1, @public_bf[100]
    @public_bf[100] = 0
    assert_equal 0, @public_bf[100]
  end

  def test_random_setting_and_unsetting
    100.times do
      index = rand(1000)
      @public_bf[index] = 1
      assert_equal 1, @public_bf[index]
      @public_bf[index] = 0
      assert_equal 0, @public_bf[index]
    end
  end
  
  def test_multiple_setting
    1.upto(999) do |pos|
      2.times { @public_bf[pos] = 1 }
      assert_equal 1, @public_bf[pos]
    end
  end

  def test_multiple_unsetting
    1.upto(999) do |pos|
      2.times { @public_bf[pos] = 0 }
      assert_equal 0, @public_bf[pos]
    end
  end
  
  def test_size
    assert_equal 1000, @public_bf.size
  end
  
  def test_to_s
    bf = BitField.new(10)
    bf[1] = 1
    bf[5] = 1
    assert_equal "0100010000", bf.to_s
  end
  
  def test_total_set
    bf = BitField.new(10)
    bf[1] = 1
    bf[5] = 1
    assert_equal 2, bf.total_set
  end
end

Count number of set bits in an integer

count = 0
count += byte & 1 and byte >>= 1 until byte == 0

upload_to_s3 - Ruby S3 upload client

Prerequisites:
gem install aws-s3
gem install main

#!/bin/env ruby

require 'rubygems'
require 'main'
require 'aws/s3'
include AWS::S3

Main {
  argument('source_filename') {
    cast :string
    description 'source filename to copy to S3'
  }

  argument('bucket_name') {
    cast :string
    description 'bucket to place the file in on S3'
  }

  option('access_key_id') {
    argument :optional
    description 'specify the access_key_id manually'
    default 'put your access key here if you want'
  }

  option('secret_access_key') {
    argument :optional
    description 'specify the secret key manually'
    default 'put your secret key here if you want'
  }

  def run
    bucket_name = params['bucket_name'].value
    source_filename = params['source_filename'].value

    Base.establish_connection!(
      :access_key_id     => params['access_key_id'].value,
      :secret_access_key => params['secret_access_key'].value
    )

    begin
      Bucket.find(bucket_name)
    rescue
      puts "Need to make bucket #{bucket_name}.."
      Bucket.create(bucket_name)

      # Confirm its existence..
      Bucket.find(bucket_name)
    end

    puts "Got bucket.."
    puts "Uploading #{File.basename(source_filename)}.."
    S3Object.store(File.basename(source_filename), open(source_filename), bucket_name)
    puts "Stored!"

    exit_success!
  end
}
« Newer Snippets
Older Snippets »
Showing 1-10 of 134 total  RSS