Never been to DZone Snippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

« Newer Snippets
Older Snippets »
Showing 1-10 of 20 total  RSS 

Turn off iPhone backup when syncing.

This really speeds up syncing with iTunes. Use at your own risk as you obviously won't have a backup.

Turn off backup
   1  
   2  defaults write com.apple.iTunes DeviceBackupsDisabled -bool true


Turn on backup
   1  
   2  defaults write com.apple.iTunes DeviceBackupsDisabled -bool false

Restore a single table from a large MySQL backup

Say, for some reason, you need to restore the entire contents of a single table from a HUGE mysqldump generated backup containing several tables. For example:

   1  
   2  create table `baz`;
   3  
   4  GIGS OF SQL YOU DON'T WANT;
   5  
   6  create table `foo`;
   7  
   8  A COUPLE THOUSAND LINES YOU DO WANT;
   9  
  10  create table `bar`;
  11  
  12  MORE SQL YOU DON'T WANT;


With a little dash 'o ruby, you can extract just the part you want:

   1  
   2  $ ruby -ne '@found=true if $_ =~ /^CREATE TABLE `foo`/i; next unless @found; exit if $_ =~ /^CREATE TABLE (?!`foo`)/i; puts $_;' giant_sql_dump.sql > foo.sql
   3  $ cat foo.sql
   4  create table `foo`;
   5  
   6  A COUPLE THOUSAND LINES YOU DO WANT;
   7  


You can then easily restore that entire table:

   1  
   2  $ mysql mydatabase -e 'drop table foo'
   3  $ mysql mydatabase < foo.sql

Active Record YAML Backup Solution - Drop in rake task and config file to backup db and directorys of your choice (user uploaded files)

EZ drop in backup rake task for your rails projects - works well - 5 - 10 min set up, one rake file and one config file
backs up db and any directory's to any number of servers with rsync


BSD License or whatever, but it would be cool if you told me your using it
2007 ISS http://industrialstrengthinc.com

this is a sample config file and a rake task you can drop into any rails project to do backups, easy to automate.
Backs up your specified environment db to activerecord yml files (one per table) and zips them up in a human readable timestamped file
syncs that and any other set of arbitrary directory's to any number of arbitrary servers with rsync
be sure to set ssh key based logins
to run: rake backup
also: rake backup:db, rake backup:restoredb (promts for a file created by backup:db), rake backup:push (to push out what u got)
note: this uses something like 30 lines of code from some blog site I got it from that I cant recall, yay for that guy, thanks

   1  
   2  ## example crontab entry:
   3  
   4  3 * * * * cd /rails_deployment_dir/current && nice rake backup RAILS_ENV=production >> /rails_deployment_dir/production_backup_system.log
   5  
   6  ## config file - goes in config/backup.yml
   7  
   8  production: 
   9    dirs: 
  10     - db/backups
  11     - public/uploaded_images
  12    servers: 
  13      -  name: backup server number 1
  14         host: gridserver.com
  15         port: 22
  16         user: blah@whatever.com
  17         dir: /home/blah/backups
  18      -  name: backup server two
  19         host: kradradio.com
  20         port: 22
  21         user: kraduser
  22         dir: /home/kraduser/backups_from_my_rails_proj
  23  
  24  
  25  development: 
  26    dirs: 
  27     - db/backups
  28     - public/uploaded_images
  29    servers: 
  30      -  name: local self
  31         host: localhost
  32         port: 5222
  33         user: oneman
  34         dir: /home/oneman/Documents/development_backup
  35  
  36  
  37  ## rake file lib/tasks/backup.rake
  38  
  39  desc "Backup Everything Specified in config/backup.yml"
  40  task :backup => [ "backup:db",  "backup:push"]
  41  
  42  namespace :backup do
  43   
  44      RAILS_APPDIR = RAILS_ROOT.sub("/config/..","")
  45      
  46     def interesting_tables
  47       ActiveRecord::Base.connection.tables.sort.reject! do |tbl|
  48         ['schema_info', 'sessions', 'public_exceptions'].include?(tbl)
  49       end
  50     end
  51    
  52     desc "Push backup to remote server"
  53     task :push  => [:environment] do 
  54        FileUtils.chdir(RAILS_APPDIR)
  55        backup_config = YAML::load( File.open( 'config/backup.yml' ) )[RAILS_ENV]
  56        for server in backup_config["servers"]
  57         puts "Backing up #{RAILS_ENV} directorys #{backup_config['dirs'].join(', ')} to #{server['name']}"
  58         puts "Time is " + Time.now.rfc2822 + "\n\n"
  59           for dir in backup_config["dirs"]
  60            local_dir = RAILS_APPDIR + "/" + dir + "/"
  61            remote_dir = server['dir'] + "/" + dir.split("/").last + "/"
  62            puts "Syncing #{local_dir} to #{server['host']}#{remote_dir}"
  63            sh "/usr/bin/rsync -avz -e 'ssh -p#{server['port']} ' #{local_dir} #{server['user']}@#{server['host']}:#{remote_dir}"
  64           end
  65         puts "Completed backup to #{server['name']}\n\n"
  66        end
  67     end
  68  
  69      task :storedb => :environment do 
  70  
  71        backupdir = RAILS_APPDIR + '/db/backup'
  72        FileUtils.mkdir_p(backupdir)
  73        FileUtils.chdir(backupdir)
  74        puts "Dumping database to activerecord yaml files in #{backupdir}"
  75        interesting_tables.each do |tbl|
  76  
  77          klass = tbl.classify.constantize
  78          puts "Writing #{tbl}..."
  79          File.open("#{tbl}.yml", 'w+') { |f| YAML.dump klass.find(:all).collect(&:attributes), f }      
  80        end
  81        puts "Database Dumped.\n\n"
  82      end
  83  
  84      desc "Dump Current Environment Db to file"    
  85      task :db => [:environment, :storedb ] do
  86        backupdir = RAILS_APPDIR + '/db/backup'
  87        archivedir = RAILS_APPDIR + '/db/backups'
  88        backup_filename = "#{RAILS_ENV}_db_backup_#{Time.now.strftime("%B.%d.%Y_at_%I.%M.%S%p_%Z")}.tar.bz2"
  89        FileUtils.mkdir_p(archivedir)
  90        puts "Archiving #{backupdir} yaml files to #{backup_filename}\n\n"
  91        `tar -C #{backupdir} -cjf #{backup_filename} *`
  92        `mv #{backup_filename} #{archivedir}`
  93      end
  94  
  95      desc "Restore Current Environment Db from a file"    
  96      task :restoredb => [:environment] do 
  97          backupdir = RAILS_APPDIR + '/db/backup'
  98          archivedir = RAILS_APPDIR + '/db/backups'
  99          print "Input a file to load into the db: #{archivedir}/"
 100          backup_filename = STDIN.gets.chomp
 101          puts "Loading backup file: #{backup_filename}"
 102          FileUtils.chdir(archivedir)
 103          `tar -xjf #{backup_filename}`
 104          `mv *.yml #{backupdir}`
 105          FileUtils.mkdir_p(backupdir)
 106          FileUtils.chdir(backupdir)
 107      
 108          interesting_tables.each do |tbl|
 109          puts "Clearing #{tbl} table.."
 110          ActiveRecord::Base.connection.execute "TRUNCATE #{tbl}"
 111          puts "Loading #{tbl} backup file..."
 112          table = YAML.load_file("#{tbl}.yml")        
 113  
 114          if table.length > 0 && table.first.key?("id")
 115              highestid = 0
 116              table.each do |fixture|
 117               if fixture["id"] > highestid
 118                  highestid = fixture["id"]
 119               end
 120              end
 121  
 122              ActiveRecord::Base.connection.execute "SELECT setval('#{tbl}_id_seq',#{highestid})"
 123              puts "Setting #{tbl}_id sequence to #{highestid}"
 124          end
 125           
 126          #klass = tbl.classify.constantize
 127          ActiveRecord::Base.transaction do 
 128          
 129            puts "Inserting #{table.length} values into #{tbl}"
 130            table.each do |fixture|
 131              ActiveRecord::Base.connection.execute "INSERT INTO #{tbl} (#{fixture.keys.join(",")}) VALUES (#{fixture.values.collect { |value| ActiveRecord::Base.connection.quote(value) }.join(",")})", 'Fixture Insert'
 132            end        
 133            puts "#{tbl} table restored.\n\n"
 134          end
 135         end
 136      end
 137  
 138   
 139  end

Python Flickr Backup Script (Untested)


   1  
   2  # flickrbackup.py - Matt Croydon - http://postneo.com
   3  import flickr # http://jamesclarke.info/projects/flickr/
   4  import BitBucket # http://www.other10percent.com/?p=15
   5  import urllib
   6  me = flickr.people_findByUsername("postneo")
   7  bucket = BitBucket.BitBucket("postneo-flickr")
   8  page = 1
   9  total_photos = found_photos = 0
  10  while 1:
  11      try:
  12          photos = flickr.people_getPublicPhotos(me.id, 100, page)
  13          for photo in photos:
  14              total_photos = total_photos + 1
  15              if bucket.has_key("%s-%s" % (photo.title, photo.id)):
  16                  pass # we already have this photo
  17              else:
  18                  data = urllib.urlretrieve("http://static.flickr.com/%s/%s_%s_o.jpg" % (photo.server, photo.id, photo.secret), "flickr.jpg")
  19                  bits = BitBucket.Bits(filename="flickr.jpg")
  20                  bucket["%s-%s" % (photo.title, photo.id)] = bits
  21                  print "saving %s" % photo.title
  22                  found_photos = found_photos + 1
  23          page = page + 1
  24      except AttributeError:
  25          break # We probably got an empty bucket
  26  print "Found %s photos, saved %s new photos" % (total_photos, found_photos)

simple backup PHP application (php files + mysql db)

// backup a folder contains all .php files and it's mysql database. a backup.ini file is required for storing mysql root login and what folder to be backed up.

   1  
   2  #!/bin/sh
   3  
   4  ########################################
   5  # simple sctipt for daily / hourly backup of PHP app + Mysql DB
   6  # Copyleft (c) Sayid Munawar. chenull@yahoo.com
   7  # LICENSE: anyone can copy / modify / distribute. whatever lah
   8  #
   9  # example of workhours backup (8 am - 5 pm. Mon - Fri)
  10  # 0 8-17 * * 1-5 /home/backup/backup.sh
  11  #
  12  # example of simple dayly backup ( 6 pm. Mon - Fri)
  13  # 0 18 * * 1-5 /home/backup/backup.sh
  14  ########################################
  15  
  16  ########################################
  17  # example of backup.ini. REMOVE ALL leading '#'s
  18  # file must be chmod 600
  19  #[main]
  20  #target = /home/backup/storage
  21  #mysql_user = root   ; root can connect to any db
  22  #mysql_pass = secret ; mysql root password
  23  #
  24  #[hukum]
  25  #path = /home/hukum
  26  #mysql_db = hukum
  27  #
  28  #[phpmyadmin]
  29  #path = /home/phpmyadmin
  30  #mysql_db = test
  31  ########################################
  32  
  33  PATH=$PATH:/usr/bin:/bin:/usr/local/bin
  34  
  35  inifile=`dirname $0`/backup.ini
  36  
  37  test ! -r $inifile && echo "ERROR! backup.ini not found nor readable" && exit 1
  38  
  39  #test apakah backup.ini bisa dibaca orang lain
  40  echo -n `stat -c '%a' $inifile` | grep -q '[0-7]00' >/dev/null 2>&1
  41  test $? -gt 0 && echo "ERROR! $inifile can be read by others" && exit 2
  42  #test `stat -c '%U' $inifile` != 'root' && echo "ERROR! $inifile ownernya bukan root" && exit 3
  43  
  44  # function to parse ini file (backup.ini)
  45  #
  46  # $1 -> file.ini
  47  # $2 -> section
  48  _parse_ini () {
  49      if [ -z "$1" ] || [ -z "$2" ]; then return 0; fi
  50      eval `cat $1 | \
  51         sed -e 's/[[:space:]]*\=[[:space:]]*/=/g' \
  52             -e 's/;.*$//' \
  53             -e 's/[[:space:]]*$//' \
  54             -e 's/^[[:space:]]*//' \
  55             -e "s/^\(.*\)=\([^\"']*\)$/\1=\"\2\"/" | \
  56         sed -n -e "/^\[$2\]/,/^\s*\[/{/^[^;].*\=.*/p;}"`
  57  }
  58  
  59  _parse_ini $inifile main
  60  
  61  # coba konek 
  62  mysql -u $mysql_user  -p${mysql_pass} -e '' > /dev/null 2>&1
  63  test $? -gt 0 && echo "ERROR! Failed to connect to mysql" && exit 4
  64  
  65  # bikin target kalo blum ada
  66  test ! -d $target && mkdir -p $target
  67  
  68  hari=`date +%A`
  69  jam=`date +%H`
  70  
  71  # get pwd
  72  cwd=`pwd`
  73  
  74  # looping
  75  for section in `cat $inifile | grep '^\[' | grep -v main | sed 's/\[//' | sed 's/\]//'`; do
  76    outdir=$target/$section/$hari/$jam
  77    echo Backing up ${section} to $outdir...
  78    # parse .ini
  79    _parse_ini $inifile $section
  80    # cek dulu
  81    test ! -d $path && echo "Warning: $path not found.  Backup process of $section skipped" && echo && continue
  82    mysql -u $mysql_user  -p${mysql_pass} -e '' $mysql_db > /dev/null 2>&1
  83    test $? -gt 0 && echo "Warning: Database $mysql_db not found.  Backup process of $section skipped" && echo && continue
  84  
  85    test ! -d $outdir && mkdir -p $outdir
  86    cd $path
  87    tar -czpf $outdir/$section.tar.gz .
  88    cd $cwd
  89    mysqldump -Q -u $mysql_user -p${mysql_pass} $mysql_db | gzip > $outdir/${mysql_db}.sql.gz
  90    ln -fs $outdir/$section.tar.gz $target/$section/latest.tar.gz
  91    ln -fs $outdir/${mysql_db}.sql.gz $target/$section/latest.sql.gz
  92  done

Backup all your del.icio.us bookmarks to XML

   1  
   2  https://api.del.icio.us/v1/posts/all
   3  curl --user accountname:password -o myDelicious.xml -O 'https://api.del.icio.us/v1/posts/all'

Simple mySQL backup script for cron

Simple mySQL backup script for cron - backs up all databases, saves the last 4 copies.
   1  
   2  #!/bin/bash
   3  
   4  # modify the following to suit your environment
   5  export DB_BACKUP="/backup/mysql_backup"
   6  export DB_USER="root"
   7  export DB_PASSWD="********"
   8  
   9  # title and version
  10  echo ""
  11  echo "mySQL_backup"
  12  echo "----------------------"
  13  echo "* Rotating backups..."
  14  rm -rf $DB_BACKUP/04
  15  mv $DB_BACKUP/03 $DB_BACKUP/04
  16  mv $DB_BACKUP/02 $DB_BACKUP/03
  17  mv $DB_BACKUP/01 $DB_BACKUP/02
  18  mkdir $DB_BACKUP/01 
  19  
  20  echo "* Creating new backup..."
  21  mysqldump --user=$DB_USER --password=$DB_PASSWD --all-databases | bzip2 > $DB_BACKUP/01/mysql-`date +%Y-%m-%d`.bz2
  22  echo "----------------------"
  23  echo "Done"
  24  exit 0

basic batch drive presence check

Badly formatted post broke our parser. Please edit!

upload_to_s3 - Ruby S3 upload client

Prerequisites:
gem install aws-s3
gem install main

   1  #!/bin/env ruby
   2  
   3  require 'rubygems'
   4  require 'main'
   5  require 'aws/s3'
   6  include AWS::S3
   7  
   8  Main {
   9    argument('source_filename') {
  10      cast :string
  11      description 'source filename to copy to S3'
  12    }
  13  
  14    argument('bucket_name') {
  15      cast :string
  16      description 'bucket to place the file in on S3'
  17    }
  18  
  19    option('access_key_id') {
  20      argument :optional
  21      description 'specify the access_key_id manually'
  22      default 'put your access key here if you want'
  23    }
  24  
  25    option('secret_access_key') {
  26      argument :optional
  27      description 'specify the secret key manually'
  28      default 'put your secret key here if you want'
  29    }
  30  
  31    def run
  32      bucket_name = params['bucket_name'].value
  33      source_filename = params['source_filename'].value
  34  
  35      Base.establish_connection!(
  36        :access_key_id     => params['access_key_id'].value,
  37        :secret_access_key => params['secret_access_key'].value
  38      )
  39  
  40      begin
  41        Bucket.find(bucket_name)
  42      rescue
  43        puts "Need to make bucket #{bucket_name}.."
  44        Bucket.create(bucket_name)
  45  
  46        # Confirm its existence..
  47        Bucket.find(bucket_name)
  48      end
  49  
  50      puts "Got bucket.."
  51      puts "Uploading #{File.basename(source_filename)}.."
  52      S3Object.store(File.basename(source_filename), open(source_filename), bucket_name)
  53      puts "Stored!"
  54  
  55      exit_success!
  56    end
  57  }

Dump postgres production data into development database

When bug requests come in, its great to grab a copy of the current production (or system test env) data and sync it into your development database.

Here's a capistrano recipe for postgresql:

   1  
   2  desc "Dumps target database into development db"
   3  task :sync_db do
   4    env   = ENV['RAILS_ENV'] || ENV['DB'] || 'production'
   5    file  = "#{application}.sql.bz2"
   6    remote_file = "#{shared}/log/#{file}"
   7    run "pg_dump --clean --no-owner --no-privileges -U#{db_user} -h#{db_host} #{db_name}_#{env} | bzip2 > #{file}" do |ch, stream, out|
   8      ch.send_data "#{db_password}\n" if out =~ /^Password:/
   9      puts out
  10    end
  11    puts rsync = "rsync #{user}@#{domain}:#{file} tmp"
  12    `#{rsync}`
  13    puts depackage = "bzcat tmp/#{file} | psql #{local_db_dev}"
  14    `#{depackage}`
  15  end
« Newer Snippets
Older Snippets »
Showing 1-10 of 20 total  RSS