DZone Snippets: duplicates code

Identify duplicates in an array

bshow (Bob Showalter) — Thu, 19 Apr 2007 00:21:22 GMT

Method to return items that appear more than once in an array (or Enumerable)



module Enumerable

  def dups

    inject({}) {|h,v| h[v]=h[v].to_i+1; h}.reject{|k,v| v==1}.keys

  end

end

How it works: the inject method builds up a hash with keys for the unique values in the array and
a value representing the number of times the entry is found in the array. Any entries with a
value of 1 (i.e. not duplicated) are removed, and the resulting keys represents the duplicated
entries in the original array.

Example:



arr = %w{foo bar baz bar baz qux foo zub}

puts arr.dups.inspect

# => ["baz", "foo", "bar"]

"uniq"

pasang_lhamu (Pasang Lhamu Sherpa) — Sat, 04 Jun 2005 14:56:16 GMT

you don't need an extra hash.



my @uniq = keys %{{ map { $_ => 1 } @list }};

'uniq' array

TMtm (Tony Bowden) — Mon, 30 May 2005 21:51:55 GMT

A much more idiomatic and efficient version of the 'uniq array' code at 152 is:



  # Input: @list

  # Output: @uniqed

  my %seen;

  my @uniqed = grep !$seen{$_}++, @list;

Removing duplicates from lists

keith (Keith Gaughan) — Thu, 21 Apr 2005 22:09:23 GMT

A request for this came up on the CF-Talk list recently. Here's what I came up with.

This sort of this should be familiar to all you perlmongers out there.

Efficient 'uniq' snippet

jmason (Justin Mason) — Wed, 13 Apr 2005 03:26:57 GMT

This snippet will quickly uniq an unsorted array; in other words, remove duplicates. (note: for large arrays, it may be more efficient to sort the array first and simply keep track of consecutive dups instead of using a hash.)



  # Input: @list

  # Output: @uniqed

  my %u = ();

  @uniqed = grep {defined} map {

      if (exists $u{$_}) { undef; } else { $u{$_}=undef;$_; }

    } @list;

  undef %u;