Remove duplicate lines from file using bash and perl
September11
we have a file called dupl and it has some duplicate entires
1 2 3 4 5 6 7 | $ cat dupl 39901 39901 39902 39903 39902 39003 |
we can use sort and uniq command to remove the duplicate entires
1 2 3 4 5 | $ sort dupl | uniq 39003 39901 39902 39903 |
using AWK
1 2 3 4 5 | $ awk '{a[$1]}END{for (i in a) print i}' dupl 39901 39902 39903 39003 |
Using Perl
1 2 3 4 5 | $ perl -ne '$s{$_}++||print' dupl 39901 39902 39903 39003 |
Recent Comments