Linux Tips and Tricks

Linux Tips and Tricks

Sort command and its usage

September14

we have a file called alpha.txt and it contains the below characters.

1
2
3
4
5
6
7
8
9
10

$ cat alpha.txt 
A
Z
c
N
K
O
c
p
l

Sort the data alphabetically.
Note : data’s are sorted in ASCII values. ( A-65, p-112 )

1
2
3
4
5
6
7
8
9
10

$ sort alpha.txt 
A
K
N
O
Z
c
c
l
p

Now, we are using -f to ignore the case.

1
2
3
4
5
6
7
8
9
10

$ sort -f alpha.txt 
A
c
c
K
l
N
O
p
Z

-r is used to reverse the output.

1
2
3
4
5
6
7
8
9
10

$ sort -r -f alpha.txt 
Z
p
O
N
l
K
c
c
A

when we sort the numerical values, we need to -n option.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

$ cat num.txt 
33
21
35
18
9
0
24
12
 
$ sort -n num.txt 
0
9
12
18
21
24
33
35

Store the output in a new file. ( use -o option )

1
2
3
4
5
6
7
8
9
10
11

$ sort -o mynumoutput.txt -n num.txt 
 
$ cat mynumoutput.txt 
0
9
12
18
21
24
33
35

Sort unique data ( Omit the duplicate values ) 

1
2
3
4
5
6
7
8
9
10
11
12

$ cat duplicate.txt 
A
A
B
C
B
C
 
$ sort -u duplicate.txt 
A
B
C

Lets see how to sort the data by nth column

1
2
3
4
5
6

$ cat numeric.txt 
9723	10741	8605	30693
497	4017	21864	10039
9110	20910	626	3916
18359	22324	16905	7394
15708	3364	18818	31782

sort by 2rd column ( use -k2, -n for numeric )

1
2
3
4
5
6

$ sort -n -k2 numeric.txt 
15708	3364	18818	31782
497	4017	21864	10039
9723	10741	8605	30693
9110	20910	626	3916
18359	22324	16905	7394

sort by 3rd column ( use -k3, -n for numeric )

1
2
3
4
5
6

$ sort -n -k3 numeric.txt 
9110	20910	626	3916
9723	10741	8605	30693
18359	22324	16905	7394
15708	3364	18818	31782
497	4017	21864	10039

Now, we will see how to sort the data which is delimted by : (colon)

I simply, use the same numeric.txt file and i replaced all the \t (tab seperated) to : (colon)

1
2
3
4
5
6
7
8

$ sed -i 's/\t/:/g' numeric.txt 
 
$ cat numeric.txt 
9723:10741:8605:30693
497:4017:21864:10039
9110:20910:626:3916
18359:22324:16905:7394
15708:3364:18818:31782

Now sort by the 4th column. (use -k4 and -n for sort and -t for delimeter )

1
2
3
4
5
6

$ sort -t: -n -k4 numeric.txt 
9110:20910:626:3916
18359:22324:16905:7394
497:4017:21864:10039
9723:10741:8605:30693
15708:3364:18818:31782
posted under Uncategorized | No Comments »

Remove empty lines from file

September12

Lets say we have two empty lines in the file (blank_lines.txt)

1
2
3
4
5
6

$ cat -n blank_lines.txt
     1    this is test file
     2
     3    for blank lines
     4
     5    we have two blank lines in this file

Using grep

1
2
3
4

$ grep . blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

using awk

1
2
3
4

$ awk '/./ {print}' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4

s$ awk '!/^$/ {print}'  blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

using sed

1
2
3
4

$ sed '/^$/d' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

using perl

1
2
3
4

$ perl -lane 'print $_ if($_=~/./)' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4
5
6

$ perl -pi -e "s/^\n//" blank_lines.txt
 
$ cat blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4

$ perl -lane 'print $_ if($_!~m/^$/)' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4

$ perl -lane 'print $_ if($_ ne "")' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

Note : If your blank line contains the spaces, then you need to use /^\s*$/

posted under Uncategorized | No Comments »

Remove duplicate lines from file using bash and perl

September11

we have a file called dupl and it has some duplicate entires

1
2
3
4
5
6
7

$ cat dupl
39901
39901
39902
39903
39902
39003

we can use sort and uniq command to remove the duplicate entires

1
2
3
4
5

$ sort dupl | uniq
39003
39901
39902
39903

using AWK

1
2
3
4
5

$ awk '{a[$1]}END{for (i in a) print i}' dupl
39901
39902
39903
39003

Using Perl

1
2
3
4
5

$  perl -ne '$s{$_}++||print' dupl
39901
39902
39903
39003
posted under Uncategorized | No Comments »

Calculate Average of the column using bash and perl

September11

Create a file with some numbers

1
2
3
4
5
6
7
8
9
10
11
12

$ echo "5
6.5
8
9.2
> 13" > calculate.txt
 
$ cat calculate.txt 
5
6.5
8
9.2
13

Using AWK

1
2

$ awk '{sum+=$1}END{print sum/NR}' calculate.txt 
8.34

Using Perl

1
2

$ perl -lane '$a+=$_;END{print $a/$.}' calculate.txt
8.34
posted under Uncategorized | No Comments »

Print Line Numbers and File contents

September10

Create a file called printNum.txt

1
2
3
4
5
6

$ echo "A
> B
> C
> D
> E
> F" > printNum.txt

Display the file using the cat command

1
2
3
4
5
6
7

$ cat printNum.txt 
A
B
C
D
E
F

Put numbers in all the lines using cat command

1
2
3
4
5
6
7

$ cat -n printNum.txt 
     1	A
     2	B
     3	C
     4	D
     5	E
     6	F

Put number in all the lines using awk command

1
2
3
4
5
6
7

$ awk '{print NR,$0}' printNum.txt 
1 A
2 B
3 C
4 D
5 E
6 F

Using Perl

1
2
3
4
5
6
7

$ perl -lane 'print "$. $_" ;' printNum.txt
1 A
2 B
3 C
4 D
5 E
6 F

Using grep command

1
2
3
4
5
6
7

$ cat my_test
NUM_SCENARIOS=100
REP_NUM_SCENARIOS=10
 
$ grep -n . my_test
1:NUM_SCENARIOS=100
2:REP_NUM_SCENARIOS=10
posted under Uncategorized | No Comments »

Lowercase to Uppercase in shell script and perl

September10

Shell Script

1
2
3
4
5
6
7
8
9
10
11
12
13

$ echo "abcd" | tr '[:lower:]' '[:upper:]'
ABCD
 
$ echo "Test Message" > ltou.txt
 
$ cat ltou.txt
Test Message
 
$ tr '[:lower:]' '[:upper:]' < ltou.txt
TEST MESSAGE
 
$ tr '[:upper:]' '[:lower:]' < ltou.txt
test message

Using Perl

1
2
3
4
5
6
7
8

$ echo "abcd" | perl -lane 'print uc($_);' 
ABCD 
 
$ perl -lane 'print uc($_);' ltou.txt 
TEST MESSAGE
 
$ perl -lane 'print lc($_);' ltou.txt
test message
posted under Uncategorized | No Comments »

Palindrome in Shell Script and Perl

September10

Shell Script

Make sure you have the rev command in your system

1
2
3
4
5
6
7
8
9
10
11
12
13

#!/bin/sh
 
echo -n "Enter the Word : "
read word
 
reverse=$(echo $word | rev)
 
if [ "$word" = "$reverse" ]
then
	echo "The given word \"$word\" is Palindrome"
else
	echo "The given word \"$word\" is not a Palindrome"
fi

Using Perl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

#!/usr/bin/perl
 
print "Enter the Word : ";
 
#Get Input from user
chomp($word = <>); 
 
#reverse the word
$reverseWord = reverse ($word);
 
#check the reversed word and user input
if ( $reverseWord eq $word )
{
   print "$word is Palindrome\n";
}
else
{
  print "$word is not Palindrome\n";
}

Perl One Liner

1
2
3
4
5

$ echo "MadaM" | perl -lane 'print $_ if(reverse($_) eq $_)';
MadaM
 
$ echo "Test" | perl -lane 'print $_ if(reverse($_) eq $_)';
$
posted under Uncategorized | No Comments »

Swap the column values

August25

$ cat swap.txt
a b
c d
e f
g h
i j
k l
$ awk ‘{print $2,$1}’ swap.txt
b a
d c
f e
h g
j i
l k

 

posted under Uncategorized | No Comments »

Find empty Directory and Files

August25

$ find . -empty -ls

536650 4 drwxr-xr-x 2 kamaraj kamaraj 4096 Aug 25 20:56 ./saa
540876 0 -rw-r–r– 1 kamaraj kamaraj 0 Jan 16 2011 ./9
520317 4 drwxr-xr-x 2 kamaraj kamaraj 4096 Apr 10 02:27 ./.abc
520314 0 -rw-r–r– 1 kamaraj kamaraj 0 Jul 31 16:41 ./empty.txt
536649 4 drwxr-xr-x 2 kamaraj kamaraj 4096 Aug 25 20:56 ./empty
521312 0 -rw-r–r– 1 kamaraj kamaraj 0 Jan 16 2011 ./9.5
540800 0 -rw-r–r– 1 kamaraj kamaraj 0 Jan 16 2011 ./core

posted under Uncategorized | No Comments »

List the Directory Name

March11

We can use the below commands to display only the directory names

$ echo */
Scripts/ unix-commands-cheat-sheet_files/

$ printf “%s\n” */
Scripts/
unix-commands-cheat-sheet_files/

$ ls -d */
Scripts/  unix-commands-cheat-sheet_files/

$ ls -l | grep ^d
drwxr-xr-x 2 kamaraj kamaraj    4096 2011-03-11 21:25 Scripts
drwxr-xr-x 4 kamaraj kamaraj    4096 2011-02-09 01:11 unix-commands-cheat-sheet_file

$ find . -maxdepth 1 -type d
.
./unix-commands-cheat-sheet_files
./Scripts

posted under Uncategorized | No Comments »
« Older EntriesNewer Entries »

Recent Comments

    Categories