Linux Tips and Tricks

Linux Tips and Tricks

Bash Tricks

September25

Bash Tricks

list the alphabets ‘A’ to ‘Z’

1
2

$ echo {A..Z}
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

list the alphabets ‘Z’ to ‘A’

1
2

$ echo {Z..A}
Z Y X W V U T S R Q P O N M L K J I H G F E D C B A

list the alphabets ‘a’ to ‘z’

1
2

$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z

list the alphabets ‘z’ to ‘a’

1
2

$ echo {z..a}
z y x w v u t s r q p o n m l k j i h g f e d c b a

list the numbers 1 to 15

1
2

$ echo {1..15}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Adding number in the back

1
2

$ echo {1..15}1
11 21 31 41 51 61 71 81 91 101 111 121 131 141 151

Adding number in front and back.

1
2

$ echo 2{1..15}1
211 221 231 241 251 261 271 281 291 2101 2111 2121 2131 2141 2151

Apply it for copy command

1
2
3
4
5

$ cp {A,C}.txt
 
$ ls -lrt [AC].txt
-rw-r--r-- 1 kamaraj kamaraj 12 Sep  6 23:08 A.txt
-rw-r--r-- 1 kamaraj kamaraj 12 Sep 25 22:52 C.txt
posted under Uncategorized | 2 Comments »

Compare md5 of two files in shell script

September21

What is MD5?
The MD5 Message-Digest Algorithm is a widely used cryptographic hash function that produces a 128-bit (16-byte) hash value. Specified in RFC 1321, MD5 has been employed in a wide variety of security applications, and is also commonly used to check data integrity. <<More>>

md5deep or md5sum commands are used to get the MD5 value for the files.

If you dont have these commands in your ubuntu, then type the below command and install the md5deep package.

1
2
3
4
5
6
7
8
9
10
11
12
13

$ sudo apt-get install md5deep
[sudo] password for kamaraj:
Reading package lists... Done
Building dependency tree       
Reading state information... Done
................
......................
Selecting previously deselected package md5deep.
(Reading database ... 251462 files and directories currently installed.)
Unpacking md5deep (from .../md5deep_3.4-3_i386.deb) ...
Processing triggers for man-db ...
Setting up md5deep (3.4-3) ...
$

Once installed, type the below command and check whether you have md5deep and md5sum commands in your system.

1
2
3
4
5
6
7
8
9
10
11

$ md5sum --version
md5sum (GNU coreutils) 7.4
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Written by Ulrich Drepper, Scott Miller, and David Madore.
 
$ md5deep -v       
3.4

Now create some text files or take any existing file. Now, we will find out the MD5 value for that file.

I have a file called A.txt and now we will find out the MD5 value for that file.

1
2
3
4
5
6
7
8
9
10
11
12
13

$ cat A.txt
A
B
C
D
E
F
 
$ md5sum A.txt
6af4cbb084064df395d07ecac556de95  A.txt
 
$ md5deep A.txt
6af4cbb084064df395d07ecac556de95  /home/kamaraj/Desktop/Scripts/A.txt

How to compare the two MD5 values?

1
2
3
4
5
6
7
8
9
10
11

$ a=$(md5deep -q A.txt)
 
$ b=$(md5deep -q B.txt)
 
$ [ "$a" = "$b" ] && echo  "Equal" || echo "Not Equal"
Equal
 
$ b=$(md5deep -q  C.txt)
 
$ [ "$a" = "$b" ] && echo  "Equal" || echo "Not Equal"
Not Equal
posted under Uncategorized | No Comments »

Sending email via gmail in linux

September19

Sending email via gmail in linux

1) Make sure you have Heirloom mailx. How to test it ?
simply enter the below code in your command prompt

1
2
3

$mailx -v
mail: invalid option -- v
Try `mail --help' or `mail --usage' for more information.

If you get the above error message, then you need to install the Heirloom mailx.

2) How to install it ? (i am using ubuntu)

1
2
3
4
5
6
7
8
9
10
11
12
13
14

$ sudo apt-get install heirloom-mailx
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libgsasl7 libmailutils2 libntlm0
................
.............
................
The following packages will be REMOVED:
mailutils
The following NEW packages will be installed:
heirloom-mailx
................

3) Once the package is installed, check whether you have any file called .mailrc in your home directory.

1
2

$ ls -l ~/.mailrc
ls: cannot access /home/kamaraj/.mailrc: No such file or directory

4) If you have any file, then just open and put the below contents. Otherwise create a new file called .mailrc

1
2
3
4
5
6
7
8

account gmail {
set smtp-use-starttls
set smtp=smtp://smtp.gmail.com:587
set smtp-auth=login
set smtp-auth-user=abcd@gmail.com
set smtp-auth-password=mypassword
set from="ABCD "
}

5) Just change the contents of the below lines.

1
2
3

set smtp-auth-user=YOUR-GMAIL-ID
set smtp-auth-password=YOUR-GMAIL-PASSWORD
set from="NAME"

6) Its the time to test. Now trigger the below command from your command prompt.

1

 
1
2
3
4
5
6
7
8
9
10
11

$ echo "Test Email" | mailx -v -A gmail -s "test" yyyyyyyy@gmail.com
Resolving host smtp.gmail.com . . . done.
Connecting to 74.125.45.109 . . . connected.
250 2.0.0 OK 1316447647 s8sm2760907ani.3
 
...............
............
............
 
 >>> QUIT
 221 2.0.0 closing connection s8sm2760907ani.3

7) Now check your gmail inbox (sent items).

posted under Uncategorized | 4 Comments »

Passwordless FTP setup – Linux

September15

Normally when you are using ftp command, it will ask username and password to connect to that specified ftp host.

we will see how to setup the password less FTP setup.

Today i installed the FTP server in my ubuntu.

In the terminal type the below command and provide the password to install the FTP server.

1

$ sudo apt-get install vsftpd

After that open the below file and modify the anonymous_enable value to YES

1
2
3
4

$ sudo vi /etc/vsftpd.conf
 
# Allow anonymous FTP? (Disabled by default)
anonymous_enable=YES

Now restart the vsftpd service by using the below command.

1
2
3
4
5
6
7
8
9
10

#before restart - PID is 3599
$ ps -ef | grep vsftpd
root      3599     1  0 23:25 ?        00:00:00 /usr/sbin/vsftpd
 
$ sudo restart vsftpd
vsftpd start/running, process 3691
 
#after restart - PID is 3691
$ ps -ef | grep vsftpd
root      3691     1  0 23:33 ?        00:00:00 /usr/sbin/vsftpd

All set to go. Try to connect to your localhost.

username : anonymous
password : anonymous

1
2
3
4
5
6
7
8
9
10

$ ftp localhost
Connected to localhost.
220 (vsFTPd 2.2.2)
Name (localhost:kamaraj): anonymous
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

So, now successfully connected to our localhost ftp server.

Now, we will see how to setup passwordless ftp configuration.
It is very simple.

create a file .netrc under your home directory.And put the contents which is shown as below.

The format is : machine login password 

1
2
3
4
5

$ pwd
/home/kamaraj
 
$ cat .netrc 
machine localhost login anonymous password anonymous

Note: Make sure .netrc is atleast having the read permission (400) permission for the user.

That’s all.

Now, we will try to login into localhost.

1
2
3
4
5
6
7
8

$ ftp localhost
Connected to localhost.
220 (vsFTPd 2.2.2)
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

This time ftp didn’t ask the username and password.

Have Fun 🙂

posted under Uncategorized | No Comments »

Sort command and its usage

September14

we have a file called alpha.txt and it contains the below characters.

1
2
3
4
5
6
7
8
9
10

$ cat alpha.txt 
A
Z
c
N
K
O
c
p
l

Sort the data alphabetically.
Note : data’s are sorted in ASCII values. ( A-65, p-112 )

1
2
3
4
5
6
7
8
9
10

$ sort alpha.txt 
A
K
N
O
Z
c
c
l
p

Now, we are using -f to ignore the case.

1
2
3
4
5
6
7
8
9
10

$ sort -f alpha.txt 
A
c
c
K
l
N
O
p
Z

-r is used to reverse the output.

1
2
3
4
5
6
7
8
9
10

$ sort -r -f alpha.txt 
Z
p
O
N
l
K
c
c
A

when we sort the numerical values, we need to -n option.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

$ cat num.txt 
33
21
35
18
9
0
24
12
 
$ sort -n num.txt 
0
9
12
18
21
24
33
35

Store the output in a new file. ( use -o option )

1
2
3
4
5
6
7
8
9
10
11

$ sort -o mynumoutput.txt -n num.txt 
 
$ cat mynumoutput.txt 
0
9
12
18
21
24
33
35

Sort unique data ( Omit the duplicate values ) 

1
2
3
4
5
6
7
8
9
10
11
12

$ cat duplicate.txt 
A
A
B
C
B
C
 
$ sort -u duplicate.txt 
A
B
C

Lets see how to sort the data by nth column

1
2
3
4
5
6

$ cat numeric.txt 
9723	10741	8605	30693
497	4017	21864	10039
9110	20910	626	3916
18359	22324	16905	7394
15708	3364	18818	31782

sort by 2rd column ( use -k2, -n for numeric )

1
2
3
4
5
6

$ sort -n -k2 numeric.txt 
15708	3364	18818	31782
497	4017	21864	10039
9723	10741	8605	30693
9110	20910	626	3916
18359	22324	16905	7394

sort by 3rd column ( use -k3, -n for numeric )

1
2
3
4
5
6

$ sort -n -k3 numeric.txt 
9110	20910	626	3916
9723	10741	8605	30693
18359	22324	16905	7394
15708	3364	18818	31782
497	4017	21864	10039

Now, we will see how to sort the data which is delimted by : (colon)

I simply, use the same numeric.txt file and i replaced all the \t (tab seperated) to : (colon)

1
2
3
4
5
6
7
8

$ sed -i 's/\t/:/g' numeric.txt 
 
$ cat numeric.txt 
9723:10741:8605:30693
497:4017:21864:10039
9110:20910:626:3916
18359:22324:16905:7394
15708:3364:18818:31782

Now sort by the 4th column. (use -k4 and -n for sort and -t for delimeter )

1
2
3
4
5
6

$ sort -t: -n -k4 numeric.txt 
9110:20910:626:3916
18359:22324:16905:7394
497:4017:21864:10039
9723:10741:8605:30693
15708:3364:18818:31782
posted under Uncategorized | No Comments »

Remove empty lines from file

September12

Lets say we have two empty lines in the file (blank_lines.txt)

1
2
3
4
5
6

$ cat -n blank_lines.txt
     1    this is test file
     2
     3    for blank lines
     4
     5    we have two blank lines in this file

Using grep

1
2
3
4

$ grep . blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

using awk

1
2
3
4

$ awk '/./ {print}' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4

s$ awk '!/^$/ {print}'  blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

using sed

1
2
3
4

$ sed '/^$/d' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

using perl

1
2
3
4

$ perl -lane 'print $_ if($_=~/./)' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4
5
6

$ perl -pi -e "s/^\n//" blank_lines.txt
 
$ cat blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4

$ perl -lane 'print $_ if($_!~m/^$/)' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file
1
2
3
4

$ perl -lane 'print $_ if($_ ne "")' blank_lines.txt
this is test file
for blank lines
we have two blank lines in this file

Note : If your blank line contains the spaces, then you need to use /^\s*$/

posted under Uncategorized | No Comments »

Remove duplicate lines from file using bash and perl

September11

we have a file called dupl and it has some duplicate entires

1
2
3
4
5
6
7

$ cat dupl
39901
39901
39902
39903
39902
39003

we can use sort and uniq command to remove the duplicate entires

1
2
3
4
5

$ sort dupl | uniq
39003
39901
39902
39903

using AWK

1
2
3
4
5

$ awk '{a[$1]}END{for (i in a) print i}' dupl
39901
39902
39903
39003

Using Perl

1
2
3
4
5

$  perl -ne '$s{$_}++||print' dupl
39901
39902
39903
39003
posted under Uncategorized | No Comments »

Calculate Average of the column using bash and perl

September11

Create a file with some numbers

1
2
3
4
5
6
7
8
9
10
11
12

$ echo "5
6.5
8
9.2
> 13" > calculate.txt
 
$ cat calculate.txt 
5
6.5
8
9.2
13

Using AWK

1
2

$ awk '{sum+=$1}END{print sum/NR}' calculate.txt 
8.34

Using Perl

1
2

$ perl -lane '$a+=$_;END{print $a/$.}' calculate.txt
8.34
posted under Uncategorized | No Comments »

Print Line Numbers and File contents

September10

Create a file called printNum.txt

1
2
3
4
5
6

$ echo "A
> B
> C
> D
> E
> F" > printNum.txt

Display the file using the cat command

1
2
3
4
5
6
7

$ cat printNum.txt 
A
B
C
D
E
F

Put numbers in all the lines using cat command

1
2
3
4
5
6
7

$ cat -n printNum.txt 
     1	A
     2	B
     3	C
     4	D
     5	E
     6	F

Put number in all the lines using awk command

1
2
3
4
5
6
7

$ awk '{print NR,$0}' printNum.txt 
1 A
2 B
3 C
4 D
5 E
6 F

Using Perl

1
2
3
4
5
6
7

$ perl -lane 'print "$. $_" ;' printNum.txt
1 A
2 B
3 C
4 D
5 E
6 F

Using grep command

1
2
3
4
5
6
7

$ cat my_test
NUM_SCENARIOS=100
REP_NUM_SCENARIOS=10
 
$ grep -n . my_test
1:NUM_SCENARIOS=100
2:REP_NUM_SCENARIOS=10
posted under Uncategorized | No Comments »

Lowercase to Uppercase in shell script and perl

September10

Shell Script

1
2
3
4
5
6
7
8
9
10
11
12
13

$ echo "abcd" | tr '[:lower:]' '[:upper:]'
ABCD
 
$ echo "Test Message" > ltou.txt
 
$ cat ltou.txt
Test Message
 
$ tr '[:lower:]' '[:upper:]' < ltou.txt
TEST MESSAGE
 
$ tr '[:upper:]' '[:lower:]' < ltou.txt
test message

Using Perl

1
2
3
4
5
6
7
8

$ echo "abcd" | perl -lane 'print uc($_);' 
ABCD 
 
$ perl -lane 'print uc($_);' ltou.txt 
TEST MESSAGE
 
$ perl -lane 'print lc($_);' ltou.txt
test message
posted under Uncategorized | No Comments »
« Older Entries

TheLinuxTips.com >> Place to share your ideas and tricks


Recent Comments

    Categories