Awk and Grep

we are going to see how the grep and awk works for the same functionality

1
2
3
4
5
6
7
8
9
10
11

$ cat test.txt
one
two
three
four
five
six
seven
eight
nine
ten

1) Grep the pattern using grep and awk

1
2
3
4
5
6
7
8
9

$ grep "o" test.txt
one
two
four
 
$ awk '/o/' test.txt
one
two
four

2) Print the pattern line number using grep and awk

1
2
3
4
5
6
7
8
9

$ grep -n "o" test.txt
1:one
2:two
4:four
 
$ awk '/o/ {print NR":"$0}' test.txt
1:one
2:two
4:four

3) Print the number of occurences of in a file using grep and awk

1
2
3
4
5

$ grep -c "o" test.txt
3
 
$ awk '/o/ {i++}END{print i}' test.txt
3

4) Invert the sense of matching, to select non-matching lines using grep and awk

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

$ grep -v "o" test.txt
three
five
six
seven
eight
nine
ten
 
$ awk '!/o/' test.txt
three
five
six
seven
eight
nine
ten

5) Print the filename if pattern found in the file.

1
2
3
4
5
6
7
8
9
10
11

$ grep -l "one" *
armstrong
armstrong1
test.pl
test.txt
 
$ awk '$0~/one/{a[FILENAME]=1}END{for(i in a)print i}' *
armstrong1
test.pl
test.txt
armstrong

6) grep multiple patterns using grep and awk

1
2
3
4
5
6
7

$ grep -E "one|two" test.txt
one
two
 
$ awk '/one/ || /two/' test.txt
one
two

Do’s and Dont’s in shell scripting

1) Dont use Grep and Cat command together

Wrong way :

1

$cat filename | grep "pattern"

Correct way :

1

$grep "pattern" filename

2) Dont use cat,grep and wc commands together

Wrong way :

To count the number of occurence

1

$cat filename | grep "pattern" | wc -l

Correct way :

1

$grep -c "pattern" filename

3) dont use sort and uniq together ( to eliminate the duplicate entires )

Wrong way :

1

$sort filename | uniq

Correct way :

1

$sort -u filename

4) Dont use grep and awk together

wrong way :

1

$ grep "pattern" filename | awk '{print $1}'

correct way :

1

$ awk '/pattern/ {print $1}' filename

5) Dont use ls in for loop ( to manipulate the files )

wrong way :

1
2

for i in `ls` ; do echo $i; done
for i in `ls *.txt`; do echo $i; done

Correct way :

1
2

for i in *; do echo $i; done
for i in *.txt; do echo $i; done

Hi,

we all started to celebrate the Christmas. Today we will see the command xsnow

In the ubuntu system, type the below command to install the xsnow command

$sudo apt-get install xsnow

Once the package is downloaded and installed, open the console and type the below command.

$xsnow

Once you executed the command, you can see wonderful snow flakes in your desktop and santa claus will be traveling in his moose vehicle and you can see Christmas trees.

you have the blow options in this command.

1) change the snow flakes count ( default is 100 )
2) change the snow flakes color
3) change the tree color
4) background color
5) santa size
6) no trees, no santa

If you want to execute this command in background, then type the below command.

$xsnow &

If you want to stop the snow, then type kill the xsnow PID

ps -ef | awk '/xsnow$/ {print $2}'

Once you found the PID of the xsnow, then issue the kill command.

kill -i PID

Happy Christmas 🙂

aria2 is a lightweight multi-protocol & multi-source download utility operated in command-line. It supports HTTP/HTTPS, FTP, BitTorrent and Metalink. aria2 has built-in JSON-RPC and XML-RPC interface. You can manipulate aria2 via these interfaces.

How to install the aria2 ?

In ubuntu system type the below command.

1

$ sudo apt-get install aria2

Once the aria2 is installed, you can use this to download the files from the url.

Download the awk one liner script

1

$ aria2c http://www.pement.org/awk/awk1line.txt

Download from 2 websites ( http & ftp )

1

$ aria2c http://one/a.txt ftp://two/b.txt

Download using 2 connections per host

1

$ aria2c -x2 http://one/a.txt

BitTorrent

1

$ aria2c http://test.com/test.torrent

Metalink

1

$ aria2c http://test.org/one.metalink

Download URIs found in text file

1

$ aria2c -i url.txt

Today we are going to see, how to create a ringtone from the MP3 file.

we are going to use the command called “cutmp3”

open the Terminal and type “cutmp3” ( without double quotes )

1
2
3
4
5
6

$ cutmp3
 
cutmp3  version 2.0.2  
 
Usage:  cutmp3 -i file.mp3 [-a inpoint] [-b outpoint] [-f timetable]
               [-o outputprefix] [-e] [-c] [-C] [-q]

If cutmp3 is not installed in your system, then type the below command ( in ubuntu )

1

$ sudo apt-get install cutmp3

Now, we can see, how to cut the mp3 file.

1
2

$ cutmp3 -i test.mp3 -a 0:01 -b 0:20
  saved 0:01.00 - 0:20.00 to 'result0001.mp3'.

-i input file. ( mp3 file )
-a starting point
-b ending point

So, in the above example, i extracted the data from 0:01 second to 0:20 seconds. And the resultant output is saved as “result0001.mp3”

If you want to give the output filename, then use -O (upper case) option.

1
2
3

 
$ cutmp3 -i test.mp3 -a 0:01 -b 0:20 -O output.mp3
  saved 0:01.00 - 0:20.00 to 'output.mp3'.

we have some other aruments also

1
2

-I print the information of the mp3 file.
-f filename ( filename should have the valid time frame. eg: 0:01 1:03 )

we can use cat command and od commands to see the non-printable characters.

Let me explain how to create some non printable characters.

1) open the console and type the below.
2) echo “Hello <ctrl + v> <ctrl +m> World” > myfile.txt

Note :– <ctrl+v> means press control key and v in the keyboard.

3) Once the file is created, use cat command to view the file contents

1
2
3
4

$ echo "Hello ^M World" > myfile.txt
 
$ cat myfile.txt
 World

4)As you notice, Hello word is disappeared.

5) use the -v option in the cat command.

1
2

$ cat -v myfile.txt
Hello ^M World

6) Now you can see the word “Hello” but after that you can see some special character (^M) which is not printed while using the cat command
7) now we can use od command to see the special characters.

1
2
3

$ od -c myfile.txt 
0000000   H   e   l   l   o      \r       W   o   r   l   d  \n
0000016

You can clearly see there is come special character \r present between the Hello and World.

Control-M is carriage return without linefeed.

How to remove the special characters ?

In this case, we can remove \r (which is clearly found by od -c command )

1
2
3
4
5
6
7

$ cat -v myfile.txt 
Hello ^M World
 
$ sed -i 's/\r//' myfile.txt 
 
$ cat -v myfile.txt 
Hello  World