Execute Shell in Jenkins - Update Permissions Example

Hello guys. How's it going? In today's post, let us look into an example of how to execute a shell command in Jenkins.

For illustration, let us consider a Java based project which is under IBM Clearcase source control and built using Maven on a Linux machine. Now we want to execute a bash command to update the permissions of all pom.xml files (i.e Parent pom's and Child pom's).

Check a few sample commands for reference :

find $WORKSPACE/$CLEARCASE_VIEWNAME/PROJECT_SRC_CVOB/SRC -type f -name pom.xml -exec chmod 755 {} \;

find $WORKSPACE/$CLEARCASE_VIEWNAME/PROJECT_SRC_CVOB/SRC -type f -name pom.xml -exec chmod 444 {} \;

find $WORKSPACE/$CLEARCASE_VIEWNAME/PROJECT_SRC_CVOB/SRC -type f -name pom.xml -exec chmod 777 {} \;

Hope it helps. Ciao!

Automate regular backups of SonarQube's DB using bash script

Guys, this post will be useful if you are using SonarQube for code analysis and quality control. In today's post we will look into how to automate taking regular backups of SonarQube's back-end database.

For illustration, we will use MySql DB to store SonarQube's metadata which includes analysis parameters like code smells, bugs, code coverage, unit test coverage etc. In this case we host Sonar on a linux machine, so we write a bash script to take regular backups of the MySql DB. Take a look at a simple sample script below :

#!/bin/bash
# script for taking database dump of sonar

user="root";
hostname="localhost";
db_name="sonar";
dateformat=`date +%d-%m-%y`;

mysqldump --defaults-extra-file="/usr/local/cron_scripts/.my.cnf" -u $user -h $hostname sonar | gzip > /usr/local/mysql_dumps/sonar_dump_$dateformat.sql.gz

chmod 755 /usr/local/mysql_dumps/sonar_dump_$dateformat.sql.gz

As you see, we have used mysqldump to take dump of the database, zip it, append it with the current date and store it at the said location.

Taking dumps of Sonar can be extremely useful for the purpose of importing and exporting of a Sonar instance from one machine to another without losing the analysis history.

Print a triangle using * in a Bash script

Hello coders! Hows it going? Today we'll see a basic simple shell script to print a triangle using asterisk * only. Here's the code to do that :

#!/bin/bash

# Simple program to print a triangle using * of height 10

star="*"

for i in {1..10}

do

echo "$star";

star="$star *"

done

You see basically, all we are doing is append a * to itself recursively. Here we are using a basic for-loop to print a triangle of height 10. You can change the height as per your requirement, as needed.

Output :

*

* *

* * *

* * * *

* * * * *

* * * * * *

* * * * * * *

* * * * * * * *

* * * * * * * * *

* * * * * * * * * *

Cheers!

Identify exact port number used by a process or app on Mac OSX

Guys, if you ever wanted to find out the exact specific port number used by a process/application on Unix/Mac OSX/Linux, type the following in Terminal :

top | awk '{print $2, $7}'

The output would be something like :

COMMAND #PORTS

awk 12

top 22+

top 19

This will show only the specific Command and Port columns from the top command and will skip other details like Process ID, % CPU utilisation , Memory usage etc.

Alternatively if you wanted to just find the port number used by a specific application, say for e.g. TextEdit, you could use the following command :

top | awk '{print $2, $7}' | grep TextEdit

TextEdit 140+

TextEdit 140

TextEdit 140

Cheers guys!

Read file line by line and count no. of lines using Shell script

Guys, in today's post we'll see how to parse a file line by line and count the number of lines in it.

We will simply use redirection to read the file. Note that we will pass the filename as a runtime parameter to the script.

Alright here we go. Suppose I have a file TextFile1.txt with the following contents. I want to print the total number of lines in it.

ABCD

DEFG

GHI

JKL

MNO

PQR

STU

VWXYZ

Now, here's the script part.

#!/bin/bash

#Sript to count number of lines in a file

File=$1

count=0

while read LINE

do

let count++

echo "$count $LINE"

done < $File

echo -e "Total $count Lines read\n"

Here's the output :

1 ABCD

2 DEFG

3 GHI

4 JKL

5 MNO

6 PQR

7 STU

8 VWXYZ

Total 8 Lines read

Cheers it has correctly displayed the count of lines in the file. In case of any questions/doubts, kindly leave your comments.

Search for string in a file and print the line : Unix basics

Guys, lets look into a very basic topic today. Today we'll see how to search for a string in a file and print the line in Unix/Linux/Mac OS X

Suppose I have a file SedText.rtf with the following text :

1.Here is a cute little tutorial for

2.trying out different things using

3.the Stream Editor or Sed in Unix.

4.The author of this tutorial is

5.none other than IroncladWriter

6.himself. The tutorial aims to

7.teach folks and educate how

8.they could unlock the power

9.of Unix to the fullest. Am posting

10.this on the blog IronCladZone

11.The best blog on the whole of

12.internet, where you can learn

13.technical stuff as well enjoy 

14.entertaining tidbits like movies,

15.music, television, fashion, food,

16.shopping, travel, trends etc.

17.Just sit back, relax and

18.ENJOY THE RIDE :)

Now I want to search this file for the keyword "Unix" and print all the lines. All we have to do is use the grep statement. Check this out :

grep Unix SedText.rtf

3.the Stream Editor or Sed in Unix.\

9.of Unix to the fullest. Am posting\

Character Combinations using Shell Script

In today's post we'll look into a very simple entry level bash script in which we form all possible alphanumeric combinations using just 2 characters and save it in a file. You could tweak the program to suit your needs - e.g. all combinations of 6 or 7 or 8 characters or maybe include special characters as well. The file in which all these combinations are stored would be more like a dictionary.

#!/bin/bash

# Program to print all possible 2-character combinations

echo {a..z}{0..9} > /Users/ironcladzone/Documents/xyz.txt

echo {0..9}{a..z} >> /Users/ironcladzone/Documents/xyz.txt

Here's the output of xyz.txt :

a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 g0 g1 g2 g3 g4 g5 g6 g7 g8 g9 h0 h1 h2 h3 h4 h5 h6 h7 h8 h9 i0 i1 i2 i3 i4 i5 i6 i7 i8 i9 j0 j1 j2 j3 j4 j5 j6 j7 j8 j9 k0 k1 k2 k3 k4 k5 k6 k7 k8 k9 l0 l1 l2 l3 l4 l5 l6 l7 l8 l9 m0 m1 m2 m3 m4 m5 m6 m7 m8 m9 n0 n1 n2 n3 n4 n5 n6 n7 n8 n9 o0 o1 o2 o3 o4 o5 o6 o7 o8 o9 p0 p1 p2 p3 p4 p5 p6 p7 p8 p9 q0 q1 q2 q3 q4 q5 q6 q7 q8 q9 r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 t0 t1 t2 t3 t4 t5 t6 t7 t8 t9 u0 u1 u2 u3 u4 u5 u6 u7 u8 u9 v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 w0 w1 w2 w3 w4 w5 w6 w7 w8 w9 x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 y0 y1 y2 y3 y4 y5 y6 y7 y8 y9 z0 z1 z2 z3 z4 z5 z6 z7 z8 z9
0a 0b 0c 0d 0e 0f 0g 0h 0i 0j 0k 0l 0m 0n 0o 0p 0q 0r 0s 0t 0u 0v 0w 0x 0y 0z 1a 1b 1c 1d 1e 1f 1g 1h 1i 1j 1k 1l 1m 1n 1o 1p 1q 1r 1s 1t 1u 1v 1w 1x 1y 1z 2a 2b 2c 2d 2e 2f 2g 2h 2i 2j 2k 2l 2m 2n 2o 2p 2q 2r 2s 2t 2u 2v 2w 2x 2y 2z 3a 3b 3c 3d 3e 3f 3g 3h 3i 3j 3k 3l 3m 3n 3o 3p 3q 3r 3s 3t 3u 3v 3w 3x 3y 3z 4a 4b 4c 4d 4e 4f 4g 4h 4i 4j 4k 4l 4m 4n 4o 4p 4q 4r 4s 4t 4u 4v 4w 4x 4y 4z 5a 5b 5c 5d 5e 5f 5g 5h 5i 5j 5k 5l 5m 5n 5o 5p 5q 5r 5s 5t 5u 5v 5w 5x 5y 5z 6a 6b 6c 6d 6e 6f 6g 6h 6i 6j 6k 6l 6m 6n 6o 6p 6q 6r 6s 6t 6u 6v 6w 6x 6y 6z 7a 7b 7c 7d 7e 7f 7g 7h 7i 7j 7k 7l 7m 7n 7o 7p 7q 7r 7s 7t 7u 7v 7w 7x 7y 7z 8a 8b 8c 8d 8e 8f 8g 8h 8i 8j 8k 8l 8m 8n 8o 8p 8q 8r 8s 8t 8u 8v 8w 8x 8y 8z 9a 9b 9c 9d 9e 9f 9g 9h 9i 9j 9k 9l 9m 9n 9o 9p 9q 9r 9s 9t 9u 9v 9w 9x 9y 9z

Area & Circumference of a circle - Shell script for Mac OS X

Guys in todays post let us look into a simple basic Mathematics problem. Let us automate the calculation of area and circumference of a circle, using a simple bash script. It's an easy script useful for learning some basic shell scripting.

Note : Value of pi used is a constant value rounded off to 3.142

#!/bin/bash

#program to calculate area and circumference of a circle

echo "=========================================="

echo "Area & Circumference of Circle Calculator"

echo "=========================================="

pi=3.142

echo "Please enter the radius of the circle : "

read radius

circumference=`expr "2*$pi*$radius"|bc`

echo "Circumference of circle : $circumference"

area=`expr "$pi*$radius*$radius"|bc`

echo "Area of circle : $area"

Output :

./AreaOfCircle.sh

==========================================

Area & Circumference of Circle Calculator

==========================================

Please enter the radius of the circle : 

5

Circumference of circle : 31.420

Area of circle : 78.550

Also do check out my previous post on Simple Interest Tool for additional scripting reference.

Script to check if a server / machine is up and running

Guys you might often come across some situations where you want to check if a server or machine is physically up and running. Quite obviously you would use the ping command to do so.

But in today's post let us write a simple bash script to automate this.

#!/bin/bash

#program to check if a server/machine is up and running

ipaddr=192.168.1.34

ping -c 4 $ipaddr >> /dev/null

if [[ $? -ne 0 ]]

then

echo "FAIL: Server seems to be down and not responding" 

else

echo "SUCCESS : Server is up and running"

fi

Note : $? is a special Unix variable to check if the previous command was executed successfully or not. It returns a numerical value - either 0 or 1. If the command was executed successfully, it returns 0, else if not, it returns 1.

In above script, simply replace the variable 'ipaddr' value to your server's ip address and check.

Also note that by redirecting the output of the ping command to /dev/null, we are suppressing and silencing the output i.e the ping o/p will not be shown on terminal. If you want to see the exact output of ping, remove the ">> /dev/null" part.

Output :

./PingServer.sh

SUCCESS : Server is up and running

Cheers!

Simple Interest Calculation : Shell Script for Mac OS X

Guys, today we'll look into a very basic simple shell script to calculate the Simple Interest. For this we need 3 essential details :

Principal Amount
Rate of Interest
Period i.e No. of years

Here's the bash script to calculate the Simple Interest.

#!/bin/bash

#program to calculate simple interest

echo "================================"

echo "Simple Interest Calculation Tool"

echo "================================"

echo "Enter principal amount : "

read principal

echo "Enter Rate of Interest : "

read roi;

echo "Enter No. of years : "

read years;

simple_interest=`expr "($principal*$years*$roi)/100"|bc`

echo "Simple Interest = $simple_interest"

Output :

./SimpleInterest.sh

================================

Simple Interest Calculation Tool

================================

Enter principal amount : 

5000

Enter Rate of Interest : 

9.5

Enter No. of years : 

5

Simple Interest = 2375

If you note, we have used the bc command to evaluate our expression i.e the Simple Interest formula. More on bc can be read here.

How to identify available shells on Mac OS X

Guys you may be aware that you can find your current shell in Mac OSX or any Unix machine by typing the following in Terminal.

echo $SHELL

/bin/bash

But to identify the available shells on Mac OS X type the following :

cat /etc/shells

# List of acceptable shells for chpass(1).

# Ftpd will not allow users to connect who are not using

# one of these shells.

/bin/bash

/bin/csh

/bin/ksh

/bin/sh

/bin/tcsh

/bin/zsh

Also, if you ant to change your default shell, type this :

chsh

Changing shell for ironcladzone.

Password for ironcladzone: 

Hope it helps. Cheers!

Double digit Arguments Passed to Shell script - Tricky Scenario

Guys, you know that $# is a special variable that gives the arguments passed to the shell script.

$1 will give the 1st argument passed, $4 will give the 4th argument passed to the script.

Let's consider a tricky scenario. 
Note : This is a tricky interview question.

Let's say you want to pass 12 arguments to the script.

Did you notice that after 9th argument, all arguments passed would be of 2 digits... I mean the 10th, 11th and 12th argument.

So let's say to get the 12th argument, you may tend to use $12. But hey look again! Thats not right! The script doesn't understand that. Check out a practical example to know what I mean.

Let's take a simple bash script and compare the output of Wrong and Correct versions.

[WRONG VERSION]

#!/bin/bash

# Call this script with 12 parameters

echo "1st parameter is $1";

echo "2nd parameter is $2";

echo "3rd paramter is $3";

echo "4th parameter is $4";

echo "5th paramter is $5";

echo "6th parameter is $6";

echo "7th parameter is $7";

echo "8th paramter is $8";

echo "9th parameter is $9";

echo "10th parameter is $10";

echo "11th parameter is $11";

echo "12th paramter is $12";

Now lets say you execute the script with 12 arguments. Watch the output :

./Parameters_Pass.sh 5 10 15 20 25 30 32 35 42 45 50 65

1st parameter is 5

2nd parameter is 10

3rd paramter is 15

4th parameter is 20

5th paramter is 25

6th parameter is 30

7th parameter is 32

8th paramter is 35

9th parameter is 42

10th parameter is 50

11th parameter is 51

12th paramter is 52

See how the 10th 11th and 12t argument that we passed got screwed up. Instead of 45, 50, 65 what we see is 50, 51 and 52. Thats's so wrong, right?

Now here's the correct version of the script.

[CORRECT VERSION]

#!/bin/bash

# Call this script with 12 parameters

echo "1st parameter is $1";

echo "2nd parameter is $2";

echo "3rd paramter is $3";

echo "4th parameter is $4";

echo "5th paramter is $5";

echo "6th parameter is $6";

echo "7th parameter is $7";

echo "8th paramter is $8";

echo "9th parameter is $9";

echo "10th parameter is ${10}";

echo "11th parameter is ${11}";

echo "12th paramter is ${12}";

Observe the brace brackets used for 10th, 11th and 12th argument. Now if you execute the script with 12 arguments again, we'll get the correct output as expected.

./Parameters_Pass.sh 5 10 15 20 25 30 32 35 42 45 50 65

1st parameter is 5

2nd parameter is 10

3rd paramter is 15

4th parameter is 20

5th paramter is 25

6th parameter is 30

7th parameter is 32

8th paramter is 35

9th parameter is 42

10th parameter is 45

11th parameter is 50

12th paramter is 65

How to change Bash History Size on Mac OS X

Guys, I hope you must be aware, that the commands that you type in terminal get saved in the history. On Mac OS X, the default size of commands history is 500 i.e it saves the most recent 500 commands that you typed in terminal in a special file named "bash_history".

Open Terminal and check for yourself.

vi ~/.bash_history

Hit escape and type this to see the history limit
:set number

Now to change this default limit of 500 to something else, lets do the following in Terminal.

sudo vi ~/.bash_profile

Enter your password.

Add this line to the file :

#history

HISTFILESIZE=1000

That's it. It means from now on, it will save upto 1000 latest commands that you typed in history. Ciao!

How to debug shell / bash scripts

While executing the bash scripts, if you want to debug them i.e you want to trace the exact sequence of steps that get executed, continue reading this post below.

Just below the shebang line on top of every script, type this :

set -x

That's it.

Let's take the example of killing a process / application using shell script that we saw in our previous post. Assuming you have opened the application - TextWrangler and want to close it using a bash script.

#!/bin/bash

set -x

kill -9 `ps -ef | grep TextWrangler | awk '{print $2}'`;

Now here's the output for the same :

++ ps -ef

++ grep TextWrangler

++ awk '{print $2}'

+ kill -9 24677 24678 24682

Note that 24677 24678 and 24682 are all PID's of all processes associated with TextWrangler. In this way you can track the flow of the script in which the tasks get executed. Debugging can be of great help if you have a complex flow of program and need to identify the sequence of steps in which they are executed.

Kill process or application using Apache Ant

Hello friends. In our last post we saw how to execute a shell script from Apache Ant. In today's post lets see if we could use Ant to kill some running processes.

For example lets open the TextWrangler app. Now lets try to close it using Ant build.

Consider a shell script kill-wrangler.sh as below :

kill -9 `ps -ef | grep TextWrangler | awk '{print $2}'`;

Now we'll execute the above bash script and kill the corresponding PID (process ID) for the TextWrangler app using the Ant build. Consider the following build.xml

<?xml version="1.0" encoding="UTF-8"?>

<project name="AntBuildTest" default="kill-process" basedir=".">

<target name="kill-process">

<exec executable="/bin/bash">

<arg value="kill-wrangler.sh"/>

</exec>

</target>

</project>

In this we execute the above bash script kill-wrangler.sh and close the application. You could grep (search) the running processes for any other program/string. For instance you could kill some Java process or maybe stop some application. Okay guys, signing out for now. Will post some more cool stuff in the forthcoming days. Stay tuned...

How to open Terminal from within Eclipse on Mac OS X

In Eclipse, from the Help menu -> Eclipse Marketplace, lookup for Terminal. Install the plugin TM Terminal that you'll come across as below:

Once installed, simply mention the shell interpreter location in the Preferences for e.g. /bin/bash and also choose the initial working directory as either your User Home or Eclipse Workspace.

Now go to Window -> Show View and choose Terminal to show it within Eclipse and click the following blue button to open a new session.

This way you can run shell scripts from within Eclipse itself. Cheers!

Call shell script from Apache Ant build on Mac OS X

Guys, you might come across some situations where you need to invoke shell and execute a bash script while building a project using Apache Ant. In today's example let's see how to run a shell script from within an Ant build.

For illustration, lets create a simple bash script to list the disk usage of our system. Let's name this file as "DiskUsage.sh"

#!/bin/bash

# Sample shell script to be invoked from an Ant build

echo "Disk usage is as follows : \n"

echo "=========================== \n"

df -h

Now let's create a simple build-test.xml using the exec command to invoke the shell as follows :

<?xml version="1.0" encoding="UTF-8"?>

<project name="AntBuildTest" default="call-shell" basedir=".">

<target name="call-shell">

<exec executable="/bin/bash">

<arg value="DiskUsage.sh"/>

</exec>

</target>

</project>

Run the ant build from command line as ant -f build-test.xml. Note that the shell script and build.xml happen to be at the same location/hierarchy within the project. If your shell script lies elsewhere, you may perhaps want to refer to its path in the build.properties instead.

You could also invoke the "expect" scripting prompt in a similar fashion to execute an expect script that we discussed in our earlier post.

More information on the exec can be read at the official Ant manual.

Change text Case using shell script

In today's post we cover a simple problem - change the case of text i.e either from lowercase to uppercase or vice versa. Let's look at a few examples.

Let's say I have a file SedText.rtf as follows :

1.Here is a cute little tutorial for

2.trying out different things using

3.the Stream Editor or Sed in Unix.

4.The author of this tutorial is

5.none other than IroncladWriter

6.himself. The tutorial aims to

7.teach folks and educate how

8.they could unlock the power

9.of Unix to the fullest. Am posting

10.this on the blog IronCladZone

11.The best blog on the whole of

12.internet, where you can learn

13.technical stuff as well enjoy 

14.entertaining tidbits like movies,

15.music, television, fashion, food,

16.shopping, travel, trends etc.

17.Just sit back, relax and

18.ENJOY THE RIDE :)

Now for converting the file into all Uppercase from the command line, use the following :

tr a-z A-Z < SedText.rtf 

Output : 

1.HERE IS A CUTE LITTLE TUTORIAL FOR\

2.TRYING OUT DIFFERENT THINGS USING\

3.THE STREAM EDITOR OR SED IN UNIX.\

4.THE AUTHOR OF THIS TUTORIAL IS\

5.NONE OTHER THAN IRONCLADWRITER\

6.HIMSELF. THE TUTORIAL AIMS TO\

7.TEACH FOLKS AND EDUCATE HOW\

8.THEY COULD UNLOCK THE POWER\

9.OF UNIX TO THE FULLEST. AM POSTING\

10.THIS ON THE BLOG IRONCLADZONE\

11.THE BEST BLOG ON THE WHOLE OF\

12.INTERNET, WHERE YOU CAN LEARN\

13.TECHNICAL STUFF AS WELL ENJOY \

14.ENTERTAINING TIDBITS LIKE MOVIES,\

15.MUSIC, TELEVISION, FASHION, FOOD,\

16.SHOPPING, TRAVEL, TRENDS ETC.\

17.JUST SIT BACK, RELAX AND\

18.ENJOY THE RIDE :)

Similarly for converting from uppercase to lowercase,  use the following command :

tr A-Z a-z < SedText.rtf 

Add suffix to strings in a file using Awk

Guys, in my previous post yesterday, we discussed how to add a prefix to strings in a file using Awk. Today lets work on the same example to add a suffix to the strings i.e append the strings with characters in the end.

We'll use the same set of phone numbers for formatting :

2127841212
3458922345
7713398403
6461228847

Let's add some characters to the these numbers in the end. How about something like " Extn : 7142". Here's the Awk code to do it below :

awk '{print $0 " Extn : 7142"}' Phones_unformatted.txt

Output :

2127841212 Extn : 7142

3458922345 Extn : 7142

7713398403 Extn : 7142

6461228847 Extn : 7142

Add a prefix to strings in a file using Awk

Suppose you have a file in Unix / Mac OS X which has phone numbers data. I mean lets say I have a text file named Phones_unformatted.txt which has some phone numbers as follows :

2127841212
3458922345
7713398403
6461228847

Now I want to format the phone numbers to have a prefix maybe like a country code +44- . something like :

+44-2127841212
+44-3458922345
+44-7713398403
+44-6461228847

If you observe I need to only append the phone number with some characters in the beginning. We can achieve the same using awk.

A simple awk code will do the trick as follows :

awk '{print "+44-" $0}' Phones_unformatted.txt

Output :

+44-2127841212

+44-3458922345

+44-7713398403

+44-6461228847

Viola!