Novell Home

HowTo: A simple Linux shell script database

Novell Cool Solutions: Feature
By Stomfi

Digg This - Slashdot This

Posted: 1 Oct 2004
 

StomfiLearning to use Linux at Home and Work
Welcome to my ongoing series of HowTo articles designed to help Linux newbies get comfortable with Linux. Before trying any of these HowTos, take a few minutes to study the prerequisites so you can hit the ground running.
--Stomfi

Like all things Linux, there are many choices available and there are quite a few shells you can use. The one we are using in these HowTos is called bash.

Type
$ man bash

for lots of technical information about bash. Don't try to understand it now. All you need to know will be revealed in easy steps.

Of course you won't be able to use man or info when you are typing shell commands inside a text editor, so open up another terminal window so you can find things out whenever possible. If you want to write shell scripts "properly" get your local library to order Linux Shell Scripting with Bash by Ken O. Burtch or Sams Teach Yourself Shell Programming in 24 hours by Sriranga Veeraraghavan. The latter book contains information about other tools besides bash.

This HowTo Aims and Purposes

The functional aim of this HowTo is to build a very simple text database and reporting system using nothing but an editor and simple shell scripts and tools. You will use what you learn here for more sophisticated tasks later so try not to skip anything. The first thing to do is to make a directory or folder in your home directory to save everything.

From your main menu -- usually found by left clicking the farthest left icon on the menu bar, - choose

  • System Tools > Terminal (in GNOME)

  • or
  • System > Terminal > Konsole (in KDE)

A white or black window will open with a flashing cursor bar. Note that you usually have to keep your mouse pointer somewhere in this window to be able to type commands.

IMPORTANT Each command must be ended with the Enter key. Also, commands are all lower case. Upper case commands don't work.

Type
$ mkdir bin
$ mkdir bin/develop
$ cd bin/develop

These three simple commands have made a directory or folder called bin in our home directory, a sub directory of bin called develop and changed our active directory (ie., the one where new files will be created) to bin/develop.

If you are going to use gedit, kedit or kate, start these up from the main menu -- probably located in Accessories -- and immediately save the empty file in your home directory under the subdirectory bin/develop and call it entry.sh.

If you are going to use vi,
Type
$ vi entry.sh

We call our file entry because we are going to write a script to let us enter information into a text database. The ending .sh tells us this is a shell script. This convention makes it easier to identify our scripts when we want to make them executable or move them in a bunch.

If you are using vi you have to explicitly enter insert mode before you can start typing. Do this by using the i command. (Tip: If you want to find out how to use vi, start vi without a file name and once inside the program type :help)

By now we should be inside our editor in a file called entry.sh ready to start typing. Don't forget that all (except a few special cases) Linux commands are lower case. File names can be mixed case but "Entry.sh" will be seen by Linux as a different file from "entry.sh". This is totally different from MS Windows.

As will be seen, a script uses special names to remember things it wants to use. These names are called variables and by convention are usually upper case.

The first line of any script is a special line to tell Linux what shell to use. The first letter of this line is a special shell symbol that tells the shell it is a comment or remark.

Type
#!/bin/bash

The next line, by convention, tells us the name of the script program and if it needs any extra input information. In this case it doesn't.

Type
#entry.sh

The next line is an explanation of the script program.

Type
#Text database information entry

Now is a good time to save the file. Frequent saving between blocks of typing lets us recall the previously saved version in case we make a mistake. In vi you will have to press the Esc key to escape out of insert mode, then press :w to write all you have typed so far. Don't forget to press i again to re-enter insert mode.

What will be our subject? Since humour is a great part of Linux (for example, the bash shell stands for Bourne Again Shell, a reference to the original inventor), we shall make a Limerick and humourous poem database.

So now to type our poem input scripting. At the end of the poem we shall use a marker to tell our reporting program that it has reached the end of a poem. We shall use %% We will use the line below to ask the user to respond (Don't type it yet.) echo "Enter a short poem. When you have finished type %% on a line by itself"

Notice the quotes around the echo text. These are mandatory for the Linux echo command.

We will use the command "read" to read our poem. Read allows you to enter text until you press the enter key. Our poem will have several lines so we have to use a shell loop structure to keep entering each line until we get to our end of poem marker input.

I am going to insert some comments to let you know what we are doing. This is so we can understand these commands for re-use later on if we need to. (There's an 80% chance that we will.)

Type

#
#Initialise a variable for the until loop
GOON=0
#Initialise a filename holding variable
POEMS="$HOME/poems.txt"
#
#Make sure there is a file called poems.txt in the home directory
#Use the if control structure with test
#if test ([]) not exist (! -e) the filename which 
#  has the value ($) of POEMS then
if [ ! -e "$POEMS" ]
then
	#indent inside a control structure for clarity
	#Create the file
	#The Linux command "touch" is the best way of creating an empty file
	#although it sometimes doesn't work, we try this first.
	touch $POEMS
	#Check again and if not there we will
	#create the file with an initial null string
	if [ ! -e "$POEMS" ]
	then
	   #The > sends the output of echo to overwrite or create a file
	   #This is called a redirection as the output from a shell command
	   #normally goes to the terminal screen.
	   echo "" > $POEMS
      fi
	#End of if control structure
fi
#End of if control structure
#
#End of initialisations
#
#Start of the program loop
#until test ([]) the value of ($) GOON equals a number (-eq) 1, do the following
until [ $GOON -eq 1 ]
do 
	#Indent lines inside a control structure for clarity
	#read a line and put it into variable named NEXTLINE
	echo "Enter Next Line of Poem or %% when finished"
	read NEXTLINE
	#Append the line to the text file called poems.txt
	#The >> appends the redirected output of echo onto the end of the file
	echo "$NEXTLINE" >>  $POEMS
	#if test value of NEXTLINE not equal to a pattern (!=) %%, then 
	if [ "$NEXTLINE" != "%%" ]
	then
		#Another indent for this control structure
		#Continue reading lines at the "do" statement
		continue
	else
		#Else it is %% so we are at the end of our poem
		#Set GOON to 1 so the loop finishes normally
		GOON=1
	fi
	#fi is the end of the if control structure
done
#done is the end of the until do control structure	

Save the file again. We can test this bit with the following Limerick

There was a young man called Tomsk
Who's hobby was making bomsk
He lit a short fuse
Tripped over his shoes
And blew up in a mighty kerwumpsk
%%

You get the idea!

In vi we can test the file inside the editor (one of the many reasons I like it.)

Gedit and kate users will have to go to their shell terminal window. They will do the following.
Type
$ pwd

This says Print Working Directory. The response should be /home/YOU/bin/develop This is where your file is or should be. YOU is the name of your own home directory. For me it would be /home/stomfi/bin/develop

Type
$ ls

This says list the current directory contents. You should see entry.sh If this is all okay it is time to test your first program.

Type
$ sh -x entry.sh

If you are in vi you should press ESC again. Then

Type
:!sh -x %

This will start a shell with the test -x option and run entry.sh (the % in vi is the name of the currently edited file), which will printout the commands and responses and wait for your first line. If it doesn't work you have a mistake in your typing. The -x test should let you know where the problem lies.

Type in each line pressing the enter key at the end of each line. When you enter the line %% the program should stop and either return you to the shell prompt or give a "press enter to continue" message in vi.

Did it work? Well, let's quit the editor and have a look. To quit vi, press Esc to make sure you are not in insert mode.

Type
:wq

This will save the file and quit back to the shell.

Type
$ cat $HOME/poems.txt

You should see all your typed poem lines on the terminal.

The next thing is to make our script executable.

Type
$ chmod +x entry.sh

This will change the mode of the file entry.sh to executable. We can make it run in the current working directory with the commands

Type
$ sh entry.sh
or
$ ./entry.sh

This is a bit inconvenient. It would be better if it was in our executable path. This is the place Linux looks for executable commands when we type their name at a shell prompt.

To find out your path

Type
$ echo $PATH

Does your path look something like mine?

/usr/bin:/bin:/usr/X11R6/bin:/usr/local/bin:/home/stomfi/bin

You will see /home/stomfi/bin in my path. That means I can copy the file entry.sh into this directory with the command

Type
$ cp ./entry.sh $HOME/bin

Now I can type
$ entry.sh and it works.

If you don't see your bin directory in your path you can temporarily put it there with the command

Type
$ export PATH=$PATH:/home/YOU/bin

Of course you replace YOU with your own home name which is shown by the command

Type
$ echo $HOME

You will have to edit your .bashrc file in your home directory to put it there permanently. Try this as an exercise, but first save your .bashrc to .oldbashrc
Type
$ cp $HOME/.bashrc $HOME/.oldbashrc
Use the editor to add export PATH=$PATH:/home/YOU/bin to the end of the file. Save and quit the file. Test it with the source command ". "
Type
$ . $HOME/.bashrc
$ echo $PATH

If it worked, all is okay. If it didn't and you are not sure what you are doing wrong, Replace it by the command
$ cp $HOME/.oldbashrc $HOME/.bashrc

Now all we have got to do is report the lines we typed in, which were saved into a text file called $HOME/poems.txt. Hopefully with all your testing you've already got 2 or 3 more poems, but just in case here is an elegy from a grave stone in the UK.

Here lies Fred
Who was alive and is dead
Had it been his Father
I would have much rather
Had it been his Mother
Still better than the other
Had it been his Sister
No one would have missed her
But since it's only Fred
Who was alive and is dead
There's nothing more to be said
%%

Now you have at least two poems. Let's have a look with the command
Type
$ cat $HOME/poems.txt

You should see the poems. If not there is something wrong with your entry.sh script. Fix it and try again.

Our simple report script will print out a poem selected by its number in the list of poems.

Open a file called poem.sh in the directory bin/develop
Type

#!/bin/bash
#poem.sh
#select and print on the terminal a poem from the file $HOME/poems.txt
#
POEMS="$HOME/poems.txt"
#
#Find out how many poems there are by setting a variable with the output of the
#grep program. Grep looks for patterns in files. The -c returns a count
#We look to see how many %%s there are
#The back quote (`) is used to tell the shell to run the program.
#
NUMPOEMS=`grep -c "%%" $POEMS`
#
ANUMBER=0
#
#Read and Check that the number is in range and is valid
#Checking input is good practice in shell scripting to avoid problems that
#may stop your program from working
#
#$ANUMBER must be greater than 0 
#and $ANUMBER must be less than or equal to $NUMPOEMS
#Set a loop control variable
GOON=0
while [ $GOON -lt 1 ]
do
	echo "Enter a number between 1 and $NUMPOEMS"
	read ANUMBER
	#check the size of the input is greater than 0
	if [ ${#ANUMBER} -lt 1 ]
	then
		#The construct ${#VariableName} returns the size of the Variable.
		#Obviously if it is less than 1 the user must have 
		#pressed Enter without typing a number so go round the loop again
		continue
	fi
	if [ "$ANUMBER" -gt 0 -a "$ANUMBER" -le "$NUMPOEMS" ]
	then
		#Got a number in the range so set GOON to exit the loop
		GOON=1
	else
		#Out of range so go round loop again
		continue
	fi
done
#
#Now use $ANUMBER to select a poem
#So we first find line in the poem file until the next %% pattern.
#for this we shall use a nifty program called awk
#awk finds patterns in files and performs actions. Awk has its own way of 
#doing things which are different to the shell. Actions are enclosed in
#a single quote ('). each action is enclosed in curly braces ({}) The BEGIN 
#action below tells awk that %% is the record separator. The if tests for the 
#record number (NR) equal to ANUM and only prints each line of that record
#Notice that awk uses == to do its equals tests
#we also have to tell awk about the $ANUMBER variable with -v
#
awk -v ANUM=$ANUMBER 'BEGIN { RS = "%%" }{if( NR == ANUM ) print $0 }' $POEMS

Stop typing. Save and quit this file. Make it executable and put a copy into the bin directory like you did before. Test it with
$ sh -x poem.sh

There is a lot more that awk can do besides this simple program and we will be using its many abilities as these HowTos continue.

Instead of entering a poem number you could use the shell variable "RANDOM" and $NUMPOEMS to get a division remainder + 1, using this to automatically select a poem. Use "let ANUMBER = YOU FILL IN THE REST" as a practice exercise.

This would work like the "fortune" program which prints a random fortune cookie.
Type
$ fortune

In the report generator, awk normally looks at every line in the file. A good exercise is to limit where awk does its processing. If record number is less than ANUM then go to the next record, if record number is greater than ANUM then exit the awk program. The awk commands can go into an awk script file thus:

awk -v ANUM=$ANUMBER -f $HOME/bin/poem.awk $POEMS

You can split the awk commands in this file into manageable lines, but each line, apart from the last, must end with a back slash (\). This tells awk that the commands are all on one line. The \ escapes the shell meaning of the end of line marker. Prior to Microsoft just about every computer used the back slash as a way of escaping the control meaning of a special character. The traditional directory path character was usually / just like you see in use on the Internet. We shall see more uses of the back slash as we progress in this series.

For example, the file poem.awk for the original example could contain

BEGIN \
{ RS = "%%" } \
{ if( NR = ANUM ) print $0 }

Don't forget, GNU/Linux is meant to be fun and easy, so if it's not, you know you should try a different way.

By the way GNU stands for GNU's Not Unix, which goes on forever, another bit of that ubiquitous humour.

Coming Next

The next article in two parts will show you how to use RunRev to turn some of this knowledge into a windowing version with buttons and fields.

For more information about Runtime Revolution visit http://www.novell.com/coolsolutions/feature/1863.html


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell