Novell Home

HowTo: Using the cron shell tool to automate procedures - Part 2 of 3

Novell Cool Solutions: Feature
By Stomfi

Digg This - Slashdot This

Posted: 9 Feb 2005
 

StomfiLearning to use Linux at Home and Work
Welcome to my ongoing series of HowTo articles designed to help Linux newbies get comfortable with Linux. Before trying any of these HowTos, take a few minutes to study the prerequisites so you can hit the ground running.
--Stomfi

In the first part of this HowTo we wrote several shell scripts to automate procedures. This is the last script which controls the sending of emails.

This is a long winded script and because it is not written in a complex manner you should be able to understand it and apply its knowledge to your other needs. Scripts like this replicate the checking tasks many administrative workers do. Like checking to see if someone has been sent an invoice and if they have paid or if not, have they been sent a reminder, and if so put them in the bad payers list, and send them a stronger one.

 

 

 

#!/bin/bash

#sendletter.sh

#

#Set Application home

AHOME="$HOME/newsletter"

#Set the temporary directory and make sure it exists

ATMP="$AHOME/tmp"

mkdir $ATMP

#The Issues file contains all the issues and dates created

ISSUES="$AHOME/Issues"

#The RDate file contains the address and dates renewed

RDATE="$AHOME/Rdate"

#The RSent file contains the address and issue

RSENT="$AHOME/Rsent"

#Make sure it exists

if [ ! -e $RSENT ]

then

   touch $RSENT

fi 

#The RConfirm file contains the address and date a confirmation email was sent

RCONFIRM="$AHOME/Rconfirm"

#Make sure it exists

if [ ! -e $RCONFIRM ]

then

   touch $RCONFIRM

fi 

#The Rlist file contain the addresses of current readers

RLIST="$AHOME/Rlist"

#

#Find out the issue number and date the prior issue but one was created 

#print the last 2 lines of the ISSUES file to awk which prints 

#the issue number and date field of the first line then exits 

PISSUE=`tail -2 $ISSUES | awk '{print $1 "#" $2 ; exit}' ` 

#

#And the current issue

CISSUE=`tail -1 $ISSUES | awk '{print $1 "#" $2 }' ` 

#

#Set the two fields in separate name holders (variables) with cut

PISNO=`echo $PISSUE | cut -d"#" -f2`

PISDATE=`echo $PISSUE | cut -d"#" -f1`

CISNO=`echo $CISSUE | cut -d"#" -f2`

CISDATE=`echo $CISSUE | cut -d"#" -f1`

#

#One of the problems in comparing dates is that we've got to chop it up into

#its day month year components and compare each one and so on.

#The shell date tool supplies a much better way of doing this. It 

#calculates the number of seconds since the birth of UNIX on 1/1/1970 

#for any supplied #date, which gives us numbers to compare. 

#This is called the epoch date. Really simple and common sense.

PISEPOCH=`date -d $PISDATE +%s`

CISEPOCH=`date -d $CISDATE +%s`

#

#Delete the temporary files

rm -f $ATMP/pconfirm.tmp 

rm -f $ATMP/nsend.$CISNO 

#

#We process the Rlist file looking for the last Rdate for each address.

#Make a list of addresses from the Rlist file

PRLIST=`awk -F"#" '{print $1}' $RLIST`

#For each address item do the following action

for ADDR in $PRLIST

do

   #Find out the last date renewed from the Rdate file

   #and the last entry for any address will be the last date renewed.

   #The quotes around $ADDR stop the shell from interpreting any 

   #embedded control characters in the address.

   #find all entries for ADDR

   grep "$ADDR" $RDATE > $ATMP/names

   #sort them by date 

   sort -t"#" -k 2,2n $ATMP/names > $ATMP/names2

   #get the last line and print the date field into LRDATE

   LRDATE=`tail -1 $ATMP/names2 | awk -F"#" '{ print $2 }' `

   #

   #Change it to epoch seconds

   LREPOCH=`date -d $LRDATE +%s`

   #If the renewal date is not current, then check the Rconfirm file 

   #for the address.

   #It is not current if it is prior to the issue before the current one

   if [ $LREPOCH -lt $PISEPOCH ]

   then

      #If it isn't current find out if a please confirm has been sent.

      #We do the same script bit as above

      #find all entries for ADDR

      grep "$ADDR" $RDATE > $ATMP/names

      #sort them by date 

      sort -t"#" -k 2,2n $ATMP/names > $ATMP/names2

      #get the last line and print the date field into CRDATE

      CRDATE=`tail -1 $ATMP/names2 | awk -F"#" '{ print $2 }' `

      #

      #Check that the date exists

      if [ ${#CRDATE} -lt 1 ]

      then 

         #Size of CRDATE is zero so there isn't a date

         #If there isn't one, but the reader got the last issue, 

         #then send a please confirm

         #Check that the reader got the last issue

         #$ADDR#$PISNO should be in the Rsent file if they got it

         LISSUE="$ADDR#$PISNO"

         GOTIT=`grep "$LISSUE" $RSENT` 

         if [ ${#GOTIT} -gt 0 ]

         then

            #We've got to send a please confirm but we don't want 

            #to stop processing this script while we do it, 

            #so we save the ADDR in a temporary file and send all 

            #of them after the loop exits.

            /bin/echo "$ADDR" >> $ATMP/pconfirm.tmp

         else

             #The reader didn't get the last issue so really 

             #shouldn't be here

             #Delete this reader from the Rlist

             grep -v "$ADDR" $RLIST > $ATMP/drlist

             mv -f $ATMP/drlist $RLIST

         fi 

      else

         #There is a date so change it to epoch seconds

         CREPOCH=`date -d $CRDATE +%s`

         #

         #Check the date is after the previous issue

         if [ $CREPOCH -gt $PISEPOCH ]

         then

            #They didn't send a renewal so delete them

            grep -v "$ADDR" $RLIST > $ATMP/drlist

            mv -f $ATMP/drlist $RLIST

         fi

      fi

      #This is the end of the 'renewal date check is not current' procedures

      #Now we check to see if it was sent after the prior issue

      #Which means it is current for the current issue

   else

      if [ $LREPOCH -gt $PISEPOCH ]

      then

         #The date is current check the Rsent file against the current issue

         THISSUE="$ADDR#$CISNO"

         GOTNU=`grep "$THISSUE" $RSENT`

         if [ ${#GOTNU} -lt 1 ]

         then

            #It hasn't been sent so echo it to the to be sent file. 

            /bin/echo "$ADDR" >> $ATMP/nsend.$CISNO

         fi

      fi

   fi

   #end of all renewal checks for the current ADDR

   #The loop will repeat for each item in the list until all are processed

done

#End of the do list for the Rlist addresses

#Now send any please confirm emails

if [ -e $ATMP/pconfirm.tmp ]

then

   $AHOME/bin/pconfirm.sh &

fi

#Send any newsletters

if [ -e $ATMP/nsend.$CISNO ]

then

   $AHOME/bin/mletter.sh $CISNO &

fi

#end of this script so exit

exit

Can you see how you can use a script like this to do your office work?

Don't forget to look up the man pages to see how things work.

Before you can test this you must configure your postfix email MTA so it can send mail to your ISP.

I use Webmin for configurations and this is the picture of the general setup.

You will see that the outbound mail domain is set to the domain of your ISP mail. i.e. The part after your name and the @ sign.

The other part that needs to be set is where to send outgoing mail. This is set to your ISP's mail upload site.

Make sure that postfix is running. You may have to change some other configurations if you've got a dial up system. The postfix documentation will show you how to do this.

If you edit this file by hand, you can find it in "/etc/postfix/main.cf"

That's all the scripts for the application. Now for the tests to see if they work!

Make sure that you have saved all your scripts in the $HOME/newsletter/bin folder and you have made them executable with the command:

$ chmod +x $ HOME/newsletter/bin/*

Send yourself a few emails as specified and run each script in the following order.

$HOME/newsletter/bin/issues.sh
$HOME/newsletter/bin/extract.sh readers reader
$HOME/newsletter/bin/extract.sh renewals renewal
$HOME/newsletter/bin/sendletter.sh

Fix up any typing errors and retest. When it all works you can go to the next step.

The cron table entries.

The scripts that go into the cron table (crontab) are as follows:

$HOME/newsletter/bin/extract.sh renewals renewal Run once per day on week days
$HOME/newsletter/bin/extract.sh readers reader Run once per day on week days
$HOME/newsletter/bin/issues.sh Run once per day in the first week of any month
$HOME/newsletter/bin/sendletter.sh Run once per week except in the first week

HowTo put an entry into crontab can be found in the manual page by giving the command:

$ man 5 crontab 

This is an extract from that page

The time and date fields are:

field allowed values

----- --------------

minute 0-59

hour 0-23

day of month 1-31

month 1-12 (or names, see below)

day of week 0-7 (0 or 7 is Sun, or use names)



A field may be an asterisk (*), which always stands for "first-last''.



Ranges of numbers are allowed. Ranges are two numbers separated with a hyphen.

The specified range is inclusive. For example, 8-11 for an "hours'' entry

specifies execution at hours 8, 9, 10 and 11.



Lists are allowed. A list is a set of numbers (or ranges) separated by commas.

Examples: "1,2,5,9'', "0-4,8-12''.



Step values can be used in conjunction with ranges. Following a range with

"/<number>'' specifies skips of the number's value through the range. For

example, "0-23/2'' can be used in the hours field to specify command execution

every other hour (the alternative in the V7 standard is

"0,2,4,6,8,10,12,14,16,18,20,22''). Steps are also permitted after an aster-

isk, so if you want to say "every two hours'', just use "*/2''.



Names can also be used for the "month'' and "day of week'' fields. Use the

first three letters of the particular day or month (case doesn't matter).

Ranges or lists of names are not allowed.



The "sixth'' field (the rest of the line) specifies the command to be run.

The entire command portion of the line, up to a newline or % character, will be

executed by /bin/sh or by the shell specified in the SHELL variable of the

cronfile. Percent-signs (%) in the command, unless escaped with backslash (\),

will be changed into newline characters, and all data after the first % will be

sent to the command as standard input.



Note: The day of a command's execution can be specified by two fields — day of

month, and day of week. If both fields are restricted (ie, aren't *), the com-

mand will be run when either field matches the current time. For example,

"30 4 1,15 * 5'' would cause a command to be run at 4:30 am on the 1st and

15th of each month, plus every Friday.

EXAMPLE CRON FILE

# use /bin/sh to run commands, no matter what /etc/passwd says

SHELL=/bin/sh

# mail any output to "paul', no matter whose crontab this is

MAILTO=paul

#

# run five minutes after midnight, every day

5 0 * * * $HOME/bin/daily.job >> $HOME/tmp/out 2>&1

# run at 2:15pm on the first of every month -- output mailed to paul

15 14 1 * * $HOME/bin/monthly

# run at 10 pm on weekdays, annoy Joe

0 22 * * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?%

23 0-23/2 * * * echo "run 23 minutes after midn, 2am, 4am ..., everyday"

5 4 * * sun echo "run at 5 after 4 every sunday"

AUTHOR

Paul Vixie <vixie@isc.org>

To edit your own cron table use the command.

$ crontab -e

This opens the vi editor for your crontab file.

Start typing by typing the "i" key to put you in "insert" mode. You can use the "delete" and the arrow keys to fix mistakes. To escape from insert mode use the ESC key. Be careful not to type anything in the ESC mode unless you know how to use vi, because the keys do editing commands in this mode. e.g. typing the x key deletes the letter under the cursor.

I presume you have typed the "i" key. The word "Insert" should appear at the bottom of the page.

Now type the following lines and press the ESC key when finished.

SHELL=/bin/bash

#Run at 1pm every week day

0 13 * * 1-5 $HOME/newsletter/bin/extract.sh renewals renewal

#Run at 3pm every week day

0 15 * * 1-5 $HOME/newsletter/bin/extract.sh readers reader

#Run once on Wednesday at 10am in the first week of any month

* 10 1-7 * 3 $HOME/newsletter/bin/issues.sh 

#Run once on Tuesday at 9.15 except in the first week of any month 

15 9 8-31 * 2 $HOME/newsletter/bin/sendletter.sh

Now hold down the Shift key and type ZZ . As long as you pressed the ESC key this will save and quit the vi editor.

If you made some email cards with the GIMP HowTo, you can use a variation of these scripts, without the renewal deletion parts, to automatically send your cards for various occasions. Your Rlist file could contain just the addresses, and the Rdate file could be the address and the dates of each event in that person's life for which you could send a card. Once you set it up, you can spend all your time making wonderful cards.

The Rlist, Rdate, Rsent and Rconfirm files all contain information that can be analysed. You can use GNUPlot to chart the results and cron to start the process each month. Say if you stop running the sendletter script on the 27th of the month, you can run the analyses script on the 28th.

I shall be showing you how to create automatic GNUPlot scripts and reports in the last part of this HowTo. And we shall layout a method for automatically generating the newsletter.

These first two parts have demonstrated the use of cron for automating administrative tasks in a general work and home environment. They have also shown how simply written UNIX/Linux shell scripts can get a computer to do very powerful automatic information processing of the type usually done by manual labour in a Microsoft environment.

I welcome lots of real world working example scripts and the cron tables that drive them as feedback on the topic of home and office task automation, as I am sure the whole community will benefit from your applications, no matter how simple. The less manual work we do, the less we need to be slaves to a computer and the more it becomes a slave to us.


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell