How can I use cron scripts to monitor if the GroupWise Agents are running and to restart them if needed.

The GroupWise agents stop or crash on occasion and the customer does not want to use the GWHA process to restart them.  How can this be automated thru scripts and a cron job.  GroupWise agent crashes should be investigated by Micro Focus GroupWise Support, but until a solution is found, consider this technical document for a temporary workaround.

Note: If any customer wants to disregard the information in this document and use the GWHA process to restart the GroupWise Agents they can go to :

https://www.novell.com/documentation/groupwise18/gw18_guide_install/data/inst_create_agents_linux.html#inst_create_agents_linux_gwha

https://support.microfocus.com/kb/doc.php?id=7016275#

Otherwise continue.

*** NOTE:  If you will implement the procedure in this document, do it on a Friday after hours to not affect the GroupWise user base, and to confirm that all agents reload properly and that you receive an e-mail notification.

The cron job e-mail notification send, on the GroupWise/SLES server, depends on postfix, if postfix is on the same server as the GroupWise GWIA, make sure postfix is using a port other than 25, to avoid a port conflict.

You can "google" how to install and configure postfix, if you need.

However to change the postfix port :

a.  On the server with postfix, make a back-up copy of "master.cf".

b.  Edit the file /etc/postfix/master.cf

c.  Search for the line that says: "smtp      inet  n       -       n       -       -       smtpd"

d.  Copy this line of text to the clipboard.

e.  Place a "#" character in front of this line to remark it out.

f.  Create a new blank line under it and Paste this copied text.

g.  At the beginning of this new line, change "SMTP" to "26", no quotes.

h.  Save the file and restart postfix with "rcpostfix restart".  no quotes.

i.  You can verify postfix is running with "rcpostfix status"

j.  To verify postfix is using port 26, you can do "netstat -tnlp | grep :2” (SLES11 or SLES12), or “ss -tnlp | grep :2” (SLES15), no quotes.

k.  If this is a GroupWise linux server with GWIA, you will see port 25 and likely 2 references to port 26.  That is normal.

Note:

The examples in this document may not exactly match your situation but you will have all the information you need to help you customize for your environment.  Adjust as needed for your environment.

Assumptions in document :

1.  The GroupWise 18.1.1 system is on Linux ( SLES11, SLES12, or SLES15 )

2.  There are 2 GroupWise Domains

3.  There are 2 GroupWise Post Offices

4.  There is 1 GroupWise GWIA

5.  There is 1 GroupWise DVA ( Document viewer agent )

6.  That the comments in the scripts will provide guidance on how to use and customize the agents scripts in this document, for your environment.

7.  Log files will be created automatically to capture a historical record of the restart events of the MTA, POA, GWIA, and DVA agents.

  - They will be stored in the /var/log/novell/groupwise/ directory

  - Called gwdvachk.log, gwgwiachk.log, gwmtachk.log and gwpoachk.log

8.  It is also assumed that you will create in this same above directory, text files that will contain basic alert information about the agent that has had a problem and was restarted.  This will be used to send an e-mail to the desired GroupWise administrator notifying him or her of the agent restart event.

  - For the purposes of this document, these text files will be called 

     smtpalert-dva.txt, smtpalert-gwia.txt, smtpalert-mta.txt, and smtpalert-poa.txt

     The content basically states:  ( change as needed )

     "Hello,

     This is to inform you that your GroupWise <Groupwise Agent Name> had a problem and shutdown and was

     restarted.

     Regards,

     Your GroupWise Administrator"

9.  Crontab information:

If you are not sure how to create, edit for view a crontab file, look it up on google.com.  or look at this : 

https://docs.oracle.com/cd/E23824_01/html/821-1451/sysrescron-24589.html

Note:  At this point, make sure that the cron daemon is not running.  At the linux terminal as "root" execute the command "rccron stop" , no quotes.

Note:  If for some reason you need to shut down a GroupWise agent and you temporarily don't want to have it automatically restart, then first run "rccron stop".  When you are ready to have the agents auto restarted again then do "rccron start".  If you are not sure if the cron daemon is running you can always execute the command "rccron status" to check.  No quotes of course.

Create the below information, about the 6 scripts, in your crontab file and save : ( edit: crontab -e) :

* * * * * /root/Documents/Cron-Scripts/domain1.sh

* * * * * /root/Documents/Cron-Scripts/domain2.sh

* * * * * /root/Documents/Cron-Scripts/po1.sh

* * * * * /root/Documents/Cron-Scripts/po2.sh

* * * * * /root/Documents/Cron-Scripts/gwia.sh

* * * * * /root/Documents/Cron-Scripts/dva.sh

10.  It is assumed in this document that you want to monitor for the all the agentsmentioned in this document.  However change as you feel appropriate.

 With the information in this document above and the information in the comments of the below

 scripts you now have a solution that will auto-restart GroupWise agents if they go down, and

 you will be notified by e-mail of the restart event :

11.  It is assumed that you will copy and paste the content of the scripts below

 into the below individual files and place them in the /root/Documents/Cron-Scripts/ directory.  They should

 have execute permissions on these scripts ( chmod +x <scriptName>) They are called :

   - domain1.sh

   - domain2.sh

   - po1.sh

   - po2.sh

   - dva.sh

   - gwia.sh

############# domain1.sh ##################################

  #!/bin/bash

  declare -i CNT=0

  LOGFILE="/var/log/novell/groupwise/gwmtachk.log"

  TIMESTAMP=`date "+%m-%d-%Y %H:%M:%S"`

  # Modify the path and name of the GroupWise MTA agent

  # startup file to match the actual name.

  # You can find this out by typing this at the terminal

  # as root on the MTA server :

  # ps -eaf | grep gwmta

  # You should see directly after "/gwmta ", in my case

  #  "@/groupwise/domain1/domain1.mta" , no quotes.

  CNT=`ps -aef | grep -v grep | grep -c "@/groupwise/domain1/domain1.mta"`

# make sure you have spaces after [ and before ]

  if [ $CNT -eq 0 ]

  then

# change the string as appropriate for you, mine is

#  "Restarting the domain1 MTA"

  echo "$TIMESTAMP - Restarting the domain1 MTA" >> $LOGFILE

  # Modify the name of the domain to match the actual

  # name.  You can check the name

  # by running "rcgrpwise status", no quotes.

  # The name of the domain is in square brackets.

  #  mine is "domain1" , no quotes.

  /etc/init.d/grpwise start domain1 &

  # Change the Subject line as appropriate for you, mine is

  #  "GroupWise bperez140 domain1 MTA restarted - $TIMESTAMP"

  # Don't change $TIMESTAMP.

  # Change the e-mail address as appropriate, mine is

  #  "bperez@microfocus.com"

  # My path and  smtp alert message body file is

  #  "/var/log/novell/groupwise/smtpalert-mta.txt"

  # Create a file called "smtpalert-mta.txt" in the

  # path listed with a message about the mta had a problem,

  # shut down and was restarted.

mail -s "GroupWise bperez140 domain1 MTA restarted - $TIMESTAMP" bperez@microfocus.com < /var/log/novell/groupwise/smtpalert-mta.txt

  else

# Change the below string as appropriate for you

  echo "The GroupWise domain1 MTA is running"

 fi

exit 0

####################################################################

 

############# domain2.sh ############################################

  #!/bin/bash

  declare -i CNT=0

  LOGFILE="/var/log/novell/groupwise/gwmtachk.log"

  TIMESTAMP=`date "+%m-%d-%Y %H:%M:%S"`

  # Modify the path and name of the GroupWise MTA

  # agent startup file to match the actual name.

  # You can find this out by typing this at the

  # terminal as root on the MTA server :

  #   ps -eaf | grep gwmta

  # You should see directly after "/gwmta ", in my case

  #  "@/groupwise/domain2/domain2.mta" , no quotes.

  CNT=`ps -aef | grep -v grep | grep -c "@/groupwise/domain2/domain2.mta"`

# make sure you have spaces after [ and before ]

  if [ $CNT -eq 0 ]

  then

# change the string as appropriate for you, mine is "Restarting the domain2 MTA"

  echo "$TIMESTAMP - Restarting the domain2 MTA" >> $LOGFILE

  # Modify the name of the domain to match the actual name.

  # You can check the name

  # by running "rcgrpwise status", no quotes.  The name

  # of the domain is in square brackets.

  # mine is "domain2" , no quotes.

  /etc/init.d/grpwise start domain2 &

  # Change the Subject line as appropriate for you,

  # mine is "GroupWise bperez140 domain2 MTA restarted - $TIMESTAMP"

  # Don't change $TIMESTAMP.

  # Change the e-mail address as appropriate,

  # mine is "bperez@microfocus.com".

  #  My path and  smtp alert message body file is

  # "/var/log/novell/groupwise/smtpalert-mta.txt".

  # Create a file called "smtpalert-mta.txt" in the path

  # listed with a message about the mta had a problem,

  # shut down and was restarted.

mail -s "GroupWise bperez140 domain2 MTA restarted - $TIMESTAMP" bperez@microfocus.com < /var/log/novell/groupwise/smtpalert-mta.txt

else

# Change the below string as appropriate for you

  echo "The GroupWise domain2 MTA is running"

 fi

exit 0

####################################################################

 

############# po1.sh ###############################################

 #!/bin/bash

 declare -i CNT=0

 LOGFILE="/var/log/novell/groupwise/gwpoachk.log"

 TIMESTAMP=`date "+%m-%d-%Y %H:%M:%S"`

  # Modify the path and name of the GroupWise POA

  # agent startup file to match the actual name.

  # You can find this out by typing this at the

  # terminal as root on the POA server :

  #  ps -eaf | grep gwpoa.

  # You should see directly after "/gwpoa ",

  # in my case "@/groupwise/po1/po1.poa" , no quotes.

 CNT=`ps -aef | grep -v grep | grep -c "@/groupwise/po1/po1.poa"`

 # make sure you have spaces after [ and before ]

 if [ $CNT -eq 0 ]

 then

 # change the string as appropriate for you,

 #  mine is "Restarting the po1 POA"

 echo "$TIMESTAMP - Restarting the po1 POA" >> $LOGFILE

 # Modify the name of the post office and domain to

 # match the actual name.  You can check the name

 # by running "rcgrpwise status", no quotes. 

 # The name of the post office is in square brackets.

 # mine is "po1.domain1" , no quotes.

 /etc/init.d/grpwise start po1.domain1 &

 # Change the Subject line as appropriate for you,

 # mine is

 # "GroupWise bperez140 po1 POA restarted - $TIMESTAMP".

 # Don't change $TIMESTAMP.

 # Change the e-mail address as appropriate,

 # mine is "bperez@microfocus.com".

 # Change the path and name of the smtp alert message

 # body file, mine is

 # "/var/log/novell/groupwise/smtpalert-poa.txt".

 # Create a file called "smtpalert-poa.txt" in the path

 # listed with a message about the POA had a

 #  problem, shut down and was restarted.

 mail -s "GroupWise bperez140 po1 POA restarted - $TIMESTAMP" bperez@microfocus.com < /var/log/novell/groupwise/smtpalert-poa.txt

 else

 # Change the below string as appropriate for you

 echo "The GroupWise po1 POA is running"

 fi

exit 0

#########################################################################

 

############# po2.sh ####################################################

 #!/bin/bash

 declare -i CNT=0

 LOGFILE="/var/log/novell/groupwise/gwpoachk.log"

 TIMESTAMP=`date "+%m-%d-%Y %H:%M:%S"`

 # Modify the path and name of the GroupWise POA

 # agent startup file to match the actual name.

 # You can find this out by typing this at the

 # terminal as root on the POA server :

 #  ps -eaf | grep gwpoa.

 # You should see directly after "/gwpoa ",

 # in my case "@/groupwise/po2/po2.poa",

 # no quotes.

 CNT=`ps -aef | grep -v grep | grep -c "@/groupwise/po2/po2.poa"`

 # make sure you have spaces after [ and before ] #

 if [ $CNT -eq 0 ]

 then

 # change the string as appropriate for you,

 # mine is "Restarting the po2 POA".

 echo "$TIMESTAMP - Restarting the po2 POA" >> $LOGFILE

 # Modify the name of the post office and domain

 # to match the actual name.  You can check the name

 # by running "rcgrpwise status", no quotes.

 # The name of the post office is in square brackets.

 # Mine is "po2.domain2" , no quotes.

 /etc/init.d/grpwise start po2.domain2 &

 # Change the Subject line as appropriate for you,

 # mine is

 # "GroupWise bperez140 po2 POA restarted - $TIMESTAMP".

 # Don't change $TIMESTAMP.

 # Change the e-mail address as appropriate,

 # mine is "bperez@microfocus.com".

 # Change the path and name of the smtp alert message

 # body file, mine is

 # "/var/log/novell/groupwise/smtpalert-poa.txt".

 # Create a file called "smtpalert-poa.txt" in the path

 # listed with a message about the POA had

 # a problem, shut down and was restarted.

 mail -s "GroupWise bperez140 po2 POA restarted - $TIMESTAMP" bperez@microfocus.com < /var/log/novell/groupwise/smtpalert-poa.txt

 else

 # Change the below string as appropriate for you

 echo "The GroupWise po2 POA is running"

 fi

exit 0

#####################################################################

 

############# dva.sh ###############################################

  #!/bin/bash

  declare -i CNT=0

  LOGFILE="/var/log/novell/groupwise/gwdvachk.log"

  TIMESTAMP=`date "+%m-%d-%Y %H:%M:%S"`

  # Modify the path and name of the GroupWise DVA

  # agent startup file to match the actual name.

  # You can find this out by typing this at the terminal

  # as root on the DVA server :

  #   ps -eaf | grep gwdva

  # You should see directly after topmost "/gwdva ",

  # in my case "@/opt/novell/groupwise/agents/share/po1.dva"

  # , no quotes. 

CNT=`ps -aef | grep -v grep | grep -c "@/opt/novell/groupwise/agents/share/po1.dva"`

# make sure you have spaces after [ and before ]

  if [ $CNT -eq 0 ]

  then

# change the string as appropriate for you,

# mine is "Restarting the gwdva.poa DVA"

  echo "$TIMESTAMP - Restarting the gwdva.po1 DVA" >> $LOGFILE

  # Modify the name of the dva.postoffice to match

  # the actual name.  You can check the name

  # by running "rcgrpwise status", no quotes. 

  # The name in will be in square brackets.

  #   mine is "gwdva.po1"

  /etc/init.d/grpwise start gwdva.po1 &

  # Change the Subject line as appropriate for you,

  # mine is "GroupWise bperez140 gwdva.po1 DVA

  # restarted - $TIMESTAMP"

  # Don't change $TIMESTAMP.

  # Change the e-mail address as appropriate,

  # mine is "bperez@microfocus.com".

  # MY path and smtp alert message body file is

  # "/var/log/novell/groupwise/smtpalert-dva.txt".

  # Create a file called "smtpalert-dva.txt" in the path

  # listed with a message about the POA had a

  # problem, shut down and was restarted.

  mail -s "GroupWise bperez140 gwdva.po1 DVA restarted - $TIMESTAMP" bperez@microfocus.com < /var/log/novell/groupwise/smtpalert-dva.txt

  else

  echo "The GroupWise gwdva.po1 DVA is running"

  fi

exit 0

################################################################

 

 ############# gwia.sh ##########################################

 #!/bin/bash

 declare -i CNT=0

 LOGFILE="/var/log/novell/groupwise/gwgwiachk.log"

 TIMESTAMP=`date "+%m-%d-%Y %H:%M:%S"`

 # Modify the path and name of the GroupWise GWIA agent

 # startup file to match the actual name.

 # You can find this out by typing this at the terminal

 # as root on the GWIA server :

 #  ps -eaf | grep gwia

 # You should see directly after "/gwia ", in my case

 # "@/groupwise/domain1/wpgate/gwia/gwia.cfg" , no quotes.

 CNT=`ps -aef | grep -v grep | grep -c "@/groupwise/domain1/wpgate/gwia/gwia.cfg"`

 # make sure you have spaces after [ and before ]

 if [ $CNT -eq 0 ]

 then

 # change the string as appropriate for you, mine is "Restarting the GWIA"

 echo "$TIMESTAMP - Restarting the GWIA" >> $LOGFILE

 # Modify the name of the gwia and domain to match the actual

 # name.  You can check the name

 # by running "rcgrpwise status", no quotes.  The name

 # of the gwia.domain is in square brackets.

 # mine is "gwia.domain1" , no quotes.

 

 /etc/init.d/grpwise start gwia.domain1 &

 #Change the Subject line as appropriate for you, mine is

 #  "GroupWise bperez140 GWIA restarted - $TIMESTAMP"

 # Don't change $TIMESTAMP.

 # Change the e-mail address as appropriate, mine is

 # "bperez@microfocus.com"

 # My smtp alert message body file is

 # "/var/log/novell/groupwise/smtpalert-gwia.txt"

 # Create a file called "smtpalert-gwia.txt" in the path

 # listed with a message about the gwia had a problem, shut  

 # down and was restarted

mail -s "GroupWise bperez140 GWIA restarted - $TIMESTAMP" bperez@microfocus.com < /var/log/novell/groupwise/smtpalert-gwia.txt

 else

# Change the below string as appropriate for you

 echo "The GroupWise GWIA is running"

 fi

exit 0

 ############################################################## 

 NOTE:  You are done .  You can now start the cron daemon (rccron start) and test by manually shutting down the individual agents one at a time, and verify if the agent restarts properly.  They take between 30 to 60 seconds for my agents to reload.  If you find that either the agents are not restarting or you do not get an e-mail notification, try to run "dos2unix <scriptFileName>".  No quotes, it may be that there are hidden characters in the bash script causing the script to not execute properly.