HowTo: Create a Factory Rejects Data Recording system Part 3

Novell Cool Solutions: Feature
By Stomfi

Reader Rating from 1 ratings

Digg This - Slashdot This

Posted: 9 Mar 2005

Learning to use Linux at Home and Work
Welcome to my ongoing series of HowTo articles designed to help Linux newbies get comfortable with Linux. Before trying any of these HowTos, take a few minutes to study the prerequisites so you can hit the ground running.
--Stomfi

This HowTo explains the creation of a turn key system for collecting and analysing factory reject data.

First a conventional data entry system using collected hand filled out forms will be developed and tested.

Second a factory floor point of origin data entry system using conventional keyboards and screens, will be developed, using the previous data collection files and analysis system.

Third a replacement wireless pocket sized hand held data entry and read out device will be designed to replace conventional keyboards and screens with multi unit base stations for each work area.

Runtime Revolution and the Linux shell tools are the tools used to develop and create this system without requiring any specialised programming skill. Reading HowTos and Linux manuals is as heavy as it gets. When using RunRev you'll find it works best in the Gnome desktop.

The use of Linux has let this system be developed. Any other solution platform would put the cost way out of the reach of the small businesses for whom it was designed.

To simplify things each program module has been developed as a separate RunRev program stack, although RunRev has the ability to combine several stacks into one program. We shall have a look at this aspect at the end of each development.

This is the program stack window which calls the other stacks. It is named "RejCtl"."

In the first part we developed this stack and the stack for the New Rejects Entry Screen. We combined both stacks into a single application.

In the second part we developed the stack for the administration of products. We combined the three stacks developed into the single application.

In this part of this how to we shall build the Operators admin and Charting stacks and the shell scripts that do the processing, and add these to our RejCtl application. This will finish the first aim of this HowTo.

Open RunRev and open the RejCtl stack.

This is the script for the OPERATORS button:

on mouseUp

   open stack "Operators"

end mouseUp

And this script for the CHARTING button:

on mouseUp

   open stack "CHARTING"

end mouseUp

Save the stack and close it. Now open a new one and name it "Operators". Make it look like this:

This is the Application Browser showing the names of the fields on this card:

Here are the scripts.

This is the card script:

on openCard

   global SOP

   global OpList

   put ($HOME & "/frejects/lists/operators.txt") into OpList

   replace return with empty in OpList

   open file OpList for read

   read from file OpList until EOF

   put it into field "SLIST"

   close file OpList

   put empty into field "OPERATOR"

   put empty into field "RESPONSE"

end openCard

This is the QUIT button script:

on mouseUp

   go stack "RejCtl"

   close stack "Operators"

end mouseUp

This is the SLIST field script:

on mouseDown

   put the selectedText of me into HOP

   put HOP into field "OPERATOR"

end mouseDown

This is the DELETE button script:

on mouseUp

   put ($HOME & "/frejects/lists/operators.txt") into OpList

   replace return with empty in OpList

   put field "OPERATOR" into DOP

   put ($HOME & "/frejects/bin/delop.sh " & DOP) into DelOp

   replace return with empty in DelOp

   answer "Really delete this Operator?" with "Yes" or "No"

   put it into SURE

   if SURE = "Yes"

   then

      open process DelOp for read

      read from process DelOp until EOF

      put it into field "RESPONSE"

      close process DelOp

   end if

   open file OpList for read

   read from file OpList until EOF

   close file OpList

   put it into field "SLIST"

end mouseUp

This is the ADD button script:

on mouseUp

   ask "Enter Full Operator Name"

   put it into NEWOP

   put NEWOP into field "OPERATOR"

   put ($HOME & "/frejects/bin/newop.sh " & NEWOP) into NewOp

   replace return with empty in NewOp

   open process NewOp for read

   read from process NewOp until EOF

   close process NewOp

   put it into field "SLIST"

   put "Operator Added" into field "RESPONSE"

end mouseUp

This is the MODIFY button script:

on mouseUp

   put field "OPERATOR" into OLDOP

   ask "Enter New Operator Name"

   put it into NOP

   put ($HOME & "/frejects/bin/delop.sh " & OLDOP) into DelOp

   replace return with empty in DelOp

   answer "Really modify this Operator?" with "Yes" or "No"

   put it into SURE

   if SURE = "Yes"

   then

      open process DelOp for read

      read from process DelOp until EOF

      put it into field "RESPONSE"

      close process DelOp

      put ($HOME & "/frejects/bin/newop.sh " & NOP) into NewOp

      replace return with empty in NewOp

      open process NewOp for read

      read from process NewOp until EOF

      close process NewOp

      put it into field "SLIST"

      put "Operator Added" into field "RESPONSE"

   end if

end mouseUp

As can been seen the Modify script uses the delete and add shell scripts to achieve its task.

This is the delop.sh shell script:

!/bin/bash

#delop.sh operator

#

#Change product into RE

OPERATOR='echo $@'

#

mv -f $HOME/frejects/lists/operators.txt $HOME/frejects/lists/
doperators.txt 

grep -v "$OPERATOR" $HOME/frejects/lists/doperators.txt > $HOME/
frejects/lists/eoperators.txt 

if [ ! -s $HOME/frejects/lists/eoperators.txt ]

then

   #didn't work as size is less than 1

   mv -f $HOME/frejects/lists/doperators.txt $HOME/frejects/lists/
   operators.txt 

   echo "Problem with delete. Original list restored"

   exit

else

   mv -f $HOME/frejects/lists/eoperators.txt $HOME/frejects/lists/
   operators.txt 

fi

echo "Operator Deleted"

exit

And this is the newop.sh shell script:

#!/bin/bash

#newop.sh operator

#

#Change operator into RE

OPERATOR='echo $@'

#

mv -f $HOME/frejects/lists/operators.txt $HOME/frejects/lists/
aoperators.txt 

echo -e "\n$OPERATOR" >> $HOME/frejects/lists/aoperators.txt

sort $HOME/frejects/lists/aoperators.txt|uniq > $HOME/frejects/
lists/boperators.txt 

sed -e /^$/d $HOME/frejects/lists/boperators.txt |tee $HOME/frejects/
lists/operators.txt 

exit

Save the stack and make the shell scripts executable in the $HOME/frejects/bin folder. Close the stack.

Now make a new stack and name it "CHARTING". This one looks like this:

A bit more colourful from my usual. This is the Application Overview.

As you can see there are 4 more cards in this stack. The scripts for the card and buttons are shown below.

on openCard

   global SDATE

   global EDATE

   global OPR

   global WC

   global PROD

   put OPR into field "operator"

   put WC into field "workcentre"

   put PROD into field "product"

end openCard

Every time this card is returned to from one of the button cards, it puts whatever value is returned into the button field. The cards set the values of the global variables when you click their BACK buttons.

on mouseUp

   global SDATE

   global EDATE

   ask "Enter Start Date in D/M/YY format"

   put it into SDATE

   put SDATE into field "sdate"

   ask "Enter End Date in D/M/YY format"

   put it into EDATE

   put EDATE into field "edate"

end mouseUp


on mouseUp

   go card "operators"

end mouseUp


on mouseUp

   go card "workcentres"

end mouseUp


on mouseUp

   go card "products"

end mouseUp


on mouseUp

   go card "plot"

end mouseUp


on mouseUp

   go stack "RejCtl"

   close stack "CHARTING"

end mouseUp

I think you can work out what script goes with what button.

This is the "operators" card:

This is its objects:

This is the "workcentres" card:

And this is its objects:

This is the "products" card:

And this is its objects:

This is the "plot" card:

And this is its objects:

These are the scripts for each card.

The operators:

on mouseDown

   put the hilitedText of me into field "ThisOP"

end mouseDown


on mouseUp

   global OPR

   put empty into OPR

   if field "ThisOP" <> empty

   then

      put field "ThisOP" into OPR

   end if

   go card "charting"

end mouseUp

The work centres:

on mouseDown

   put the hilitedText of me into field "ThisWC"

   end mouseDown

   on mouseUp

   global WC

   put empty into WC

if field "ThisWC" <> empty

   then

      put field "ThisWC" into WC

   end if

   go card "charting"

end mouseUp

The products:

on mouseDown

   put the hilitedText of me into field "ThisProd"

end mouseDown


on mouseUp

   global PROD

   put empty into PROD

   if field "ThisProd" <> empty

   then

      put field "ThisProd" into PROD

   end if

   go card "charting"

end mouseUp

As you can see they all work the same way.

These are the plot card scripts. The plot is generated as soon as the card is opened, allowing the user to use one of the buttons to either save a plot as a printable jpeg file or to return to to the plot selection card, getting rid of the current plot on the way back.

on openCard

   global SDATE

   global EDATE

   global OPR

   global WC

   global PROD

   #check that something is not empty

   if SDATE <> empty or OPR <> empty or WC <> empty or PROD <> empty

   then

      put (quote & SDATE & "#" & EDATE & "#" & OPR & "#" & WC & "#" & 
      PROD & quote) into SEL

      replace return with empty in SEL

      put ($HOME & "/frejects/bin/plot.sh" && SEL) into PLOTTER

      replace return with empty in PLOTTER

      open process PLOTTER for read

      read from process PLOTTER until EOF

      close process PLOTTER

   end if

end openCard


on mouseUp

   put ($HOME & "/frejects/bin/printplot.sh") into PLOTPRINT

   replace return with empty in PLOTPRINT

   put the shell of PLOTPRINT into DUPRINT

end mouseUp


on mouseUp

   put the shell of ("killall gnuplot_x11") into KILLTHIS

   go card "charting"

end mouseUp

Save this stack and close it. We will build the complete application from all the stacks we have developed when we have created and tested the plotting shell scripts.

This is the plot.sh shell script. It is huge.

#!/bin/bash

#plot.sh Selection_info

#If you need to print this file use lpr plot.sh

#

#uses gnuplot to create a chart from frejects reject data which is

#preselected by this script using supplied Selection_info.

#Selection_info is a string in the form 

# SDATE#EDATE#OPERATOR#WORKCENTRE#PRODUCT

#At least one must contain data for this script to be called

#

#First assign values

PSTRING="$1"

SDATE='echo $PSTRING | cut -d"#" -f1'

EDATE='echo $PSTRING | cut -d"#" -f2'

OPR='echo $PSTRING | cut -d"#" -f3'

WC='echo $PSTRING | cut -d"#" -f4'

PROD='echo $PSTRING | cut -d"#" -f5'

REJFILE="$HOME/frejects/database/savedrejs.txt"

FTMP="$HOME/frejects/tmp"

#

#Now comes the hard part

#Do the easy stuff first

if [ ${#PROD} -gt 0 ]

then

   #select all this product

   grep "$PROD" $REJFILE > $FTMP/cprods.txt

   GOTPROD=1

else

   GOTPROD=0

fi

#

#Find out if its an OPR 

if [ ${#OPR} -gt 0 ]

then

   if [ $GOTPROD -gt 0 ]

   then

      grep "$OPR" $FTMP/cprods.txt > $FTMP/coprs.txt

      GOTOPR=1

   else

      grep "$OPR" $REJFILE > $FTMP/coprs.txt

      GOTOPR=1

   fi

else

   GOTOPR=0

fi

#

#Find out if its a WC

if [ ${#WC} -gt 0 ]

then

   if [ $GOTPROD -gt 0 ]

   then

      if [ $GOTOPR -gt 0 ]

      then

         grep $WC $FTMP/coprs.txt > $FTMP/cwcs.txt

         GOTWC=1

      else

         grep $WC $FTMP/cprods.txt > $FTMP/cwcs.txt

         GOTWC=1

      fi

   else

      grep $WC $REJFILE > $FTMP/cwcs.txt

      GOTWC=1

   fi

else

   GOTWC=0

fi 

#

#find out if its a date range

if [ ${#SDATE} -gt 0 -o ${#EDATE} -gt 0 ]

then

   #Check if we have one date and make blank date equal the other

   if [ ${#SDATE} -gt 0 -a ${#EDATE} -lt 1 ]

   then

      EDATE=$SDATE

   fi

   if [ ${#EDATE} -gt 0 -a ${#SDATE} -lt 1 ]

   then

      SDATE=$EDATE

   fi

   #

   #Change dates to epoch seconds format

   SEPOCH='date -d $SDATE +%s'

   EEPOCH='date -d $EDATE +%s'

   #now as above except use Epoch range

   if [ $GOTPROD -gt 0 ]

   then

      if [ $GOTOPR -gt 0 ]

      then

         if [ $GOTWC -gt 0 ]

         then

            #special date range selector

            awk -F# -v SDATE=$SEPOCH -v EDATE=$EEPOCH\

            '{if( $7 >= SDATE && $7 <= EDATE) print $0}'\

            $FTMP/cwcs.txt > $FTMP/cdates.txt

            GOTDATE=1

         else

            #special date range selector

            awk -F# -v SDATE=$SEPOCH -v EDATE=$EEPOCH\

            '{if( $7 >= SDATE && $7 <= EDATE) print $0}'\

            $FTMP/coprs.txt > $FTMP/cdates.txt

            GOTDATE=1

         fi

      else

         #special date range selector

         awk -F# -v SDATE=$SEPOCH -v EDATE=$EEPOCH\

         '{if( $7 >= SDATE && $7 <= EDATE) print $0}'\

         $FTMP/cprods.txt > $FTMP/cdates.txt

         GOTDATE=1

      fi

   else

      #special date range selector

      awk -F# -v SDATE=$SEPOCH -v EDATE=$EEPOCH\

      '{if( $7 >= SDATE && $7 <= EDATE) print $0}'\

      $REJFILE > $FTMP/cdates.txt

      GOTDATE=1

   fi

else

   GOTDATE=0

fi

#

#Got the data. Now find out which file to process

if [ $GOTDATE -gt 0 ]

then

   mv -f $FTMP/cdates.txt $FTMP/chartsel.txt

else

   if [ $GOTWC -gt 0 ]

   then

      mv -f $FTMP/cwcs.txt $FTMP/chartsel.txt

   else

      if [ $GOTOPR -gt 0 ]

      then

         mv -f $FTMP/coprs.txt $FTMP/chartsel.txt

      else

         #The only possible one left

         mv -f $FTMP/cprods.txt $FTMP/chartsel.txt

      fi

   fi

fi

#Now we've got a file to process

#We've got to sort it so we can total each reject type count

#This is all we are interested in. We can always refine the selection

#to get more specific

sort -t"#" -k5,5 -o $FTMP/chartsrt.txt $FTMP/chartsel.txt

#

#Now we use awk to compare the the 5th field 

#and output the reject type and the 5th field totals

#

#The logic of this may be a bit hard to work out in awk, so a good way

#is to write what is called stomfi's psuedo code

#

#for each line in input file

#do the following actions

# if its the first line, then

# set OLDTYPE = Field 5

# set OLDCOUNT = Field 6

# else (not the first line)

# if Field 5 = OLDTYPE, then

# OLDCOUNT = OLDCOUNT plus Field 6

# else (its a different type)

# Print OLDTYPE "#" OLDCOUNT appended to output file

# set OLDTYPE = Field 5

# set OLDCOUNT = Field 6

# end if

# end if

#done

#

#This is how the psuedo code looks when translated into awk

#Since awk reads each line at a time we leave out the for do done bit

#Awk also uses += to add to OLDCOUNT and ; to separate multiple 
statements

#Remember the \ is used to tell the shell this is all on one line

#

awk -F# '{if(NR == 1){ OLDFIELD = $5; OLDCOUNT = $6}\

else { if($5 == OLDFIELD) { OLDCOUNT += $6}\

else {print $5"#"OLDCOUNT;\

OLDCOUNT = $6;\

OLDFIELD = $5}\

}\

}' $FTMP/chartsrt.txt > $FTMP/chartawk.txt 

#

#Now we sort this file numerically descending on the count field

sort -t"#" -k 2 -n -r -o $FTMP/chartsort.txt $FTMP/chartawk.txt

#

#We use the reject names for the chart bar labels

#We build the gnuplot config file

#

#manipulate data for pareto chart


#For the x labels use this awk command to create a file with 

#name position pairs

awk -F# '{print $1 "#" NR}' $FTMP/chartsort.txt > $FTMP/xlabels1.dat

#

#Word smith each line from xlabels1.dat into a gnuplot label format

#This is a one line file with each name, position pair separated by a 
comma

#and each name surrounded by quotes.

awk -F# 'BEGIN{ORS=", "};{print "\"" $1 "\" " $2}' $FTMP/xlabels1.dat > 
$FTMP/xlabels2.dat

#

#substitute the last "," with ")" avoiding any spaces at the end

#We will use sed the stream editor and a Regular Expression pattern

#The s means substitute /First pattern/for second pattern/

#The first pattern is the comma , then [\ ][\ ]* for one or more 
spaces,

#the backslash says interpret the next character as its literal 
meaning ie space

#next is $ meaning the end of the line.

#So our pattern means a comma and one or more spaces at the end of 
the line

#

#The substitute pattern is the literal meaning of )

#

sed -e s/,[\ ][\ ]*$/\)/ $FTMP/xlabels2.dat > $FTMP/xlabels3.dat

#

#add leading "set xtics rotate (" to tell Gnuplot to put the labels
sideways

awk '{print "set xtics rotate (" $0}' $FTMP/xlabels3.dat > 
$FTMP/xlabels4.dat

#

#We create the Title from the Selection criteria using the GOT
variables

if [ $GOTDATE -gt 0 ]

then

   PTITLE="$SDATE to $EDATE "

fi

if [ $GOTWC -gt 0 ]

then

   PTITLE="$PTITLE$WC "

fi

if [ $GOTOPR -gt 0 ]

then

   PTITLE="$PTITLE$OPR "

fi

if [ $GOTPROD -gt 0 ]

then

   PTITLE="$PTITLE$PROD "

fi

#

#We need to create the pareto settings file to include the bar
labels

#and the Title

#The pareto2.dem file is a template which contains the Gnuplot
settings

cp $HOME/frejects/info/pareto2.dem $FTMP/pareto.dem 

#

#This next line says echo ctrl@. 

#To insert this symbol in the vi editor press Ctrl-v and then Ctrl-@

#If you need to print this file use lpr plot.sh

echo "Ctrl@" >> $FTMP/pareto.dem

echo "set title \"Rejects for $PTITLE\"" >> $FTMP/pareto.dem 

echo "Ctrl@" >> $FTMP/pareto.dem

cat $FTMP/xlabels4.dat >> $FTMP/pareto.dem 

echo "Ctrl@" >> $FTMP/pareto.dem

echo "plot \"$FTMP/pareto.dat\"" >> $FTMP/pareto.dem 

#

#For the plot data use this awk command which creates a file

#with line numbers tabs and reject counts, telling gnuplot the

#number of each bar column and its height

awk -F# '{print NR "\t" $2}' $FTMP/chartsort.txt > $FTMP/pareto.dat

#

#Use Gnuplot with screen window, boxes option to chart file

#

#Make the plot in an X11 window using the pareto.dem config file

gnuplot -background wheat -noraise -persist -geometry 650x540+370+0
$FTMP/pareto.dem

#

#End of plot script

Wow. That's a shell script and a half. I've kept it na?ve, so there is a lot of if else statements. I'm sure the clever one's amongst you can improve on it. Send in your improvements and I sure to get to see them. Those Ctrl@ things are a bit tricky to include in a file. It is best to do it with vi in a shell. In insert mode press Ctrl and the v key, then press Ctrl and the @ key.

This is the $HOME/frejects/info/pareto2.dem file.

set nokey

set data style boxes

set ylabel "Count"

set yrange [0:]

set timestamp

#Title, output file and the xtic labels will be put in by the plot.sh script.

This is the printplot.sh script:

#!/bin/bash

#printplot.sh

#

#Save the current Gnuplot window

#

#Set Plot folder

PLOTDIR="$HOME/frejects/plots"

#Create a unique reference from the date and time

PLOTNO='date +%y%m%d%M%S '

#Create plot name

PLOTSHOT="plot$PLOTNO.jpeg"

#Use import to save the window

import -display :0.0 -window Gnuplot $PLOTDIR/$PLOTSHOT

#End of script

You will see that these scripts depend on the programs "GNUPlot" and "import". You will have to install them if they are not on your system.

Test both scripts out making sure that all necessary folders are created and the scripts are made executable. Use some dummy data to do the tests. When all works as expected, try it out from the RunRev card. You should see a plot like this appear on the screen.

Now we build the application as a stand alone.

First open each stack in turn and use the property window to lock the text in every text field. Saving and closing each stack as its done. Then open the RejCtl stack.

Next click File -- Standalone Application Settings

Give a name for your application in the General window.

Click on the Stacks icon at the top.

And add all the stacks in the development.

There should be 5 as shown below.

Mine will have different names as this application was developed for an fmea system.

Now click File -- Save as Standalone Application.

Test your application, going to each card to check that the display is OK. Rebuild it if you need to.

This is the last part of this part of this HowTo. Part 4 will develop collecting the reject data at the point of origin on the factory floor from key strokes using standard keyboards, screens and computers. The RunRev rawkey function will be used to do this.

For more information about Runtime Revolution visit http://www.novell.com/coolsolutions/feature/1863.html

Like what you see?
Sign up for our weekly newsletter.

Want to contribute?
It could earn you a nano! Learn more.

Like Wikis?
Join the Cool Solutions Wiki.

Interested?
Request a sales call

Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

HowTo: Create a Factory Rejects Data Recording system Part 3

Novell Cool Solutions: FeatureBy Stomfi

Novell Cool Solutions: Feature
By Stomfi