demoIterator.job

demoIterator                                # Total: 129 lines
|-- demoIterator.jdl                        #   68 lines
`-- demoIterator.policy                     #   61 lines

demoIterator.jdl

 1  import time, random 
 2  
 3  # 
 4  # Add to the 'examples' group on deployment 
 5  # 
 6  if __mode__ == "deploy": 
 7      try: 
 8          jobgroupname = "examples" 
 9          jobgroup = getMatrix().getGroup(TYPE_JOB, jobgroupname) 
10          if jobgroup == None: 
11              jobgroup = getMatrix().createGroup(TYPE_JOB, jobgroupname) 
12          jobgroup.addMember(__jobname__) 
13      except: 
14          exc_type, exc_value, exc_traceback = sys.exc_info() 
15          print "Error adding %s to %s group: %s %s" % (__jobname__, jobgroupname, exc_type, exc_value) 
16  
17  class demoIteratorJob(Job): 
18           
19      def job_started_event(self): 
20          print 'job_started_event' 
21          self.completed = 0 
22  
23          # Launch the joblets 
24          numJoblets = self.getFact("jobargs.numJoblets") 
25          print 'Launching ', numJoblets, ' joblets' 
26  
27          pspace = ParameterSpace() 
28          i = 1 
29          while i <= numJoblets: 
30              pspace.appendRow({'name':'joblet'+str(i)}) 
31              i += 1 
32          pspace.maxJobletSize = 1 
33          self.schedule(demoIteratorJoblet,pspace,{}) 
34  
35      def joblet_completed_event(self, jobletnumber, node): 
36          self.completed += 1 
37          self.setFact("jobinstance.memo", "Tests run: %s" % (self.completed)) 
38  
39  class demoIteratorJoblet(Joblet): 
40  
41      def joblet_started_event(self): 
42          print "Hi from joblet ", self.getFact("joblet.number") 
43          time.sleep(random.random() * 15) 
44  
45          cmd = self.getFact("jobargs.cmd") 
46          if len(cmd) > 0: 
47              system(cmd) 
48   
49              # Example on more sophisticated exec 
50              # e.g.  e.signal("SIGUSR1") 
51              """ 
52              e = Exec() 
53              e.setCommand(cmd) 
54              #e.setStdoutFile("cmd.out") 
55              e.writeStdoutToLog() 
56              e.writeStderrtoLog() 
57              #try: 
58              e.execute() 
59              #except: 
60                  #raise RetryElsewhere, "example error" 
61              """

demoIterator.policy

 1  <policy> 
 2     <constraint type="accept" reason="Too busy for more work. Try again later!"> 
 3        <or> 
 4           <lt fact="job.instances.queued" value="4" /> 
 5           <contains fact="user.groups" value="superuser" /> 
 6        </or> 
 7     </constraint> 
 8  
 9     <constraint type="start" reason="Waiting on queue"> 
10        <or> 
11           <lt fact="job.instances.active" value="2" /> 
12           <contains fact="user.groups" value="superuser" /> 
13        </or> 
14     </constraint> 
15  
16     <jobargs> 
17        <fact name="numJoblets" 
18              type="Integer" 
19              description="joblets to run" 
20              value="100" 
21              visible="true" /> 
22  
23        <fact name="cmd" 
24              type="String" 
25              description="Simple command to execute" 
26              value="" /> 
27  
28        <fact name="os" 
29              type="String" 
30              description="Regular expression match for Operating System" 
31              value=".*" /> 
32  
33        <fact name="cpu" 
34              type="String" 
35              description="Regular expression match for CPU architecture" 
36              value=".*" /> 
37     </jobargs> 
38  
39     <constraint type="resource" reason="Does not match"> 
40        <and> 
41           <eq fact="resource.os.family" factvalue="jobargs.os" match="regexp" /> 
42           <eq fact="resource.cpu.architecture" factvalue="jobargs.cpu" match="regexp"/> 
43  
44           <or> 
45              <and> 
46                 <defined fact="env.VENDOR" /> 
47                 <eq fact="resource.os.vendor" factvalue="env.VENDOR" match="regexp" /> 
48              </and> 
49              <undefined fact="env.VENDOR" /> 
50           </or> 
51        </and> 
52     </constraint> 
53  
54     <job> 
55        <fact name="description" 
56              type="String" 
57              value="This example job is a reference for a simple test iterator. It is useful for demonstrating how policies and job args can be used to target the job to a particular resource." /> 
58     </job> 
59  </policy>

Definitions:

Job

A representation of a running job instance.

Joblet

Defines execution on the resource.

MatrixInfo

A representation of the matrix grid object, which provides operations for retrieving and creating grid objects in the system. MatrixInfo is retrieved using the built-in getMatrix() function. Write capability is dependent on the context in which getMatrix() is called. For example, in a joblet process on a resource, creating new grid objects is not supported.

GroupInfo

A representation of Group grid objects. Operations include retrieving the group member lists and adding/removing from the group member lists, and retrieving and setting facts on the group.

Exec

Used to manage command line execution on resources.

ParameterSpace

Defines a parameter space to be used by the scheduler to create a Joblet set. A parameter space might consist of rows of columns or a list of columns that is expanded and can be turned into a cross product.

zosadmin deploy

The deployment for the DemoIterator job is performed by lines 3-15 of demoIterator.jdl. When jobs are deployed into the grid, they can optionally be organized into and easy reference. In this case, the demoIterator job will be added to the group named examples, and will show up in the ZENworks Orchestrator Console in the Explorer panel at the location:

/ZOS/YOUR_GRID/Jobs/examples

For a general overview of how jobs are added to groups during deployment, see Walkthrough: Deploy a Sample Job.

job_started_event

When the DemoIterator job receives a job_started_event, it creates a ParameterSpace object, and adds the number of rows as indicated by the value of the argument numJoblets (see lines 27-31 in demoIterator.jdl). A ParameterSpace object is like a spreadsheet, containing rows and columns of information that might all be given to one joblet or sliced up across many joblets at schedule time. In this case, the ParameterSpace is told that maxJobletSize is 1 (see line 32), meaning a joblet instance is created for each row in the ParameterSpace during job scheduling (see line 33).

Not shown in this example is the fact that a joblet can get access to this “spreadsheet” of information by calling self.getParameterSpace(), and calling hasNext() and next() to enumerate through each row of information. To learn more about putting information in a ParameterSpace object from a job and obtaining that information from the JobletParameterSpace object from a joblet, see ParameterSpace.

The resource that runs the joblet is determined from the resource constraint specified in lines 2-14 and 39-52 of demoIterator.policy, and from the values specified for the parameters os and cpu supplied on the command line. If these parameters are not specified on the command line, the default value for both is the regular expression .*, which means to include everything.

The constraints at lines 2-14 in demoIterator.policy define the work load for the resources. In this case, resources do not accept jobs if there are already four jobs queued up, and are not to run jobs if there are two or more jobs currently in progress.

To learn more about setting start, resource, or accept constraints in a policy file, see Defining Job Elements.

joblet_started_event

As the DemoIterator joblet is executed on a particular resource, it receives a joblet_started_event. When this happens, the DemoIterator joblet simply sleeps for a random amount of time to stagger the execution of the joblets, and then sends a command to the operating system, if one was supplied as a job argument. The command is executed on the target operating system using the built-in function system(), which is an alternative to using the more feature-rich class Exec.

For more information on sending commands to the operating system using the Exec class, see Exec .

After the joblet is finished running, a joblet_completed_event is sent to demoIteratorJob, which increments the variable completed, and posts the updated value to the job fact jobinstance.memo (see lines 35-37 in demoIterator.jdl). You can see the text for the memo displayed on the Job Log tab in the list of running jobs in the ZENworks Orchestrator Console.

For more information, see Starting and Stopping the ZENworks Orchestrator Console.