Reference implementation for a simple test iterator. Several concepts are demonstrated: 1) Using policy constraints and job arguments to restrict joblet execution to a specific resource, 2) Scheduling joblets using a ParameterSpace, and 3) An example of executing a command on a resource.
> zos login --user zenuser Please enter current password for 'zenuser': Logged into grid as zenuser > zos jobinfo --detail demoIterator Jobname/Parameters Attributes ------------------ ---------- demoIterator Desc: This example job is a reference for a simple test iterator. It is useful for demonstrating how policies and job args can be used to target the job to a particular resource. numJoblets Desc: joblets to run Type: Integer Default: 100 cmd Desc: Simple command to execute Type: String Default: os Desc: Regular expression match for Operating System Type: String Default: .* cpu Desc: Regular expression match for CPU architecture Type: String Default: .*
The files that make up the DemoIterator job include:
demoIterator # Total: 156 lines |-- demoIterator.jdl # 79 lines `-- demoIterator.policy # 77 lines
1 # ----------------------------------------------------------------------------- 2 # Copyright © 2008 Novell, Inc. All Rights Reserved. 3 # 4 # NOVELL PROVIDES THE SOFTWARE "AS IS," WITHOUT ANY EXPRESS OR IMPLIED 5 # WARRANTY, INCLUDING WITHOUT THE IMPLIED WARRANTIES OF MERCHANTABILITY, 6 # FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGMENT. NOVELL, THE AUTHORS 7 # OF THE SOFTWARE, AND THE OWNERS OF COPYRIGHT IN THE SOFTWARE ARE NOT LIABLE 8 # FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 9 # TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE 10 # OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 11 # ----------------------------------------------------------------------------- 12 # $Id: demoIterator.jdl,v 1.4 2008/03/05 20:05:48 ray Exp $ 13 # ----------------------------------------------------------------------------- 14 15 import time, random 16 17 # 18 # Add to the 'examples' group on deployment 19 # 20 if __mode__ == "deploy": 21 try: 22 jobgroupname = "examples" 23 jobgroup = getMatrix().getGroup(TYPE_JOB, jobgroupname) 24 if jobgroup == None: 25 jobgroup = getMatrix().createGroup(TYPE_JOB, jobgroupname) 26 jobgroup.addMember(__jobname__) 27 except: 28 exc_type, exc_value, exc_traceback = sys.exc_info() 29 print "Error adding %s to %s group: %s %s" % (__jobname__, jobgroupname, exc_type, exc_value) 30 31 32 class demoIteratorJob(Job): 33 34 def job_started_event(self): 35 print 'job_started_event' 36 self.completed = 0 37 38 # Launch the joblets 39 numJoblets = self.getFact("jobargs.numJoblets") 40 print 'Launching ', numJoblets, ' joblets' 41 42 pspace = ParameterSpace() 43 i = 1 44 while i <= numJoblets: 45 pspace.appendRow({'name':'joblet'+str(i)}) 46 i += 1 47 pspace.maxJobletSize = 1 48 self.schedule(demoIteratorJoblet,pspace,{}) 49 50 def joblet_completed_event(self, jobletnumber, node): 51 self.completed += 1 52 self.setFact("jobinstance.memo", "Tests run: %s" % (self.completed)) 53 54 55 class demoIteratorJoblet(Joblet): 56 57 def joblet_started_event(self): 58 print "Hi from joblet ", self.getFact("joblet.number") 59 time.sleep(random.random() * 15) 60 61 cmd = self.getFact("jobargs.cmd") 62 if len(cmd) > 0: 63 system(cmd) 64 65 66 67 # Example of more sophisticated exec 68 # e.g. e.signal("SIGUSR1") 69 """ 70 e = Exec() 71 e.setCommand(cmd) 72 #e.setStdoutFile("cmd.out") 73 e.writeStdoutToLog() 74 e.writeStderrToLog() 75 #try: 76 e.execute() 77 #except: 78 #self.retry("retryable example error") 79 """
1 <!-- 2 *============================================================================= 3 * Copyright (c) 2008 Novell, Inc. All Rights Reserved. 4 * 5 * NOVELL PROVIDES THE SOFTWARE "AS IS," WITHOUT ANY EXPRESS OR IMPLIED 6 * WARRANTY, INCLUDING WITHOUT THE IMPLIED WARRANTIES OF MERCHANTABILITY, 7 * FITNESS FOR A PARTICULAR PURPOSE, AND NON INFRINGMENT. NOVELL, THE AUTHORS 8 * OF THE SOFTWARE, AND THE OWNERS OF COPYRIGHT IN THE SOFTWARE ARE NOT LIABLE 9 * FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 10 * TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE 11 * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 12 *============================================================================= 13 * $Id: demoIterator.policy,v 1.2 2008/02/27 20:49:34 john Exp $ 14 *============================================================================= 15 --> 16 17 <policy> 18 <constraint type="accept" reason="Too busy for more work. Try again later!"> 19 <or> 20 <lt fact="job.instances.queued" value="4" /> 21 <contains fact="user.groups" value="superuser" /> 22 </or> 23 </constraint> 24 25 <constraint type="start" reason="Waiting on queue"> 26 <or> 27 <lt fact="job.instances.active" value="2" /> 28 <contains fact="user.groups" value="superuser" /> 29 </or> 30 </constraint> 31 32 <jobargs> 33 <fact name="numJoblets" 34 type="Integer" 35 description="joblets to run" 36 value="100" 37 visible="true" /> 38 39 <fact name="cmd" 40 type="String" 41 description="Simple command to execute" 42 value="" /> 43 44 <fact name="os" 45 type="String" 46 description="Regular expression match for Operating System" 47 value=".*" /> 48 49 <fact name="cpu" 50 type="String" 51 description="Regular expression match for CPU architecture" 52 value=".*" /> 53 </jobargs> 54 55 <constraint type="resource" reason="Does not match"> 56 <and> 57 <eq fact="resource.os.family" factvalue="jobargs.os" match="regexp" /> 58 <eq fact="resource.cpu.architecture" factvalue="jobargs.cpu" match="regexp"/> 59 60 <or> 61 <and> 62 <defined fact="env.VENDOR" /> 63 <eq fact="resource.os.vendor" factvalue="env.VENDOR" match="regexp" /> 64 </and> 65 <undefined fact="env.VENDOR" /> 66 </or> 67 </and> 68 </constraint> 69 70 <job> 71 <fact name="description" 72 type="String" 73 value="This example job is a reference for a simple test iterator. It is useful for demonstrating how policies and job args can be used to target the job to a particular resource." /> 74 </job> 75 76 </policy> 77
A representation of a running job instance.
Defines execution on the resource.
A representation of the matrix grid object, which provides operations for retrieving and creating grid objects in the system. MatrixInfo is retrieved using the built-in getMatrix() function. Write capability is dependent on the context in which getMatrix() is called. For example, in a joblet process on a resource, creating new grid objects is not supported.
A representation of Group grid objects. Operations include retrieving the group member lists and adding/removing from the group member lists, and retrieving and setting facts on the group.
Used to manage command line execution on resources.
Defines a parameter space to be used by the scheduler to create a Joblet set. A parameter space might consist of rows of columns or a list of columns that is expanded and can be turned into a cross product.
The following sections describe the DemoIterator job:
The deployment for the DemoIterator job is performed by lines 3-15 of demoIterator.jdl. When jobs are deployed into the grid, they can optionally be organized for grouping. In this case, the demoIterator job is added to the group named examples, and can be displayed in the ZENworks Orchestrator Console in the Explorer view at the location:
/ZOS/YOUR_GRID/Jobs/examples
For a general overview of how jobs are added to groups during deployment, see Walkthrough: Deploy a Sample Job
in the Novell ZENworks Orchestrator 1.3 Installation and Getting Started Guide.
When the DemoIterator job receives a job_started_event, it creates a ParameterSpace JDL class and adds the number of rows as indicated by the value of the argument numJoblets (see lines 27-31 in demoIterator.jdl). A ParameterSpace object is like a spreadsheet, containing rows and columns of information that might all be given to one joblet or sliced up across many joblets at schedule time. In this case, the ParameterSpace is told that maxJobletSize is 1 (see line 32), meaning a joblet instance is created for each row in the ParameterSpace during job scheduling (see line 33).
Not shown in this example is the fact that a joblet can get access to this “spreadsheet” of information by calling self.getParameterSpace(), and calling hasNext() and next() to enumerate through each row of information. To learn more about putting information in a ParameterSpace object from a job and obtaining that information from the JobletParameterSpace object from a joblet, see ParameterSpace.
The resource that runs the joblet is determined from the resource constraint specified in lines 2-14 and 39-52 of demoIterator.policy, and from the values specified for the parameters os and cpu supplied on the command line. If these parameters are not specified on the command line, the default value for both is the regular expression .*, which means to include everything.
The constraints at lines 2-14 in demoIterator.policy define the work load for the resources. In this case, resources do not accept jobs if there are already four jobs queued up, and are not to run jobs if there are two or more jobs currently in progress.
To learn more about setting start, resource, or accept constraints in a policy file, see Defining Job Elements.
As the DemoIterator joblet is executed on a particular resource, it receives a joblet_started_event. When this happens, the DemoIterator joblet simply sleeps for a random amount of time to stagger the execution of the joblets and then sends a command to the operating system, if one was supplied as a job argument. The command is executed on the target operating system using the built-in function system(), which is an alternative to using the more feature-rich class Exec.
For more information on sending commands to the operating system using the Exec class, see Exec .
After the joblet is finished running, a joblet_completed_event is sent to demoIteratorJob, which increments the variable completed, and posts the updated value to the job fact jobinstance.memo (see lines 35-37 in demoIterator.jdl). You can see the text for the memo displayed on the Job Log tab in the list of running jobs in the ZENworks Orchestrator Console.
For more information, see Starting and Stopping the ZENworks Orchestrator Console
.
Execute the following commands to deploy and run demoIterator.job:
Deploy demoIterator.job into the grid:
> zosadmin deploy demoIterator.job
Display the list of deployed jobs:
> zos joblist
should appear in this list.
Run the job on the first available resource without regard to OS or CPU, and use the default value for number of joblets, which is 100:
> zos run demoIterator
Run 10 joblets on Intel Windows resources, and launch the Notepad* application on each one:
> zos run demoIterator numJoblets=10 cmd=notepad os=Windows cpu=i386
NOTE:If a resource with the matching OS is not available, the job remains in the “waiting” state.
Here is an example that runs the pwd command on three joblets on the Linux operating system:
> zos run demoIterator numJoblets=3 cmd=pwd os=linux JobID: zenuser.demoIterator.417 zos log zenuser.demoIterator.417 job_started_event Launching 3 joblets [freeze] Hi from joblet 1 [freeze] /var/opt/novell/zenworks/zos/agent/node.default/freeze/zenuser.demoIterator.417.1 [skate] Hi from joblet 0 [skate] /var/opt/novell/zenworks/zos/agent/node.default/skate/zenuser.demoIterator.417.0 [melt] Hi from joblet 2 [melt] /var/opt/novell/zenworks/zos/agent/node.default/melt/zenuser.demoIterator.417.2
Setting Constraints Using Policies (see Section 4.4, Policy Management and Section 5.0, Developing Policies).
Adding Jobs to Groups During Deployment (see how the JDL code can print the ID of group of jobs in factJunction.job).
quickie.job demonstrates how a job starts up multiple instances of a joblet on one or more resources. The Joblet class defines how a joblet is executed on a resource.
Setting default parameter values using policies
Configuring constraints in a policy file
Naming conventions for policy facts (see Section 3.1.1, Naming Orchestrator Job Files)
Facts provided by the ZENworks Orchestrator system that can be referenced within a JDL file
Using zos
Running commands using the Exec class