7.1 Understanding JDL

The Orchestrator Grid Management system uses an embedded Python-based language for describing jobs (called the Job Definition Language or JDL). This scripting language is used to control the job flow, request resources, handle events and generally interact with the Grid server as jobs proceed.

Jobs run in an environment that expects facts (information) to exist about available resources. These facts are either set up manually through configuration or automatically discovered via discovery jobs. Both the end-user jobs and the discovery jobs have the same structure and language. The only difference is in how they are scheduled.

The job JDL controls the complete life cycle of the job. JDL is a scripting language, so it does not provide compile-time type checking. There are no checks for infinite loops, although various precautions are available to protect against runaway jobs, including job and joblet timeouts, maximum resource consumption, quotas, and limited low-priority JDL thread execution.

As noted, the JDL language is based on the industry standard Python language, which was chosen because of its widespread use for test script writing in QA departments, its performance, its readability of code, and ease to learn.

The Python language has all the familiar looping and conditional operations as well as some powerful operations. There are various books on the language including O’Reilly’s Python in a Nutshell and Learning Python. Online resources are available at http://www.python.org

Within the Orchestrator Server and grid jobs, JDL not only adds a suite of new commands but also provides an event-oriented programming environment. A job is notified of every state change or activity by calling an appropriately named event handler method.

A job only defines handlers for events it is interested in. In addition to built-in events (such as, joblet_started_event, job_completed_event, job_cancelled_event, and job_started_event) it can define handlers for custom events caused by incoming messages. For example, if a job (Job class) defines a method as follows:

    def my_custom_event(self, job, params):
      print \u201cGot a my_custom event carrying ", params)

And the joblet (Joblet class) sends an event/message as follows:

    self.sendEvent(“my_custom”, {“arg1”:”one”})

The following line is added to the job log:

    Got a my_custom event carrying arg1=”one”

JDL can also define timer events (periodic and one-time) with similar event handlers.

Each event handler can run in a separate thread for parallel execution or can be synchronized to a single thread. A separate thread results in better performance, but also incurs the development expense of ensuring that shared data structures are thread safe.