JobControlManager (LCSim 3.1.7 API)

java.lang.Object
- org.lcsim.job.JobControlManager

```
public class JobControlManager
extends Object
```
This class provides a front end for running and managing LCSim event processing jobs using XML steering files.
More details about this XML format can be found at the
LCSim XML Confluence Page.
The command line syntax is:
java org.lcsim.job.JobManager steeringFile.xml [options]
To see the available command line options with descriptions, run with "-h" as the only option.
Command-line parameters that can be defined using switches are overridden by the corresponding settings in the job XML file, if they are present. This means that if these parameters are to be taken from the CL, the matching settings should be left out of the XML job file. This is not the case, however, for input files specified by the "-i" option, which are appended to the ones listed in the steering file.

Version:

$Id: JobControlManager.java,v 1.65 2013/02/14 22:30:07 jeremy Exp $

Author:

Jeremy McCormick

Field Summary

Fields
Modifier and Type	Field and Description
`private File`	`cacheDirectory` Root directory for file caching.
`private ClassLoader`	`classLoader` The class loader that will be used for the job.
`(package private) org.apache.commons.cli.CommandLine`	`commandLine` Command line that is setup from `main` arguments.
`private Map<String,Double>`	`constantsMap` Map of constants definitions.
`private String`	`detectorName` User supplied detector name.
`private DriverAdapter`	`driverAdapter` A driver adapter created on the fly in case it is needed by an external program.
`private List<Driver>`	`driverExec` List of drivers to execute in the job.
`private Map<String,Driver>`	`driverMap` Map of driver names to objects.
`private boolean`	`dryRun` Enable dry run so no events are processed.
`private boolean`	`dummyDetector` Setup a "dummy" detector in the conditions system.
`private Long`	`eventPrintInterval` Event printing interval (null means no event printing).
`private JDOMExpressionFactory`	`factory` JDOM expression factory for variables.
`private FileCache`	`fileCache` File cache.
`private List<File>`	`inputFiles` List of input LCIO files.
`private boolean`	`isSetup` Flag set to `true` after setup is performed.
`private long`	`jobEnd` The job end timestamp in ms.
`private long`	`jobStart` The job start timestamp in ms.
`protected static Logger`	`LOGGER` Initialize the logger which uses the package name.
`private LCSimLoop`	`loop` The LCIO record loop.
`private int`	`numberOfEvents` Number of events to run before stopping job.
`private static org.apache.commons.cli.Options`	`OPTIONS` The command line options.
`private ParameterConverters`	`paramConverter` Helper for converting Driver parameters.
`private boolean`	`printDriverStatistics` Set to `true` to print out driver statistics at the end of the job.
`private File`	`rewriteFile` Path for rewriting steering file with variables resolved.
`private boolean`	`rewriteSteering` Enable rewriting of the steering file to a new path with variables resolved.
`private org.jdom.Element`	`root` The root node of the XML document providing config to the manager.
`private Integer`	`runNumber` User supplied run number.
`private int`	`skipEvents` Number of events to skip at start of job.
`private boolean`	`useSteeringResource` Interpret steering file argument as a resource rather than file path.
`private static Pattern`	`VARIABLE_PATTERN` The regular expression for extracting the variables from an XML file.
`private Map<String,String>`	`variableMap` Map of variable names to their values.

Constructor Summary

Constructors
Constructor and Description

JobControlManager()
The default constructor.

Constructors
Constructor and Description
`JobControlManager()` The default constructor.

Method Summary

Methods
Modifier and Type	Method and Description
`private void`	`addDriver(String name, Driver driver)` Add a Driver to the internal Driver map.
`void`	`addInputFile(File inputFile)` Add an input LCIO file to be proceesed.
`void`	`addVariableDefinition(String key, String value)` Add a variable definition to be substituted into the job's XML file.
`void`	`configure()` Configure start of job (usually done automatically).
`private static org.apache.commons.cli.Options`	`createCommandLineOptions()` Create the command line options.
`private void`	`createDriverAdapter()` Create a driver adapter.
`private void`	`createDriverExecList()` Create the `Driver` execution list.
`void`	`enableHeadlessMode()` Turn on the batch analysis factory so plots are not shown on the screen even when `Plotter.show()` is called.
`void`	`finish()` Activate end of job hooks (usually done automatically).
`protected String`	`getDetectorName()` Get the detector name set by the command line options.
`DriverAdapter`	`getDriverAdapter()` Get a `DriverAdapter` from the currently configured Driver list.
`List<Driver>`	`getDriverExecList()` Return a list of Drivers to be executed.
`LCSimLoop`	`getLCSimLoop()` Get the `LCSimLoop` of this JobManager.
`private static Class`	`getPrimitiveType(String name)` Get the Java primitive type class from a type name.
`protected Integer`	`getRunNumber()` Get the run number set by the command line options.
`private List<Method>`	`getSetterMethods(Class klass)` Get a list of a class's setter methods.
`protected void`	`initializeConditions()` Initialize the conditions system before the job starts.
`private void`	`initializeLoop()` Initialize the `LCSimLoop`.
`static void`	`main(String[] args)` Run from the command line.
`void`	`parse(String[] args)` Parse command-line options and setup job state from them.
`private static void`	`printHelp()` Print help and exit.
`private void`	`printInputFileList()` Print the list of input files.
`private void`	`printUserClasspath()` Print out extra URLs added to the classpath from the XML.
`private void`	`processConstants()` Create the constants from the XML file.
`private void`	`processDriverParameters(Class driverClass, Driver newDriver, List<org.jdom.Element> parameters)` A fairly ugly method to process the provided XML parameters on a `Driver`.
`void`	`processEvent(EventHeader event)` Process a single event.
`private File`	`processFileElement(org.jdom.Element fileElement, List<File> fileList)` Create a `File` object from the text in an XML element.
`private File`	`processFileText(String fileText, List<File> fileList)` Cleanup file text and add to the file list.
`private String`	`processPath(String path)` Process the file path string to substitute in the user's home directory for the "~" character.
`private void`	`rewriteXMLSteering(org.jdom.Document doc)` Rewrite the XML steering file after resolving variables (done externally).
`boolean`	`run()` Execute a job using the current parameters.
`void`	`setDryRun(boolean dryRun)` Set whether a dry run should be performed which will only perform setup and not process any events.
`void`	`setNumberOfEvents(int numberOfEvents)` Set the number of events to run on the loop before ending the job.
`private void`	`setup(org.jdom.Document xmlDocument)` Setup the job parameters from an XML Document.
`void`	`setup(File file)` Setup job parameters from a `File`.
`void`	`setup(InputStream in)` Setup job parameters from an `InputStream` that should be valid XML text.
`void`	`setup(String resourceURL)` Setup job parameters from an embedded resource.
`private void`	`setupClassLoader()` Setup the manager's class loader.
`private void`	`setupDrivers()` Create the drivers from the XML.
`private void`	`setupFileCache()` Setup the file cache.
`private void`	`setupInputFiles()` Setup the list of input files to be processed from the XML job file.
`private void`	`setupJobControlParameters()` Setup the job control parameters using the XML steering document.
`private void`	`setupUnits()` Setup the system of units.
`private void`	`substituteVariables(org.jdom.Document doc)` Perform variable substitution within all text data in a entire document.
`private void`	`substituteVariables(org.jdom.Element element)` Substitute values from the `variableMap` into an XML element and all its children, recursively.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOGGER
```
protected static final Logger LOGGER
```
    Initialize the logger which uses the package name.
  - OPTIONS
```
private static final org.apache.commons.cli.Options OPTIONS
```
    The command line options.
  - VARIABLE_PATTERN
```
private static final Pattern VARIABLE_PATTERN
```
    The regular expression for extracting the variables from an XML file.
  - cacheDirectory
```
private File cacheDirectory
```
    Root directory for file caching.
  - classLoader
```
private ClassLoader classLoader
```
    The class loader that will be used for the job.
  - commandLine
```
org.apache.commons.cli.CommandLine commandLine
```
    Command line that is setup from main arguments.
  - constantsMap
```
private final Map<String,Double> constantsMap
```
    Map of constants definitions.
  - detectorName
```
private String detectorName
```
    User supplied detector name.
  - driverAdapter
```
private DriverAdapter driverAdapter
```
    A driver adapter created on the fly in case it is needed by an external program.
  - driverExec
```
private final List<Driver> driverExec
```
    List of drivers to execute in the job.
  - driverMap
```
private final Map<String,Driver> driverMap
```
    Map of driver names to objects.
  - dryRun
```
private boolean dryRun
```
    Enable dry run so no events are processed.
  - dummyDetector
```
private boolean dummyDetector
```
    Setup a "dummy" detector in the conditions system.
  - eventPrintInterval
```
private Long eventPrintInterval
```
    Event printing interval (null means no event printing).
  - factory
```
private final JDOMExpressionFactory factory
```
    JDOM expression factory for variables.
  - fileCache
```
private FileCache fileCache
```
    File cache.
  - inputFiles
```
private final List<File> inputFiles
```
    List of input LCIO files.
  - isSetup
```
private boolean isSetup
```
    Flag set to true after setup is performed.
  - jobEnd
```
private long jobEnd
```
    The job end timestamp in ms.
  - jobStart
```
private long jobStart
```
    The job start timestamp in ms.
  - loop
```
private LCSimLoop loop
```
    The LCIO record loop.
  - numberOfEvents
```
private int numberOfEvents
```
    Number of events to run before stopping job.
  - paramConverter
```
private final ParameterConverters paramConverter
```
    Helper for converting Driver parameters.
  - printDriverStatistics
```
private boolean printDriverStatistics
```
    Set to true to print out driver statistics at the end of the job.
  - rewriteFile
```
private File rewriteFile
```
    Path for rewriting steering file with variables resolved.
  - rewriteSteering
```
private boolean rewriteSteering
```
    Enable rewriting of the steering file to a new path with variables resolved.
  - root
```
private org.jdom.Element root
```
    The root node of the XML document providing config to the manager.
  - runNumber
```
private Integer runNumber
```
    User supplied run number.
  - skipEvents
```
private int skipEvents
```
    Number of events to skip at start of job.
  - useSteeringResource
```
private boolean useSteeringResource
```
    Interpret steering file argument as a resource rather than file path.
  - variableMap
```
private final Map<String,String> variableMap
```
    Map of variable names to their values.
- Constructor Detail
  - JobControlManager
```
public JobControlManager()
```
    The default constructor.
- Method Detail
  - createCommandLineOptions
```
private static org.apache.commons.cli.Options createCommandLineOptions()
```
    Create the command line options.
    
    Returns:
    The command line options for the manager.
  - getPrimitiveType
```
private static Class getPrimitiveType(String name)
```
    Get the Java primitive type class from a type name.
    
    Parameters:
    name - The name of the type.
    
    Returns:
    The primitive type class.
  - main
```
public static void main(String[] args)
```
    Run from the command line.
    Takes command-line options (use -h option to see them).
    
    Parameters:
    args - the command line arguments
  - printHelp
```
private static void printHelp()
```
    Print help and exit.
  - addDriver
```
private void addDriver(String name,
             Driver driver)
```
    Add a Driver to the internal Driver map.
    
    Parameters:
    name - the unique name of the Driver
    driver - the instance of the Driver
  - addInputFile
```
public void addInputFile(File inputFile)
```
    Add an input LCIO file to be proceesed.
    
    Parameters:
    inputFile - The input LCIO file.
  - addVariableDefinition
```
public void addVariableDefinition(String key,
                         String value)
```
    Add a variable definition to be substituted into the job's XML file. This method is public so that caller's not using the CL can still define necessary variables for the steering file.
    
    Parameters:
    key - The variable name.
    value - The variable's value.
  - configure
```
public void configure()
```
    Configure start of job (usually done automatically).
  - createDriverAdapter
```
private void createDriverAdapter()
```
    Create a driver adapter.
  - createDriverExecList
```
private void createDriverExecList()
```
    Create the Driver execution list.
  - enableHeadlessMode
```
public void enableHeadlessMode()
```
    Turn on the batch analysis factory so plots are not shown on the screen even when Plotter.show() is called.
  - finish
```
public void finish()
```
    Activate end of job hooks (usually done automatically).
  - getDriverAdapter
```
public DriverAdapter getDriverAdapter()
```
    Get a DriverAdapter from the currently configured Driver list.
    
    Returns:
    the driver adapter
  - getDriverExecList
```
public List<Driver> getDriverExecList()
```
    Return a list of Drivers to be executed. This can be used from an external framework like JAS3. The list will be empty unless the setup() method has been called.
    
    Returns:
    A List of Drivers.
  - getLCSimLoop
```
public LCSimLoop getLCSimLoop()
```
    Get the LCSimLoop of this JobManager.
    
    Returns:
    The LCSimLoop.
  - getSetterMethods
```
private List<Method> getSetterMethods(Class klass)
```
    Get a list of a class's setter methods.
    
    Parameters:
    klass - The class.
    
    Returns:
    A list of setter methods.
  - initializeLoop
```
private void initializeLoop()
```
    Initialize the LCSimLoop.
  - parse
```
public void parse(String[] args)
```
    Parse command-line options and setup job state from them. This method calls setup(File) to load the steering paramters from an XML file, after processing other command line options. This method is private so that callers must all use the main(String[]) routine as the primary entry point.
    
    Parameters:
    args - The command line arguments.
  - printInputFileList
```
private void printInputFileList()
```
    Print the list of input files.
  - printUserClasspath
```
private void printUserClasspath()
```
    Print out extra URLs added to the classpath from the XML.
  - processConstants
```
private void processConstants()
```
    Create the constants from the XML file.
  - processDriverParameters
```
private void processDriverParameters(Class driverClass,
                           Driver newDriver,
                           List<org.jdom.Element> parameters)
```
    A fairly ugly method to process the provided XML parameters on a Driver.
    
    Parameters:
    driverClass - the Java class of the driver
    newDriver - the instantiated Driver
    parameters - the list of XML parameters
  - processEvent
```
public void processEvent(EventHeader event)
```
    Process a single event.
    
    Parameters:
    event -
  - processFileElement
```
private File processFileElement(org.jdom.Element fileElement,
                      List<File> fileList)
```
    Create a File object from the text in an XML element.
    
    Parameters:
    fileElement - The element containing a file path or URL.
    fileList - List to append new File.
    
    Returns:
    The File object.
  - processFileText
```
private File processFileText(String fileText,
                   List<File> fileList)
```
    Cleanup file text and add to the file list.
    
    Parameters:
    fileText - the text containing the file's path from the XML
    fileList - the list of files to which the new file will be appended
    
    Returns:
    the file that was created from the text
  - processPath
```
private String processPath(String path)
```
    Process the file path string to substitute in the user's home directory for the "~" character. This is needed if running on Windows.
    
    Parameters:
    path - The original path.
    
    Returns:
    The path with home dir substitution.
  - rewriteXMLSteering
```
private void rewriteXMLSteering(org.jdom.Document doc)
```
    Rewrite the XML steering file after resolving variables (done externally). The output path is set by command line argument to "-w".
    
    Parameters:
    doc - The XML steering doc with variables substituted.
  - run
```
public boolean run()
```
    Execute a job using the current parameters.
  - setDryRun
```
public void setDryRun(boolean dryRun)
```
    Set whether a dry run should be performed which will only perform setup and not process any events.
    
    Parameters:
    dryRun - true to enable a dry run
  - setNumberOfEvents
```
public void setNumberOfEvents(int numberOfEvents)
```
    Set the number of events to run on the loop before ending the job. This should be called after the setup(File) method is called or it will be overridden.
  - setup
```
private void setup(org.jdom.Document xmlDocument)
```
    Setup the job parameters from an XML Document.
    This method contains the primary logic for setting up job parameters from an XML file. The other setup methods such as setup(InputStream), setup(String) and setup(File) all call this method.
    
    Parameters:
    xmlDocument - The lcsim recon XML document describing the job.
  - setup
```
public void setup(File file)
```
    Setup job parameters from a File.
    
    Parameters:
    file - the path to the XML file
  - setup
```
public void setup(InputStream in)
```
    Setup job parameters from an InputStream that should be valid XML text.
    
    Parameters:
    in - the XML input stream
  - setup
```
public void setup(String resourceURL)
```
    Setup job parameters from an embedded resource. This method calls setup(InputStream).
  - setupClassLoader
```
private void setupClassLoader()
```
    Setup the manager's class loader.
  - setupDrivers
```
private void setupDrivers()
```
    Create the drivers from the XML.
  - setupFileCache
```
private void setupFileCache()
```
    Setup the file cache.
  - setupInputFiles
```
private void setupInputFiles()
```
    Setup the list of input files to be processed from the XML job file.
  - setupJobControlParameters
```
private void setupJobControlParameters()
```
    Setup the job control parameters using the XML steering document.
  - setupUnits
```
private void setupUnits()
```
    Setup the system of units.
  - substituteVariables
```
private void substituteVariables(org.jdom.Document doc)
```
    Perform variable substitution within all text data in a entire document.
    
    Parameters:
    doc - The XML document.
  - substituteVariables
```
private void substituteVariables(org.jdom.Element element)
```
    Substitute values from the variableMap into an XML element and all its children, recursively.
    
    Parameters:
    element - The XML element.
  - initializeConditions
```
protected void initializeConditions()
                             throws ConditionsManager.ConditionsNotFoundException
```
    Initialize the conditions system before the job starts.
    Sub-classes should override this if custom conditions system initialization is required.
    
    Throws:
    
    ConditionsManager.ConditionsNotFoundException - if some conditions were not found
  - getRunNumber
```
protected Integer getRunNumber()
```
    Get the run number set by the command line options.
    
    Returns:
    the run number or null if not set
  - getDetectorName
```
protected String getDetectorName()
```
    Get the detector name set by the command line options.
    
    Returns:
    the detector name or null if not set

Class JobControlManager

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOGGER

OPTIONS

VARIABLE_PATTERN

cacheDirectory

classLoader

commandLine

constantsMap

detectorName

driverAdapter

driverExec

driverMap

dryRun

dummyDetector

eventPrintInterval

factory

fileCache

inputFiles

isSetup

jobEnd

jobStart

loop

numberOfEvents

paramConverter

printDriverStatistics

rewriteFile

rewriteSteering

root

runNumber

skipEvents

useSteeringResource

variableMap

Constructor Detail

JobControlManager

Method Detail

createCommandLineOptions

getPrimitiveType

main

printHelp

addDriver

addInputFile

addVariableDefinition

configure

createDriverAdapter

createDriverExecList

enableHeadlessMode

finish

getDriverAdapter

getDriverExecList

getLCSimLoop

getSetterMethods

initializeLoop

parse

printInputFileList

printUserClasspath

processConstants

processDriverParameters

processEvent

processFileElement

processFileText

processPath

rewriteXMLSteering

run

setDryRun

setNumberOfEvents

setup

setup

setup

setup

setupClassLoader

setupDrivers

setupFileCache

setupInputFiles

setupJobControlParameters

setupUnits