General interface to the Data Grid. See Section 3.0, Setting Up a Development Grid.
Interactions with the Data Grid revolve around instances of this class. Operations include copying files from the DataGrid down to the resource for joblet usage and likewise uploading files from a resource to the DataGrid.
A Data Grid URL is the convention for denoting paths in the Data Grid. The Data Grid URL convention is the form grid:///. The gridID is optional and if absent means the default grid. The ^ symbol can be used as a shortcut to the directory either standalone indicating the current job or followed by the jobid to identify a particular job. The ! symbol can be used as a shortcut to the deployed job’s home directory either standalone indicating the current jobs type or followed by the deployed jobs name. The ~ symbol can be used as a shortcut to the users home directory (in the datagrid) either standalone indicating the current user or followed by the desired user id to identify a particular user.
DataGrid(GridObjectInfo): Construct a DataGrid instance.
Appends the string data to the end of the grid file referenced by this path.
Parameters: destfile - File to append to. value - String data to append
Raises: Exception - Thrown if a system or configuration error prevents the append of data to the grid file.
Obtains the specified source file to the destination.A recursive copy is attempted if setRecursive(True) is set. The default is a single file copy. A multicast is attempted if setMulticast(True) is set. The default is to do a unicast copy.
Deletes the file or directory referenced by this path. If this path references a file, then the file is removed from the grid. If a directory is referenced, then the directory is recursively deleted.
Parameters: path - File or directory to delete.
Raises: Exception.
Retrieves the Boolean.setting for using cached data or not; if caching, true; otherwise, false.
Retrieves the Boolean value of debug logging. If debug logging is on, true; otherwise false..
Returns Boolean true if operation is to attempt multicast, otherwise false.
If true then fallback to unicast if multicast does not meer the setMulticastMin requirement; false to not fallback.
Retrieves an integer indicating the minimum number of receivers needed in order to multicast. If minimum number is not reached, then unicast is used.
Retreives the multicast minimum file size; returns the minimum file size for multicasting.
Retrieves the number of receivers that constitute a quorum and allow multicast to start.
Retrieves the requested data rate in bytes per second for multicast. Returns the requested data rate in bytes per second.
Retrieves the maximum amount of time a receiver is prepared to wait before start of multicast. Returns the wait time in milliseconds (ms).
Recursively creates a new empty directory. If no component in the path referenced by this object is a regular file, and the current user has permission to do so, then a directory is created that reflects the current data grid path. Any intermediate subdirectories are automatically created as well.
Parameter: dir - Directory path to create
Raises: Exception
Moves a file or directory from one path to another.
Parameter: source - The source to move. dest - Destination of move
Raises: Exception
Deletes the file or directory referenced by this path. If this path references a file, then the file is removed from the grid. If a directory is referenced, then the directory is recursively deleted.
Parameter: dir - Directory to delete
Raises: Exception
To use cached data or not.\.
Parameter: caching - true to use caching, false to not.
Attempts to obtain a file as part of a combined multicast transfer. This only applies to the copy() operation.
Behaves the same as a unicast copy(), except that an initial attempt is made to fetch the file as part of a multicast to this and potentially many other hosts at the same time. If successful, the total network bandwith consumed by all hosts can be greatly reduced.
If multicasting is not available or not enough hosts are willing to multicast the same file in the specified wait period, then this operation transparently falls back to a normal unicast copy() request.
Multicasting is intended for use by a set of nodes running the same job and requesting the same large file from the data grid. The multicastMin, multicastQuorum, and multicatWait are ignored for requests after the first to join any given multicast session.
Multicast also honors caching requests if caching is used.
Paramters: mcast - true to attempt multicast. Default is false.
If <>true, then a multicast attempt that does not meet the setMulticastMin requirement will fall back to a unicast file copy.
Parameters: mcastFallBack - true to fallback to unicast, false to not.
Sets the minimum number of receivers required in order to proceed with the multicast. If this number is not reached within the "setMulticastWait" time limit, then the receivers will either fall back to unicast or fail, depending on the "setMulticastFallback" option..
Returns: The minimum number of receivers.
Sets the multicast minimum file size. If file size is equal to or larger than this limit then it is considered for multcast. If it is smaller, then it is always unicast, despite the setting of setMulticast().
Parameters: minFileSize - The filesize in bytes. Default is 10,000,000 (10M)
Sets the number of receivers that constitute a quorum and allow multicast to start.
Once this many receivers join a multicast, the server is free to start sending immediately. A negative or zero value means there is no quorum. The wait time is allowed to fully expire before sending begins.
Parameters: mcastQuorum - The number of receivers.
Sets the multicast send rate limit in bytes per second. If zero, then the default rate limit defined on the server is used. This value may be capped by a maximum rate limit on the server, as well.
Parameters: mcastRate - The requested data rate in bytes per second.
Set the number of milliseconds to wait for a more receivers to join the multicast. The multicast will begin if this time expires, or if the "setMulticastQuorum" number of receivers is reached. This allows other hosts to join in the requested multicast. Zero wait means the default wait of 15 seconds. -1 means to wait infinitely.
Parameters: mcastWait - Wait time in milliseconds.
Performs recursive copy. Default is to not recurse.
Parameters: recursive - True for the copy() to do a recursive copy, including creating any directories.
Copies a file from the a Datagrid directory to a local file. This example assumes you have created the directory “newdir” in the DataGrid and have uploaded a file named “results.txt” there:
DataGrid().copy("grid:///newdir/results.txt","local.txt")
Copies a file from the job instance directory to a local file:
DataGrid().copy("grid:///^/results.txt","local.txt")
Copies a file from a named job instance directory to a local file:
DataGrid().copy("grid:///^user.myjob.1024/results.txt","local.txt")
Copies a local file into the job directory for job 'myjob':
DataGrid().copy("local.txt","grid:///!myjob")
Copies a local file into a subdir of the job directory:
DataGrid().copy("local.txt","grid:///!myjob/subdir")
Copies a local file into the current job instance directory:
DataGrid().copy("local.txt","grid:///^/")
Javadoc: DataGrid