edu.isi.pegasus.planner.transfer.mapper.impl

Class AbstractFileFactoryBasedMapper

java.lang.Object

edu.isi.pegasus.planner.transfer.mapper.impl.AbstractFileFactoryBasedMapper

All Implemented Interfaces:

OutputMapper

Direct Known Subclasses:

Flat, Hashed

public abstract class AbstractFileFactoryBasedMapper
extends java.lang.Object
implements OutputMapper

The abstract class that serves as the base class for the File Factory based mappers.

Author:

Karan Vahi

Field Summary

Fields

Modifier and Type	Field and Description
protected org.griphyn.vdl.euryale.FileFactory	mFactory
protected LogManager	mLogger The handle to the logger.
protected java.lang.String	mOutputSite The output site where the data needs to be placed.
protected SiteStore	mSiteStore Handle to the Site Catalog contents.
protected Directory	mStageoutDirectory The stage out directory where the outputs are staged to.

Fields inherited from interface edu.isi.pegasus.planner.transfer.mapper.OutputMapper

VERSION

Constructor Summary

Constructors

Constructor and Description
AbstractFileFactoryBasedMapper() The default constructor.

Method Summary

Modifier and Type	Method and Description
protected void	complainForStorageFileServer(FileServerType.OPERATION operation, java.lang.String site) Complains for a missing head node storage file server on a site for a job
protected java.lang.String	constructURL(FileServer server, java.lang.String addOn) Returns the full path on remote output site, where the lfn will reside, using the FileServer passed.
abstract java.lang.String	createAndGetAddOn(java.lang.String lfn, java.lang.String site, boolean existing) Returns the addOn part that is retrieved from the File Factory.
protected java.lang.String	getErrorMessagePrefix() Returns the prefix message to be attached to an error message
abstract java.lang.String	getShortName() Returns the short name for the implementation class.
void	initialize(PegasusBag bag, ADag workflow) Initializes the mappers.
abstract org.griphyn.vdl.euryale.FileFactory	instantiateFileFactory(PegasusBag bag, ADag workflow) Method that instantiates the FileFactory
protected Directory	lookupStorageDirectory(java.lang.String site) Looks up the site catalog to return the storage directory for a site
java.lang.String	map(java.lang.String lfn, java.lang.String site, FileServerType.OPERATION operation) Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location.
java.lang.String	map(java.lang.String lfn, java.lang.String site, FileServerType.OPERATION operation, boolean existing) Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location.
java.util.List<java.lang.String>	mapAll(java.lang.String lfn, java.lang.String site, FileServerType.OPERATION operation) Maps a LFN to a location on the filsystem of a site and returns all the possible equivalent externally accessible URL corresponding to that location.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.isi.pegasus.planner.transfer.mapper.OutputMapper

description

Field Detail

mLogger

protected LogManager mLogger

The handle to the logger.

mFactory

protected org.griphyn.vdl.euryale.FileFactory mFactory

mSiteStore

protected SiteStore mSiteStore

Handle to the Site Catalog contents.

mOutputSite

protected java.lang.String mOutputSite

The output site where the data needs to be placed.

mStageoutDirectory

protected Directory mStageoutDirectory

The stage out directory where the outputs are staged to.

Constructor Detail

AbstractFileFactoryBasedMapper

public AbstractFileFactoryBasedMapper()

The default constructor.

Method Detail

initialize

public void initialize(PegasusBag bag,
                       ADag workflow)
                throws MapperException

Initializes the mappers.

Specified by:

initialize in interface OutputMapper

Parameters:

bag - the bag of objects that is useful for initialization.

workflow - the workflow refined so far.

Throws:

MapperException - if unable to construct URL for any reason

instantiateFileFactory

public abstract org.griphyn.vdl.euryale.FileFactory instantiateFileFactory(PegasusBag bag,
                                                                           ADag workflow)

Method that instantiates the FileFactory

Parameters:

bag - the bag of objects that is useful for initialization.

workflow - the workflow refined so far.

Returns:

the handle to the File Factory to use

getShortName

public abstract java.lang.String getShortName()

Returns the short name for the implementation class.

Returns:

createAndGetAddOn

public abstract java.lang.String createAndGetAddOn(java.lang.String lfn,
                                                   java.lang.String site,
                                                   boolean existing)

Returns the addOn part that is retrieved from the File Factory. It creates a new file in the factory for the LFN and returns it.

Parameters:

lfn - the LFN to be used

site - the site at which the LFN resides

existing - indicates whether to create a new location/placement for a file, or rely on existing placement on the site.

Returns:

constructURL

protected java.lang.String constructURL(FileServer server,
                                        java.lang.String addOn)
                                 throws MapperException

Returns the full path on remote output site, where the lfn will reside, using the FileServer passed. This method creates a new File in the FileFactory space.

Parameters:

server - the file server to use

addOn - the addOn part containing the LFN

Returns:

the URL for the LFN

Throws:

MapperException

map

public java.lang.String map(java.lang.String lfn,
                            java.lang.String site,
                            FileServerType.OPERATION operation)
                     throws MapperException

Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location. If the storage directory for the site has multiple file servers associated with it, it selects one randomly and returns a URL accessible from that FileServer

Specified by:

map in interface OutputMapper

Parameters:

lfn - the lfn

site - the output site

operation - whether we want a GET or a PUT URL

Returns:

the URL to file that was mapped

Throws:

MapperException - if unable to construct URL for any reason

map

public java.lang.String map(java.lang.String lfn,
                            java.lang.String site,
                            FileServerType.OPERATION operation,
                            boolean existing)
                     throws MapperException

Maps a LFN to a location on the filsystem of a site and returns a single externally accessible URL corresponding to that location.

Specified by:

map in interface OutputMapper

Parameters:

lfn - the lfn

site - the output site

operation - whether we want a GET or a PUT URL

existing - indicates whether to create a new location/placement for a file, or rely on existing placement on the site.

Returns:

externally accessible URL to the mapped file.

Throws:

MapperException - if unable to construct URL for any reason

mapAll

public java.util.List<java.lang.String> mapAll(java.lang.String lfn,
                                               java.lang.String site,
                                               FileServerType.OPERATION operation)
                                        throws MapperException

Maps a LFN to a location on the filsystem of a site and returns all the possible equivalent externally accessible URL corresponding to that location. For example, if a file on the filesystem is accessible via multiple file servers it should return externally accessible URL's from all the File Servers on the site.

Specified by:

mapAll in interface OutputMapper

Parameters:

lfn - the lfn

site - the output site

operation - whether we want a GET or a PUT URL

Returns:

List of externally accessible URLs to the mapped file.

Throws:

MapperException - if unable to construct URL for any reason

lookupStorageDirectory

protected Directory lookupStorageDirectory(java.lang.String site)
                                    throws MapperException

Looks up the site catalog to return the storage directory for a site

Parameters:

site - the site

Returns:

Throws:

MapperException

complainForStorageFileServer

protected void complainForStorageFileServer(FileServerType.OPERATION operation,
                                            java.lang.String site)

Complains for a missing head node storage file server on a site for a job

Parameters:

operation - the file server operation

site - the site

getErrorMessagePrefix

protected java.lang.String getErrorMessagePrefix()

Returns the prefix message to be attached to an error message

Returns: