Class JythonTopLevelDataSetHandlerV2<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>

java.lang.Object
ch.systemsx.cisd.etlserver.AbstractTopLevelDataSetRegistrator
ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator<T>
ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T>
ch.systemsx.cisd.etlserver.registrator.api.v2.JythonTopLevelDataSetHandlerV2<T>
All Implemented Interfaces:
ch.systemsx.cisd.common.filesystem.FaultyPathDirectoryScanningHandler.IFaultyPathDirectoryScanningHandlerDelegate, ch.systemsx.cisd.common.filesystem.IPathHandler, ch.systemsx.cisd.common.filesystem.IStopSignaler, ch.systemsx.cisd.common.utilities.ISelfTestable, ch.systemsx.cisd.etlserver.ITopLevelDataSetRegistrator, ch.systemsx.cisd.etlserver.registrator.v2.IOmniscientEntityRegistrator<T>

public class JythonTopLevelDataSetHandlerV2<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation> extends ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T>
A top-level data set handler that runs a python (jython) script to register data sets.
  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    JythonTopLevelDataSetHandlerV2.JythonDataSetRegistrationService<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
     
    static enum 
     
    static class 
    JythonTopLevelDataSetHandlerV2.ProgrammableDropboxObjectFactory<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
     

    Nested classes/interfaces inherited from class ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler

    ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler.RecoveryHookAdaptor

    Nested classes/interfaces inherited from class ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator

    ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator.DoNothingDelegatedAction, ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator.NoOpDelegate, ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator.OmniscientTopLevelDataSetRegistratorState, ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator.PostRegistrationCleanUpAction
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected static final String
     
    protected static final String
    The name of the local variable under which the incoming directory is made available to the script.
    static final String
     
    protected final ch.systemsx.cisd.common.jython.IJythonInterpreterFactory
     
    static final String
     
    protected final File
     
    protected static final String
    The name of the local variable under which the service is made available to the script.
    protected static final String
    The name of the local variable under which the global state
    protected static final String
    The name of the local variable under which the transaction is made available to the script.

    Fields inherited from class ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator

    notificationLog, operationLog, state
  • Constructor Summary

    Constructors
    Constructor
    Description
    JythonTopLevelDataSetHandlerV2(ch.systemsx.cisd.etlserver.TopLevelDataSetRegistratorGlobalState globalState)
    Constructor.
  • Method Summary

    Modifier and Type
    Method
    Description
    protected ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T>
    createDataSetRegistrationService(ch.systemsx.cisd.etlserver.registrator.DataSetFile incomingDataSetFile, ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation callerDataSetInformationOrNull, ch.systemsx.cisd.common.action.IDelegatedActionWithResult<Boolean> cleanAfterwardsAction, ch.systemsx.cisd.etlserver.ITopLevelDataSetRegistratorDelegate delegate)
    Create a registration service that includes a python interpreter (we need the interpreter in the service so we can use it in error handling).
    protected ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T>
    createJythonDataSetRegistrationServiceV2(ch.systemsx.cisd.etlserver.registrator.DataSetFile incomingDataSetFile, ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation userProvidedDataSetInformationOrNull, ch.systemsx.cisd.common.action.IDelegatedActionWithResult<Boolean> cleanAfterwardsAction, ch.systemsx.cisd.etlserver.ITopLevelDataSetRegistratorDelegate delegate, ch.systemsx.cisd.common.jython.IJythonInterpreter pythonInterpreter, ch.systemsx.cisd.etlserver.TopLevelDataSetRegistratorGlobalState globalState)
    Create a Jython registration service that includes access to the interpreter.
    protected void
    executeJythonScript(ch.systemsx.cisd.etlserver.registrator.DataSetFile dataSetFile, String scriptString, JythonTopLevelDataSetHandlerV2.JythonDataSetRegistrationService<T> service)
     
    protected ch.systemsx.cisd.common.jython.IJythonInterpreter
    getInterpreterFromService(ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> service)
     
    protected ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T>.ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler.RecoveryHookAdaptor
     
    getV2DropboxProgram(ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> service)
     
    void
    handleDataSet(ch.systemsx.cisd.etlserver.registrator.DataSetFile dataSetFile, ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> genericService)
     
    protected boolean
     
    boolean
    V2 registration framework -- do not put files that are scheduled for recovery into the faulty paths.
    static ch.systemsx.cisd.common.jython.IJythonFunction
    tryJythonFunction(ch.systemsx.cisd.common.jython.IJythonInterpreter interpreter, JythonTopLevelDataSetHandlerV2.JythonHookFunction functionDefinition)
     
    protected void
    verifyEvaluatorHookFunctions(ch.systemsx.cisd.common.jython.IJythonInterpreter interpreter)
     

    Methods inherited from class ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler

    asSerializableException, createJythonDataSetRegistrationService, createObjectFactory, didCommitTransaction, didPostRegistration, didPreRegistration, didRollbackTransaction, executeProcessFunctionWithRetries, handleRecovery, waitTheRetryPeriod, waitUntilApplicationIsReady, wrapTransaction

    Methods inherited from class ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator

    check, didEncounterSecondaryTransactionErrors, getRegistrationLock, getRegistratorState, getRollBackStackParentFolder, handle, handle, handle, handleValidationErrors, isRemote, isStopped, registerDataSetInApplicationServer, rollback, updateStopped

    Methods inherited from class ch.systemsx.cisd.etlserver.AbstractTopLevelDataSetRegistrator

    getGlobalState, isUseIsFinishedMarkerFile

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface ch.systemsx.cisd.etlserver.registrator.v2.IOmniscientEntityRegistrator

    getGlobalState
  • Field Details

    • jythonInterpreterFactory

      protected final ch.systemsx.cisd.common.jython.IJythonInterpreterFactory jythonInterpreterFactory
    • JYTHON_VERSION

      public static final String JYTHON_VERSION
      See Also:
    • FACTORY_VARIABLE_NAME

      protected static final String FACTORY_VARIABLE_NAME
      See Also:
    • SERVICE_VARIABLE_NAME

      protected static final String SERVICE_VARIABLE_NAME
      The name of the local variable under which the service is made available to the script.
      See Also:
    • STATE_VARIABLE_NAME

      protected static final String STATE_VARIABLE_NAME
      The name of the local variable under which the global state
      See Also:
    • INCOMING_DATA_SET_VARIABLE_NAME

      protected static final String INCOMING_DATA_SET_VARIABLE_NAME
      The name of the local variable under which the incoming directory is made available to the script.
      See Also:
    • TRANSACTION_VARIABLE_NAME

      protected static final String TRANSACTION_VARIABLE_NAME
      The name of the local variable under which the transaction is made available to the script.
      See Also:
    • SCRIPT_PATH_KEY

      public static final String SCRIPT_PATH_KEY
      See Also:
    • scriptFile

      protected final File scriptFile
  • Constructor Details

    • JythonTopLevelDataSetHandlerV2

      public JythonTopLevelDataSetHandlerV2(ch.systemsx.cisd.etlserver.TopLevelDataSetRegistratorGlobalState globalState)
      Constructor.
      Parameters:
      globalState -
  • Method Details

    • createDataSetRegistrationService

      protected ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> createDataSetRegistrationService(ch.systemsx.cisd.etlserver.registrator.DataSetFile incomingDataSetFile, ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation callerDataSetInformationOrNull, ch.systemsx.cisd.common.action.IDelegatedActionWithResult<Boolean> cleanAfterwardsAction, ch.systemsx.cisd.etlserver.ITopLevelDataSetRegistratorDelegate delegate)
      Create a registration service that includes a python interpreter (we need the interpreter in the service so we can use it in error handling).
      Overrides:
      createDataSetRegistrationService in class ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
    • createJythonDataSetRegistrationServiceV2

      protected ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> createJythonDataSetRegistrationServiceV2(ch.systemsx.cisd.etlserver.registrator.DataSetFile incomingDataSetFile, ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation userProvidedDataSetInformationOrNull, ch.systemsx.cisd.common.action.IDelegatedActionWithResult<Boolean> cleanAfterwardsAction, ch.systemsx.cisd.etlserver.ITopLevelDataSetRegistratorDelegate delegate, ch.systemsx.cisd.common.jython.IJythonInterpreter pythonInterpreter, ch.systemsx.cisd.etlserver.TopLevelDataSetRegistratorGlobalState globalState)
      Create a Jython registration service that includes access to the interpreter.
      Parameters:
      pythonInterpreter -
    • executeJythonScript

      protected void executeJythonScript(ch.systemsx.cisd.etlserver.registrator.DataSetFile dataSetFile, String scriptString, JythonTopLevelDataSetHandlerV2.JythonDataSetRegistrationService<T> service)
    • shouldNotAddToFaultyPathsOrNull

      public boolean shouldNotAddToFaultyPathsOrNull(File file)
      V2 registration framework -- do not put files that are scheduled for recovery into the faulty paths.
      Specified by:
      shouldNotAddToFaultyPathsOrNull in interface ch.systemsx.cisd.common.filesystem.FaultyPathDirectoryScanningHandler.IFaultyPathDirectoryScanningHandlerDelegate
      Specified by:
      shouldNotAddToFaultyPathsOrNull in class ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
    • hasRecoveryMarkerFile

      protected boolean hasRecoveryMarkerFile(File incoming)
      Overrides:
      hasRecoveryMarkerFile in class ch.systemsx.cisd.etlserver.registrator.v2.AbstractOmniscientTopLevelDataSetRegistrator<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
    • getV2DropboxProgram

      protected IJavaDataSetRegistrationDropboxV2 getV2DropboxProgram(ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> service)
      Specified by:
      getV2DropboxProgram in class ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
    • getRecoveryHookAdaptor

      protected ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T>.ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler.RecoveryHookAdaptor getRecoveryHookAdaptor(File incoming)
      Specified by:
      getRecoveryHookAdaptor in class ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
    • handleDataSet

      public void handleDataSet(ch.systemsx.cisd.etlserver.registrator.DataSetFile dataSetFile, ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> genericService) throws Throwable
      Specified by:
      handleDataSet in class ch.systemsx.cisd.etlserver.registrator.v2.AbstractProgrammableTopLevelDataSetHandler<T extends ch.systemsx.cisd.openbis.dss.generic.shared.dto.DataSetInformation>
      Throws:
      Throwable
    • tryJythonFunction

      public static ch.systemsx.cisd.common.jython.IJythonFunction tryJythonFunction(ch.systemsx.cisd.common.jython.IJythonInterpreter interpreter, JythonTopLevelDataSetHandlerV2.JythonHookFunction functionDefinition)
    • verifyEvaluatorHookFunctions

      protected void verifyEvaluatorHookFunctions(ch.systemsx.cisd.common.jython.IJythonInterpreter interpreter)
    • getInterpreterFromService

      protected ch.systemsx.cisd.common.jython.IJythonInterpreter getInterpreterFromService(ch.systemsx.cisd.etlserver.registrator.v2.DataSetRegistrationService<T> service)