Infrastructure » CN REST

Handling Catastrophic failures of SyncObjectTask

If the SyncObjectTaskManager recieves an exception that disables the running of SyncObjectTask, then all of synchronization should be halted, and a notification Or log message sent regarding the issue.

The impact will be on quartz scheduling and any quartz jobs that are running.

Quartz jobs run the ObjectListHarvestTask. Any ObjectListHarvestTask job this is running when the exception happens should halt it's processing and return.

since quartz jobs are created by quartz scheduling, an observer pattern can not be applied, since we are not able to track the job instantiations in an observer class.

If SyncObjectTaskManager goes down, then HarvestSchedulingManager should shutdown all of its jobs, never to be rescheduled.

NodeTopicListener calls HarvestSchedulingManager.manageHarvest method when it recieves a message about an updated node.

In DataONE CN Common, there is a class named ComponentActiviationUtility that tracks the activation state of d1_processing components. syncrhonizationIsActive() is a method call that returns the evaluation of a private method sychronizationComponentActive(). It only returns the status of the property Synchronization.active. Add an additional AtomicBoolean set by SyncObjectTaskManager. Add a method to disbleSynchronization by setting the boolean to false, also conjunct Synchronization.active to the AtomicBoolen when calling syncrhonizationIsActive().

Update HarvestSchedulingManager.manageHarvest to review the scheduler.isShutdown state before rescheduling tasks. Create halt method that will shutdown the scheduler, waiting or any active jobs to complete.

SyncObjectTaskManager will need a reference to the HarvestSchedulingManager in order to call halt when exception occurs.