Story #975: Refactor to support synchronization across multiple MNs
Distributed job execution
A job will execute on a given schedule. Though a trigger will execute on a home machine, the execution of the job may be distributed to a remote machine. Hazelcast allows for distributed executor service.
SyncObjectExecutor creates a single thread on each CN that runs a SyncObjectTask. SyncObjectTask polls on the syncObjectQueue (a hazelcast queue) for synchronization tasks to run. Each CN will run a maximum configurable set of tasks for each membernode being synchronized. A sync task is associated with an object. For each object that needs synchronization a TransferObjectTask thread is executed.
an object that fails to synchronize is submitted to the MN via the sync failure interface.