Story #593
design the synchronization service for CNs
100%
Description
As a developer, I need to design the synchronization service for coordinating nodes
This service provides a periodic polling mechanism to ask MNs for new metadata objects (using the MN.listObjects), and when needed, pulls and inserts those objects into the CN store using MN.get(). The service needs to pick up new objects at least daily, but would be even better if it can be more frequent (so that user objects show up in search soon after insertion) without overwhelming MNs. The service should be fast and only update new objects during most operations (i.e., by asking for changes since the last sync), while occasionally sweeping through all objects to be sure old stuff was not missed. This will require a semi-intelligent scheduler. A smart service would also allow the 3 coordinating nodes to do harvesting in parallel, each taking a portion of the MNs, and using the CN-to-CN replication to sync data, although this is not required from the beginning (one CN could do it all).
-- involves revising use cases to reflect needed functionality
-- involves revising sequence diagrams to make sure they are correct
-- involves developing class diagram for CN sync implementation
-- involves specifying both private and public sync methods
History
#1 Updated by Dave Vieglais over 14 years ago
- Status changed from New to Closed
This is documented in the architecture docs and implemented in the CN using a simple cron driven approach.
There is an ongoing need for additional design and implementation work on this issue that should be apparent as the project milestones are re-defined for the next phase of activties.