Project

General

Profile

Story #593

design the synchronization service for CNs

Added by Matthew Jones about 14 years ago. Updated over 13 years ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
Documentation
Target version:
Start date:
Due date:
% Done:

100%

Story Points:
Sprint:

Description

As a developer, I need to design the synchronization service for coordinating nodes

This service provides a periodic polling mechanism to ask MNs for new metadata objects (using the MN.listObjects), and when needed, pulls and inserts those objects into the CN store using MN.get(). The service needs to pick up new objects at least daily, but would be even better if it can be more frequent (so that user objects show up in search soon after insertion) without overwhelming MNs. The service should be fast and only update new objects during most operations (i.e., by asking for changes since the last sync), while occasionally sweeping through all objects to be sure old stuff was not missed. This will require a semi-intelligent scheduler. A smart service would also allow the 3 coordinating nodes to do harvesting in parallel, each taking a portion of the MNs, and using the CN-to-CN replication to sync data, although this is not required from the beginning (one CN could do it all).
-- involves revising use cases to reflect needed functionality
-- involves revising sequence diagrams to make sure they are correct
-- involves developing class diagram for CN sync implementation
-- involves specifying both private and public sync methods

History

#1 Updated by Dave Vieglais over 13 years ago

  • Status changed from New to Closed

This is documented in the architecture docs and implemented in the CN using a simple cron driven approach.

There is an ongoing need for additional design and implementation work on this issue that should be apparent as the project milestones are re-defined for the next phase of activties.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)