Project

General

Profile

Story #1280

Refactor MN-CN Synchronization

Added by Robert Waltz about 13 years ago. Updated over 12 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Robert Waltz
Category:
d1_synchronization
Target version:
-
Start date:
2011-01-30
Due date:
% Done:

100%

Story Points:
Sprint:

Description

With the current mechanism, it is possible that not all objects will be synchronized if a failure should occur. There is a very real possibility that if a failure should stop synchronization, the next time it starts, it will not begin from where it left off. Spring Batch has persistence and fail over capabilities.

Create a persistent mechanism that will determine if a failure is repeated due do to a problem with the system or a problem with a document. If an object is unable to be synchronized do to object format failure, then a message needs to be sent to administrator.

Determining batch size may be an issue as well. If search services do not order result sets, then more robust persistence measures should be taken to insure recovery after failure. Currently, ordering of search results is unspecified. It may be that MNs will not implement such a measure. In fact, this problem may well extend beyond simple ordering of sets based on dataSysModified...

We also will need to create a background process that double checks every doc in metacat that can be index has been indexed. This is incase a message is lost or an error occurs that allows a document to be created/updated in metacat but not pushed to the index triggering mechanism. or, if the triggering mechanism fails.


Related issues

Related to Infrastructure - Story #975: Refactor to support synchronization across multiple MNs Closed 2011-07-05

History

#1 Updated by Robert Waltz about 13 years ago

  • Milestone changed from CCI-0.8 to CCI-0.6

#2 Updated by Robert Waltz about 13 years ago

  • Milestone deleted (CCI-0.6)

We also will need to create a background process that double checks every doc in metacat that can be index has been indexed. This is incase a message is lost or an error occurs that allows a document to be created/updated in metacat but not pushed to the index triggering mechanism. or, if the triggering mechanism fails.

#3 Updated by Robert Waltz over 12 years ago

  • Milestone set to None

#4 Updated by Dave Vieglais over 12 years ago

  • Position set to 8

#5 Updated by Dave Vieglais over 12 years ago

  • Position deleted (11)
  • Position set to 1

#6 Updated by Robert Waltz over 12 years ago

  • Status changed from New to Closed

Alternative story has been completed that fulfills the objectives of this story.

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)