Story #7716
Updated by Rob Nahf about 8 years ago
Currently, when CNs harvest objects from MNs, they only compare compared the last modified time stamp on the system metadata with the last harvest date. If the last modified stamp is earlier than the last harvest date, the object wouldn't be put on the synchronization queue.
This causes an issue: if the synchronization for this object objects failed in the previous synchornizations and the issue of the failure was corrected (e.g. a misconfiguration miss configuration on the MN and the last modified time stamp should NOT be changed), the harvest will not pick up the object. We have to manually to either reset the harvest date, or submit a v2.CN.synchronize(PID,NodeID) date in order to pick up the object.
MemberNode operators are not well versed in how to submit these requests, so some sort of monitoring and management tool for MemberNode operators (and DataONE administrators) is proposed to simplify this type of action.
We are thinking to persist the information for the failed objects on a database table -e.g. failedSynchronization and it contains:
identifier, mn_id, last_failed_time, message.
Message can be separated to another table to use the foreign key.
Once Since we have the failure information, we can work on decide how to use it to automate the monitoring and management tool. re-harvest process.
This causes an issue: if the synchronization for this object objects failed in the previous synchornizations and the issue of the failure was corrected (e.g. a misconfiguration miss configuration on the MN and the last modified time stamp should NOT be changed), the harvest will not pick up the object. We have to manually to either reset the harvest date, or submit a v2.CN.synchronize(PID,NodeID) date in order to pick up the object.
MemberNode operators are not well versed in how to submit these requests, so some sort of monitoring and management tool for MemberNode operators (and DataONE administrators) is proposed to simplify this type of action.
We are thinking to persist the information for the failed objects on a database table -e.g. failedSynchronization and it contains:
identifier, mn_id, last_failed_time, message.
Message can be separated to another table to use the foreign key.
Once Since we have the failure information, we can work on decide how to use it to automate the monitoring and management tool. re-harvest process.
