https://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2017-08-23T18:42:13ZDataONE TasksInfrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=290772017-08-23T18:42:13ZRob Nahfrnahf@epscor.unm.edu
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/diff/29077?detail_id=41414">diff</a>)</li></ul> Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=290912017-08-25T16:12:03ZRob Nahfrnahf@epscor.unm.edu
<ul></ul><p>IARC also wants to turn off replication.</p>
Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=290922017-08-25T17:41:49ZRob Nahfrnahf@epscor.unm.edu
<ul></ul><p>The check for potential target nodes happens in a private method under ReplicationManager.createAndQueueTasks(Identifier pid). It seems to happen near the time of the request, but, there is a shift from direct execution of mn.replicate to submitting replicate requests by targetMN. Tasks can be for Pids (make more replicas) and PID+targetNode (call mn.replicate),</p>
<p>in Replication repository:<br>
pid tasks can be in : NEW or IN_PROCESS state<br>
in the object's systemMetadata<br>
pid+target tasks can be : QUEUED, REQUESTED, FAILED, COMPLETED </p>
<p>ReplicationEventListener.entryUpdated() / .entryAdded()<br>
(a Hz systemetadata map listener)<br>
- if the pid's authoritativeMN is listed as a replica with status COMPLETE<br>
create new replica task in the task repository unless there already is one<br>
(if there are more than one task for the pid, delete existing ones and submit a new one)</p>
<p>(triggered every two minutes, trigger set up in ReplicationManager<br>
ReplicationTaskProcessor.run()<br>
move a page of tasks from NEW to IN_PROCESS status<br>
- markInProcess<br>
- call createAndQueueTasks (pid)</p>
<p>createAndQueueTasks (pid)<br>
(lockPid)<br>
processPid<br>
removeReplicationTasks if sysmeta disallows replication or number of replicas are sufficient<br>
determine potential target nodes (from NodeList and sysmeta)<br>
createAndQueueTasks(pid, potentialTargets,desiredNumber)<br>
loop to create a few per-target-node tasks<br>
cnReplication.updateReplicationMetadata<br>
if success,<br><br>
requeueReplicationTask(pid) #return the pid to the NEW state because we don't assume the requisite number of replicas have been created<br>
ReplicationTaskQueue.processAllTasksForMN(targetMN)<br>
(lock the targetNode)<br>
get replication tasks in the QUEUED state for this target node<br>
foreach task, requestReplication (ReplicationService.requestQueuedReplication(pid,target)<br>
targetMn.replicate(pid)<br>
(unlock the targetNode)<br>
(unlockPid)</p>
Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=292082017-10-24T16:46:22ZDave Vieglaisdave.vieglais@gmail.com
<ul><li><strong>Target version</strong> changed from <i>CCI-2.3.5</i> to <i>CCI-2.3.7</i></li></ul> Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=292332017-10-31T17:46:37ZDave Vieglaisdave.vieglais@gmail.com
<ul><li><strong>Target version</strong> changed from <i>CCI-2.3.7</i> to <i>CCI-2.3.8</i></li></ul> Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=295032018-01-17T18:40:35ZDave Vieglaisdave.vieglais@gmail.com
<ul><li><strong>Sprint</strong> set to <i>Infrastructure backlog</i></li></ul> Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=295042018-01-17T18:40:53ZDave Vieglaisdave.vieglais@gmail.com
<ul><li><strong>Sprint</strong> changed from <i>Infrastructure backlog</i> to <i>CCI-2.3.8</i></li></ul> Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=299912018-03-02T21:47:33ZDave Vieglaisdave.vieglais@gmail.com
<ul><li><strong>Target version</strong> changed from <i>CCI-2.3.8</i> to <i>CCI-2.3.10</i></li></ul> Infrastructure - Story #8162: Replication tasks can contain stale potential target nodeshttps://redmine.dataone.org/issues/8162?journal_id=305682018-07-04T11:18:53ZDave Vieglaisdave.vieglais@gmail.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-1 priority-4 priority-default parent" href="/issues/8639">Story #8639</a>: Replication performance is too slow to service demand</i> added</li></ul>