Task #3916: Dryad object list contains pids that throw NotFound at mn/meta/{pid} mn/object/{pid} - Member Nodes - DataONE Tasks

Task #3916

MNDeployment #3118: Dryad Member Node

Dryad object list contains pids that throw NotFound at mn/meta/{pid} mn/object/{pid}

Added by Skye Roseboom over 11 years ago. Updated almost 11 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Skye Roseboom

Target version:

Operational

Start date:

2013-08-13

Due date:

% Done:

100%

Estimated time:

0.00 h

Story Points:

Sprint:

Description

During sync test run on cn-dev for Dryad MN, found 113 Not Found exceptions in the cn-sync log files.

These pids should be generated from the MN's object list - so it is peculiar sync would be catching Not Found exceptions, asking Dryad MN for the object.

Example - this example is strange - the pid reported in the ERROR line does not match the pid in the xml error document. The end of the pid is missing. Appears the string is being truncated at the '&' character. Not sure if this is a reporting/logging issue or cause of not found. If this was systematic, would expect to see more than 113 or so of these errors.

[ERROR] 2013-08-13 17:13:38,810 (TransferObjectTask:retrieveSystemMetadata:268) Task-urn:node:mnTestDRYAD-http://dx.doi.org/10.5061/dryad.v0j89?ver=2012-10-31T15:34:55.222-0400
<?xml version="1.0" encoding="UTF-8"?>

The specified object does not exist on this node.

Another similiar example:
[ERROR] 2013-08-13 17:13:39,567 (TransferObjectTask:retrieveSystemMetadata:268) Task-urn:node:mnTestDRYAD-http://dx.doi.org/10.5061/dryad.v0j89?format=d1rem&ver=2012-10-31T15:34:55.222-0400
<?xml version="1.0" encoding="UTF-8"?>

The specified object does not exist on this node.

However there are also a number of errors where the pid seems intact:

[ERROR] 2013-08-13 17:39:17,240 (TransferObjectTask:retrieveSystemMetadata:268) Task-urn:node:mnTestDRYAD-http://dx.doi.org/10.5061/dryad.43874/3/bitstream
<?xml version="1.0" encoding="UTF-8"?>

The specified object does not exist on this node.

[ERROR] 2013-08-13 17:41:53,819 (TransferObjectTask:retrieveSystemMetadata:268) Task-urn:node:mnTestDRYAD-http://dx.doi.org/10.5061/dryad.37gr1/4/bitstream
<?xml version="1.0" encoding="UTF-8"?>

The specified object does not exist on this node.

Would like to confirm that this is not a D1 synchronization issue. Would be nice to know why the pid appears to be munged in some of the example output above. If we can confirm these pids appear in the MN's object list then we should assign to Dryad dev (ryan).

History

#1 Updated by Skye Roseboom over 11 years ago

cn-dev-ucsb-1 was the only CN running synchronization - so all error messages can be found in its logs.

#2 Updated by Ryan Scherle about 11 years ago

This should now be fixed on the Dryad dev system.

#3 Updated by Robert Waltz about 11 years ago

translation missing: en.field_remaining_hours set to 0.0
Status changed from New to Closed

#4 Updated by Skye Roseboom about 11 years ago

Estimated time set to 0.00
Assignee changed from Robert Waltz to Ryan Scherle
Status changed from Closed to In Progress

Re-opening issue as CN and dryad are reporting different counts for ORE documents. Investigating CN logs, it appears there are identifiers for ORE docs that appear in the object list of dryad - but are not retrieved by calls to /meta/{pid} or /object/{pid} on dryad.

pids that appear in dryad object list but result in not found from mn/meta and mn/object on dryad:

Not sure what the pattern is here, this is likely not the complete list of ids that appear on the object list but dont seem to exist on the MN.

See redmine #4140 for description of sync/index results on 10/28 test sync run.

#5 Updated by Skye Roseboom about 11 years ago

Subject changed from CN sync throwing Not Found Errors to Dryad object list contains pids that throw NotFound at mn/meta/{pid} mn/object/{pid}

#6 Updated by Ryan Scherle about 11 years ago

These problems were due to dirty metadata on our development server. This data has been cleaned up.

Please re-test.

#7 Updated by Ryan Scherle about 11 years ago

Assignee changed from Ryan Scherle to Skye Roseboom

#8 Updated by Skye Roseboom about 11 years ago

Status changed from In Progress to Testing

#9 Updated by Skye Roseboom about 11 years ago

Status changed from Testing to Closed

This issue looks to be corrected. closing.

#10 Updated by Laura Moyers almost 11 years ago

Target version changed from Deploy by end of Y5Q2 to Deploy by end of Y5Q3

#11 Updated by Laura Moyers almost 11 years ago

Target version changed from Deploy by end of Y5Q3 to Operational

Also available in: Atom PDF

Project

General

Profile

Member Nodes

Issues

Custom queries