Project

General

Profile

Bug #7918

SEAD object only partially synchronized - missing autogen.2016092916012224122 document from /var/metacat/documents

Added by Dave Vieglais about 5 years ago. Updated about 5 years ago.

Status:
New
Priority:
Normal
Assignee:
Category:
cn_metacat
Target version:
-
Start date:
2016-10-21
Due date:
% Done:

0%

Story Points:
Sprint:

Description

While updating #7222 and #7914. Checked for content for those, and it is accessible through CN. May still be an issue however as the SEAD MN reports 69 objects, though
there are 68 reported in the search index.

Counting on the MN:

curl "http://seadva.d2i.indiana.edu:8081/sead/rest/mn/v1/object" | xml fo

returns 69 objects.

Counting on the CN::

curl "https://cn.dataone.org/cn/v2/object?nodeId=urn:node:SEAD" | xml fo

returns 69 objects.

Counting on the CN using the search index::

https://cn.dataone.org/cn/v1/query/solr/?start=0&rows=10&fl=id%2Ctitle%2CformatId&q=datasource%3A%22urn%5C%3Anode%5C%3ASEAD%22

returns 68 objects.

List of identifiers in search index::

curl "https://cn.dataone.org/cn/v1/query/solr/?start=0&rows=100&fl=id&q=datasource%3A%22urn%5C%3Anode%5C%3ASEAD%22" | xml sel -t -m "//doc/str[@name='id']" -v . -n | sort > SEAD_index_pids.txt

List of identifiers on CN::

curl "https://cn.dataone.org/cn/v2/object?nodeId=urn:node:SEAD" | xml sel -t -m "//objectInfo/identifier" -v . -n | sort > SEAD_cn_pids.txt

Missing PID::

diff SEAD_cn_pids.txt SEAD_index_pids.txt
60d59
< seadva-c918e4ff-2861-496a-a907-d2cb382ddb30

System Metadata for PID::

<?xml version="1.0"?>

1
seadva-c918e4ff-2861-496a-a907-d2cb382ddb30
FGDC-STD-001-1998
6553
ff3d4641669b4e08c9f8b978b85d6113a4c1bab8
SEAD
CN=urn:node:SEAD,DC=dataone,DC=org


public
read



sead-Martin-John-f1dbc3df-c27c-4647-b05a-4b1f05c99a24
2016-08-17T12:41:34.03Z
2016-08-17T14:43:58.677Z
urn:node:SEAD
urn:node:SEAD

urn:node:SEAD
completed
2016-09-29T23:01:21.235Z


urn:node:CN
completed
2016-09-29T23:01:21.241Z

/ns1:systemMetadata

Confirm object can be retieved::

curl "http://seadva.d2i.indiana.edu:8081/sead/rest/mn/v1/object/seadva-c918e4ff-2861-496a-a907-d2cb382ddb30" | xml fo

System Metadata for obsoleted object::

<?xml version="1.0"?>

1
sead-Martin-John-f1dbc3df-c27c-4647-b05a-4b1f05c99a24
FGDC-STD-001-1998
7443
ebd481ad8268f9714f23d772c15fb62e61384486
CN=urn:node:SEAD,DC=dataone,DC=org
CN=urn:node:SEAD,DC=dataone,DC=org


public
read



seadva-c918e4ff-2861-496a-a907-d2cb382ddb30
2013-10-24T18:41:31.213Z
2016-08-17T14:43:58.677Z
urn:node:SEAD
urn:node:SEAD

urn:node:CN
completed
2013-10-24T23:00:04.684Z


urn:node:SEAD
completed
2013-10-24T23:00:04.579Z

/ns1:systemMetadata

autogenid for seadva-c918e4ff-2861-496a-a907-d2cb382ddb30::

psql metacat
select * from identifier where guid='seadva-c918e4ff-2861-496a-a907-d2cb382ddb30';

guid docid rev
seadva-c918e4ff-2861-496a-a907-d2cb382ddb30 autogen.2016092916012224122 1

$ ls /var/metacat/data/autogen.2016092916012224122*
ls: cannot access /var/metacat/data/autogen.2016092916012224122: No such file or directory


Related issues

Related to Infrastructure - Bug #7917: Fix the issue that the system data exists but the bytes doesn't exist for an object on CN after synchronization Closed 2016-10-21
Related to CN REST - Bug #7222: SEAD object only partially synchronized - missing autogen.2015061616000265251 document from /var/metacat/documents Closed 2015-06-18
Related to CN REST - Task #7914: SEAD object only partially synchronized in CN production - missing autogen.2016092916013111425 document from /var/metacat/documents Closed 2016-10-17

History

#1 Updated by Dave Vieglais about 5 years ago

  • Related to Bug #7917: Fix the issue that the system data exists but the bytes doesn't exist for an object on CN after synchronization added

#2 Updated by Dave Vieglais about 5 years ago

  • Related to Bug #7222: SEAD object only partially synchronized - missing autogen.2015061616000265251 document from /var/metacat/documents added

#3 Updated by Dave Vieglais about 5 years ago

  • Related to Task #7914: SEAD object only partially synchronized in CN production - missing autogen.2016092916013111425 document from /var/metacat/documents added

#4 Updated by Dave Vieglais about 5 years ago

SQL to manipulate the database, removing the identifier entries and associated system metadata.

\set v_pid 'seadva-c918e4ff-2861-496a-a907-d2cb382ddb30'
\set v_docid 'autogen.2016092916012224122'
\set v_rev '1'

BEGIN;
DELETE FROM xml_queryresult WHERE docid = :'v_docid';
DELETE FROM xml_path_index WHERE docid = :'v_docid';
DELETE FROM xml_index WHERE docid = :'v_docid';
DELETE FROM xml_accesssubtree WHERE docid = :'v_docid';
DELETE FROM xml_documents WHERE docid = :'v_docid';
DELETE FROM xml_nodes WHERE docid = :'v_docid';
DELETE FROM xml_revisions WHERE docid = :'v_docid';
DELETE FROM xml_nodes_revisions WHERE docid = :'v_docid';
DELETE FROM access_log WHERE docid = :'v_docid'||'.'||:'v_rev';

DELETE FROM smmediatypeproperties WHERE guid = :'v_pid';
DELETE FROM smreplicationpolicy WHERE guid = :'v_pid';
DELETE FROM smreplicationstatus WHERE guid = :'v_pid';
DELETE FROM systemmetadata WHERE guid = :'v_pid';
DELETE FROM xml_access WHERE guid = :'v_pid';
DELETE FROM identifier WHERE guid = :'v_pid';
COMMIT;

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)