Project

General

Profile

Story #7946

historical schema incompliant entries in CN storage causing CN_Read errors

Added by Rob Nahf over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Metacat
Target version:
Start date:
2016-11-30
Due date:
% Done:

100%

Story Points:
Sprint:

Description

The CI 2.3 deployment rolled out JAXB schema validation (replacing JiBX schema validation), which is stricter.

Empty ReplicationPolicy preferred and blockMemberNode entries were in the postgres db, and needed to be removed to avoid future Marshalling errors upon retreival.

We need to check the rest of the postgres db systemmetadata and related tables for similar empty entries.

History

#1 Updated by Rob Nahf over 3 years ago

using psql, I checked the system metadata tables for persisted empty values. Found no problems except for already known about spaces in identifiers (#3492).

Tables searched:
systemmetadata
xml_access
smreplicationpolicy
smreplicationstatus

searches:

WHERE IS NULL;
WHERE BTRIM(, ' ') = ''; # this finds any field which is empty or 1 or more space characters.
WHERE LIKE '% %'; # find fields that contain internal white space

against all text-based fields.

Also ran checks to make sure that system metadata could be retrieved when a record from xml_access, smreplicationpolicy, or smreplicationstatus
tables could not be joined because of no matching entry in the 'sub' tables.

  1. found exemplar pid using query similar to this:

    select systemmetadata.guid, xml_access.guid from systemmetadata LEFT OUTER JOIN xml_access ON (systemmetadata.guid = xml_access.guid) where xml_access.guid IS NULL;

  2. using curl from the CN and using client .pem file, did curl command against cn.dataone.org/cn/v2/meta/ to make sure it returned properly.

no problems found.

#2 Updated by Rob Nahf over 3 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)