Task #3138
Story #3136: Inconsistent data in production 1.0.3 release
Identify and correct missing xml_documents or xml_revisions table entries on ORC, UNM and UCSB
100%
Description
The identifiers table entries are missing corresponding entries in xml_documents or xml_revisions table. If there is an entry in systeMetadata table that has a corresponding entry in the identifiers table, then there should be an entry in either the xml_documents or xml_revisions table.
Use the following query on the 3 production machines:
su postgres -c "psql -d metacat -c \"select count(guid) from identifier i \
where not exists (select docid, rev from xml_documents x where x.docid = i.docid and x.rev = i.rev \
union select docid, rev from xml_revisions r where r.docid = i.docid and r.rev = i.rev)\""
It will result in 14920 on ORC, 14901 on UNM and 18 on UCSB.
The errors on UCSB are related to earlier releases. The problems causing these errors may have already been solved and the data may just need to be corrected on UCSB. No determination has been made about the problems on ORC and UNM, but most of the problems appear to be related to CDL Resource Maps not being replicated correctly.
identifiersMissingUCSB-Documents.out (701 Bytes)
identifiersMissingUCSB-Documents.analysis.txt 
(15.8 KB)
identifiersMissingUNM-Documents.out (626 KB)
identifiersMissingORC-Documents.out (627 KB)
Related issues
History
#1 Updated by Ben Leinfelder over 11 years ago
For the missing docs on ORC and UNM, it looks like Metacat replication is timing out when retrieving the list of docs from UCSB.
knb 20120816-17:11:20: [ERROR]: ReplicationHandler.update - Failed to get updated doc list for server cn-ucsb-1.dataone.org/knb/servlet/replication because Read timed out [ReplicationLogging]
We should up the configured timeout to 5 minutes since it is currently taking 1:02 and will only grow and grow:
replication.client.timeout=300000
#2 Updated by Ben Leinfelder over 11 years ago
- Status changed from New to In Progress
I also fixed a problem with metacat's force replication when a data object (ORE map) was inserted in Metacat using the DataONE API. It had been incorrectly using the force replication action with "create" when it should have been "update". This has essentially prevented us from replicating ORE documents on initial insert since day one. The fix will be included in Metacat 1.0.4
#3 Updated by Ben Leinfelder over 11 years ago
- Status changed from In Progress to Closed
The ORC and UNM numbers have been reconciled. UCSB remains 18 documents shy -- in a different task.
