DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2020-09-16T20:42:07ZDataONE Tasks
Redmine Infrastructure - Story #8869 (New): Equivalent identities show owning different amount of package...https://redmine.dataone.org/issues/88692020-09-16T20:42:07ZJing Taotao@nceas.ucsb.edu
<p>Matt reported he could see 188 packages listed as <code>My Dataset</code> on the profile page of KNB, which including two packages he created as his ORCID, when he logged in by his ldap account. However, he only could see two packages when he logged in by his ORCID even though the two identities are set to be equivalent:<br>
<a href="https://cn.dataone.org/cn/v2/accounts/http%3A%2F%2Forcid.org%2F0000-0003-0077-4738">https://cn.dataone.org/cn/v2/accounts/http%3A%2F%2Forcid.org%2F0000-0003-0077-4738</a></p>
<p>It seems the equivalent identities are not commutative.</p>
<p>Matt also noticed that the equivalent ldap id for his ORCID is <code>UID=jones,O=NCEAS,DC=ecoinformatics,DC=org</code>, which uses <code>UID</code> rather than <code>uid</code>. He suspected this is the issue to cause the problem.</p>
CN REST - Bug #8867 (New): CNCore.listChecksumAlgorithms() returns incorrect listhttps://redmine.dataone.org/issues/88672020-08-06T00:06:07ZMatthew Jonesjones@nceas.ucsb.edu
<p>The definition of the ChecksumAlgorithm type in SystemMetadata allows any checksum algorithm listed in the Library of Congress vocab. But the current CNCore.listChecksumAlgorithms() implementation only returns two, MD5 and SHA-1. Need to correct this to include the full list of supported algorithms (see <a href="http://id.loc.gov/vocabulary/preservation/cryptographicHashFunctions.html">http://id.loc.gov/vocabulary/preservation/cryptographicHashFunctions.html</a>).</p>
<p>The implementation of this is in a property file, which needs to be updated with the correct list. The file (d1_cn_rest/src/test/resources/org/dataone/configuration/node.properties) currently contains:</p>
<p><code>cn.checksumAlgorithmList=SHA-1;MD5</code></p>
<p>But it should contain all of the other valid algorithms as well from the LoC.</p>
CN REST - Story #8864 (New): Sychronization does not register authoritative replica entry correctlyhttps://redmine.dataone.org/issues/88642020-06-17T21:49:55ZChris Jonescjones@nceas.ucsb.edu
<p>When objects are synchronized to the CN, the <code>d1_synchronization</code> component will fetch the system metadata <br>
for each object and will add a <code><replica></code> entry for the origin node (like <code>urn:node:ESS_DIVE</code>, <br>
as well as entries for other copies (for instance for science metadata copied to the CN, <br>
a <code><replica>urn:node:CN</replica></code> will be added.</p>
<p>In some instances, the origin replica instance is not added to the replica list.<br><br>
This causes downstream problems for the <code>d1_replication</code> component because it relies on the origin node <br>
replica entry to be present in order to set up a replication request to a target node. I'm seeing errors like:</p>
<pre>/var/log/dataone/replicate/cn-replication.log.90:[ERROR] 2020-06-04 05:18:30,179 [pool-15-thread-1] (MNCommunication:requestReplication:34) Could not determine replication source node for replication request for pid: ess-dive-eb6cbb22c605506-20200122T170607966. Replication request failed.
</pre>
<p>Looking back in the logs, this is the case for the following objects:</p>
<pre>ess-dive-3947e68e9825233-20180621T213650539
ess-dive-3b8d9f4513e02f9-20180621T214221437
ess-dive-467a6c3dda4dc88-20180621T211148554
ess-dive-51f345daca126f7-20180328T160350610716
ess-dive-53b37ae5d8c0f51-20200219T211634419654
ess-dive-6b688fab5524c46-20200121T210154766
ess-dive-7a31346c154f02b-20200127T155012488
ess-dive-a1fb05cbd903309-20200130T190835651
ess-dive-b420b097851c716-20180523T161714606
ess-dive-ba81a8a8e0bef31-20180727T200828345
ess-dive-bfaf3d6d6fd038c-20180716T154005175903
ess-dive-c2ef5f3af108c9c-20180621T220020545
ess-dive-eb6cbb22c605506-20200122T170607966
ess-dive-f3238db16593de5-20180621T215956950
</pre>
<p>We need to fix this issue in <code>d1_synchronization</code> so replication runs correctly and monthly <br>
replica auditing (done by ESS_DIVE) doesn't flag these issues.</p>
CN REST - Story #8770 (New): Issue with CN handling of encoded identifiers in object/ meta/ node/...https://redmine.dataone.org/issues/87702019-03-05T19:37:13ZRoger Dahldahl@unm.edu
<p>Works:<br>
<a href="http://cn.dataone.org/cn/v2/object/doi:10.6073/AA/knb-lter-bes.298.37">http://cn.dataone.org/cn/v2/object/doi:10.6073/AA/knb-lter-bes.298.37</a><br>
<a href="https://cn.dataone.org/cn/v2/node/urn:node:LTER">https://cn.dataone.org/cn/v2/node/urn:node:LTER</a></p>
<p>Does not work:<br>
<a href="http://cn.dataone.org/cn/v2/object/doi%3A10.6073%2FAA%2Fknb-lter-bes.298.37">http://cn.dataone.org/cn/v2/object/doi%3A10.6073%2FAA%2Fknb-lter-bes.298.37</a><br>
<a href="https://cn.dataone.org/cn/v2/node/urn%3Anode%3ALTER">https://cn.dataone.org/cn/v2/node/urn%3Anode%3ALTER</a></p>
<p>Note: Behavior differs between HTTP / HTTPS.</p>
CN REST - Bug #8733 (New): Exception handling in ForesiteResourceMap needs improvementhttps://redmine.dataone.org/issues/87332018-10-12T20:09:55ZChris Jonescjones@nceas.ucsb.edu
<p>We're seeing some errors from the <code>org.dataone.cn.indexer.resourcemap.ForesiteResourceMap</code> constructor when indexing resource maps in Metacat 2.9.0, d1_cn_index_processor 2.3.4. A general <code>Exception</code> is being caught and re-thrown as an <code>OREParserException</code> on line 104:</p>
<pre>metacat 20181012-10:53:02: [INFO]: done queueing doc index for pid urn:uuid:5a860679-159e-45a6-bcb6-beb67f4166c2 [edu.ucsb.nceas.metacat.MetacatHandler:handleReindexAction:2777]
metacat 20181012-10:53:02: [DEBUG]: ================= in the finally statement [edu.ucsb.nceas.metacat.MetacatHandler:handleReindexAction:2808]
metacat 20181012-10:53:02: [DEBUG]: ================= in the finally statement which out is not null [edu.ucsb.nceas.metacat.MetacatHandler:handleReindexAction:2810]
org.dspace.foresite.OREParserException: java.lang.NullPointerException
at org.dataone.cn.indexer.resourcemap.ForesiteResourceMap.<init>(ForesiteResourceMap.java:104)
at org.dataone.cn.indexer.resourcemap.ResourceMapFactory.buildResourceMap(ResourceMapFactory.java:45)
at edu.ucsb.nceas.metacat.index.resourcemap.ResourceMapSubprocessor.processResourceMap(ResourceMapSubprocessor.java:97)
at edu.ucsb.nceas.metacat.index.resourcemap.ResourceMapSubprocessor.processDocument(ResourceMapSubprocessor.java:86)
at edu.ucsb.nceas.metacat.index.SolrIndex.process(SolrIndex.java:236)
at edu.ucsb.nceas.metacat.index.SolrIndex.insert(SolrIndex.java:426)
at edu.ucsb.nceas.metacat.index.SolrIndex.update(SolrIndex.java:637)
at edu.ucsb.nceas.metacat.index.SolrIndex.update(SolrIndex.java:592)
at edu.ucsb.nceas.metacat.index.SystemMetadataEventListener.entryUpdated(SystemMetadataEventListener.java:146)
at edu.ucsb.nceas.metacat.index.SystemMetadataEventListener.entryAdded(SystemMetadataEventListener.java:119)
at com.hazelcast.client.impl.EntryListenerManager.notifyListeners(EntryListenerManager.java:148)
at com.hazelcast.client.impl.EntryListenerManager.notifyListeners(EntryListenerManager.java:130)
at com.hazelcast.client.impl.ListenerManager.customRun(ListenerManager.java:88)
at com.hazelcast.client.ClientRunnable.run(ClientRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
</pre>
<p>The resource map that is failing to index is <a href="https://knb.ecoinformatics.org/knb/d1/mn/v2/object/urn%3Auuid%3A5a860679-159e-45a6-bcb6-beb67f4166c2">https://knb.ecoinformatics.org/knb/d1/mn/v2/object/urn%3Auuid%3A5a860679-159e-45a6-bcb6-beb67f4166c2</a></p>
<p>This resource map parses fine using <code>com.hp.hpl.jena.rdf.model.Model.read()</code> outside of the index processor code.</p>
<p>This <code>ForesiteResourceMap</code> constructor is taking an <code>org.w3c.dom.Document</code> as input, converting it to an in-memory <code>ByteArrayOutputStream</code>, and converting that to an in-memory <code>String</code>, to finally get a <code>ReaderInputStream</code> to be passed to <code>_init()</code>. We may need to address the handling inefficiency as a secondary improvement to the code, if feasible.</p>
<p>We also need to not catch a general <code>Exception</code> on line 117, but rather catch the more specific exception that is thrown, which will help with troubleshooting where the exception is coming from. Debug logging statements would help too.</p>
<p>The exception is a <code>NullPointerException</code>, so we ultimately need to track down where this is coming from.</p>
<p>It doesn't look like this code has changed in the 2.3.6 version of the library, so an upgrade wouldn't make a difference. I did look on <code>cn-orc-1</code> to see if we are having similar issues outside of a Metacat MN context, and see errors like these:</p>
<pre>cn-index-processor-daemon.log.9-[ERROR] 2018-10-12 06:42:44,977 (ErrorHandlerFactory$ErrorLogger:logError:84) {E201} rdf:resource not allowed as attribute here.
cn-index-processor-daemon.log.9:[ERROR] 2018-10-12 06:42:44,977 (ForesiteResourceMap:_init:163) Unable to parse ORE document:
cn-index-processor-daemon.log.9-org.apache.jena.riot.RiotException: {E201} rdf:resource not allowed as attribute here.
cn-index-processor-daemon.log.9- at org.apache.jena.riot.system.ErrorHandlerFactory$ErrorHandlerStd.error(ErrorHandlerFactory.java:128)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.lang.LangRDFXML$ErrorHandlerBridge.error(LangRDFXML.java:241)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.ARPSaxErrorHandler.error(ARPSaxErrorHandler.java:38)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.warning(XMLHandler.java:206)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.warning(XMLHandler.java:183)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.warning(XMLHandler.java:178)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.ParserSupport.warning(ParserSupport.java:199)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.AttributeLexer.error(AttributeLexer.java:187)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.QNameLexer.lookup(QNameLexer.java:80)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.AttributeLexer.processSpecials(AttributeLexer.java:71)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.states.WantDescription.startElement(WantDescription.java:60)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.states.AbsWantLiteralValueOrDescription.startElement(AbsWantLiteralValueOrDescription.java:84)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.states.WantLiteralValueOrDescription.startElement(WantLiteralValueOrDescription.java:56)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.startElement(XMLHandler.java:121)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLNamespaceBinder.handleStartElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLNamespaceBinder.emptyElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.RDFXMLParser.parse(RDFXMLParser.java:151)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.ARP.load(ARP.java:119)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.lang.LangRDFXML.parse(LangRDFXML.java:142)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTFactoryImpl$1.read(RDFParserRegistry.java:142)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFDataMgr.process(RDFDataMgr.java:818)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:258)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:244)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.adapters.RDFReaderRIOT.read(RDFReaderRIOT.java:69)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.model.impl.ModelCom.read(ModelCom.java:274)
cn-index-processor-daemon.log.9- at org.dspace.foresite.jena.JenaOREParser.parseToModel(JenaOREParser.java:119)
cn-index-processor-daemon.log.9- at org.dspace.foresite.jena.JenaOREParser.parse(JenaOREParser.java:69)
cn-index-processor-daemon.log.9- at org.dataone.ore.ResourceMapFactory.parseResourceMap(ResourceMapFactory.java:444)
cn-index-processor-daemon.log.9: at org.dataone.cn.indexer.resourcemap.ForesiteResourceMap._init(ForesiteResourceMap.java:160)
cn-index-processor-daemon.log.9: at org.dataone.cn.indexer.resourcemap.ForesiteResourceMap.<init>(ForesiteResourceMap.java:73)
cn-index-processor-daemon.log.9- at org.dataone.cn.indexer.resourcemap.ResourceMapFactory.buildResourceMap(ResourceMapFactory.java:41)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessor.getNextIndexTask(IndexTaskProcessor.java:641)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessor.processFailedIndexTaskQueue(IndexTaskProcessor.java:229)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessor.processIndexTaskQueue(IndexTaskProcessor.java:191)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessorJob.execute(IndexTaskProcessorJob.java:57)
</pre>
<p>This error is also related to <code>_init</code>, but isn't an NPE, so <code>_init()</code> might be a good place to look first.</p>
Infrastructure - Bug #8722 (New): Object in search index but systemmetadata is not available.https://redmine.dataone.org/issues/87222018-10-01T18:38:25ZDave Vieglaisdave.vieglais@gmail.com
<p>The object: <code>urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2</code> is a CSV file. </p>
<p>Resolve or getSystemMetadata return a 404 for that object. It is however, available in the search index:</p>
<p><a href="https://cn.dataone.org/cn/v2/query/solr/?start=0&rows=10&fl=*&q=id%3A%22urn%5C%3Auuid%5C%3A4923cca4%5C-c155%5C-4edc%5C-b901%5C-f6e3b4f2e7b2%22">https://cn.dataone.org/cn/v2/query/solr/?start=0&rows=10&fl=*&q=id%3A%22urn%5C%3Auuid%5C%3A4923cca4%5C-c155%5C-4edc%5C-b901%5C-f6e3b4f2e7b2%22</a></p>
<p>cn-synchronization.log* reports:</p>
<pre>cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:32,418 [ProcessDaemonTask2] (SyncObjectTask:executeTransferObjectTask:293) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 received
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:32,418 [ProcessDaemonTask2] (SyncObjectTask:executeTransferObjectTask:310) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 submitted for execution
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:32,418 [SynchronizeTask322] (V2TransferObjectTask:call:202) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Locking task, attempt 1
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:32,469 [SynchronizeTask322] (V2TransferObjectTask:call:207) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Processing SyncObject
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:32,575 [SynchronizeTask322] (V2TransferObjectTask:retrieveMNSystemMetadata:317) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Retrieved SystemMetadata Identifier:urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 from node urn:node:KNB for ObjectInfo Identifier urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:36,547 [SynchronizeTask322] (V2TransferObjectTask:createObject:730) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Start CreateObject
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:40,388 [SynchronizeTask322] (V2TransferObjectTask:call:234) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Unlocked Pid.
cn-synchronization.log.32:[ERROR] 2018-09-21 15:37:40,388 [SynchronizeTask322] (V2TransferObjectTask:call:269) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - UnrecoverableException: urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 cn.createObject failed: The identifier is already in use by an existing object. - InvalidRequest - The identifier is already in use by an existing object.
cn-synchronization.log.32:org.dataone.cn.batch.exceptions.UnrecoverableException: urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 cn.createObject failed: The identifier is already in use by an existing object.
cn-synchronization.log.32:[ WARN] 2018-09-21 15:37:40,389 [SynchronizeTask322] (SyncFailedTask:submitSynchronizationFailed:116) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - SynchronizationFailed: detail code: 6001 id:urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 nodeId:urn:node:CNUCSB1 description:Synchronization task of [PID::] urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 [::PID] failed. Cause: InvalidRequest: The identifier is already in use by an existing object.
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:40,460 [SynchronizeTask322] (V2TransferObjectTask:call:294) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - exiting with callState: FAILED
cn-synchronization.log.32:[ INFO] 2018-09-21 15:37:40,461 [ProcessDaemonTask2] (SyncObjectTask:reapFutures:372) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 SyncObjectState: FAILED
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:17,203 [ProcessDaemonTask2] (SyncObjectTask:executeTransferObjectTask:293) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 received
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:17,204 [ProcessDaemonTask2] (SyncObjectTask:executeTransferObjectTask:310) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 submitted for execution
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:17,205 [SynchronizeTask37] (V2TransferObjectTask:call:202) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Locking task, attempt 1
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:17,255 [SynchronizeTask37] (V2TransferObjectTask:call:207) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Processing SyncObject
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:17,462 [SynchronizeTask37] (V2TransferObjectTask:retrieveMNSystemMetadata:317) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Retrieved SystemMetadata Identifier:urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 from node urn:node:KNB for ObjectInfo Identifier urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:21,034 [SynchronizeTask37] (V2TransferObjectTask:createObject:730) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Start CreateObject
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:25,672 [SynchronizeTask37] (V2TransferObjectTask:call:234) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - Unlocked Pid.
cn-synchronization.log.33:[ERROR] 2018-09-21 13:37:25,673 [SynchronizeTask37] (V2TransferObjectTask:call:269) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - UnrecoverableException: urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 cn.createObject failed: The identifier is already in use by an existing object. - InvalidRequest - The identifier is already in use by an existing object.
cn-synchronization.log.33:org.dataone.cn.batch.exceptions.UnrecoverableException: urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 cn.createObject failed: The identifier is already in use by an existing object.
cn-synchronization.log.33:[ WARN] 2018-09-21 13:37:25,674 [SynchronizeTask37] (SyncFailedTask:submitSynchronizationFailed:116) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - SynchronizationFailed: detail code: 6001 id:urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 nodeId:urn:node:CNUCSB1 description:Synchronization task of [PID::] urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 [::PID] failed. Cause: InvalidRequest: The identifier is already in use by an existing object.
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:25,740 [SynchronizeTask37] (V2TransferObjectTask:call:294) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 - exiting with callState: FAILED
cn-synchronization.log.33:[ INFO] 2018-09-21 13:37:25,740 [ProcessDaemonTask2] (SyncObjectTask:reapFutures:372) Task-urn:node:KNB-urn:uuid:4923cca4-c155-4edc-b901-f6e3b4f2e7b2 SyncObjectState: FAILED
</pre>
<p>Need to determine how an object could be added to the search index, apparently be replicated, but not exist in the system metadata.</p>
CN REST - Story #8582 (New): Replica Auditing service is throwing errorshttps://redmine.dataone.org/issues/85822018-05-01T19:15:35ZChris Jonescjones@nceas.ucsb.edu
<p>Replica auditing should be auditing objects every 90 days for fixity, and setting the <code>replicaStatus</code> appropriately. The <code>/var/log/dataone/cn-replication-audit.log*</code> files are showing many errors:</p>
<pre>cjones@cn-ucsb-1:replicate$ grep ERROR cn-replication-audit.log* | grep "Cannot update replica status" | wc -l
437601
</pre>
<p>Determine if this is a configuration issue or a code issue and fix it as needed. Also, fix the code to call <code>Identifier.getValue()</code> when logging these errors to avoid printing the memory location of the object like <code>org.dataone.service.types.v1.Identifier@7e90f2e8</code>. There are multiple places where <code>getValue()</code> needs to be added.</p>
Infrastructure - Bug #8500 (New): Geohash not calculated properlyhttps://redmine.dataone.org/issues/85002018-03-14T17:06:52ZDave Vieglaisdave.vieglais@gmail.com
<p>The metadata with pid = <code>{39A912EE-E2DC-4F50-A3DE-C0EF04BC1E88}</code> has bounding coords correctly indexed:</p>
<pre> <float name="eastBoundCoord">166.657</float>
<float name="westBoundCoord">-178.378</float>
<float name="southBoundCoord">-14.5596</float>
<float name="northBoundCoord">28.4536</float>
</pre>
<p>but the computed geohash is wrong:</p>
<pre> <arr name="geohash_9">
<str>ec5zd8st0</str>
</arr>
<arr name="geohash_1">
<str>e</str>
</arr>
<arr name="geohash_2">
<str>ec</str>
</arr>
<arr name="geohash_3">
<str>ec5</str>
</arr>
<arr name="geohash_4">
<str>ec5z</str>
</arr>
<arr name="geohash_5">
<str>ec5zd</str>
</arr>
<arr name="geohash_6">
<str>ec5zd8</str>
</arr>
<arr name="geohash_7">
<str>ec5zd8s</str>
</arr>
<arr name="geohash_8">
<str>ec5zd8st</str>
</arr>
</pre>
<p>Investigate if this is a systemic issue or peculiar to certain types of metadata or representation of coordinates within the metadata.</p>
CN REST - Task #8469 (In Progress): evaluate if ORCID API will continue to work after 1.2 is depr...https://redmine.dataone.org/issues/84692018-03-02T00:37:55ZMatthew Jonesjones@nceas.ucsb.edu
<p>We use ORCID to authenticate users via OAuth. ORCID has announced that it will completely sunset their version 1.2 API on March 1, 2018 (today). See <a href="https://github.com/ORCID/ORCID-Source/blob/master/orcid-model/src/main/resources/README.md">https://github.com/ORCID/ORCID-Source/blob/master/orcid-model/src/main/resources/README.md</a> The API has been deprecated since early 2017. </p>
<p>We use OAuth portions of the API which do not seem to be affected by the XSD version changes in the ORCID API, but we should evaluate whether this will affect us. In particular, I note that they state that the proper endpoints for OAuth are:</p>
<ul>
<li><a href="http://orcid.org/oauth/authorize">http://orcid.org/oauth/authorize</a></li>
<li><a href="https://orcid.org/oauth/token">https://orcid.org/oauth/token</a></li>
</ul>
<p>However, for the second of these, our configuration file (<a href="https://repository.dataone.org/software/cicore/trunk/d1_portal_servlet/src/main/webapp/WEB-INF/portal.properties">https://repository.dataone.org/software/cicore/trunk/d1_portal_servlet/src/main/webapp/WEB-INF/portal.properties</a>) indicates that we use:</p>
<ul>
<li><a href="https://pub.orcid.org/oauth/token">https://pub.orcid.org/oauth/token</a></li>
</ul>
<p>I think the <code>pub</code> endpoints have been deprecated, and we may need to change our configuration to use the established endpoint. Evaluate and possibly change this if needed.</p>
Infrastructure - Story #8173 (New): add checks for retrograde systemMetadata changeshttps://redmine.dataone.org/issues/81732017-09-01T19:42:33ZRob Nahfrnahf@epscor.unm.edu
<p>with the ability to prioritize and the introduction of parallelized index task processing, the effective queue is not guaranteed to be time-ordered. If there are two valid system metadata changes resulting in two tasks and the second change hits the index first, the earlier task should be rejected, as its changes are out of date.</p>
Infrastructure - Task #8026 (Testing): Portal reaches the maximum capacity in a poolhttps://redmine.dataone.org/issues/80262017-02-27T17:00:59ZJing Taotao@nceas.ucsb.edu
<p>After portal runs for a while, the ci-logon can't work since it reaches the maximum capacity of a pool. We have to restart tomcat.<br>
type Exception report</p>
<p>message An exception occurred processing JSP page /setup.jsp at line 15</p>
<p>description The server encountered an internal error that prevented it from fulfilling this request.</p>
<p>exception</p>
<p>org.apache.jasper.JasperException: An exception occurred processing JSP page /setup.jsp at line 15</p>
<p>12: <%<br>
13: <br>
14: // get the certificate, if we have it<br>
15: X509Certificate certificate = PortalCertificateManager.getInstance().getCertificate(request);<br>
16: PrivateKey key = PortalCertificateManager.getInstance().getPrivateKey(request);<br>
17: <br>
18: // if we don't have a certificate, then we aren't logged in</p>
<p>Stacktrace:<br>
org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:568)<br>
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:470)<br>
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395)<br>
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)<br>
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)<br>
com.hazelcast.web.WebFilter.doFilter(WebFilter.java:598)</p>
<p>root cause</p>
<p>edu.uiuc.ncsa.security.core.util.PoolException: Error: Maximum capacity of 1000 elements has been exceeded<br>
edu.uiuc.ncsa.security.core.util.Pool.realCreate(Pool.java:75)<br>
edu.uiuc.ncsa.security.core.util.Pool.pop(Pool.java:63)<br>
edu.uiuc.ncsa.security.storage.sql.SQLDatabase.getConnection(SQLDatabase.java:23)<br>
edu.uiuc.ncsa.security.storage.sql.SQLStore.get(SQLStore.java:194)<br>
edu.uiuc.ncsa.security.storage.sql.SQLStore.get(SQLStore.java:35)<br>
edu.uiuc.ncsa.myproxy.oa4mp.client.storage.AssetStoreUtil.get(AssetStoreUtil.java:43)<br>
edu.uiuc.ncsa.myproxy.oa4mp.client.storage.SQLAssetStore.get(SQLAssetStore.java:34)<br>
org.dataone.portal.PortalCertificateManager.getCredentials(PortalCertificateManager.java:196)<br>
org.dataone.portal.PortalCertificateManager.getCredentials(PortalCertificateManager.java:235)<br>
org.dataone.portal.PortalCertificateManager.getCertificate(PortalCertificateManager.java:157)<br>
org.apache.jsp.account_jsp._jspService(account_jsp.java:100)<br>
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)<br>
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)<br>
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)<br>
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:395)<br>
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:339)<br>
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)<br>
com.hazelcast.web.WebFilter.doFilter(WebFilter.java:598)</p>
CN REST - Bug #8010 (New): CN.archive fails with 401 Unauthorized when using either MN or CN clie...https://redmine.dataone.org/issues/80102017-02-01T20:04:46ZMark Servillamark.servilla@gmail.com
<p>CN.archive fails with 401 Unauthorized when using either MN or CN client certificate for PID with authoritative MN as urn:node:LTER - </p>
<p>MN attempt:<br>
<br>
curl -i -E ./urn_node_LTER-1.pem -X PUT <a href="https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35">https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35</a><br>
HTTP/1.1 401 Unauthorized<br>
Date: Wed, 01 Feb 2017 19:23:20 GMT<br>
Server: Apache/2.4.7 (Ubuntu)<br>
Content-Type: text/xml<br>
Content-Length: 291<br>
Access-Control-Allow-Origin: <br>
Access-Control-Allow-Credentials: true<br>
Access-Control-Allow-Headers: Authorization, Content-Type, Location, Content-Length, x-annotator-auth-token<br>
Access-Control-Expose-Headers: Content-Length, Content-Type, Location<br>
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE<br>
<?xml version="1.0" encoding="UTF-8"?><br>
The Coordinating Node is not authorized to make systemMetadata changes on this object. Please make changes directly on the authoritative Member Node.<br>
</p>
<p>CN attempt:<br>
<br>
curl -i -E ./urn_node_CNUCSB1.pem -X PUT <a href="https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35">https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35</a><br>
HTTP/1.1 401 Unauthorized<br>
Date: Wed, 01 Feb 2017 19:29:29 GMT<br>
Server: Apache/2.4.7 (Ubuntu)<br>
Content-Type: text/xml<br>
Content-Length: 291<br>
Access-Control-Allow-Origin: <br>
Access-Control-Allow-Credentials: true<br>
Access-Control-Allow-Headers: Authorization, Content-Type, Location, Content-Length, x-annotator-auth-token<br>
Access-Control-Expose-Headers: Content-Length, Content-Type, Location<br>
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE<br>
<?xml version="1.0" encoding="UTF-8"?><br>
The Coordinating Node is not authorized to make systemMetadata changes on this object. Please make changes directly on the authoritative Member Node.<br>
</p>
<p>Object System Metadata:<br>
<br>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><br>
<br>
3<br>
doi:10.6073/AA/knb-lter-bes.437.35<br>
eml://ecoinformatics.org/eml-2.0.1<br>
10881<br>
ecfee77e0d5297ddb74a796fb4c36f02<br>
uid=BES,o=LTER,dc=ecoinformatics,dc=org<br>
uid=BES,o=LTER,dc=ecoinformatics,dc=org<br>
<br>
<br>
uid="BES",o=lter,dc=ecoinformatics,dc=org<br>
read<br>
write<br>
changePermission<br>
<br>
<br>
public<br>
read<br>
<br>
<br>
<br>
doi:10.6073/AA/knb-lter-bes.437.34<br>
doi:10.6073/AA/knb-lter-bes.437.36<br>
false<br>
2010-01-08T00:00:00.000+00:00<br>
2015-08-14T21:03:34.343+00:00<br>
urn:node:LTER<br>
urn:node:LTER<br>
<br>
urn:node:CN<br>
completed<br>
2015-08-14T21:03:31.858+00:00<br>
<br>
<br>
urn:node:LTER<br>
completed<br>
2015-12-25T09:37:20.314+00:00<br>
<br>
<a href="/ns3:systemMetadata">/ns3:systemMetadata</a></p>
CN REST - Task #7903 (New): Need to implement/support the default http methods - HEAD and GET ...https://redmine.dataone.org/issues/79032016-10-07T23:59:15ZMatthew Jonesjones@nceas.ucsb.edu
<p>Developers on the Whole Tale project at NCSA reported a bug in the HTTP HEAD request for our resolve service URIs. Example output below to reproduce the error. </p>
<p>Expected: a status code of 200</p>
<p>xarth@shakuras ~ $ curl --head <a href="https://cn.dataone.org/cn/v2/resolve/urn%3Auuid%3Ae9ff8bfe-f12d-4630-a6f1-f3eab740be6f">https://cn.dataone.org/cn/v2/resolve/urn%3Auuid%3Ae9ff8bfe-f12d-4630-a6f1-f3eab740be6f</a><br>
HTTP/1.1 500 Internal Server Error<br>
Date: Fri, 07 Oct 2016 23:00:14 GMT<br>
Server: Apache/2.2.22 (Ubuntu)<br>
Content-Length: 260<br>
Access-Control-Allow-Origin: <br>
Access-Control-Allow-Credentials: true<br>
Access-Control-Allow-Headers: Authorization, Content-Type, Location, Content-Length, x-annotator-auth-token<br>
Access-Control-Expose-Headers: Content-Length, Content-Type, Location<br>
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE<br>
Vary: Accept-Encoding<br>
Connection: close<br>
Content-Type: text/xml;charset=UTF-8</p>
<p>xarth@shakuras ~ $ curl --head <a href="https://knb.ecoinformatics.org/knb/d1/mn/v2/object/urn%3Auuid%3Ae9ff8bfe-f12d-4630-a6f1-f3eab740be6f">https://knb.ecoinformatics.org/knb/d1/mn/v2/object/urn%3Auuid%3Ae9ff8bfe-f12d-4630-a6f1-f3eab740be6f</a><br>
HTTP/1.1 200 OK<br>
Date: Fri, 07 Oct 2016 23:00:48 GMT<br>
Server: Apache/2.4.7 (Ubuntu)<br>
Set-Cookie: JSESSIONID=7DC18368F71D5D9948371B3C33437E8B; Path=/knb/; Secure<br>
DataONE-Checksum: SHA-1,927a11b6e46b771c9922083814f6ee8e5b09f696<br>
Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT<br>
DataONE-ObjectFormat: application/octet-stream<br>
DataONE-SerialVersion: 0<br>
Content-Length: 2809655736<br>
Access-Control-Allow-Origin: <br>
Access-Control-Allow-Credentials: true<br>
Content-Type: text/xml</p>
Infrastructure - Bug #4674 (New): Ask Judith, Mike and Virgina Perez.2.1 to obsolete those pids w...https://redmine.dataone.org/issues/46742014-03-31T18:02:41ZJing Taotao@nceas.ucsb.edu
<p>doi:10.5063/AA/Virginia Perez.2.1<br>
judith botha.1.1<br>
judith botha.2.1<br>
judith kruger.1.1<br>
judith kruger.2.1<br>
judith kruger.3.1<br>
judith kruger.4.1<br>
judith kruger.5.1<br>
doi:10.6085/AA/ SHLX00_XXXITV2XLSR03_20111128.40.1 (PISCO)</p>
Infrastructure - Bug #3492 (In Progress): Invalid PIDs in production (whitespace)https://redmine.dataone.org/issues/34922013-01-17T15:13:44ZDave Vieglaisdave.vieglais@gmail.com
<p>Recording this for future reference. </p>
<p>There are nine PIDs in the production environment that contain whitespace. This appears to have no functional effect - sysmeta and objects can be retrieved so no action is required other than to ensure no more sneak in.</p>
<p>The PIDs in question are:</p>
<a name="guid"></a>
<h2 > guid <a href="#guid" class="wiki-anchor">¶</a></h2>
<p>doi:10.5063/AA/Virginia Perez.2.1<br>
judith kruger.3.1<br>
judith kruger.4.1<br>
judith botha.1.1<br>
judith kruger.1.1<br>
judith kruger.2.1<br>
judith kruger.5.1<br>
judith botha.2.1<br>
resourceMap_Lin Cheng-Jung.1.1<br>
resourceMap_Lin Cheng-Jung.1.2<br>
resourceMap_Lin Cheng-Jung.1.3<br>
Lin Cheng-Jung.1.1<br>
Lin Cheng-Jung.1.2<br>
Lin Cheng-Jung.1.3<br>
doi:10.6085/AA/ SHLX00_XXXITV2XLSR03_20111128.40.1</p>