DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2020-08-06T00:06:07ZDataONE Tasks
Redmine CN REST - Bug #8867 (New): CNCore.listChecksumAlgorithms() returns incorrect listhttps://redmine.dataone.org/issues/88672020-08-06T00:06:07ZMatthew Jonesjones@nceas.ucsb.edu
<p>The definition of the ChecksumAlgorithm type in SystemMetadata allows any checksum algorithm listed in the Library of Congress vocab. But the current CNCore.listChecksumAlgorithms() implementation only returns two, MD5 and SHA-1. Need to correct this to include the full list of supported algorithms (see <a href="http://id.loc.gov/vocabulary/preservation/cryptographicHashFunctions.html">http://id.loc.gov/vocabulary/preservation/cryptographicHashFunctions.html</a>).</p>
<p>The implementation of this is in a property file, which needs to be updated with the correct list. The file (d1_cn_rest/src/test/resources/org/dataone/configuration/node.properties) currently contains:</p>
<p><code>cn.checksumAlgorithmList=SHA-1;MD5</code></p>
<p>But it should contain all of the other valid algorithms as well from the LoC.</p>
CN REST - Bug #8860 (New): /token endpoint doesn't set a content-type and character encodinghttps://redmine.dataone.org/issues/88602020-02-29T01:00:11ZBryce Mecummecum@nceas.ucsb.edu
<p>On Firefox only, requests to the /portal/token endpoint (i.e., the one MetacatUI and other clients use to fetch their auth tokens, like <a href="https://cn.dataone.org/portal/token">https://cn.dataone.org/portal/token</a>) result in errors in the browser console.</p>
<p>When you access the URL via an XHR request, you see:</p>
<blockquote>
<p>XML Parsing Error: syntax error<br>
Location: <a href="https://cn-stage.test.dataone.org/portal/token">https://cn-stage.test.dataone.org/portal/token</a><br>
Line Number 1, Column 1:</p>
</blockquote>
<p>When you access the URL directly in Firefox:</p>
<blockquote>
<p>The character encoding of the plain text document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the file needs to be declared in the transfer protocol or file needs to use a byte order mark as an encoding signature.</p>
</blockquote>
<p>I had a hunch that this error would go away if the response simply had the <code>Content-Type</code> header set to <code>text/plain; charset=utf-8</code> so I spun up <code>mitmproxy</code>, made that edit to the intercepted response, and saw that the error does go away.</p>
<p>I think we should modify the portal code to set the <code>Content-Type</code> header like above so the error goes away.</p>
CN REST - Bug #8740 (New): CN resolve service returning 404 for some pidshttps://redmine.dataone.org/issues/87402018-11-08T00:21:45ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>A user has reported that certain pids (all for metadata objects) return an http 404 status for the resolve service. Here is the complete list that the user tried that didn't resolve:</p>
<p><a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-arc.1343.2">https://cn.dataone.org/cn/v2/resolve/knb-lter-arc.1343.2</a><br>
<a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-arc.1212.2">https://cn.dataone.org/cn/v2/resolve/knb-lter-arc.1212.2</a><br>
<a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-arc.1477.2">https://cn.dataone.org/cn/v2/resolve/knb-lter-arc.1477.2</a><br>
<a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-bnz.442.8">https://cn.dataone.org/cn/v2/resolve/knb-lter-bnz.442.8</a><br>
<a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-pie.15.5">https://cn.dataone.org/cn/v2/resolve/knb-lter-pie.15.5</a><br>
<a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-pie.248.1">https://cn.dataone.org/cn/v2/resolve/knb-lter-pie.248.1</a><br>
<a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-sbc.6.13">https://cn.dataone.org/cn/v2/resolve/knb-lter-sbc.6.13</a><br>
<a href="https://cn.dataone.org/cn/v2/resolve/knb-lter-sbc.15.23">https://cn.dataone.org/cn/v2/resolve/knb-lter-sbc.15.23</a></p>
<p>The CNRead.getSystemMetadata() service returns metadata for each of these pids.</p>
CN REST - Bug #8733 (New): Exception handling in ForesiteResourceMap needs improvementhttps://redmine.dataone.org/issues/87332018-10-12T20:09:55ZChris Jonescjones@nceas.ucsb.edu
<p>We're seeing some errors from the <code>org.dataone.cn.indexer.resourcemap.ForesiteResourceMap</code> constructor when indexing resource maps in Metacat 2.9.0, d1_cn_index_processor 2.3.4. A general <code>Exception</code> is being caught and re-thrown as an <code>OREParserException</code> on line 104:</p>
<pre>metacat 20181012-10:53:02: [INFO]: done queueing doc index for pid urn:uuid:5a860679-159e-45a6-bcb6-beb67f4166c2 [edu.ucsb.nceas.metacat.MetacatHandler:handleReindexAction:2777]
metacat 20181012-10:53:02: [DEBUG]: ================= in the finally statement [edu.ucsb.nceas.metacat.MetacatHandler:handleReindexAction:2808]
metacat 20181012-10:53:02: [DEBUG]: ================= in the finally statement which out is not null [edu.ucsb.nceas.metacat.MetacatHandler:handleReindexAction:2810]
org.dspace.foresite.OREParserException: java.lang.NullPointerException
at org.dataone.cn.indexer.resourcemap.ForesiteResourceMap.<init>(ForesiteResourceMap.java:104)
at org.dataone.cn.indexer.resourcemap.ResourceMapFactory.buildResourceMap(ResourceMapFactory.java:45)
at edu.ucsb.nceas.metacat.index.resourcemap.ResourceMapSubprocessor.processResourceMap(ResourceMapSubprocessor.java:97)
at edu.ucsb.nceas.metacat.index.resourcemap.ResourceMapSubprocessor.processDocument(ResourceMapSubprocessor.java:86)
at edu.ucsb.nceas.metacat.index.SolrIndex.process(SolrIndex.java:236)
at edu.ucsb.nceas.metacat.index.SolrIndex.insert(SolrIndex.java:426)
at edu.ucsb.nceas.metacat.index.SolrIndex.update(SolrIndex.java:637)
at edu.ucsb.nceas.metacat.index.SolrIndex.update(SolrIndex.java:592)
at edu.ucsb.nceas.metacat.index.SystemMetadataEventListener.entryUpdated(SystemMetadataEventListener.java:146)
at edu.ucsb.nceas.metacat.index.SystemMetadataEventListener.entryAdded(SystemMetadataEventListener.java:119)
at com.hazelcast.client.impl.EntryListenerManager.notifyListeners(EntryListenerManager.java:148)
at com.hazelcast.client.impl.EntryListenerManager.notifyListeners(EntryListenerManager.java:130)
at com.hazelcast.client.impl.ListenerManager.customRun(ListenerManager.java:88)
at com.hazelcast.client.ClientRunnable.run(ClientRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
</pre>
<p>The resource map that is failing to index is <a href="https://knb.ecoinformatics.org/knb/d1/mn/v2/object/urn%3Auuid%3A5a860679-159e-45a6-bcb6-beb67f4166c2">https://knb.ecoinformatics.org/knb/d1/mn/v2/object/urn%3Auuid%3A5a860679-159e-45a6-bcb6-beb67f4166c2</a></p>
<p>This resource map parses fine using <code>com.hp.hpl.jena.rdf.model.Model.read()</code> outside of the index processor code.</p>
<p>This <code>ForesiteResourceMap</code> constructor is taking an <code>org.w3c.dom.Document</code> as input, converting it to an in-memory <code>ByteArrayOutputStream</code>, and converting that to an in-memory <code>String</code>, to finally get a <code>ReaderInputStream</code> to be passed to <code>_init()</code>. We may need to address the handling inefficiency as a secondary improvement to the code, if feasible.</p>
<p>We also need to not catch a general <code>Exception</code> on line 117, but rather catch the more specific exception that is thrown, which will help with troubleshooting where the exception is coming from. Debug logging statements would help too.</p>
<p>The exception is a <code>NullPointerException</code>, so we ultimately need to track down where this is coming from.</p>
<p>It doesn't look like this code has changed in the 2.3.6 version of the library, so an upgrade wouldn't make a difference. I did look on <code>cn-orc-1</code> to see if we are having similar issues outside of a Metacat MN context, and see errors like these:</p>
<pre>cn-index-processor-daemon.log.9-[ERROR] 2018-10-12 06:42:44,977 (ErrorHandlerFactory$ErrorLogger:logError:84) {E201} rdf:resource not allowed as attribute here.
cn-index-processor-daemon.log.9:[ERROR] 2018-10-12 06:42:44,977 (ForesiteResourceMap:_init:163) Unable to parse ORE document:
cn-index-processor-daemon.log.9-org.apache.jena.riot.RiotException: {E201} rdf:resource not allowed as attribute here.
cn-index-processor-daemon.log.9- at org.apache.jena.riot.system.ErrorHandlerFactory$ErrorHandlerStd.error(ErrorHandlerFactory.java:128)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.lang.LangRDFXML$ErrorHandlerBridge.error(LangRDFXML.java:241)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.ARPSaxErrorHandler.error(ARPSaxErrorHandler.java:38)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.warning(XMLHandler.java:206)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.warning(XMLHandler.java:183)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.warning(XMLHandler.java:178)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.ParserSupport.warning(ParserSupport.java:199)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.AttributeLexer.error(AttributeLexer.java:187)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.QNameLexer.lookup(QNameLexer.java:80)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.AttributeLexer.processSpecials(AttributeLexer.java:71)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.states.WantDescription.startElement(WantDescription.java:60)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.states.AbsWantLiteralValueOrDescription.startElement(AbsWantLiteralValueOrDescription.java:84)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.states.WantLiteralValueOrDescription.startElement(WantLiteralValueOrDescription.java:56)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.startElement(XMLHandler.java:121)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLNamespaceBinder.handleStartElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLNamespaceBinder.emptyElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
cn-index-processor-daemon.log.9- at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.impl.RDFXMLParser.parse(RDFXMLParser.java:151)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.arp.ARP.load(ARP.java:119)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.lang.LangRDFXML.parse(LangRDFXML.java:142)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTFactoryImpl$1.read(RDFParserRegistry.java:142)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFDataMgr.process(RDFDataMgr.java:818)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:258)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:244)
cn-index-processor-daemon.log.9- at org.apache.jena.riot.adapters.RDFReaderRIOT.read(RDFReaderRIOT.java:69)
cn-index-processor-daemon.log.9- at com.hp.hpl.jena.rdf.model.impl.ModelCom.read(ModelCom.java:274)
cn-index-processor-daemon.log.9- at org.dspace.foresite.jena.JenaOREParser.parseToModel(JenaOREParser.java:119)
cn-index-processor-daemon.log.9- at org.dspace.foresite.jena.JenaOREParser.parse(JenaOREParser.java:69)
cn-index-processor-daemon.log.9- at org.dataone.ore.ResourceMapFactory.parseResourceMap(ResourceMapFactory.java:444)
cn-index-processor-daemon.log.9: at org.dataone.cn.indexer.resourcemap.ForesiteResourceMap._init(ForesiteResourceMap.java:160)
cn-index-processor-daemon.log.9: at org.dataone.cn.indexer.resourcemap.ForesiteResourceMap.<init>(ForesiteResourceMap.java:73)
cn-index-processor-daemon.log.9- at org.dataone.cn.indexer.resourcemap.ResourceMapFactory.buildResourceMap(ResourceMapFactory.java:41)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessor.getNextIndexTask(IndexTaskProcessor.java:641)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessor.processFailedIndexTaskQueue(IndexTaskProcessor.java:229)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessor.processIndexTaskQueue(IndexTaskProcessor.java:191)
cn-index-processor-daemon.log.9- at org.dataone.cn.index.processor.IndexTaskProcessorJob.execute(IndexTaskProcessorJob.java:57)
</pre>
<p>This error is also related to <code>_init</code>, but isn't an NPE, so <code>_init()</code> might be a good place to look first.</p>
CN REST - Bug #8698 (New): CN Performance degradationhttps://redmine.dataone.org/issues/86982018-09-13T21:05:40ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Calling simple services on the CN can take a long time to execute. For example, calling </p>
<p><a href="https://cn.dataone.org/cn/v2/resolve/aekos.org.au/collection/nsw.gov.au/nsw_atlas/vis_flora_module/KM_CUDM.20160202">https://cn.dataone.org/cn/v2/resolve/aekos.org.au/collection/nsw.gov.au/nsw_atlas/vis_flora_module/KM_CUDM.20160202</a></p>
<p>can take about 8-10 seconds, while calling the MN directly (where the resolve service redirects to) takes about 1-2 seconds:</p>
<p><a href="https://dataone.tern.org.au/mn/v2/object/aekos.org.au%2Fcollection%2Fnsw.gov.au%2Fnsw_atlas%2Fvis_flora_module%2FKM_CUDM.20160202">https://dataone.tern.org.au/mn/v2/object/aekos.org.au%2Fcollection%2Fnsw.gov.au%2Fnsw_atlas%2Fvis_flora_module%2FKM_CUDM.20160202</a></p>
CN REST - Bug #8630 (New): equivalentIdentity values use uppercase letters when the same person s...https://redmine.dataone.org/issues/86302018-06-26T14:15:00ZLauren Walkerwalker@nceas.ucsb.edu
<p>I would expect a username to appear exactly the same everywhere in the identity service. </p>
<p>Here is an example where the <code>equivalentIdentity</code> uses uppercase letters for the subject (UID=corinalogan,O=unaffiliated,DC=ecoinformatics,DC=org) when the <code>person</code>><code>subject</code> uses lowercase letters (uid=corinalogan,o=unaffiliated,dc=ecoinformatics,dc=org):</p>
<p><a href="https://cn.dataone.org/cn/v2/accounts/http%3A%2F%2Forcid.org%2F0000-0002-5944-906X">https://cn.dataone.org/cn/v2/accounts/http%3A%2F%2Forcid.org%2F0000-0002-5944-906X</a></p>
<pre><ns2:subjectInfo xmlns:ns2="http://ns.dataone.org/service/types/v1">
<person>
<subject>http://orcid.org/0000-0002-5944-906X</subject>
<givenName>Corina</givenName>
<familyName>Logan</familyName>
<equivalentIdentity>
UID=corinalogan,O=unaffiliated,DC=ecoinformatics,DC=org
</equivalentIdentity>
<verified>false</verified>
</person>
<person>
<subject>
uid=corinalogan,o=unaffiliated,dc=ecoinformatics,dc=org
</subject>
<givenName>Corina</givenName>
<familyName>Logan</familyName>
<equivalentIdentity>http://orcid.org/0000-0002-5944-906X</equivalentIdentity>
<verified>false</verified>
</person>
</ns2:subjectInfo>
```~~~
</pre> CN REST - Bug #8010 (New): CN.archive fails with 401 Unauthorized when using either MN or CN clie...https://redmine.dataone.org/issues/80102017-02-01T20:04:46ZMark Servillamark.servilla@gmail.com
<p>CN.archive fails with 401 Unauthorized when using either MN or CN client certificate for PID with authoritative MN as urn:node:LTER - </p>
<p>MN attempt:<br>
<br>
curl -i -E ./urn_node_LTER-1.pem -X PUT <a href="https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35">https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35</a><br>
HTTP/1.1 401 Unauthorized<br>
Date: Wed, 01 Feb 2017 19:23:20 GMT<br>
Server: Apache/2.4.7 (Ubuntu)<br>
Content-Type: text/xml<br>
Content-Length: 291<br>
Access-Control-Allow-Origin: <br>
Access-Control-Allow-Credentials: true<br>
Access-Control-Allow-Headers: Authorization, Content-Type, Location, Content-Length, x-annotator-auth-token<br>
Access-Control-Expose-Headers: Content-Length, Content-Type, Location<br>
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE<br>
<?xml version="1.0" encoding="UTF-8"?><br>
The Coordinating Node is not authorized to make systemMetadata changes on this object. Please make changes directly on the authoritative Member Node.<br>
</p>
<p>CN attempt:<br>
<br>
curl -i -E ./urn_node_CNUCSB1.pem -X PUT <a href="https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35">https://cn.dataone.org/cn/v2/archive/doi:10.6073/AA/knb-lter-bes.437.35</a><br>
HTTP/1.1 401 Unauthorized<br>
Date: Wed, 01 Feb 2017 19:29:29 GMT<br>
Server: Apache/2.4.7 (Ubuntu)<br>
Content-Type: text/xml<br>
Content-Length: 291<br>
Access-Control-Allow-Origin: <br>
Access-Control-Allow-Credentials: true<br>
Access-Control-Allow-Headers: Authorization, Content-Type, Location, Content-Length, x-annotator-auth-token<br>
Access-Control-Expose-Headers: Content-Length, Content-Type, Location<br>
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE<br>
<?xml version="1.0" encoding="UTF-8"?><br>
The Coordinating Node is not authorized to make systemMetadata changes on this object. Please make changes directly on the authoritative Member Node.<br>
</p>
<p>Object System Metadata:<br>
<br>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><br>
<br>
3<br>
doi:10.6073/AA/knb-lter-bes.437.35<br>
eml://ecoinformatics.org/eml-2.0.1<br>
10881<br>
ecfee77e0d5297ddb74a796fb4c36f02<br>
uid=BES,o=LTER,dc=ecoinformatics,dc=org<br>
uid=BES,o=LTER,dc=ecoinformatics,dc=org<br>
<br>
<br>
uid="BES",o=lter,dc=ecoinformatics,dc=org<br>
read<br>
write<br>
changePermission<br>
<br>
<br>
public<br>
read<br>
<br>
<br>
<br>
doi:10.6073/AA/knb-lter-bes.437.34<br>
doi:10.6073/AA/knb-lter-bes.437.36<br>
false<br>
2010-01-08T00:00:00.000+00:00<br>
2015-08-14T21:03:34.343+00:00<br>
urn:node:LTER<br>
urn:node:LTER<br>
<br>
urn:node:CN<br>
completed<br>
2015-08-14T21:03:31.858+00:00<br>
<br>
<br>
urn:node:LTER<br>
completed<br>
2015-12-25T09:37:20.314+00:00<br>
<br>
<a href="/ns3:systemMetadata">/ns3:systemMetadata</a></p>
CN REST - Bug #7918 (New): SEAD object only partially synchronized - missing autogen.201609291601...https://redmine.dataone.org/issues/79182016-10-21T17:20:29ZDave Vieglaisdave.vieglais@gmail.com
<p>While updating <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: SEAD object only partially synchronized - missing autogen.2015061616000265251 document from /var/... (Closed)" href="https://redmine.dataone.org/issues/7222">#7222</a> and <a class="issue tracker-5 status-5 priority-4 priority-default closed" title="Task: SEAD object only partially synchronized in CN production - missing autogen.2016092916013111425 do... (Closed)" href="https://redmine.dataone.org/issues/7914">#7914</a>. Checked for content for those, and it is accessible through CN. May still be an issue however as the SEAD MN reports 69 objects, though<br>
there are 68 reported in the search index.</p>
<p>Counting on the MN:</p>
<p>curl "<a href="http://seadva.d2i.indiana.edu:8081/sead/rest/mn/v1/object">http://seadva.d2i.indiana.edu:8081/sead/rest/mn/v1/object</a>" | xml fo</p>
<p>returns 69 objects.</p>
<p>Counting on the CN::</p>
<p>curl "<a href="https://cn.dataone.org/cn/v2/object?nodeId=urn:node:SEAD">https://cn.dataone.org/cn/v2/object?nodeId=urn:node:SEAD</a>" | xml fo</p>
<p>returns 69 objects.</p>
<p>Counting on the CN using the search index::</p>
<p><a href="https://cn.dataone.org/cn/v1/query/solr/?start=0&rows=10&fl=id%2Ctitle%2CformatId&q=datasource%3A%22urn%5C%3Anode%5C%3ASEAD%22">https://cn.dataone.org/cn/v1/query/solr/?start=0&rows=10&fl=id%2Ctitle%2CformatId&q=datasource%3A%22urn%5C%3Anode%5C%3ASEAD%22</a></p>
<p>returns 68 objects.</p>
<p>List of identifiers in search index::</p>
<p>curl "<a href="https://cn.dataone.org/cn/v1/query/solr/?start=0&rows=100&fl=id&q=datasource%3A%22urn%5C%3Anode%5C%3ASEAD%22">https://cn.dataone.org/cn/v1/query/solr/?start=0&rows=100&fl=id&q=datasource%3A%22urn%5C%3Anode%5C%3ASEAD%22</a>" | xml sel -t -m "//doc/str[@name='id']" -v . -n | sort > SEAD_index_pids.txt</p>
<p>List of identifiers on CN::</p>
<p>curl "<a href="https://cn.dataone.org/cn/v2/object?nodeId=urn:node:SEAD">https://cn.dataone.org/cn/v2/object?nodeId=urn:node:SEAD</a>" | xml sel -t -m "//objectInfo/identifier" -v . -n | sort > SEAD_cn_pids.txt</p>
<p>Missing PID::</p>
<p>diff SEAD_cn_pids.txt SEAD_index_pids.txt<br>
60d59<br>
< seadva-c918e4ff-2861-496a-a907-d2cb382ddb30</p>
<p>System Metadata for PID::</p>
<p><?xml version="1.0"?><br>
<br>
1<br>
seadva-c918e4ff-2861-496a-a907-d2cb382ddb30<br>
FGDC-STD-001-1998<br>
6553<br>
ff3d4641669b4e08c9f8b978b85d6113a4c1bab8<br>
SEAD<br>
CN=urn:node:SEAD,DC=dataone,DC=org<br>
<br>
<br>
public<br>
read<br>
<br>
<br>
<br>
sead-Martin-John-f1dbc3df-c27c-4647-b05a-4b1f05c99a24<br>
2016-08-17T12:41:34.03Z<br>
2016-08-17T14:43:58.677Z<br>
urn:node:SEAD<br>
urn:node:SEAD<br>
<br>
urn:node:SEAD<br>
completed<br>
2016-09-29T23:01:21.235Z<br>
<br>
<br>
urn:node:CN<br>
completed<br>
2016-09-29T23:01:21.241Z<br>
<br>
<a href="/ns1:systemMetadata">/ns1:systemMetadata</a></p>
<p>Confirm object can be retieved::</p>
<p>curl "<a href="http://seadva.d2i.indiana.edu:8081/sead/rest/mn/v1/object/seadva-c918e4ff-2861-496a-a907-d2cb382ddb30">http://seadva.d2i.indiana.edu:8081/sead/rest/mn/v1/object/seadva-c918e4ff-2861-496a-a907-d2cb382ddb30</a>" | xml fo</p>
<p>System Metadata for obsoleted object::</p>
<p><?xml version="1.0"?><br>
<br>
1<br>
sead-Martin-John-f1dbc3df-c27c-4647-b05a-4b1f05c99a24<br>
FGDC-STD-001-1998<br>
7443<br>
ebd481ad8268f9714f23d772c15fb62e61384486<br>
CN=urn:node:SEAD,DC=dataone,DC=org<br>
CN=urn:node:SEAD,DC=dataone,DC=org<br>
<br>
<br>
public<br>
read<br>
<br>
<br>
<br>
seadva-c918e4ff-2861-496a-a907-d2cb382ddb30<br>
2013-10-24T18:41:31.213Z<br>
2016-08-17T14:43:58.677Z<br>
urn:node:SEAD<br>
urn:node:SEAD<br>
<br>
urn:node:CN<br>
completed<br>
2013-10-24T23:00:04.684Z<br>
<br>
<br>
urn:node:SEAD<br>
completed<br>
2013-10-24T23:00:04.579Z<br>
<br>
<a href="/ns1:systemMetadata">/ns1:systemMetadata</a><br></p>
<p>autogenid for <code>seadva-c918e4ff-2861-496a-a907-d2cb382ddb30</code>::</p>
<p>psql metacat<br>
select * from identifier where guid='seadva-c918e4ff-2861-496a-a907-d2cb382ddb30';</p>
<table><thead>
<tr>
<th>guid</th>
<th>docid</th>
<th>rev</th>
</tr>
</thead><tbody>
<tr>
<td>seadva-c918e4ff-2861-496a-a907-d2cb382ddb30</td>
<td>autogen.2016092916012224122</td>
<td>1</td>
</tr>
</tbody></table>
<p>$ ls /var/metacat/data/autogen.2016092916012224122*<br>
ls: cannot access /var/metacat/data/autogen.2016092916012224122: No such file or directory</p>
CN REST - Bug #7687 (New): Synchronization unable to change BaseURLs upon updateNodeCapabilities ...https://redmine.dataone.org/issues/76872016-03-23T18:34:41ZRobert Waltz
<p>Synchronization does not change the baseurl of a membernode after the membernode has called updateNodeCapabilities in order to update its baseurl.</p>
<p>Synchronization keeps a pool of d1clients associated with a membernode Id. </p>
<p>NodeCommObjectListHarvestFactory and NodeCommSyncObjectFactory will each need to be notified if a membernode has changed its baseurl node. Those objects can then purge and re-initialize the d1clients responsible for maintaining communication to the membernode.</p>
CN REST - Bug #7489 (New): processing daemon common-logging misconfiguredhttps://redmine.dataone.org/issues/74892015-11-16T16:32:08ZRobert Waltz
<p>the commons-logging.properties file appears misconfigured. It is missing the line:</p>
<p>org.apache.commons.logging.Log=org.apache.commons.logging.impl.Log4JLogger</p>
<p>Hopefully, adding this line will allow all the apache commons logging calls to go to the log4j loggers.</p>
CN REST - Bug #7440 (New): Non-discernable error during synchronization affecting (mostly) urn:no...https://redmine.dataone.org/issues/74402015-10-16T17:40:44ZMark Servillamark.servilla@gmail.com
<p>Multiple (366) non-discernable errors (see attachment) with message only stating "Cline is shutdown" occurred on 14 Oct 2015 on cn-stage.test.dataone.org (cn-stage-ucsb-1); sampling of objects/system metadata indicate all is retrievable from the MN <a href="https://dataone-dev.ecoinformatics.org.au/mn">https://dataone-dev.ecoinformatics.org.au/mn</a>. This error had also occurred (24 events) for urn:node:mnTestLTER.</p>
<p>For example:</p>
<p>cn-synchronization.log.1-[ERROR] 2015-10-14 11:33:02,749 (TransferObjectTask:write:606) Task-urn:node:mnTestAEKOS-aekos.org.au/collection/nsw.gov.au/nsw_atlas/vis_flora_module/ABERBALDIE.20150515<br>
cn-synchronization.log.1:Client is shutdown.</p>
CN REST - Bug #7301 (New): CN-stage allows connections to a MN that is operating a self-signed SS...https://redmine.dataone.org/issues/73012015-08-18T18:12:14ZMark Servillamark.servilla@gmail.com
<p>CN-stage supports connections to a MN that is operating a self-signed SSL server certificate - this should not be allowed since the connection could occur with a rogue non-verified server.</p>
<p>This instance occurred with dataone-dev.ecoinformatics.org.au:443 on 18 August 2015:</p>
<p>Certificate chain<br>
0 s:/CN=dataone-dev.ecoinformatics.org.au<br>
i:/CN=dataone-dev.ecoinformatics.org.au</p>
<p>Issuer: CN=dataone-dev.ecoinformatics.org.au<br>
Validity<br>
Not Before: Aug 11 04:56:19 2015 GMT<br>
Not After : Aug 8 04:56:19 2025 GMT<br>
Subject: CN=dataone-dev.ecoinformatics.org.au</p>
CN REST - Bug #7161 (New): TERN object fails to be indexed by Solr, but successfully synchronizedhttps://redmine.dataone.org/issues/71612015-06-05T18:02:10ZMark Servillamark.servilla@gmail.com
<p>TERN object aekos.org.au/collection/nsw.gov.au/nsw_atlas/vis_flora_module/V_ILLAWDB3.20150515 fails to by indexed by Solr on cn.dataone.edu (cn-ucsb-1) even though it was successfully synchronized. Investigation indicates it was not added to the HazelCast ObjectPathMap structure according to Skye Roseboom:</p>
<p>Im not sure why this pid is not appearing in the hazelcast ObjectPathMap structure.</p>
<p>I looked into the metacat database schema a bit and noticed the pid does not seem to appear in the ‘identifier_mapping’ table:</p>
<p>select * from identifier_mapping where guid='aekos.org.au/collection/nsw.gov.au/nsw_atlas/vis_flora_module/V_ILLAWDB3.20150515';</p>
<table><thead>
<tr>
<th>guid</th>
<th>docid</th>
<th>rev</th>
</tr>
</thead><tbody>
</tbody></table>
<p>(0 rows)</p>
<p>Without the ‘docid’ or ‘localid’ as metacat calls them, Im don’t thing the pid could be added to the objectPathMap in hazelcast.</p>
CN REST - Bug #5739 (New): LDAP upgrades fail with purge of dataone-cn-os-corehttps://redmine.dataone.org/issues/57392014-07-16T20:47:31ZRobert Waltz
<p>open ldap should continue to function normally on a host after dataone-cn-os-core has been purged (apt-get remove --purge dataone-cn-os-core)</p>
<p>We should wipe out /etc/ldap/slapd.d, restore it to a simpler configuration, backup and then remove the dataone entries from the openldap database.</p>
<p>We will need to complete this before migrating to 14.04 of ubuntu</p>
CN REST - Task #2415 (New): Implement exceptions for log endpointhttps://redmine.dataone.org/issues/24152012-02-27T21:56:12ZSkye Roseboomsroseboo@dataone.unm.edu
<p><a href="http://mule1.dataone.org/ArchitectureDocs-current/apis/CN_APIs.html#CNCore.getLogRecords">http://mule1.dataone.org/ArchitectureDocs-current/apis/CN_APIs.html#CNCore.getLogRecords</a></p>
<p>not currently provided by mod_rewrite</p>