DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2019-11-05T19:25:48ZDataONE Tasks
Redmine Infrastructure - Story #8849 (New): During sync, the CN does not detect error returned from getCh...https://redmine.dataone.org/issues/88492019-11-05T19:25:48ZRoger Dahldahl@unm.edu
<p>Due to a bug, GMN returned 500 on some getChecksum() calls. The CN did not detect the 500 return status and proceeded with the sync, using "null" as the checksum.</p>
Infrastructure - Story #8848 (New): A minor difference of annotation index between CN and MNhttps://redmine.dataone.org/issues/88482019-11-01T21:37:01ZJing Taotao@nceas.ucsb.edu
<p>The solr index on CN is:</p>
<pre><arr name="sem_annotation">
<str>http://purl.dataone.org/odo/ECSO_00000512</str>
<str>
http://ecoinformatics.org/oboe/oboe.1.2/oboe-core.owl#MeasurementType
</str>
<str>http://purl.dataone.org/odo/ECSO_00001102</str>
<str>http://purl.dataone.org/odo/ECSO_00001243</str>
<str>http://purl.dataone.org/odo/ECSO_00000629</str>
<str>http://purl.dataone.org/odo/ECSO_00000518</str>
<str>http://www.w3.org/2000/01/rdf-schema#Resource</str>
<str>http://purl.dataone.org/odo/ECSO_00000516</str>
<str>http://purl.obolibrary.org/obo/UO_0000301</str>
</arr>
</pre>
<p>The mn is:</p>
<pre><arr name="sem_annotation">
<str>http://purl.dataone.org/odo/ECSO_00000512</str>
<str>
http://ecoinformatics.org/oboe/oboe.1.2/oboe-core.owl#MeasurementType
</str>
<str>http://purl.dataone.org/odo/ECSO_00001102</str>
<str>http://purl.dataone.org/odo/ECSO_00001243</str>
<str>http://purl.dataone.org/odo/ECSO_00000629</str>
<str>http://purl.dataone.org/odo/ECSO_00000518</str>
<str>http://purl.dataone.org/odo/ECSO_00000516</str>
<str>http://purl.obolibrary.org/obo/UO_0000301</str>
</arr>
</pre>
<p>The cn has an extra <code><str>http://www.w3.org/2000/01/rdf-schema#Resource</str></code><br>
Bryce and I discussed it and thought it wouldn't affect the feature. But we still need to figure it out.</p>
Infrastructure - Story #8806 (New): Cleanup from OS upgradeshttps://redmine.dataone.org/issues/88062019-05-21T12:45:17ZDave Vieglaisdave.vieglais@gmail.com
<p>There's a few items that need to be addressed after the OS upgrades from 14.04 to 18.04.</p>
Infrastructure - Story #8796 (New): Various issues with service access after upgrade to 18.04https://redmine.dataone.org/issues/87962019-05-14T23:57:48ZDave Vieglaisdave.vieglais@gmail.com
<p>Users have reported some issues with CNs after upgrades to 18.04. See individual issues for details.</p>
CN REST - Story #8749 (New): Fix log aggregation events from the CN without associated CN IPshttps://redmine.dataone.org/issues/87492018-11-16T20:39:55ZChris Jonescjones@nceas.ucsb.edu
<p>The robots list used to filter out usage events includes the IP addresses of the CNs, so events logged during synchronization don't show up as true hits. Because of the SSL infrastructure at lbl.gov, the ESS-DIVE group doesn't see the public IP of an incoming request, but rather an internal private IP assigned by lbl.gov infrastructure. You can see the impact of this on the <a href="https://data.ess-dive.lbl.gov/#profile" class="external">ESS-DIVE profile page</a>. The spike of 11,000+ downloads in August 2018 was the CN synchronizing content.</p>
<p>Rushiraj summarized these events in a <a href="https://gist.github.com/rushirajnenuji/847d8239acf68a108bda30e04af0406b" class="external">gist</a></p>
<p>There are multiple <code>10.42.x.x</code> IP associated with the CN requests. These events all need to be updated in the <code>logsolr</code> core and changed to an actual CN IP. For future synchronizations, perhaps we need to add <code>10.42.0.0/16</code> to the robots list? </p>
Infrastructure - Story #8208 (New): Review and adjust memory allocated to d1-index-task-generator...https://redmine.dataone.org/issues/82082017-10-24T16:09:06ZDave Vieglaisdave.vieglais@gmail.com
<p>Configuration for these services is located at:<br>
<br>
/etc/default/d1-index-task-generator<br>
/etc/default/d1-index-task-processor</p>
<p>d1-index-task-generator is probably fine, however d1-index-task-processor may benefit from increased allocation especially for processing resource maps.</p>
Infrastructure - Story #8207 (New): review and adjust memory allocated to d1-processinghttps://redmine.dataone.org/issues/82072017-10-24T16:06:24ZDave Vieglaisdave.vieglais@gmail.com
<p>d1-processing settings are under @/etc/default/d1-processing@</p>
<p>Currently runs like: <br>
<br>
jsvc.exec -home /usr/lib/jvm/java-1.8.0-openjdk-amd64 -cp /usr/share/java/commons-daemon.jar:/usr/share/j<br>
ava/d1_process_daemon.jar -debug -outfile /var/log/dataone/daemon/d1-processing-jsvc.log -errfile /var/log/dataone/daemon/d1-process<br>
ing-jsvc.err -pidfile /var/run/d1-processing.pid -user tomcat7 -Djava.awt.headless=true -XX:UseParallelGC -Xmx4096M -Xms1024M -Xss12<br>
80k -XX:MaxPermSize=512M org.dataone.cn.batch.daemon.SchedulerDaemon</p>
Infrastructure - Story #8204 (New): Adjust memory allocation for services running under JVM on CNshttps://redmine.dataone.org/issues/82042017-10-24T15:29:28ZDave Vieglaisdave.vieglais@gmail.com
<p>Most services on CNs run in JVM instances, each with separate restrictions on memory use.</p>
<p>Current configurations were mostly based on defaults and have not changed much despite significantly higher memory use for systems such as Solr and Hazelcast.</p>
<p>The goal of this story is to evaluate the memory configuration of each JVM instance on the CNs and tune as necessary with the primary goal of avoiding out of memory errors, secondary goal is improving performance.</p>
<p>Note that with increased heap allocation, garbage collection may become a significant bottleneck, with application freeze in the order of several minutes possible or likely with higher heap allocations (e.g. 16GB)</p>
Infrastructure - Story #7882 (In Progress): Tune CN logfile managementhttps://redmine.dataone.org/issues/78822016-09-09T20:13:07ZDave Vieglaisdave.vieglais@gmail.com
<p>There are many log files generated on the Coordinating Nodes which can make diagnostics challenging. Some logs also appear to be misconfigured or set to log at DEBUG level even on production systems, resulting in extremely verbose logs.</p>
<p>The goal of this activity is to streamline logging to make it easier to find useful information in the logs by reducing verbosity, consolidating where possible, and perhaps refining some log messages.</p>
Infrastructure - Story #7859 (New): Add formatID for the STL 3d model file formathttps://redmine.dataone.org/issues/78592016-08-04T19:02:58ZBryce Mecummecum@nceas.ucsb.edu
<p>The STL file format is a domain standard file format for storing 3d models and is the most common way I've managed 3d models used while 3d printing. Given that 3d printing is seeing increased usage in the sciences, I would say this is a good candidate for inclusion in the controlled list of format ids.</p>
<p>Type: DATA<br>
Id: STL<br>
Name: StereoLithography File Format<br>
Media type: application/sla (unofficial)<br>
Extension: .stl</p>
<p>There is an ASCII form and a Binary form of this format. They don't see to be distinguished according to any standard. What do we do in this case?</p>
<p>References: <br>
- <a href="https://en.wikipedia.org/wiki/STL_(file_format)">https://en.wikipedia.org/wiki/STL_(file_format)</a><br>
- <a href="https://reference.wolfram.com/language/ref/format/STL.html">https://reference.wolfram.com/language/ref/format/STL.html</a></p>
Infrastructure - Story #7810 (New): Need to avoid buffer overflow condition during HTTP client TL...https://redmine.dataone.org/issues/78102016-05-16T23:33:37ZChris Jonescjones@nceas.ucsb.edu
<p>In <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Error -1205 "Client Certificate Rejected" by Safari (Closed)" href="https://redmine.dataone.org/issues/2693">#2693</a>, <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: completely unable to access cn.dataone.org from Safari 7.1 if user has any certificates installed (Closed)" href="https://redmine.dataone.org/issues/6539">#6539</a>, and other tickets we've documented issues with Safari failing to connect to CN and MN services when the server side is configured with 'SSLVerifyClient optional' and Safari has certificates installed in the Keychain that happen to not be trusted by the CN or MN. The workaround is to conditionally set the SSLVerifyClient directive.</p>
<p>However, a side affect of this is that HTTP clients look to be forced to buffer the request on the server during the TLS renegotiation phase. The default buffer size is 128K, and therefore most all POSTs to MNs and CNs that involve data files exceed this limit, and the connection fails due to a buffer overflow condition. An example on the Arctic Data Center deployment shows:</p>
<p>./arcticdata.io.error.log.1:[Fri May 13 13:24:10.681521 2016] [ssl:error] [pid 9096] [client 98.228.75.248:54733] AH02018: request body exceeds maximum size (131072) for SSL buffer, referer: <a href="https://arcticdata.io/catalog/">https://arcticdata.io/catalog/</a><br>
</p>
<p>The main workaround, as mentioned in this "Stack Overflow thread":<a href="http://stackoverflow.com/a/15394058/4200841">http://stackoverflow.com/a/15394058/4200841</a>, is to have clients leverage the HTTP 1.1 'Expect' header feature. By setting this header value to '100-continue', it cues the server that a large payload is coming (> 128K!) and to do the renegotiation without buffering the request.</p>
<p>This story is a placeholder to add tasks for each DataONE client product we manage to include this header: d1_libclient_java, d1_libclient_python, matlab-dataone (covered by the Java fix, but needs incorporation), rdataone, and MetacatUI.</p>
<p>Note that a temporary fix on the server side is to increase the buffer size on the MNs and CNs using the 'SSLRenegBufferSize' directive, but the memory consequences need to be considered, as well as the possibility of DOS exposure.</p>
DataONE API - Story #6759 (New): ObjectFormat Managementhttps://redmine.dataone.org/issues/67592015-01-13T20:12:14ZRob Nahfrnahf@epscor.unm.edu
<p>There currently are not any API methods for managing the collection of objectFormats registered to a dataone environment. There is a "bootstrap" resource that constitutes a the list in either d1_libclient_java or d1_common_java that can be used in testing environments. There's also a different resource in the cn-os-core project that is used in production.</p>
<p>These 2 resources are difficult to maintain (keep synchronized), and there isn't a defined process for adding formats to production.</p>
<p>We discussed the inclusion of an "addFormat(...) method in V2, but it is not currently in the API. (It would be part of the CNCore API).</p>
<p>It would be good to review the situation with a focused discussion to at least troubleshoot the existing informal management practices and formalize them; and then consider if more infrastructure is needed.</p>
Infrastructure - Story #4091 (New): ESRI GeoPortal MN stackhttps://redmine.dataone.org/issues/40912013-10-15T13:36:56ZBruce Wilsonbwilso27@utk.edu
<p>The objective is to design, develop, and implement a MN Stack to integrate with the ESRI GeoPortal server (<a href="http://www.esri.com/software/arcgis/geoportal">http://www.esri.com/software/arcgis/geoportal</a>).</p>
Infrastructure - Story #3656 (New): integration testing: what are acceptable pids for update?https://redmine.dataone.org/issues/36562013-03-12T22:31:53ZRob Nahfrnahf@epscor.unm.edu
<p>Update takes an originalPid as a parameter which is used to set the obsoletes and obsoletedBy fields for the two objects. Can the pid provided for that parameter be for an object that is:</p>
<p>a) archived - ?<br>
b) reserved - should be no<br>
c) deleted - ?<br>
d) "current" but located on different MN</p>
<p>d) may be difficult to test, it would happen only when the original member node stops hosting the original - is no longer the authoritative node - but is still in service.</p>
DataONE API - Story #1644 (New): Develop an object format creation policyhttps://redmine.dataone.org/issues/16442011-06-14T16:25:11ZChris Jonescjones@nceas.ucsb.edu
<p>The object format list in d1_common_java is thus far an ad hoc list of known object formats needed in the D1 software. Additions will be needed. We need to develop a policy on who will have write access to the realtime version of this list, when the on-disk version will be periodically updated, etc. New object formats need to be vetted, and that process should be put into place. This process should align with the object format creation process with the UDFR group when their registry is operational.</p>