DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2020-04-23T16:24:46ZDataONE Tasks
Redmine Infrastructure - Story #8862 (New): Deploy a new dataone-cn-rest releasehttps://redmine.dataone.org/issues/88622020-04-23T16:24:46ZJing Taotao@nceas.ucsb.edu
<p>We have a new d1_portal jar release which addresses the issue that restarting tomcat in CNs is needed when the LE certificates are renewed in CNs. The new d1_portal jar file has been deployed to dataone-cn-portal. However, the component dataone-cn-rest was overlooked. We need to deploy it there as well.<br>
Yesterday, we did a hack fix in CNs when we restarted tomcat - dropped the d1_portal-2.3.2.jar file there. So now it should work. But we still need a formal release.</p>
Infrastructure - Story #8848 (New): A minor difference of annotation index between CN and MNhttps://redmine.dataone.org/issues/88482019-11-01T21:37:01ZJing Taotao@nceas.ucsb.edu
<p>The solr index on CN is:</p>
<pre><arr name="sem_annotation">
<str>http://purl.dataone.org/odo/ECSO_00000512</str>
<str>
http://ecoinformatics.org/oboe/oboe.1.2/oboe-core.owl#MeasurementType
</str>
<str>http://purl.dataone.org/odo/ECSO_00001102</str>
<str>http://purl.dataone.org/odo/ECSO_00001243</str>
<str>http://purl.dataone.org/odo/ECSO_00000629</str>
<str>http://purl.dataone.org/odo/ECSO_00000518</str>
<str>http://www.w3.org/2000/01/rdf-schema#Resource</str>
<str>http://purl.dataone.org/odo/ECSO_00000516</str>
<str>http://purl.obolibrary.org/obo/UO_0000301</str>
</arr>
</pre>
<p>The mn is:</p>
<pre><arr name="sem_annotation">
<str>http://purl.dataone.org/odo/ECSO_00000512</str>
<str>
http://ecoinformatics.org/oboe/oboe.1.2/oboe-core.owl#MeasurementType
</str>
<str>http://purl.dataone.org/odo/ECSO_00001102</str>
<str>http://purl.dataone.org/odo/ECSO_00001243</str>
<str>http://purl.dataone.org/odo/ECSO_00000629</str>
<str>http://purl.dataone.org/odo/ECSO_00000518</str>
<str>http://purl.dataone.org/odo/ECSO_00000516</str>
<str>http://purl.obolibrary.org/obo/UO_0000301</str>
</arr>
</pre>
<p>The cn has an extra <code><str>http://www.w3.org/2000/01/rdf-schema#Resource</str></code><br>
Bryce and I discussed it and thought it wouldn't affect the feature. But we still need to figure it out.</p>
Member Nodes - Story #8835 (New): Add ability for scanner to stop after a certain number of errorshttps://redmine.dataone.org/issues/88352019-08-12T19:16:39ZJohn Evans
<p>Right now the scanner will try to go thru the entire list of sitemap documents, regardless of whether they all fail or not. We should add the ability to abort further checks if a certain error threshold is crossed.</p>
Member Nodes - Story #8832 (New): Conflict between pyshacl and owlr on ubuntuhttps://redmine.dataone.org/issues/88322019-08-05T19:30:42ZJohn Evans
<p>This error arises when trying to test on gmn-multihost-test (ubuntu 18.04.2 LTS).<br><br>
~~~<br>
Traceback (most recent call last):<br>
File "/home/gmn/.pyenv/versions/schema_org/bin/d1-check-site", line 11, in <br>
load_entry_point('schema-org==4.0.0', 'console_scripts', 'd1-check-site')()<br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/pkg_resources/<strong>init</strong>.py", line 489, in load_entry_point<br>
return get_distribution(dist).load_entry_point(group, name)<br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/pkg_resources/<strong>init</strong>.py", line 2793, in load_entry_point<br>
return ep.load()<br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/pkg_resources/<strong>init</strong>.py", line 2411, in load<br>
return self.resolve()<br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/pkg_resources/<strong>init</strong>.py", line 2417, in resolve<br>
module = <strong>import</strong>(self.module_name, fromlist=['<strong>name</strong>'], level=0)<br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/schema_org-4.0.0-py3.7.egg/schema_org/<strong>init</strong>.py", line 1, in <br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/schema_org-4.0.0-py3.7.egg/schema_org/arm.py", line 9, in <br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/schema_org-4.0.0-py3.7.egg/schema_org/common.py", line 24, in <br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/schema_org-4.0.0-py3.7.egg/schema_org/jsonld_validator.py", line 10, in <br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/pyshacl/<strong>init</strong>.py", line 3, in <br>
from pyshacl.validate import validate, Validator<br>
File "/home/gmn/.pyenv/versions/schema_org/lib/python3.7/site-packages/pyshacl/validate.py", line 5, in <br>
import owlrl<br>
File "/home/gmn/.pyenv/versions/3.7.4/envs/schema_org/bin/owlrl.py", line 4, in <br>
from owlrl import convert_graph, RDFXML, TURTLE, JSON, AUTO, RDFA<br>
ImportError: cannot import name 'convert_graph' from 'owlrl' (/home/gmn/.pyenv/versions/3.7.4/envs/schema_org/bin/owlrl.py)<br>
Worker completed with exit code: 1<br>
~~~</p>
Infrastructure - Story #8806 (New): Cleanup from OS upgradeshttps://redmine.dataone.org/issues/88062019-05-21T12:45:17ZDave Vieglaisdave.vieglais@gmail.com
<p>There's a few items that need to be addressed after the OS upgrades from 14.04 to 18.04.</p>
Infrastructure - Story #8762 (New): Add new formats to CNhttps://redmine.dataone.org/issues/87622019-02-07T19:58:28ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Add a new formatId for Python notebooks: application/x-ipynb+json. This is the current popular Media Type for Python notebooks (<a href="https://jupyter.readthedocs.io/en/latest/reference/mimetype.html">https://jupyter.readthedocs.io/en/latest/reference/mimetype.html</a>), but as it has the 'x-' prefix it is 'unregistered' and<br>
not recognized by IANA (<a href="https://www.iana.org/assignments/media-types/media-types.xhtml">https://www.iana.org/assignments/media-types/media-types.xhtml</a>).</p>
<p>The new entry should be:<br>
<objectFormat><br>
<formatId>application/x-ipynb+json</formatId><br>
<formatName>Jupyter Notebook</formatName><br>
<formatType>DATA</formatType><br>
<mediaType name="application/x-ipynb+json"/><br>
<extension>ipynb</extension><br>
</objectFormat></p>
Member Nodes - Story #8634 (New): DAAC: Testing & Developmenthttps://redmine.dataone.org/issues/86342018-07-02T14:55:23ZAmy Forresteraforres4@utk.edu
<p><strong>Install or develop a functional member node to be registered to a non-production environment.</strong></p>
Member Nodes - Story #8589 (New): ARM: Re-Discovery & Planninghttps://redmine.dataone.org/issues/85892018-05-10T20:35:03ZAmy Forresteraforres4@utk.edu
<p><strong>5/10/2018:</strong> Meeting at ORNL - Giri Prakash expressed re-interest in becoming a MN. Following on USGS_SDC GMN implementation (~Fall 2018), Aaron Stokes can turn attention to ARM installation</p>
Member Nodes - Story #8568 (New): (CyVerse) Move to productionhttps://redmine.dataone.org/issues/85682018-04-19T18:15:01ZAmy Forresteraforres4@utk.eduMember Nodes - Story #8564 (New): (CyVerse) Testing & Development https://redmine.dataone.org/issues/85642018-04-19T18:11:37ZAmy Forresteraforres4@utk.edu
<p><strong>Install or develop a functional member node to be registered to a non-production environment.</strong></p>
Member Nodes - Story #8521 (New): CAFF: Testing & Developmenthttps://redmine.dataone.org/issues/85212018-03-26T15:07:58ZAmy Forresteraforres4@utk.edu
<p>The repository and DataONE have agreed to proceed with deployment as a member node. Install or develop a functional member node to be registered to a non-production environment.</p>
Member Nodes - Story #8225 (In Progress): Customize Indexing & View for gmd-pangaeahttps://redmine.dataone.org/issues/82252017-12-06T19:41:28ZMonica Ihliemail@monicaihli.com
<p>An example metadata record: <a href="http://cn-sandbox.test.dataone.org/cn/v2/object/doi:10.1594/PANGAEA.877809_.201711172109">http://cn-sandbox.test.dataone.org/cn/v2/object/doi:10.1594/PANGAEA.877809_.201711172109</a></p>
<p>This record in the search interface on sandbox: <a href="https://search-sandbox.test.dataone.org/#view/doi:10.1594/PANGAEA.877809_.201711172109">https://search-sandbox.test.dataone.org/#view/doi:10.1594/PANGAEA.877809_.201711172109</a></p>
<p>Currently, alternate access point is pulling the link from:<br>
/ns0:MD_Metadata/ns0:distributionInfo[ 1 ]/ns0:MD_Distribution[ 1 ]/ns0:transferOptions[ 1 ]/ns0:MD_DigitalTransferOptions[ 1 ]/ns0:onLine[ 1 ]/ns0:CI_OnlineResource[ 1 ]/ns0:linkage[ 1 ]/ns0:URL[ 1 ]</p>
<p>However, Pangaea wishes users to be directed towards a landing page where they are able to obtain METADATA in multiple formats, found in:<br>
/ns0:MD_Metadata/ns0:dataSetURI[ 1 ]/ns2:CharacterString[ 1 ]</p>
<p>The landing page for this example: <a href="https://doi.pangaea.de/10.1594/PANGAEA.877809">https://doi.pangaea.de/10.1594/PANGAEA.877809</a></p>
Infrastructure - Story #8204 (New): Adjust memory allocation for services running under JVM on CNshttps://redmine.dataone.org/issues/82042017-10-24T15:29:28ZDave Vieglaisdave.vieglais@gmail.com
<p>Most services on CNs run in JVM instances, each with separate restrictions on memory use.</p>
<p>Current configurations were mostly based on defaults and have not changed much despite significantly higher memory use for systems such as Solr and Hazelcast.</p>
<p>The goal of this story is to evaluate the memory configuration of each JVM instance on the CNs and tune as necessary with the primary goal of avoiding out of memory errors, secondary goal is improving performance.</p>
<p>Note that with increased heap allocation, garbage collection may become a significant bottleneck, with application freeze in the order of several minutes possible or likely with higher heap allocations (e.g. 16GB)</p>
Infrastructure - Story #7859 (New): Add formatID for the STL 3d model file formathttps://redmine.dataone.org/issues/78592016-08-04T19:02:58ZBryce Mecummecum@nceas.ucsb.edu
<p>The STL file format is a domain standard file format for storing 3d models and is the most common way I've managed 3d models used while 3d printing. Given that 3d printing is seeing increased usage in the sciences, I would say this is a good candidate for inclusion in the controlled list of format ids.</p>
<p>Type: DATA<br>
Id: STL<br>
Name: StereoLithography File Format<br>
Media type: application/sla (unofficial)<br>
Extension: .stl</p>
<p>There is an ASCII form and a Binary form of this format. They don't see to be distinguished according to any standard. What do we do in this case?</p>
<p>References: <br>
- <a href="https://en.wikipedia.org/wiki/STL_(file_format)">https://en.wikipedia.org/wiki/STL_(file_format)</a><br>
- <a href="https://reference.wolfram.com/language/ref/format/STL.html">https://reference.wolfram.com/language/ref/format/STL.html</a></p>
OGC-Slender Node - Story #7166 (New): Create ORE document for an NODC data packagehttps://redmine.dataone.org/issues/71662015-06-08T18:49:41ZDave Vieglaisdave.vieglais@gmail.com
<p>Each accession is treated as a data package. Need to generate a resource map document that describes the contents of an accession.</p>