DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2019-11-01T21:37:01ZDataONE Tasks
Redmine Infrastructure - Story #8848 (New): A minor difference of annotation index between CN and MNhttps://redmine.dataone.org/issues/88482019-11-01T21:37:01ZJing Taotao@nceas.ucsb.edu
<p>The solr index on CN is:</p>
<pre><arr name="sem_annotation">
<str>http://purl.dataone.org/odo/ECSO_00000512</str>
<str>
http://ecoinformatics.org/oboe/oboe.1.2/oboe-core.owl#MeasurementType
</str>
<str>http://purl.dataone.org/odo/ECSO_00001102</str>
<str>http://purl.dataone.org/odo/ECSO_00001243</str>
<str>http://purl.dataone.org/odo/ECSO_00000629</str>
<str>http://purl.dataone.org/odo/ECSO_00000518</str>
<str>http://www.w3.org/2000/01/rdf-schema#Resource</str>
<str>http://purl.dataone.org/odo/ECSO_00000516</str>
<str>http://purl.obolibrary.org/obo/UO_0000301</str>
</arr>
</pre>
<p>The mn is:</p>
<pre><arr name="sem_annotation">
<str>http://purl.dataone.org/odo/ECSO_00000512</str>
<str>
http://ecoinformatics.org/oboe/oboe.1.2/oboe-core.owl#MeasurementType
</str>
<str>http://purl.dataone.org/odo/ECSO_00001102</str>
<str>http://purl.dataone.org/odo/ECSO_00001243</str>
<str>http://purl.dataone.org/odo/ECSO_00000629</str>
<str>http://purl.dataone.org/odo/ECSO_00000518</str>
<str>http://purl.dataone.org/odo/ECSO_00000516</str>
<str>http://purl.obolibrary.org/obo/UO_0000301</str>
</arr>
</pre>
<p>The cn has an extra <code><str>http://www.w3.org/2000/01/rdf-schema#Resource</str></code><br>
Bryce and I discussed it and thought it wouldn't affect the feature. But we still need to figure it out.</p>
Infrastructure - Story #8806 (New): Cleanup from OS upgradeshttps://redmine.dataone.org/issues/88062019-05-21T12:45:17ZDave Vieglaisdave.vieglais@gmail.com
<p>There's a few items that need to be addressed after the OS upgrades from 14.04 to 18.04.</p>
Infrastructure - Story #8762 (New): Add new formats to CNhttps://redmine.dataone.org/issues/87622019-02-07T19:58:28ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Add a new formatId for Python notebooks: application/x-ipynb+json. This is the current popular Media Type for Python notebooks (<a href="https://jupyter.readthedocs.io/en/latest/reference/mimetype.html">https://jupyter.readthedocs.io/en/latest/reference/mimetype.html</a>), but as it has the 'x-' prefix it is 'unregistered' and<br>
not recognized by IANA (<a href="https://www.iana.org/assignments/media-types/media-types.xhtml">https://www.iana.org/assignments/media-types/media-types.xhtml</a>).</p>
<p>The new entry should be:<br>
<objectFormat><br>
<formatId>application/x-ipynb+json</formatId><br>
<formatName>Jupyter Notebook</formatName><br>
<formatType>DATA</formatType><br>
<mediaType name="application/x-ipynb+json"/><br>
<extension>ipynb</extension><br>
</objectFormat></p>
Infrastructure - Story #8736 (New): Decouple the index generator to the hazelcast system metadata...https://redmine.dataone.org/issues/87362018-10-23T20:01:34ZJing Taotao@nceas.ucsb.edu
<p>Currently, the index generator is a listener of the system metadata map. During the tomcat starting, the map will load data from the db table and will generate a lot of unneeded index tasks. Currently we try to use filters to filter out them but it is very expensive. </p>
Member Nodes - Story #8589 (New): ARM: Re-Discovery & Planninghttps://redmine.dataone.org/issues/85892018-05-10T20:35:03ZAmy Forresteraforres4@utk.edu
<p><strong>5/10/2018:</strong> Meeting at ORNL - Giri Prakash expressed re-interest in becoming a MN. Following on USGS_SDC GMN implementation (~Fall 2018), Aaron Stokes can turn attention to ARM installation</p>
Member Nodes - Story #8568 (New): (CyVerse) Move to productionhttps://redmine.dataone.org/issues/85682018-04-19T18:15:01ZAmy Forresteraforres4@utk.eduMember Nodes - Story #8521 (New): CAFF: Testing & Developmenthttps://redmine.dataone.org/issues/85212018-03-26T15:07:58ZAmy Forresteraforres4@utk.edu
<p>The repository and DataONE have agreed to proceed with deployment as a member node. Install or develop a functional member node to be registered to a non-production environment.</p>
Testing MN Management - Story #8381 (New): Discoveryhttps://redmine.dataone.org/issues/83812018-02-27T17:32:03ZMonica Ihliemail@monicaihli.com
<p>Discovery is about establishing contact and building a relationship with a potential new member node. In this phase, it is determined if DataONE and the repository are a good fit for one another and if the repository generally meets the requirements of DataONE member nodes. Broad discussions of deployment options may be reviewed as well.<br>
This story is complete when a determination is made to either proceed with planning a new deployment, or that joining DataONE is not an option for the repository at this time.</p>
CN REST - Story #8364 (In Progress): Ensure portal uses correct X509 certificateshttps://redmine.dataone.org/issues/83642018-02-13T20:17:25ZChris Jonescjones@nceas.ucsb.edu
<p>We've run into issues where after an upgrade of the <code>dataone-cn-portal</code> package on the CNs, the properties pointing to the public certificate and private key are incorrectly pointing to the old GeoTrust wildcard files rather than the new Lets Encrypt files:<br>
<br>
cn.server.publiccert.filename=/etc/ssl/certs/<em>.test.dataone.org.crt<br>
cn.server.privatekey.filename=/etc/ssl/private/</em>.test.dataone.org.key</p>
<p>These should be (in STAGE):</p>
<p>/etc/letsencrypt/live/cn-stage.test.dataone.org/cert.pem<br>
/etc/letsencrypt/live/cn-stage.test.dataone.org/privkey.pem</p>
<p>The issue might be that these are not being set correctly during the <code>postinst</code> script run. Jing pointed out that these values are taken from the debconf database settings that get set when <code>dataon-cn-os-core</code> is installed. So although the <code>postinst</code> script might be setting the correct values, the old cached values might still be in memory in the debconf database. If so, we'll need to clear those values during installations and upgrades.</p>
<p>Also, knowing where to look for these configuration settings can be challenging. These are referenced from <code>/var/lib/tomcat7/webapps/portal/WEB-INF/portal.properties</code>. These settings should be consolidated into <code>/etc/dataone/portal/portal.properties</code> so they also don't get blown away on war file upgrades in Tomcat.</p>
Testing MN Management - Story #8340 (New): Discoveryhttps://redmine.dataone.org/issues/83402018-02-08T15:28:09ZMonica Ihliemail@monicaihli.com
<p>Discovery is about establishing contact and building a relationship with a potential new member node. In this phase, it is determined if DataONE and the repository are a good fit for one another and if the repository generally meets the requirements of DataONE member nodes. Broad discussions of deployment options may be reviewed as well.<br>
This story is complete when a determination is made to either proceed with planning a new deployment, or that joining DataONE is not an option for the repository at this time.</p>
Infrastructure - Story #8204 (New): Adjust memory allocation for services running under JVM on CNshttps://redmine.dataone.org/issues/82042017-10-24T15:29:28ZDave Vieglaisdave.vieglais@gmail.com
<p>Most services on CNs run in JVM instances, each with separate restrictions on memory use.</p>
<p>Current configurations were mostly based on defaults and have not changed much despite significantly higher memory use for systems such as Solr and Hazelcast.</p>
<p>The goal of this story is to evaluate the memory configuration of each JVM instance on the CNs and tune as necessary with the primary goal of avoiding out of memory errors, secondary goal is improving performance.</p>
<p>Note that with increased heap allocation, garbage collection may become a significant bottleneck, with application freeze in the order of several minutes possible or likely with higher heap allocations (e.g. 16GB)</p>
Infrastructure - Story #7807 (New): cn.synchronize should support synchronization failure correct...https://redmine.dataone.org/issues/78072016-05-13T16:56:25ZRob Nahfrnahf@epscor.unm.edu
<p>cn.synchronize(session, identifier) works well for its original purpose (supporting MN-driven system metadata updates, and MN-driven push synchronization), but doesn't seem to work for manual synchronization failure workflows. The main problem is that the request can only be made by the MN itself (using the MN client certificate). </p>
<p>As we envision a centralized dashboard for monitoring failed synchronization items, how do we address this situation? </p>
<p>The synchronization processing queue needs both the pid and a nodeId from where to retrieve the object. the NodeId is not specified directly in the method call, but gleaned from the session by a reverse lookup from the certificate. (It uses the first node found in the NodeList where the Node.subject field matches the certificate subject).</p>
<p>Should we allow node.contactSubjects into the algorithm?<br>
Should we add nodeId as a parameter?</p>
Member Nodes - Story #5833 (In Progress): GBIF: Developinghttps://redmine.dataone.org/issues/58332014-07-17T21:05:32ZRoger Dahldahl@unm.edu
<p>Determine which software stack to use, etc.</p>
Infrastructure - Story #4091 (New): ESRI GeoPortal MN stackhttps://redmine.dataone.org/issues/40912013-10-15T13:36:56ZBruce Wilsonbwilso27@utk.edu
<p>The objective is to design, develop, and implement a MN Stack to integrate with the ESRI GeoPortal server (<a href="http://www.esri.com/software/arcgis/geoportal">http://www.esri.com/software/arcgis/geoportal</a>).</p>
DataONE API - Story #1644 (New): Develop an object format creation policyhttps://redmine.dataone.org/issues/16442011-06-14T16:25:11ZChris Jonescjones@nceas.ucsb.edu
<p>The object format list in d1_common_java is thus far an ad hoc list of known object formats needed in the D1 software. Additions will be needed. We need to develop a policy on who will have write access to the realtime version of this list, when the on-disk version will be periodically updated, etc. New object formats need to be vetted, and that process should be put into place. This process should align with the object format creation process with the UDFR group when their registry is operational.</p>