DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2018-07-13T11:24:16ZDataONE Tasks
Redmine Infrastructure - Bug #8655 (New): Synchronization died with OOMhttps://redmine.dataone.org/issues/86552018-07-13T11:24:16ZDave Vieglaisdave.vieglais@gmail.com
<p>d1-processing became unresponsive. cn-synchronization log showed:<br>
<code><br>
[ERROR] 2018-07-12 18:28:26,875 [ProcessDaemonTask1] (SyncObjectTaskManager:run:84) java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space<br>
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space<br>
at java.util.concurrent.FutureTask.report(FutureTask.java:122)<br>
at java.util.concurrent.FutureTask.get(FutureTask.java:192)<br>
at org.dataone.cn.batch.synchronization.SyncObjectTaskManager.run(SyncObjectTaskManager.java:76)<br>
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<br>
at java.util.concurrent.FutureTask.run(FutureTask.java:266)<br>
at java.lang.Thread.run(Thread.java:748)<br>
Caused by: java.lang.OutOfMemoryError: Java heap space<br>
[ INFO] 2018-07-12 18:28:49,862 [ProcessDaemonTask1] (SyncObjectTaskManager:run:110) SyncObjectTaskManager Complete<br>
[ WARN] 2018-07-12 20:41:15,788 [hz.client.2.Listener] (NodeTopicListener:onMessage:68) urn:node:OTS_NDC- NodeTopicListener Disabl<br>
</code></p>
<p>d1-processing is running with:<br>
<code><br>
-Djava.awt.headless=true -XX:UseParallelGC -Xmx4096M -Xms1024M -Xss1280k -XX:MaxPermSize=512M<br>
</code></p>
Testing MN Management - Task #8464 (New): test: Metadata Validationhttps://redmine.dataone.org/issues/84642018-03-01T21:04:42ZAmy Forresteraforres4@utk.edu
<p>Receive and validate sample metadata records from member node.</p>
Testing MN Management - Story #8463 (New): test: Testing & Developmenthttps://redmine.dataone.org/issues/84632018-03-01T21:04:41ZAmy Forresteraforres4@utk.edu
<p>Install or develop a functional member node to be registered to a non-production environment. </p>
Testing MN Management - Task #8351 (New): Display as Upcominghttps://redmine.dataone.org/issues/83512018-02-08T15:28:12ZMonica Ihliemail@monicaihli.com
<ul>
<li>Ensure MN logo in github (<a href="https://github.com/DataONEorg/member-node-info">https://github.com/DataONEorg/member-node-info</a>)</li>
<li>Create new entry for the MN in the Upcoming Nodes Document. (<a href="https://github.com/DataONEorg/member-node-info/tree/master/production/upcoming">https://github.com/DataONEorg/member-node-info/tree/master/production/upcoming</a>)</li>
<li>Confirm MN Dashboard detected change in github and displays the new entry as "upcoming". (<a href="https://www.dataone.org/current-member-nodes">https://www.dataone.org/current-member-nodes</a>)</li>
</ul>
Testing MN Management - Task #8350 (New): Test Registrationhttps://redmine.dataone.org/issues/83502018-02-08T15:28:12ZMonica Ihliemail@monicaihli.com
<p>Custom nodes will start off by first registering in Sandbox. Nodes using an existing MN software will register in Stage.</p>
<ul>
<li>Node contact subject approved by D1 admin in DataONE LDAP.</li>
<li>Node software configuration - synchronization enabled if applicable.</li>
<li>Node registration document generated and submitted to Sandbox or Stage CN.</li>
<li>D1 Admin approves node registration on the Sandbox or Stage CN server.</li>
<li>Monitor and verify synchronization, indexing, search behavior.</li>
<li>MN approves display of information in test search interface (<a href="https://search-sandbox.test.dataone.org/#data">https://search-sandbox.test.dataone.org/#data</a> or <a href="https://search-stage.test.dataone.org/#data">https://search-stage.test.dataone.org/#data</a>)</li>
</ul>
<p>A note for custom implemented DataONE services: It's expected that development and testing will be more iterative. When satisfied with results in sandbox, repeat the process by changing target to cn-stage.</p>
Testing MN Management - Task #8349 (New): Develop or Implement MN Softwarehttps://redmine.dataone.org/issues/83492018-02-08T15:28:12ZMonica Ihliemail@monicaihli.com
<ul>
<li><p>Change node's primary MNDeployment ticket status to Testing.</p></li>
<li><p>Identity Management:</p>
<ul>
<li>Confirm what nodeID will be used.</li>
<li>Register NCEAS LDAP (<a href="https://identity.nceas.ucsb.edu/">https://identity.nceas.ucsb.edu/</a>) because cert download auth is maintained by separate system.</li>
<li>Use Use cilogon to generate a DataONE LDAP account by logging into DataONE test env. (Sign-in link at <a href="https://search-stage.test.dataone.org">https://search-stage.test.dataone.org</a>)</li>
</ul></li>
<li><p>DataONE Certificate:</p>
<ul>
<li>Request in #CI for a sandbox or stage D1 certificate generated available to NCEAS uid.</li>
<li>Cert/key downloaded by node operator.</li>
</ul></li>
</ul>
<p><strong><em>For a custom implementation of DataONE APIs as new service:</em></strong></p>
<ul>
<li>MN implements full scope of Member Node APIs.</li>
<li>API methods systematically verified by D1 tech lead.</li>
<li>Passes web tester checks (<a href="http://mncheck.test.dataone.org:8080">http://mncheck.test.dataone.org:8080</a>)</li>
</ul>
<p><strong><em>For a DataONE Software Stack</em></strong></p>
<ul>
<li>Install software, SSL cert, and DataONE cert per installation instructions.</li>
<li>If using LE cert, schedule cron for renewal.</li>
<li>Configure software as appropriate (baseURL, Node Contact Subject, replication, etc).</li>
</ul>
<p><strong><em>For a SlenderNode:</em></strong></p>
<ul>
<li>Develop adapter.</li>
<li>Fully test movement of data from source system to target MN software.</li>
<li>Install on MN web server and schedule cron job. Verify cron is running as expected.</li>
</ul>
<p><strong><em>For both:</em></strong><br>
* Verify system metadata of test records. Ensure correct node URI value in auth/orig MN.<br>
* Verify node description document. Ensure that Contact Subject is set to the DataONE LDAP identity string.</p>
Testing MN Management - Task #8348 (New): Metadata Validationhttps://redmine.dataone.org/issues/83482018-02-08T15:28:11ZMonica Ihliemail@monicaihli.com
<p>Receive and validate sample metadata records from member node.</p>
Testing MN Management - Story #8347 (New): Testing & Developmenthttps://redmine.dataone.org/issues/83472018-02-08T15:28:11ZMonica Ihliemail@monicaihli.com
<p>Install or develop a functional member node to be registered to a non-production environment. </p>
Testing MN Management - Task #8346 (New): Training & Educationhttps://redmine.dataone.org/issues/83462018-02-08T15:28:11ZMonica Ihliemail@monicaihli.com
<ul>
<li>Live training session or video links provided for DataONE systems and architecture.</li>
<li>Synchronization training video (<a href="http://vimeo.com/200407194">http://vimeo.com/200407194</a> / <a href="https://www.dataone.org/sites/default/files/20160901_mn-synchronization.pdf">https://www.dataone.org/sites/default/files/20160901_mn-synchronization.pdf</a>)</li>
<li>Share links to appropriate technical resources as needed. These might include:
<ul>
<li>MN API (<a href="http://jenkins-1.dataone.org/jenkins/job/API_Documentation_trunk/ws/api-documentation/build/html/apis/MN_APIs.html">http://jenkins-1.dataone.org/jenkins/job/API_Documentation_trunk/ws/api-documentation/build/html/apis/MN_APIs.html</a>)</li>
<li>MetaCat installation instructions (<a href="https://knb.ecoinformatics.org/knb/docs/install.html#installing-on-linux">https://knb.ecoinformatics.org/knb/docs/install.html#installing-on-linux</a>)</li>
<li>GMN installation instructions (<a href="http://dataone-python.readthedocs.io/en/latest/gmn/index.html">http://dataone-python.readthedocs.io/en/latest/gmn/index.html</a>)</li>
<li>Python Library implementation of APIs (<a href="http://dataone-python.readthedocs.io/en/latest/common/index.html">http://dataone-python.readthedocs.io/en/latest/common/index.html</a>)</li>
</ul></li>
</ul>
Testing MN Management - Task #8345 (New): Technical Requirementshttps://redmine.dataone.org/issues/83452018-02-08T15:28:10ZMonica Ihliemail@monicaihli.com
<ul>
<li>Confirm chosen deployment path.</li>
<li>Communicate knowledge of technical requirements for selected deployment. These can include:
<ul>
<li>SSL certificate.</li>
<li>FQDN.</li>
<li>Apache web server with Python 2.7 or Java (depending on stack).</li>
</ul></li>
<li>Confirm metadata format. Communicate expectations for metadata quality.</li>
<li>Specify in advance whether a separate test installation will be maintained in parallel with production. This is important for planning the deployment. A separate test will need distinct subdomain and SSL cert if not using wildcard.</li>
</ul>
Testing MN Management - Story #8343 (New): Planninghttps://redmine.dataone.org/issues/83432018-02-08T15:28:10ZMonica Ihliemail@monicaihli.com
<p>The repository and DataONE have agreed to proceed with deployment as a member node. Decisions will be made as to how to proceed with development. Node operators will receive training.</p>
Testing MN Management - Story #8340 (New): Discoveryhttps://redmine.dataone.org/issues/83402018-02-08T15:28:09ZMonica Ihliemail@monicaihli.com
<p>Discovery is about establishing contact and building a relationship with a potential new member node. In this phase, it is determined if DataONE and the repository are a good fit for one another and if the repository generally meets the requirements of DataONE member nodes. Broad discussions of deployment options may be reviewed as well.<br>
This story is complete when a determination is made to either proceed with planning a new deployment, or that joining DataONE is not an option for the repository at this time.</p>
Infrastructure - Bug #7919 (New): unloadable system metadata in CNs by Hazelcasthttps://redmine.dataone.org/issues/79192016-10-26T16:22:56ZRob Nahfrnahf@epscor.unm.edu
<p>Looking through the metacat logs, I found a lot of instances where the HzSystemMetadataMap could not load system metadata for particular pids. Most had dryad in the pid (~1200), but another 130 are from elsewhere.</p>
<p>a random sample showed that it couldn't be retrieved via /meta although the pid could be retrieved from the Dryad MN.</p>
<p>This appears to be another type of half-created content on the CN.</p>
<p>rnahf@cn-ucsb-1:~$ grep 'could not load system metadata' /var/metacat/logs/metacat.log | cut -c60- | sort | uniq | grep -v dryad | wc -l<br>
139<br>
rnahf@cn-ucsb-1:~$ grep 'could not load system metadata' /var/metacat/logs/metacat.log | cut -c60- | sort | uniq | grep dryad | wc -l<br>
1216</p>
Member Nodes - Task #3906 (New): Update malformed Resource Mapshttps://redmine.dataone.org/issues/39062013-08-09T17:50:27ZRob Nahfrnahf@epscor.unm.edu
<p>Update all existing resource maps in Merritt and ONShare MNs so that URIs are used instead of object-literals, to create valid resource maps. </p>
Infrastructure - Task #1556 (In Progress): Interns mailing listhttps://redmine.dataone.org/issues/15562011-05-12T20:19:01ZAmber Buddenaebudden@gmail.com
<p>Can you please clear the current 'interns' mailing list in preparation for use in 2011. It currently sends out to last years interns. Once the new interns have set up plone accounts I will submit a new request to have them added.</p>