DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2016-05-13T16:56:25ZDataONE Tasks
Redmine Infrastructure - Story #7807 (New): cn.synchronize should support synchronization failure correct...https://redmine.dataone.org/issues/78072016-05-13T16:56:25ZRob Nahfrnahf@epscor.unm.edu
<p>cn.synchronize(session, identifier) works well for its original purpose (supporting MN-driven system metadata updates, and MN-driven push synchronization), but doesn't seem to work for manual synchronization failure workflows. The main problem is that the request can only be made by the MN itself (using the MN client certificate). </p>
<p>As we envision a centralized dashboard for monitoring failed synchronization items, how do we address this situation? </p>
<p>The synchronization processing queue needs both the pid and a nodeId from where to retrieve the object. the NodeId is not specified directly in the method call, but gleaned from the session by a reverse lookup from the certificate. (It uses the first node found in the NodeList where the Node.subject field matches the certificate subject).</p>
<p>Should we allow node.contactSubjects into the algorithm?<br>
Should we add nodeId as a parameter?</p>
Python GMN - Story #7095 (New): Support HTTP redirect capability for GMN within the Vendor Specif...https://redmine.dataone.org/issues/70952015-05-12T16:20:39ZMark Servillamark.servilla@gmail.com
<p>GMN provides proxy capabilities to integrate existing content residing in a back-end data repository through its Vendor Specific Extension model. Only minimal support exists within the GMN-VSE model for HTTP redirects. Although the use of HTTP redirects is more of an exception within the the GMN-VSE model, it can provide a flexible mechanism for MNs who dynamically manage data and metadata objects.</p>
Python Libraries - Story #5468 (New): Remove dead code from D1 Python stackhttps://redmine.dataone.org/issues/54682014-06-06T18:57:00ZRoger Dahldahl@unm.edu
<p><a href="https://pypi.python.org/pypi/vulture">https://pypi.python.org/pypi/vulture</a></p>
Infrastructure - Story #4188 (In Progress): dataone Exception definition and implementation requi...https://redmine.dataone.org/issues/41882013-11-27T21:34:20ZDave Vieglaisdave.vieglais@gmail.com
<p>The documentation describes the properties and serialization of DataONE exceptions:</p>
<p><a href="http://mule1.dataone.org/ArchitectureDocs-current/apis/Exceptions.html">http://mule1.dataone.org/ArchitectureDocs-current/apis/Exceptions.html</a></p>
<p>However, the definition in the schema:</p>
<p><a href="https://repository.dataone.org/software/cicore/tags/D1_SCHEMA_1_1_1/dataoneErrors.xsd">https://repository.dataone.org/software/cicore/tags/D1_SCHEMA_1_1_1/dataoneErrors.xsd</a></p>
<p>differs, and so presents an inconsistent reference for implementations.</p>
<p>The Java code appears to follow the documentation, however the python implementation uses the schema to generate exception messages, and so follows the schema definition.</p>
<p>The schema and python code need to be updated to reflect the description in the documentation. Also, all implementations of MN and client software need to be informed of the issue and how they may be impacted.</p>
Infrastructure - Story #4091 (New): ESRI GeoPortal MN stackhttps://redmine.dataone.org/issues/40912013-10-15T13:36:56ZBruce Wilsonbwilso27@utk.edu
<p>The objective is to design, develop, and implement a MN Stack to integrate with the ESRI GeoPortal server (<a href="http://www.esri.com/software/arcgis/geoportal">http://www.esri.com/software/arcgis/geoportal</a>).</p>
Infrastructure - Story #4052 (In Progress): OPeNDAP MN Storyhttps://redmine.dataone.org/issues/40522013-10-06T20:07:50ZBruce Wilsonbwilso27@utk.edu
<p>There are a large number of possible DataONE MN's running Data Access Protocol (DAP) compliant servers, particularly servers based on the OPeNDAP Hyrax and UCAR THREDDS servers. The objective of this work is to develop a MN stack that can be used with at least one of these DAP-compliant software stacks as a low-barrier route to becoming a Tier 1 MN.</p>
Infrastructure - Story #3720 (New): resource maps should be validatedhttps://redmine.dataone.org/issues/37202013-04-19T20:59:36ZRob Nahfrnahf@epscor.unm.edu
<p>well-formed rdf-xml resource maps can still be unreadable by DataONE tools, which requires certain relationships to be present to be useful for DataONE. see Data Packaging architecture document (<a href="http://mule1.dataone.org/ArchitectureDocs-current/design/DataPackage.html#generating-resource-maps">http://mule1.dataone.org/ArchitectureDocs-current/design/DataPackage.html#generating-resource-maps</a>)</p>
<p>Therefore, resource maps should be validated against the DataONE requirements to prevent "silent" errors from making content undiscoverable.</p>
<p>At a minimum, validation methods should be built in d1_libclient_java, so that validation can be done prior to submission, on the MN (during MN.create/MN.update, or on the CN (during sync). </p>
<p>Additionally, facilities for interrogating the resource map to pull out relationships should be developed, using RDFS Reasoners to recover from missing inverse relationships, and existing services using resource maps (the indexer) should make use of them.</p>
<p>Development of validation services should be considered to help clients validate prior to submission.</p>
Java Client - Story #3666 (In Progress): D1Client.listUpdateHistory() needs to handle changing ac...https://redmine.dataone.org/issues/36662013-03-15T22:51:23ZRob Nahfrnahf@epscor.unm.edu
<p>the current D1Client.listUpdateHistory() method needs to gracefully handle the situation where a NotAuthorized request is returned. the ObsoletesChain client class may need to be refactored to allow for this exception to be held so it can notify the user where appropriate.</p>
<p>Ostensibly, with a NotAuthorized, the user will not have access to either the tail or head of the chain, so can't return the head or tail, depending on how access changes.</p>
Infrastructure - Story #3656 (New): integration testing: what are acceptable pids for update?https://redmine.dataone.org/issues/36562013-03-12T22:31:53ZRob Nahfrnahf@epscor.unm.edu
<p>Update takes an originalPid as a parameter which is used to set the obsoletes and obsoletedBy fields for the two objects. Can the pid provided for that parameter be for an object that is:</p>
<p>a) archived - ?<br>
b) reserved - should be no<br>
c) deleted - ?<br>
d) "current" but located on different MN</p>
<p>d) may be difficult to test, it would happen only when the original member node stops hosting the original - is no longer the authoritative node - but is still in service.</p>
Infrastructure - Story #3591 (In Progress): Content consistency checks for new member nodeshttps://redmine.dataone.org/issues/35912013-02-18T23:24:34ZRob Nahfrnahf@epscor.unm.edu
<p>Want to be able to detect systematic content errors for new Member nodes:<br>
1) systemMetadata is parseable - existing getSystemMetadata() test should fail when it can't deserialize the systemMetadata, yes?<br>
2) resourceMaps are parseable - pull a resourceMap listObjects(formatId=, use ResourceMapFactory to deserialize <br>
3) checksum stability - content should have the same checksum every time it is pulled.</p>
Infrastructure - Story #2944 (New): Design and implement a MN kill switch mechanismhttps://redmine.dataone.org/issues/29442012-06-15T15:00:54ZDave Vieglaisdave.vieglais@gmail.com
<p>We need a mechanism to quickly and effectively de-register a MN and it's content in case there is some issue with the MN that warrants such action (e.g. malware is found on the server or some other inappropriate or malicious activity).</p>
Infrastructure - Story #2548 (New): recasting untrusted certs to public poses accessibility incon...https://redmine.dataone.org/issues/25482012-03-27T21:55:59ZRob Nahfrnahf@epscor.unm.edu
<p>KNB recasts a connection with an untrusted certificate to public, so that a client does not get "less than public" privileges.<br>
GMN throws an InvalidToken in this situation.<br>
both refuse connections from clients with expired certificates from trusted CAs.</p>
<p>This approach can cause confusion caused when the user unwittingly uses an untrusted certficate and doesn't get what they expected. If these connections were instead refused, the user would be alerted and could reconnect as a public user, if it chose.</p>
<p>brief discussion found at line 97 of : <a href="http://epad.dataone.org/20120131-authn-authz-questions">http://epad.dataone.org/20120131-authn-authz-questions</a></p>
<ul>
<li>when would honest users be in this situation?</li>
<li>elicit advantages of recasting approach</li>
<li>either way, dataone should implement uniform behavior across CN and MNs.</li>
</ul>
Infrastructure - Story #2488 (New): Changing the authoritativeMembernode will require all replica...https://redmine.dataone.org/issues/24882012-03-14T19:08:19ZRobert Waltz
<p>Modifying the authoritativeMembernode requires human intervention. A tool should be created to perform the modification and then call MNStorage.systemMetadataChanged() on all replicas of the object.</p>
DataONE API - Story #1644 (New): Develop an object format creation policyhttps://redmine.dataone.org/issues/16442011-06-14T16:25:11ZChris Jonescjones@nceas.ucsb.edu
<p>The object format list in d1_common_java is thus far an ad hoc list of known object formats needed in the D1 software. Additions will be needed. We need to develop a policy on who will have write access to the realtime version of this list, when the on-disk version will be periodically updated, etc. New object formats need to be vetted, and that process should be put into place. This process should align with the object format creation process with the UDFR group when their registry is operational.</p>
Infrastructure - Story #725 (In Progress): Create Authentication and Access control design specif...https://redmine.dataone.org/issues/7252010-08-02T21:42:24ZChad Berkleyberkley@nceas.ucsb.edu
<p>The metacat <a class="wiki-page new" href="https://redmine.dataone.org/projects/d1/wiki/CrudService">CrudService</a> class contains methods for authentication and access control changes that are not part of the original D1 Crud specification. These services need to be decided on at a higher level and described in the specification so that they can be made to work with any D1 node, not just metacat.</p>