DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2015-09-28T17:34:30ZDataONE Tasks
Redmine Java Client - Task #7389 (Testing): V2 D1Object fails to download V1 contenthttps://redmine.dataone.org/issues/73892015-09-28T17:34:30ZChris Jonescjones@nceas.ucsb.edu
<p>During testing of services in the mixed V1/V2 DEV2 environment, D1Object fails to download content listed in the ObjectLocationList from a V1-only Member Node. The symptom is a null pointer exception when trying to close a non-existent temporary file where the bytes of the object should have been located. Fix download() to call V1 endpoints on V1-only MNs.</p>
Java Client - Bug #7322 (Testing): D1Object stores data in memory, causes out of memory errorshttps://redmine.dataone.org/issues/73222015-08-27T23:56:06ZChris Jonescjones@nceas.ucsb.edu
<p>When assembling DataPackage instances and populating them, the DataPackage class relies on the underlying D1Object.download() method to store members of the DataPackage locally. The current implementation calls IOUtils.toByteArray(inputstream), which of course stores all bytes in memory. With large data files, this effectively renders DataPackage useless because of OutOfMemory exceptions. The move towards using the javax.activation.DataSource interface helps with this since it provide in memory and on disk implemetations.</p>
<p>Change download() to default to the on-disk DataSource, and make the storage location configurable in d1client.properties.</p>
Java Client - Task #7120 (Testing): Fix DataPackage.insertRelationship() to handle any URI for ex...https://redmine.dataone.org/issues/71202015-05-21T16:49:34ZChris Jonescjones@nceas.ucsb.edu
<p>DataPackage currently provides two insertRelationship() methods - one to add ORE relationships between metadata and data members of the aggregation, and a second to provide any relationship using predicates from other namespaces (such as PROV). The latter method assumes that all identifiers should be treated as objects using the CN Base URL when constructing the subject and object URIs. This isn't always the case. Change or override the method to accept any URI as subject and object components of the triple, and fix any tests that use this method.</p>
Infrastructure - Task #6843 (In Progress): Update the prov instance of the RdfXmlSubprocessor to ...https://redmine.dataone.org/issues/68432015-02-06T23:18:48ZChris Jonescjones@nceas.ucsb.edu
<p>In the "sem-prov-design issue 66":<a href="https://github.com/DataONEorg/sem-prov-design/issues/66">https://github.com/DataONEorg/sem-prov-design/issues/66</a> we have renamed the provenance-based Solr fields to include 'prov_' as a prefix, and have added new fields. See also "issue 99":<a href="https://github.com/DataONEorg/sem-prov-design/issues/99">https://github.com/DataONEorg/sem-prov-design/issues/99</a> and "issue 100":<a href="https://github.com/DataONEorg/sem-prov-design/issues/100">https://github.com/DataONEorg/sem-prov-design/issues/100</a>.<br>
Modify the provRdfXmlSubprocessor bean to handle the renaming scheme, the new fields, and the inverse fields determined to be useful. Also, add these fields as static Solr fields so we can remove the '_sm' suffixes from the names.</p>
Infrastructure - Task #6250 (New): CN methods return 500 ServiceFailure when called with bogus te...https://redmine.dataone.org/issues/62502014-08-29T16:58:54ZRoger Dahldahl@unm.edu
<p>A number of the test for the Python CN Client currently fail due to 500 ServiceFailure responses from the server. The CN calls are not meant to complete successfully because they're performed with random values and without certificates, but the tests expect specific exceptions, such as 401 NotAuthorized when calling an API without a certificate.</p>
<p>While 500 ServiceFailure is not exactly a bug, it would be great to get an exception that relates to the actual issue with the request.</p>
<p>For instance, a call to CNIdentity.registerAccount() gives:</p>
<p><?xml version="1.0" encoding="UTF-8"?><br>
<br>
Could not counstruct partial tree: Invalid name: Sidalcea</p>
<p>I'm not sure how many of APIs this applies to, but it's at least several of these:</p>
<p>CNCore.setObsoletedBy()</p>
<p>CNAuthorization.setRightsHolder()<br>
CNAuthorization.isAuthorized()</p>
<p>CNIdentity.registerAccount()<br>
CNIdentity.updateAccount()<br>
CNIdentity.verifyAccount()<br>
CNIdentity.getSubjectInfo()<br>
CNIdentity.listSubjects()<br>
CNIdentity.mapIdentity()<br>
CNIdentity.removeMapIdentity()<br>
CNIdentity.requestMapIdentity()<br>
CNIdentity.confirmMapIdentity()<br>
CNIdentity.denyMapIdentity()<br>
CNIdentity.createGroup()<br>
CNIdentity.addGroupMembers()<br>
CNIdentity.removeGroupMembers()</p>
<p>CNReplication.setReplicationStatus()<br>
CNReplication.updateReplicationMetadata()<br>
CNReplication.setReplicationPolicy()<br>
CNReplication.isNodeAuthorized()</p>
<p>CNRegister.updateNodeCapabilities()<br>
CNRegister.register()</p>
Infrastructure - Task #6168 (New): CNAuthorization.setRightsHolder() returns 500 ServiceFailurehttps://redmine.dataone.org/issues/61682014-08-29T16:01:00ZRoger Dahldahl@unm.edu
<p>CNAuthorization.setRightsHolder() returns 500 ServiceFailure when called with invalid pid.</p>
Infrastructure - Task #6166 (New): CNRead.getChecksum() returns Content-Type of text/csv while it...https://redmine.dataone.org/issues/61662014-08-29T15:03:54ZRoger Dahldahl@unm.eduInfrastructure - Feature #5145 (New): Consider including cert subject(s) in NotAuthorized exceptionshttps://redmine.dataone.org/issues/51452014-04-30T13:21:36ZRoger Dahldahl@unm.edu
<p>When a call fails with a NotAuthorized, including the cert subject(s) in the description makes it easy for the client to determine if they were using the right cert.</p>
Infrastructure - Task #5137 (New): Fix DataONE CA chain file location in cn-buildouthttps://redmine.dataone.org/issues/51372014-04-25T15:06:34ZChris Jonescjones@nceas.ucsb.edu
<p>We had been using /var/local/dataone as the location for trusted CA certificates on the CN, but in 2012 we changed to using a single chain file rather than a directory. I made this change in the cn-ssl config, but (inadvertantly?) used /etc/ssl/certs for tha DataONECAChain.crt file location.</p>
<p>When this file <em>isn't</em> hashed during c_rehash, there is no duplicate hashes created for the DataONERootCA certificate, but when it is, using /etc/ssl/certs in curl operations fail. </p>
<p>The easiest fix is to move the DataONECAChain.crt file back to /var/local/dataone in cn-ssl, so there isn't the potential for a conflict.</p>
Infrastructure - Bug #4211 (New): Potential race condition between archive and replicationhttps://redmine.dataone.org/issues/42112013-12-20T18:59:04ZSkye Roseboomsroseboo@dataone.unm.edu
<p>Current implementation of CNodeService.archive() does not appear to increment the serial version of the system metadata.</p>
<p>This implies that a race condition between calls of archive (either due to sync or directly by user) could be overwritten by updates from replication (other services in the future). Since the serial version is not updated by archive - the replication process may not recognize there has been an update from archive - and potentially overwrite the archive flag.</p>
<p>A possible short term solution for protecting the 'archive' flag data, would be to implement a 'business rule' that enforces that once SystemMetadata.archive is set to 'true' - that no subsequent updates are allowed to reset the archived flag to 'false'. This would prevent the race condition between updates from over-writing the 'archive' flag. This change will also PREVENT and archived document from being 'unarchived' in the future (even as requested by user?) </p>
<p>Further discussion is likely needed to determine the best layer to insert this type of business logic enforcement.</p>
Infrastructure - Task #4136 (In Progress): Make cosmetic changes to the distribution maphttps://redmine.dataone.org/issues/41362013-10-29T15:00:56ZChris Jonescjones@nceas.ucsb.edu
<p>From feedback at the AHM:</p>
<ul>
<li><p>The DataONE orange color appears too red on the map due to the transparency of the plum-colored data symbols. Use either a lighter shade of orange, the DataONE teal, etc.</p></li>
<li><p>Move the legend to the top right corner</p></li>
<li><p>Change the title to be more descriptive, like 'Data Set Distribution'</p></li>
</ul>
Infrastructure - Task #4125 (New): Mock up an informational section of the Member Node detail viewhttps://redmine.dataone.org/issues/41252013-10-28T20:45:57ZChris Jonescjones@nceas.ucsb.edu
<p>Since the MN statistics table contains redundant information with the colored header boxes, we are removing it from the MN detail view. Mock up an informational section below the stats graph that highlights other information about the MN.</p>
Infrastructure - Task #3978 (In Progress): Add a CN reporting script that summarizes spatial data...https://redmine.dataone.org/issues/39782013-09-13T16:12:08ZChris Jonescjones@nceas.ucsb.edu
<p>Spatial data in the CN Solr search index includes per-object bounding box data. For client side mapping purposes, these data are too numerous to add to a vector map. Create a spatial summarization script that reduces the total points to summarized counts at a given cell resolution. Allow for the resolution to be configurable. Export the result as a JSON object, compatible with mapping libraries like heatmapjs and D3js.</p>
Infrastructure - Task #3328 (New): Enable a data preview from the search results pagehttps://redmine.dataone.org/issues/33282012-10-10T15:54:40ZChris Jonescjones@nceas.ucsb.edu
<p>Visualization and mapping tools for the data sets:<br>
Before the users download the data, several expressed that they wanted to “view” the data <br>
-Browsing graphics<br>
-Mapping graphics</p>
<p>this capability will need some planning and discussion, since the browsable view is entirely dependent on the format of the data file. We may be able to support some common types at first (CSV, PNG, JPEG), and later enable others (mapping, charting, etc.).</p>
Infrastructure - Task #2896 (New): Modify the admin DN in createObjectformat script per environmenthttps://redmine.dataone.org/issues/28962012-06-08T21:54:44ZChris Jonescjones@nceas.ucsb.edu
<p>The /usr/share/metacat/debian/createObjectFormat script uses a default DN to connect to Metacat. This should change on a per environment basis (particularly production). For instance, the production DN is cn=dataone_cn_metacat,dc=dataone,dc=org. Modify this script during the dataone-cn-metacat postinst script.</p>