DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2017-03-22T20:02:54ZDataONE Tasks
Redmine Infrastructure - Bug #8051 (In Progress): CORS-based CN calls fail using Internet Explorer on Win...https://redmine.dataone.org/issues/80512017-03-22T20:02:54ZChris Jonescjones@nceas.ucsb.edu
<p>As noted in <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: Error -1205 "Client Certificate Rejected" by Safari (Closed)" href="https://redmine.dataone.org/issues/2693">#2693</a>, <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: completely unable to access cn.dataone.org from Safari 7.1 if user has any certificates installed (Closed)" href="https://redmine.dataone.org/issues/6539">#6539</a>, and <a class="issue tracker-1 status-6 priority-4 priority-default closed" title="Bug: Safari 6.0 fails to connect to Metacat MN with SSLVerifyClient (Rejected)" href="https://redmine.dataone.org/issues/3255">#3255</a>, Safari does not handle TLS handshakes correctly when asked for a client X509 certificate. Similarly, IE 11 (and 10) on Windows 7 (and maybe others) does not handle TLS handshakes correctly.</p>
<p>The MetacatUI application used on certain Member Nodes (KNB, ARCTIC, ...) makes calls to the CN Identity API to get account information for users and their associated groups. This is done via an XHR, using the CORS pre-flight mechanism of calling @OPTIONS@ on the REST endpoint. During this call, the CN is returning an @HTTP 403@ Not Authorized response to IE11 on Windows, but succeeds on Firefox and Chrome on Windows. </p>
<p>It seems (but we're not sure) that IE is responding to the request for a client certificate and whatever is sent is being rejected by the CN web server. However, it's not super straight forward. When Apache is configured with:<br>
<br>
SSLVerifyClient optional<br>
SSLVerifyDepth 10</p>
<p>IE11 succeeds on the @OPTIONS@ request. However, when the CN is configured to conditionally set @SSLVerifyClient@ within a @@ block:<br>
<br>
SSLVerifyClient none<br>
<br>
<br>
SSLVerifyClient optional<br>
<br>
<br>
SSLVerifyDepth 10<br>
<br>
the request fails (which is currently the production configuration).</p>
<p>However, after testing in STAGE, IE11 works fine when not asked for a client certificate (@SSLVerifyClient none@). It seems that the interaction with Apache changes based on the conditional logic in a specific @@ block, and IE11 responds incorrectly in some way.</p>
<p>To alleviate issues with browser-based client certificate requests, I suggest that we adopt the following configuration:<br>
<br>
SSLVerifyClient none<br>
<br>
<br>
SSLVerifyClient optional<br>
<br>
<br>
SSLVerifyDepth 10<br>
<br>
This configuration excludes most desktop/handheld browser clients from being asked for an X509 certificate. However, it still allows for Java, Python, curl, R, etc. clients to connect using client-side certificates. Since we've migrated to JWT token-based browser authentication, this seems reasonable to me.</p>
<p>This is currently a blocker in production, so we should consider a manual change to the production CNs before this gets rolled into a CCI release, if agreed upon. Thoughts welcome.</p>
Search UI - Task #7498 (In Progress): Search UI deployments need to be automatedhttps://redmine.dataone.org/issues/74982015-11-20T23:19:19ZChris Jonescjones@nceas.ucsb.edu
<p>We need to be able to install the Search UI code with minimal to no manual intervention. Create a debian package for the Search UI based on the cn-buildout project. Integrate this into the Jenkins build system so the debian package can be installed from the unstable, beta, and stable channels.</p>
Infrastructure - Task #7466 (In Progress): Some objects not accessible on the CN via REST APIhttps://redmine.dataone.org/issues/74662015-11-04T18:41:38ZBryce Mecummecum@nceas.ucsb.edu
<p>While doing other work, I noticed that a good number (not sure how many) of objects listed on the CN's Solr index are not accessible via the REST API get() and resolve() methods. Instead of returning the object, they return a NotFound error. </p>
<p>To reproduce,</p>
<ol>
<li>Visit <a href="https://cn.dataone.org/cn/v1/query/solr/?fl=identifier,title,authoritativeMN,datasource&q=formatType:METADATA+AND+-obsoletedBy:*&rows=100&start=0">https://cn.dataone.org/cn/v1/query/solr/?fl=identifier,title,authoritativeMN,datasource&q=formatType:METADATA+AND+-obsoletedBy:*&rows=100&start=0</a></li>
<li>Pick a PID from the query result, e.g.</li>
</ol>
<ul>
<li>knb-lter-cap.148.9</li>
<li>CLOEBDMETADATA.10242013.1</li>
</ul>
<ol>
<li>Attempt to resolve() or get() the object via the REST API like: <a href="https://cn.dataone.org/cn/v1/object/CLOEBDMETADATA.10242013.1">https://cn.dataone.org/cn/v1/object/CLOEBDMETADATA.10242013.1</a></li>
<li>Receive a NotFound error instead of the object.</li>
</ol>
<p>Notes:</p>
<p>In IRC, Skye noticed that the objects can be retrieved via their respective MN so it appears this issue may be a Metacat replication issue.</p>
Java Client - Task #7389 (Testing): V2 D1Object fails to download V1 contenthttps://redmine.dataone.org/issues/73892015-09-28T17:34:30ZChris Jonescjones@nceas.ucsb.edu
<p>During testing of services in the mixed V1/V2 DEV2 environment, D1Object fails to download content listed in the ObjectLocationList from a V1-only Member Node. The symptom is a null pointer exception when trying to close a non-existent temporary file where the bytes of the object should have been located. Fix download() to call V1 endpoints on V1-only MNs.</p>
Java Client - Bug #7322 (Testing): D1Object stores data in memory, causes out of memory errorshttps://redmine.dataone.org/issues/73222015-08-27T23:56:06ZChris Jonescjones@nceas.ucsb.edu
<p>When assembling DataPackage instances and populating them, the DataPackage class relies on the underlying D1Object.download() method to store members of the DataPackage locally. The current implementation calls IOUtils.toByteArray(inputstream), which of course stores all bytes in memory. With large data files, this effectively renders DataPackage useless because of OutOfMemory exceptions. The move towards using the javax.activation.DataSource interface helps with this since it provide in memory and on disk implemetations.</p>
<p>Change download() to default to the on-disk DataSource, and make the storage location configurable in d1client.properties.</p>
Java Client - Task #7120 (Testing): Fix DataPackage.insertRelationship() to handle any URI for ex...https://redmine.dataone.org/issues/71202015-05-21T16:49:34ZChris Jonescjones@nceas.ucsb.edu
<p>DataPackage currently provides two insertRelationship() methods - one to add ORE relationships between metadata and data members of the aggregation, and a second to provide any relationship using predicates from other namespaces (such as PROV). The latter method assumes that all identifiers should be treated as objects using the CN Base URL when constructing the subject and object URIs. This isn't always the case. Change or override the method to accept any URI as subject and object components of the triple, and fix any tests that use this method.</p>
Infrastructure - Task #6843 (In Progress): Update the prov instance of the RdfXmlSubprocessor to ...https://redmine.dataone.org/issues/68432015-02-06T23:18:48ZChris Jonescjones@nceas.ucsb.edu
<p>In the "sem-prov-design issue 66":<a href="https://github.com/DataONEorg/sem-prov-design/issues/66">https://github.com/DataONEorg/sem-prov-design/issues/66</a> we have renamed the provenance-based Solr fields to include 'prov_' as a prefix, and have added new fields. See also "issue 99":<a href="https://github.com/DataONEorg/sem-prov-design/issues/99">https://github.com/DataONEorg/sem-prov-design/issues/99</a> and "issue 100":<a href="https://github.com/DataONEorg/sem-prov-design/issues/100">https://github.com/DataONEorg/sem-prov-design/issues/100</a>.<br>
Modify the provRdfXmlSubprocessor bean to handle the renaming scheme, the new fields, and the inverse fields determined to be useful. Also, add these fields as static Solr fields so we can remove the '_sm' suffixes from the names.</p>
Member Nodes - Task #5449 (In Progress): Resolve resource map issues for LTER-Europehttps://redmine.dataone.org/issues/54492014-05-29T19:03:54ZBruce Wilsonbwilso27@utk.eduInfrastructure - Task #4719 (In Progress): Replace CN client certificatehttps://redmine.dataone.org/issues/47192014-04-14T17:43:48ZDave Vieglaisdave.vieglais@gmail.com
<p>As part of the fallout from the Heartbleed vulnerability, the CN client certificates need to be regenerated, the old ones revoked, and new ones used in their place.</p>
<p>This should not be done until the production server SSL certificates have been replaced.</p>
Infrastructure - Task #4716 (In Progress): refresh client certificate for urn:node:DRYADhttps://redmine.dataone.org/issues/47162014-04-14T17:10:01ZDave Vieglaisdave.vieglais@gmail.com
<p>Contact is Ryan Scherle</p>
Infrastructure - Task #4715 (In Progress): Refresh client certificate for urn:node:CLOAKNhttps://redmine.dataone.org/issues/47152014-04-14T16:52:56ZDave Vieglaisdave.vieglais@gmail.com
<p>Contact is "Kevin F. Webb" kfw4 at cornell edu</p>
Infrastructure - Task #4714 (In Progress): Refresh client certificate for MN urn:node:TFRIhttps://redmine.dataone.org/issues/47142014-04-14T16:48:17ZDave Vieglaisdave.vieglais@gmail.com
<p>Contact is "Meei-ru Jeng" beerjeng at gmail com</p>
Infrastructure - Task #4210 (Testing): Metacat does not set serialVersion correctly in CNodeServi...https://redmine.dataone.org/issues/42102013-12-20T15:22:50ZChris Jonescjones@nceas.ucsb.edu
<p>For DATA and METADATA, CNodeService.archive() and D1NodeService.archive(), respectively, don't increment the serialVersion field. Check this for delete() as well. D1NodeService delegates to DocumentImpl to call the HZ put() method, so the fix needs to be there, and in CNodeService.</p>
Infrastructure - Task #3978 (In Progress): Add a CN reporting script that summarizes spatial data...https://redmine.dataone.org/issues/39782013-09-13T16:12:08ZChris Jonescjones@nceas.ucsb.edu
<p>Spatial data in the CN Solr search index includes per-object bounding box data. For client side mapping purposes, these data are too numerous to add to a vector map. Create a spatial summarization script that reduces the total points to summarized counts at a given cell resolution. Allow for the resolution to be configurable. Export the result as a JSON object, compatible with mapping libraries like heatmapjs and D3js.</p>
Infrastructure - Bug #2411 (In Progress): knb MNs and CNs allow self-signed certificates to connecthttps://redmine.dataone.org/issues/24112012-02-24T21:53:41ZRob Nahfrnahf@epscor.unm.edu
<p>I have a self-signed certificate that succeeds in mn.ping(), mn.listObjects() and mn.isAuthorized() against demo1.test.dataone.org.</p>
<p>However:<br>
a) an expired certificate for the same user (signed by dataone CA) fails to establish a connection, and<br>
b) my MobileMe certificate passed by the Safari browser fails to establish a connection</p>
<p>poses a security risk to the system.</p>
<p>also tests the same way for cn-dev, so maybe a problem with CN deployment packaging as well.</p>