DataONE Tasks: Issueshttps://redmine.dataone.org/https://redmine.dataone.org/favicon.ico2020-07-15T18:03:40ZDataONE Tasks
Redmine Infrastructure - Bug #8866 (New): Java client tools should set a custom user agent stringhttps://redmine.dataone.org/issues/88662020-07-15T18:03:40ZBryce Mecummecum@nceas.ucsb.edu
<p>Related to <a href="https://redmine.dataone.org/issues/7047">https://redmine.dataone.org/issues/7047</a></p>
<p>It looks like nowhere in <code>d1_libclient_java</code> do we set a user agent string. Aside from being best practice, it limits our ability to customize our infrastructure around it. For example, OPC is running into HTTP 413s due to overrunning their TLS renegotiation buffer and we can't effectively whitelist their requests, which come from our Java client tools, to allow them to upload large files.</p>
CN REST - Bug #8860 (New): /token endpoint doesn't set a content-type and character encodinghttps://redmine.dataone.org/issues/88602020-02-29T01:00:11ZBryce Mecummecum@nceas.ucsb.edu
<p>On Firefox only, requests to the /portal/token endpoint (i.e., the one MetacatUI and other clients use to fetch their auth tokens, like <a href="https://cn.dataone.org/portal/token">https://cn.dataone.org/portal/token</a>) result in errors in the browser console.</p>
<p>When you access the URL via an XHR request, you see:</p>
<blockquote>
<p>XML Parsing Error: syntax error<br>
Location: <a href="https://cn-stage.test.dataone.org/portal/token">https://cn-stage.test.dataone.org/portal/token</a><br>
Line Number 1, Column 1:</p>
</blockquote>
<p>When you access the URL directly in Firefox:</p>
<blockquote>
<p>The character encoding of the plain text document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the file needs to be declared in the transfer protocol or file needs to use a byte order mark as an encoding signature.</p>
</blockquote>
<p>I had a hunch that this error would go away if the response simply had the <code>Content-Type</code> header set to <code>text/plain; charset=utf-8</code> so I spun up <code>mitmproxy</code>, made that edit to the intercepted response, and saw that the error does go away.</p>
<p>I think we should modify the portal code to set the <code>Content-Type</code> header like above so the error goes away.</p>
Infrastructure - Bug #8822 (New): Account queries in STAGE-2 failing from web browserhttps://redmine.dataone.org/issues/88222019-06-19T01:44:19ZBryce Mecummecum@nceas.ucsb.edu
<p>Steps to reproduce:</p>
<ol>
<li>Visit <a href="https://search-stage-2.test.dataone.org/">https://search-stage-2.test.dataone.org/</a></li>
<li>Log in</li>
<li>Navigate to <a href="https://search-stage-2.test.dataone.org/data">https://search-stage-2.test.dataone.org/data</a></li>
<li>Observe an HTTP 500 in the Network pane of whichever browser you're using to <a href="https://search-stage-2.test.dataone.org/cn/v2/accounts/?query=%7BYOUR_DN%7D">https://search-stage-2.test.dataone.org/cn/v2/accounts/?query={YOUR_DN}</a></li>
</ol>
<p>Response body:</p>
<pre><?xml version="1.0" encoding="UTF-8"?>
<error detailCode="500" errorCode="500" name="ServiceFailure">
<description>Internal Server Error: The server encountered an unexpected condition which prevented it from fulfilling the request.</description>
</error>
</pre>
<p>Some notes:</p>
<ul>
<li>I notice this in multiple browsers</li>
<li>I notice this only when the request is issued from MetacatUI, not when I visit the URL in my browser or hit it with curl</li>
<li>I don't notice this error on search.dataone.org</li>
<li>MetacatUI on search.dataone.org doesn't even issue this request</li>
<li>MetacatUI on stage-2 is at v2.4.2 and on search is at 2.6.1</li>
</ul>
<p>It seems like a bug to me that we see a service failure but I'm not sure if this is a MetacatUI bug or an issue in the CN stack (i.e., Apache config or something) but I wanted to file it for someone to take a look.</p>
Infrastructure - Bug #8815 (New): Investigate and fix failed sync/harvest of doi:10.18739/A2CH48https://redmine.dataone.org/issues/88152019-06-04T23:26:43ZBryce Mecummecum@nceas.ucsb.edu
<p>I noticed this while doing something unrelated. There isn't a copy of the sysmeta for <a href="https://arcticdata.io/metacat/d1/mn/v2/meta/doi:10.18739/A2CH48">https://arcticdata.io/metacat/d1/mn/v2/meta/doi:10.18739/A2CH48</a> on the CN:</p>
<p>At: <a href="http://cn.dataone.org/cn/v2/meta/doi:10.18739/A2CH48">http://cn.dataone.org/cn/v2/meta/doi:10.18739/A2CH48</a></p>
<pre><code class="xml syntaxhl"><span class="CodeRay"><span class="preprocessor"><?xml version="1.0" encoding="UTF-8"?></span><span class="tag"><error</span> <span class="attribute-name">detailCode</span>=<span class="string"><span class="delimiter">"</span><span class="content">1420</span><span class="delimiter">"</span></span> <span class="attribute-name">errorCode</span>=<span class="string"><span class="delimiter">"</span><span class="content">404</span><span class="delimiter">"</span></span> <span class="attribute-name">name</span>=<span class="string"><span class="delimiter">"</span><span class="content">NotFound</span><span class="delimiter">"</span></span><span class="tag">></span>
<span class="tag"><description></span>No system metadata could be found for given PID: doi:10.18739/A2CH48<span class="tag"></description></span>
<span class="tag"></error></span>
</span></code></pre>
<p>CNs are in readonly mode at the moment so I'm filing this here so someone can a look later on.</p>
Infrastructure - Decision #8774 (New): Add new CN format for MPEG-2 or update video/mpeg format t...https://redmine.dataone.org/issues/87742019-03-08T02:04:18ZBryce Mecummecum@nceas.ucsb.edu
<p>We already have this format:</p>
<pre><objectFormat>
<formatId>video/mpeg</formatId>
<formatName>MPEG-1 Video</formatName>
<formatType>DATA</formatType>
<mediaType name="video/mpeg"/>
<extension>mpg</extension>
</objectFormat>
</pre>
<p>It clearly says MPEG-1 Video but there is also an MPEG-2 format. It'd be useful to have a proper format ID to use for MPEG-2 video. The info for MPEG-2 is largely the same as for MPEG-1 (format type, mediaType, extension). Should we add a new format or just update the description of the existing one? We also have an MP4 format type so maybe that'd be a reason to just add another to cover MPEG-2.</p>
Infrastructure - Decision #8693 (In Progress): Support Google Dataset Search on search.dataone.or...https://redmine.dataone.org/issues/86932018-09-07T00:16:59ZBryce Mecummecum@nceas.ucsb.edu
<a name="Background"></a>
<h2 >Background<a href="#Background" class="wiki-anchor">¶</a></h2>
<p>Yesterday, <a href="https://toolbox.google.com/datasetsearch" class="external">Google Dataset Search</a> launched. We previoiusly attempted to make MetacatUI (and by extension, DataONE Search) compatible with it by <a href="https://github.com/NCEAS/metacatui/issues/482" class="external">injecting Schema.org JSON-LD into appropriate pages</a>. During development and testing, we checked our compatibility with the upcoming Google Dataset Search using Google's <a href="https://search.google.com/structured-data/testing-tool" class="external">Structured Data Testing Tool</a>. During development, this was all working fine and the feature appeared to be compatible but, after launching the feature on search.dataone.org, behavior changed on Google's end making it so Google no longer saw this JSON-LD. The reason for this is likely that, because MetacatUI follows a single page application architecture and we inject the JSON-LD on the client side, Google's JSON-LD crawler only saw what was sent from the server (a nearly empty index.html) and not our full page (with JSON-LD). I was able to test this theory and, while Google's crawler does execute JavaScript, it limits execution to about or exactly five seconds and MetacatUI <em>usually</em> doesn't finish injecting JSON-LD and rendering all content until after that timeout.</p>
<p>Potential paths forward to get DataONE Search compatible with Google's Dataset Search include (none of which are mutually exclusive):</p>
<ol>
<li>The assets that make up MetacatUI and the asset loading strategies could be optimized: <a href="https://github.com/NCEAS/metacatui/issues/224">https://github.com/NCEAS/metacatui/issues/224</a></li>
<li>Move the code (and any dependencies) that injects JSON-LD further up in the app boot so that Google sees it</li>
<li>Inject the appropriate JSON-LD on the server side to guarantee that Google sees it (originally Matt Jones' idea!)</li>
</ol>
<p>(1) is being worked on for sure, and (2) may not be needed if (1) is successful. I want to talk about option (3) because:</p>
<ul>
<li>It's a quicker solution (I already have something working) which would help get us involved in the project faster</li>
<li>It paves the way for future features and/or improvements to MetacatUI (we could be rendering more on the server side than just JSON-LD, like other metadata, more page content, etc)</li>
</ul>
<a name="What-I-did"></a>
<h2 >What I did<a href="#What-I-did" class="wiki-anchor">¶</a></h2>
<p>To test this idea, I modified a <a href="https://github.com/amoeba/backbone-pushstate-example" class="external">previous project</a> which is just a simple Node (Express.js) app that hosts MetacatUI by intercepting every request and serving the appropriate asset. In injects Schema.org JSON-LD, when appropriate, by querying the CN Solr index before sending MetacatUI's index.html to the client. <a href="https://github.com/amoeba/metacatui-ssr" class="external">Code is here</a> and its deployed <a href="http://neutral-cat.nceas.ucsb.edu/" class="external">here</a>. View source on any /view/... pages and you'll see a minimal Schema.org/Dataset description in the head. More properties can be added later. I did it quick and dirty: The app pre-loads MetacatUI's index.html as a <code>String</code> at app boot and injects the JSON-LD into it. No templating language or other magic.</p>
<a name="Things-to-address"></a>
<h2 >Things to address<a href="#Things-to-address" class="wiki-anchor">¶</a></h2>
<ul>
<li>How do we feel abouts switching from hosting MetacatUI via Apache (simple, bullet proof) to a Node based deployment just to support this feature (new territory, at least for me)?</li>
<li>If we do switch, we'd want to make really sure the Node app doesn't have weird failure cases where it doesn't return index.html (e.g., when Solr is down, or slow). The app needs to return index.html (and every other static asset) on every request and do it very fast and we should decide what the cutoff is so that it doesn't hold up app boot if Solr is slow/down.</li>
<li>Can this type of deployment easily be integrated with CN buildouts? I've deployed Node apps before by fronting them with Apache/nginx (via reverse proxy) and then keeping the node process up with Upstart</li>
<li>Is this performant enough for DataONE? I think my implementation is non-blocking but I'm not a Node expert so we'd want to code review and probably benchmark </li>
<li>We could wait on (1) and stick with our current deployment strategy</li>
</ul>
<a name="Other-notes"></a>
<h2 >Other notes<a href="#Other-notes" class="wiki-anchor">¶</a></h2>
<p>Unrelated to the Google Dataset Search issue but related to Google's crawling for Google Search, we've also identified:</p>
<ul>
<li>That the Metacat View Service is often unreasonably slow: <a href="https://github.com/NCEAS/metacat/issues/1234">https://github.com/NCEAS/metacat/issues/1234</a> and are planning to figure out why</li>
<li>That we can and should make use of sitemaps to help Google crawl our pages: <a href="https://github.com/NCEAS/metacat/issues/1263">https://github.com/NCEAS/metacat/issues/1263</a></li>
</ul>
Infrastructure - Decision #8616 (New): Consider expanding isotc211's indexing component's keyword...https://redmine.dataone.org/issues/86162018-06-15T00:13:04ZBryce Mecummecum@nceas.ucsb.edu
<p>From </p>
<p><a href="https://repository.dataone.org/software/cicore/trunk/cn/d1_cn_index_processor/src/main/resources/application-context-isotc211-base.xml">https://repository.dataone.org/software/cicore/trunk/cn/d1_cn_index_processor/src/main/resources/application-context-isotc211-base.xml</a></p>
<p>The current XPath for the <code>keyword</code> field pulls out:</p>
<pre>//gmd:identificationInfo/gmd:MD_DataIdentification/gmd:descriptiveKeywords/gmd:MD_Keywords/gmd:keyword/gmx:Anchor/text() |
//gmd:identificationInfo/gmd:MD_DataIdentification/gmd:descriptiveKeywords/gmd:MD_Keywords/gmd:keyword/gco:CharacterString/text()
</pre>
<p>ISO also defines <code>MD_DataIdentification/gmd:topicCategory</code> which is defined as "The main theme(s) of the dataset." and is required (recommended) when describing a dataset. It's conditional, and repeatable. An example from a PANGAEA doc is</p>
<pre>...
<ns0:topicCategory>
<ns0:MD_TopicCategoryCode>geoscientificInformation</ns0:MD_TopicCategoryCode>
</ns0:topicCategory>
</ns0:MD_DataIdentification>
</pre>
<p>I think it's improve recall to include in our keywords list. It appears to be a controlled vocabulary so we could even make more direct use of it. The controlled vocabulary appears to be (From the MI_Metadata workbook):</p>
<p>Domain: <br>
- farming<br>
- biota<br>
- boundaries<br>
- climatologyMeteorolgyAtmosphere<br>
- economy<br>
- elevation<br>
- environement<br>
- geoscientificInformation<br>
- health<br>
- imageryBaseMapsEarchCover<br>
- intelligenceMilitary<br>
- inlandWaters<br>
- location<br>
- oceans<br>
- planningCadastre<br>
- society<br>
- structure<br>
- transportation<br>
- utilitiesCommunicationgeoscientificInformation, health, imageryBaseMapsEarchCover, intelligenceMilitary, inlandWaters, location, oceans, planningCadastre, society, structure, transportation, utilitiesCommunication</p>
<p>Both NCEI and PANGAEA make use of this field in their ISO docs.</p>
Infrastructure - Decision #8601 (New): Decide on a URI space for DataONE resourceshttps://redmine.dataone.org/issues/86012018-06-05T00:39:42ZBryce Mecummecum@nceas.ucsb.edu
<a name="Summary"></a>
<h2 >Summary<a href="#Summary" class="wiki-anchor">¶</a></h2>
<p>In the past, we've needed to represent DataONE resources (e.g., Objects, "datasets", etc.) in Linked Open Data contexts. Currently, these resources don't have have canonical URIs.</p>
<p>For example, take a dataset on search.dataone.org with the following URL:</p>
<pre>https://search.dataone.org/#view/doi:10.18739/A28K74W2F
</pre>
<p>Candiate URIs for this resource include:</p>
<ol>
<li><a href="https://search.dataone.org/#view/doi:10.18739/A28K74W2F:">https://search.dataone.org/#view/doi:10.18739/A28K74W2F:</a> Depends on implementation details in MetacatUI's router, uses fragment URLs in the URL which we're deprecating soon anyhow</li>
<li><a href="https://cn.dataone.org/cn/v2/resolve/doi:10.18739/A28K74W2F:">https://cn.dataone.org/cn/v2/resolve/doi:10.18739/A28K74W2F:</a> Depends on implementation details of the DataONE API and is tied to a specific version of the DataONE API. e.g., When API version 3 is release, will ../v2/resolve/doi:10.18739/A28K74W2F and ../v3/resolve/doi:10.18739/A28K74W2F refer to the same resource?</li>
</ol>
<a name="Proposal"></a>
<h2 >Proposal<a href="#Proposal" class="wiki-anchor">¶</a></h2>
<p>Create a URI space that all services can integrate against. This space follows the convention of:</p>
<p><a href="https://dataone.org/%7Bresource_type%7D/%7Bresource_identifier%7D">https://dataone.org/{resource_type}/{resource_identifier}</a></p>
<p>where <code>{resource_type}</code> is a singular name of a top level DataONE resource such as "dataset", "person", or "object" and <code>{resource_identifier}</code> is a type-appropriate identifier (e.g., PID of a science metadata Object for the dataset type, DN for the person type, etc.). The collection of top level resources (e.g., datasets), follows the form <a href="https://dataone.org/%7Bresource_type_plural%7D">https://dataone.org/{resource_type_plural}</a> (e.g., "datasets").</p>
<p>Examples:</p>
<ul>
<li>Dataset: <a href="https://dataone.org/dataset/doi%3A10.18739%2FA28K74W2F">https://dataone.org/dataset/doi%3A10.18739%2FA28K74W2F</a></li>
<li>Person: <a href="https://dataone.org/person/https%3A%2F%2Forcid.org%2F0000-0002-0381-3766">https://dataone.org/person/https%3A%2F%2Forcid.org%2F0000-0002-0381-3766</a></li>
<li>Object: <a href="https://dataone.org/object/urn%3Auuid%3A3c80e9d6-277c-4a32-bc7a-d85c499f370f">https://dataone.org/object/urn%3Auuid%3A3c80e9d6-277c-4a32-bc7a-d85c499f370f</a></li>
<li>All datasets on DataONE: <a href="https://dataone.org/datasets">https://dataone.org/datasets</a></li>
</ul>
<a name="Expected-outcomes"></a>
<h2 >Expected outcomes<a href="#Expected-outcomes" class="wiki-anchor">¶</a></h2>
<ul>
<li>A normative document describing the URI space will be added to the DataONE documentation (<a href="https://releases.dataone.org/online/api-documentation-v2.0/">https://releases.dataone.org/online/api-documentation-v2.0/</a>)</li>
<li>Other project will make use of these URIs</li>
</ul>
<a name="Affected-projects"></a>
<h2 >Affected projects<a href="#Affected-projects" class="wiki-anchor">¶</a></h2>
<ul>
<li>Metacat (sitemap functionality will make use of these URIs)</li>
<li>MetacatUI (Google Structured Data integration via JSON-LD will use these URIs for resources)</li>
<li>GeoLink (DataONE LOD graph will use these URIs for resources)</li>
</ul>
<a name="Future-work"></a>
<h2 >Future work<a href="#Future-work" class="wiki-anchor">¶</a></h2>
<p>With the URI space decided, we can start working on a unified, content-negotating DataONE resolve service (different from the CN/MNStorage.resolve API method). See <a href="https://hpad.dataone.org/GYJgjAJgpgHMwFoBGBWALItBjGMEE4kBDAZgKzSWBSS2ADYoSg==#detail-object-content-negotiation-for-objects">https://hpad.dataone.org/GYJgjAJgpgHMwFoBGBWALItBjGMEE4kBDAZgKzSWBSS2ADYoSg==#detail-object-content-negotiation-for-objects</a>. How this works is not being decided in this Redmine ticket.</p>
<a name="Previous-work-discussions-chronological-order"></a>
<h2 >Previous work / discussions (chronological order):<a href="#Previous-work-discussions-chronological-order" class="wiki-anchor">¶</a></h2>
<ul>
<li><a href="https://docs.google.com/document/d/1yU-d-aFdtiSB91Wk0sFj1xthW8skBPof9qKwx00bjxE/edit?usp=sharing">https://docs.google.com/document/d/1yU-d-aFdtiSB91Wk0sFj1xthW8skBPof9qKwx00bjxE/edit?usp=sharing</a></li>
<li><a href="https://hpad.dataone.org/GYJgjAJgpgHMwFoBGBWALItBjGMEE4kBDAZgKzSWBSS2ADYoSg==">https://hpad.dataone.org/GYJgjAJgpgHMwFoBGBWALItBjGMEE4kBDAZgKzSWBSS2ADYoSg==</a></li>
</ul>
Infrastructure - Task #8499 (New): Improve rendering of http://www.isotc211.org/2005/gmd-pangaea ...https://redmine.dataone.org/issues/84992018-03-14T00:49:31ZBryce Mecummecum@nceas.ucsb.edu
<p>I offered to file this a while back but it just sat in my inbox.</p>
<p>Initial support for rendering went in with <a href="https://redmine.dataone.org/issues/8219">https://redmine.dataone.org/issues/8219</a> but we noted in Slack in #ci that the rendering could be a lot better. As an example of how much better, Pangaea's own landing pages are quite nice: <a href="https://doi.pangaea.de/10.1594/PANGAEA.511392">https://doi.pangaea.de/10.1594/PANGAEA.511392</a></p>
<p>Upon checking the first Pangaea dataset in the index, <a href="https://search.dataone.org/#view/af113afed60c98df50052feaf6cb7894">https://search.dataone.org/#view/af113afed60c98df50052feaf6cb7894</a>, I see that the view service isn't actually working at all for this document. So I guess this could be two tasks:</p>
<ol>
<li>Enable the Metacat View Service for Pangaea docs</li>
<li>Improve the XSLT</li>
</ol>
<p>2 just involve extending the existing isotc211 XSLT if we think that other isotc211 creators want those fixes but if we want to make a nice view that's Pangaea-specific, we may want to consider a separate stylesheet suite.</p>
Infrastructure - Bug #8215 (New): Consider how Subjects are compared (e.g. HTTP vs. HTTPS ORCID U...https://redmine.dataone.org/issues/82152017-11-07T01:41:51ZBryce Mecummecum@nceas.ucsb.edu
<p>I know this has come up a few times in the last few years and it has bitten us a lot in day-to-day operations at the Arctic Data Center. Some Subjects in DataONE authentication appear to be compared literally so these two subjects are not considered equivalent:</p>
<p><a href="http://orcid.org/0000-0002-0381-3766">http://orcid.org/0000-0002-0381-3766</a><br>
<a href="https://orcid.org/0000-0002-0381-3766">https://orcid.org/0000-0002-0381-3766</a></p>
<p>even though a reasonable spectator might consider them equivalent.</p>
<p>Where this causes trouble seems to be when System Metadata is authored by users, specifically the rightsHolder and accessPolicy portions, and what's in the System Metadata does not literally match the user's actual Subject. As an example, when a user logs in via ORCID, their DataONE Subject ends up being the <em>http</em> variant of their ORCID URI, so if they use the <em>https</em> variant of their ORCID URI in their System Metadata, API calls requiring read, write, and changePermission permission fail for them because the literal string comparison determines the two Subjects to non-equivalent.</p>
<p>I think this may only affect ORCIDs right now because Subjects such as LDAP DNs may already be compared in a string-insensitive fashion. Though I'm not sure on this point.</p>
<p>A few of us on the NCEAS dev team discussed this and we think it's fair to compare Subjects more intelligently, or at least be more aware of the semantic structure of the Subject string. This would have numerous benefits, such as:</p>
<ul>
<li>Prevent users from becoming hopelessly confused when they can't figure out why they can't read/write Objects (people often don't notice http vs https) which will make DataONE seem friendlier</li>
<li>Make DataONE authentication less dependent upon protocols such as HTTP/HTTPS, both of which my change (i.e., ORCID may disable HTTPS; HTTP(S) may be replaced by a future web protocol) and thus more future-proof</li>
</ul>
<p>This type of change would require changes in one or more software projects:</p>
<ul>
<li>DataONE Portal or libclient Java (I'm not entirely sure where this check is done right now)</li>
<li>Architecture documentation</li>
<li>(Potentially any/all) MN software stacks (At least a review would be needed)</li>
<li>(Potentially) Client tools (e.g., R, Python) (At least a review would be needed)</li>
</ul>
Member Nodes - Support #8209 (Closed): Five objects failed to sync to the CN from urn:node:ARCTIChttps://redmine.dataone.org/issues/82092017-10-25T18:28:11ZBryce Mecummecum@nceas.ucsb.edu
<p>I noticed this while doing other work and wanted to report the Objects in question so they can get fixed.</p>
<p>doi:10.18739/A2T091<br>
doi:10.18739/A2FV7G<br>
doi:10.18739/A28Z56<br>
doi:10.18739/A2Z55G<br>
doi:10.18739/A2PP08</p>
<p>Calls to CNRead.get() and CNRead.resolve() are giving me error that the System Metadata for each Object can't be found, e.g.,</p>
<p>No system metadata could be found for given PID: doi:10.18739/A2T091"</p>
Infrastructure - Bug #8052 (New): Geohashed value is incorrecthttps://redmine.dataone.org/issues/80522017-03-27T20:43:47ZBryce Mecummecum@nceas.ucsb.edu
<p>Adam Shepherd at BCO-DMO uploaded a test DCX doc here:</p>
<p><a href="https://search-sandbox.test.dataone.org/#view/http://lod.bco-dmo.org/id/dataset-file/682007">https://search-sandbox.test.dataone.org/#view/http://lod.bco-dmo.org/id/dataset-file/682007</a></p>
<p>which has bounding coordinates of:</p>
<p>North<br>
50.4907 degrees<br>
South<br>
20.4907 degrees<br>
East<br>
-120 degrees<br>
West<br>
120.826 degrees</p>
<p>and when we looked at the geohash values stored in the index for the record they appear to be incorrect. The bounding coordinates this DCX record is using are a bit weird but I'm not sure they're invalid. Google's JavaScript maps API calculates the centroid as 35.490700000000004,-179.58700000000002 which, according to this tool, <a href="http://www.movable-type.co.uk/scripts/geohash.html">http://www.movable-type.co.uk/scripts/geohash.html</a>, should have a geohash of 8n23ckusk but has a geohash of sn23ckusr instead, which is nearly 180 degrees longitude away from the expected location.</p>
Infrastructure - Bug #7858 (New): Obsoleting a resource map clears the resourceMap field for the ...https://redmine.dataone.org/issues/78582016-08-03T21:34:03ZBryce Mecummecum@nceas.ucsb.edu
<p>This is long-standing behavior that I consider a bug. That said, there are likely plenty of design conversations that predate me I'm unaware of.</p>
<p>When I update a Data Package by updating the metadata object and its resource map with new ones, the resourceMap field in the Solr index for the obsoleted metadata object is cleared. I expected it not to be cleared.</p>
<p>Why is this the way it works? The way I see it, clearing out the resourceMap field in the Solr index for the obsoleted metadata object reduces benefit of us versioning objects. When a package is cited by its metadata object's PID and the package is updated after the citation was published, a visitor to the dataset landing page will no longer see the package because the resource map isn't in the index. Of course they will be shown a link to the latest version of the package which does have a resource map but that's not what they cited.</p>
DataONE API - Bug #7684 (New): Call to MNStorage.update() via REST API returns java.lang.StackOve...https://redmine.dataone.org/issues/76842016-03-21T23:07:39ZBryce Mecummecum@nceas.ucsb.edu
<p>I was trying to update an object via the REST API via cURL and forgot to enter the correct URL. The cURL command I used and response is:</p>
<p>$ curl -X PUT -H "Authorization: Bearer $TOKEN" -F "pid=resourceMap_doi:10.5065/D6G44NFV" -F "object=@object.xml" -F "sysmeta=@sysmeta.xml" -F "newPid=resourceMap_doi:10.5065/D6G44NFV_v3" $URL<br>
<?xml version="1.0" encoding="UTF-8"?><br>
java.lang.StackOverflowError<br>
</p>
<p>Where $URL was '<a href="https://arcticdata.io/metacat/d1/mn/v2/object">https://arcticdata.io/metacat/d1/mn/v2/object</a>' instead of '<a href="https://arcticdata.io/metacat/d1/mn/v2/object/resourceMap_doi:10.5065/D6G44NFV">https://arcticdata.io/metacat/d1/mn/v2/object/resourceMap_doi:10.5065/D6G44NFV</a>'</p>
<p>I expected to receive some sort of warning/error that I had forgotten to specify the URL properly for this call but instead saw a StackOverflowError.</p>
DataONE API - Bug #7578 (New): Fix 404 link to d1_instance_generator folder in documentationhttps://redmine.dataone.org/issues/75782016-01-08T22:01:20ZBryce Mecummecum@nceas.ucsb.edu
<p>In the MN API documentation for MNStorage.create (<a href="https://jenkins-ucsb-1.dataone.org/job/API%20Documentation%20-%20trunk/ws/api-documentation/build/html//apis/MN_APIs.html#MNStorage.create">https://jenkins-ucsb-1.dataone.org/job/API%20Documentation%20-%20trunk/ws/api-documentation/build/html//apis/MN_APIs.html#MNStorage.create</a>), I found a the following paragraph contains a broken link to d1_instance_generator:</p>
<blockquote>
<p>"The system metadata included with the create call must contain values for the elements required to be set by clients (see System Metadata). The system metadata document can be crafted by hand or preferably with a tool such as generate_sysmeta.py which is available in the d1_instance_generator Python package. See documentation included with that package for more information on its operation."</p>
</blockquote>
<p>The link to d1_instance_generator was to the SVN folder <a href="https://repository.dataone.org/software/cicore/trunk/d1_instance_generator">https://repository.dataone.org/software/cicore/trunk/d1_instance_generator</a> which is currently a 404. I think the folder moved to /d1_test_utilities_python/src/d1_test/instance_generator.</p>