Member Nodes

Uses FGDC

Some descriptive items in e-mail from Mike Frame to John Cobb on 20130130

The best known USGS maps are the 1:24,000-scale topographic maps, also known as 7.5-minute quadrangles. More than 55,000 7.5-minute maps were made to cover the 48 conterminous States. This is the only uniform map series that covers the entire area of the United States in considerable detail.

Response from those in CSAS Working with our USGS National Geospatial Program People about some options:::

Cool. We should explore this, but do so in concert with Paul Wiese (now acting in Hope's old job) and Rob Dollison (National Map delivery guy). If we're going to do some type of official sharing with another outlet, they should be in the loop.

Technically, an API-level connection is quite doable. The new harvesting method is still dragging along with testing in beta, but we have a good portion of the current US Topo collection in production today. Here's the parent collection item:

https://www.sciencebase.gov/catalog/item/4f554236e4b018de15819c85

Looking at that in the UI, you'll see the map load up that shows you the overall spatial distribution of these products. You can view the entire set from that parent collection here (48,577 individual maps):

https://www.sciencebase.gov/catalog/items?parentId=4f554236e4b018de15819c85

At the REST API level, this can be seen here (different formats with "format" parameter but also with ACCEPT header):

JSON - https://www.sciencebase.gov/catalog/items?parentId=4f554236e4b018de15819c85&format=json
ATOM (OpenSearch) - https://www.sciencebase.gov/catalog/items?parentId=4f554236e4b018de15819c85&format=atom

An app can manipulate what it gets from the JSON output with the fields parameter - more stuff here http://goo.gl/Di6H6. We need to provide a simple layout of all "fields" parameter, but it basically corresponds to what you see as top-level objects and arrays in the JSON behind a given item like the following (e.g., "spatial" returns the item footprint information - points in this case):

https://www.sciencebase.gov/catalog/item/4f85cc66e4b01c0325950366?format=json

The weblinks collection contains links to the following for these products now:

• Link to FGDC XML full metadata
• Link to downloadable GeoPDF
• Link to a thumbnail image

The fact that this and other collections exist in the ScienceBase model means that those collections also become "interactable" items that can be accessed via WMS, WFS, and KML (see RESTful URLs):

https://www.sciencebase.gov/catalogMaps/mapping/ows/4f554236e4b018de15819c85?service=wms&request=getcapabilities&version=1.3.0
https://www.sciencebase.gov/catalogMaps/mapping/ows/4f554236e4b018de15819c85?service=wfs&request=getcapabilities&version=1.0.0
https://www.sciencebase.gov/catalogMaps/mapping/ows/4f554236e4b018de15819c85?mode=download&request=kml&service=wms&layers=children

Once we finalize the new harvesting process, the ScienceBase repository itself will have the full FGDC XML metadata that's the base documentation for individual maps along with the thumbnail image. The GeoPDFs are going to live on Amazon S3. They are still missing some key information needed in the original metadata like map dates needed for the types of faceting they want to do in the search, and that's what we're working through now.

I need to learn more about the system metadata/science metadata distinction in DataONE. From what I remember reading and hearing, the system metadata is basically the stuff about the record from the DataONE perspective (created, updated, provenance (into DataONE), etc.). In the scenario you outline, the USGS Member Node becomes a second order aggregation point for these records with ScienceBase being the first order aggregation point. So, it's essentially a copy of a copy of the original resource. NGP is using ScienceBase as its aggregation point for the 30 individual catalogs that will ultimately be harvested in order to use the above services along with the overall API interface to drive discovery and access applications. What we provide out from there to other discovery points is what we need to talk with Paul and Rob about. Is it really useful to provide out the individual topo map records for discovery through the DataONE Member Node, or would it be most useful to provide a record for the entire collection (of current US Topo maps right now) that users could interact with using map services and other end points?

Rough estimate of size of content from Mike:

"they are around 15-20MB each and right now there are about 55,000 supposedly. So, if my math is right at is about 1.05 Terabytes."

Mike Frame said in an email on 6/3/13:

I have been talking with this separate group in USGS and it looks like it will probably be Fall timeframe. They are in the process of implementing DOIs for all of these 60K or so TOPO maps and want to finish that process/updates first.

(LM)

followup from conversation with Mike Frame and John Cobb from July Meeting in Knoxville:

Once DOI's created, most likely, Topo map content will be published as a supplement to existing USGS CSAS node. So this ticket may need to be closed/merged with already operational CSAC node (Ticket # 3552)

Asked Mike Frame about progress with DOIs, etc., and he said:

"I can say though that the group has to make them available for our USGS Science Data Catalog by November 1. So, they should be more easily consumable then by DataONE shortly after that date. Some of the DataONE MN requirements (i..e System Metadata) will take some additional time, but hopefully by the end of December we can also make these available either via the Clearinghouse MN or a ScienceBase MN."

So, Y5Q2 is still feasible.