MNDeployment #7969
FEMC (Forest Ecosystem Monitoring Cooperative) - was VMC Vermont Monitoring Cooperative
0%
Description
Jim Duncan from VMC contacted us via the "Contact Us" page inquiring about participation with DataONE. See meeting notes from 11 Jan 2017 for summary: https://epad.dataone.org/pad/p/VMC_and_DataONE
From http://www.uvm.edu/vmc/about/
"The mission of the Vermont Monitoring Cooperative is to serve Vermont through improved understanding of long-term trends, annual conditions, and interdisciplinary relationships of the physical, chemical, and biological components of forested ecosystems in Vermont.
The VMC also promotes the efficient coordination of multi-disciplinary environmental monitoring and research activities among federal, state, university, and private-sector agencies with common interests in the long-term health, management, and protection of Vermont's forested ecosystems."
Subtasks
Related issues
History
#1 Updated by Laura Moyers almost 8 years ago
- % Done changed from 0 to 10
- Target version set to Deploy by end of Y3Q2
- Status changed from New to Planning
#2 Updated by Monica Ihli over 7 years ago
The server VMC is using is managed by the institution's central IT department. As of now, GMN has successfully been installed on CentOS 7: https://dataone.uvm.edu/mn/v2. This is presently a standalone installation, and will remain so until the adapter integration between their repository and GMN has been fully developed and tested.
#3 Updated by Laura Moyers over 7 years ago
- Latitude set to 44.48
- Base URL set to http://www.uvm.edu/vmc
- Longitude set to -73.21
#4 Updated by Laura Moyers over 7 years ago
- Software stack set to GMN
- NodeIdentifier changed from urn:node:VMC to urn:node:FEMC
- Assignee changed from Laura Moyers to Monica Ihli
VMC has broadened its scope to include state forestry resources/managers from neighboring states and will be known as the Forest Ecosystem Monitoring Cooperative (FEMC). Note node ID change from urn:node:VMC to urn:node:FEMC
FEMC uses EML as their metadata standard.
Jim and Mike are beginning work on the GMN adapter; next meeting 1 May.
#6 Updated by Laura Moyers over 7 years ago
- Subject changed from VMC - Vermont Monitoring Cooperative to FEMC (Forest Ecosystem Monitoring Cooperative) - was VMC Vermont Monitoring Cooperative
- Target version changed from Deploy by end of Y3Q2 to Deploy by end of Y3Q4
#7 Updated by Laura Moyers over 7 years ago
Jim provided this status update 22 June:
I just wanted to provide a status update to the whole group. Mike and Monica have been making headway on the URL endpoints and setting up our adapter, and will be having another round of work very soon. I don’t think we need to set up call immediately, I am hopeful that we will have something workable to show before the July 4 holiday, after which it would be great to talk again.
There was a bit of a problem getting access to the FEMC endpoints earlier in the month, but Monica and Mike/Jim were able to sort this out.
#8 Updated by Laura Moyers over 7 years ago
- File FEMC_logo_fulltitle_right.png added
- File Member_Node_Description_Form_FEMC_JD.pdf added
- File logo-icon-femc.png added
- File FEMC_logo_fulltitle_bottom.png added
Jim passed on some edits to their MNDD and new logos (attached).
#9 Updated by Laura Moyers over 7 years ago
- Related to Task #8134: VMC/FEMC - Test Pip Upgrade added
#10 Updated by Laura Moyers over 7 years ago
- Related to Task #8140: VMC/FEMC - Convert adapter added
#11 Updated by Laura Moyers over 7 years ago
- Related to Task #8141: VMC/FEMC - Register in Sandbox added
#12 Updated by Laura Moyers over 7 years ago
- File deleted (
Member_Node_Description_Form_FEMC_JD.pdf)
#13 Updated by Laura Moyers over 7 years ago
- File Member_Node_Description_Form_FEMC_JD.pdf added
Removed poorly formatted MNDD, added updated MNDD.
Notes from UTK DataONE meeting 8/22/17:
Scheduled working meeting for 8/23 of this week to upgrade GMN software and review the status of their data exposure endpoint. FEMC has an internal deadline of mid-September to have a DataONE MN in production. This is possible, but iffy. The plan is still to upgrade the FEMC GMC to 2.3.5 before integrated testing.
#14 Updated by Monica Ihli over 7 years ago
ABOUT RESTFUL ENDPOINTS
This operates in two parts. First there is an endpoint which presents the identifiers which have changed since a point in time. Then these identifiers are fed into a 2nd endpoint which can be used to retrieve the metadata object for that specific set of identifiers.
The first endpoint for records changed is of the following format:
https://www.uvm.edu/femc/dataone/changedDatasets/changedSince//X-API-KEY/
For example:
https://www.uvm.edu/femc/dataone/changedDatasets/changedSince/1502668800/X-API-KEY/gumboandcrawfish
will return values such as:
12345
67890
12345
09876
Which may then be used in the following way as:
/femc/dataone/datasetEML/projectID//datasetID//X-API-KEY/
Such as for the above example:
/femc/dataone/datasetEML/projectID/1065/datasetID/2294/X-API-KEY/gumboandcrawfish
The result of the specific record request will be a science metadata xml file returned.
#15 Updated by Laura Moyers over 7 years ago
- % Done changed from 10 to 50
- Status changed from Planning to Testing
#16 Updated by Laura Moyers over 7 years ago
- Start date set to 2017-01-11
setting start date based on initial meeting with VMC/FEMC in January 2017
#17 Updated by Laura Moyers over 7 years ago
- Target version changed from Deploy by end of Y3Q4 to Deploy by end of Y4Q1
#18 Updated by Monica Ihli over 7 years ago
The build of metadata records from data in the native system was using an outdated algorithm which, among other issues, prefixed the eml namespace to all elements. This invalidated the xml document. An updated script for generating eml documents has been applied. There may still be a few invalid documents to look out for here or there to look out for with small errors.
Additional Requirement:
FEMC system permits researchers to temporarily set an existing data set to be private. They are concerned as to how this will be handled in DataONE. They don't want to archive it yet because something can't be unarchived once that is done.
Proposed solution:
Use .update() to remove public access rule from system metadata to create outcome of privacy. Add a status to the FEMC endpoint which lists altered records to identify appropriate action. This could also provide a tidy way of identifying if a record has been deleted /archived in your system so that DataONE knows to archive it on our copy as well. For example:
@
20
229
ACTIVE
20
229
HIDDEN
20
229
DELETED
@
An important consideration is to ensure that, if a metadata record changes between the time it is set to private through removal of access rules and the time the author wants it to go public again, we must make sure to execute an update() instead of just .updateSystemMetadata()
#19 Updated by Monica Ihli about 7 years ago
- File GMN-Reinstall over existing.pdf added
FEMC has been upgraded to 2.3.7 using the attached process.
#20 Updated by Monica Ihli about 7 years ago
- FEMC has acquired their Stage DataONE test certificate/key.
- New certificate are installed and CRUD tests done.
- Adapter has been developed ( https://github.com/DataONEorg/SlenderNodes/tree/master/femc)
- Adapter is installed on FEMC MN server and cron setup for 3 hour intervals. Testing is in progress. The node operator will wait for initial load to complete and then will manipulate some test records so we can confirm updates are cascading to GMN.
- Will convene with Mike in 1-2 days to check progress and testing. If all looks good will complete registration at that time.
#21 Updated by Monica Ihli about 7 years ago
- Status changed from Testing to Operational
#22 Updated by Monica Ihli about 7 years ago
- MN_Date_Online set to 2017-10-02
FEMC requests support updating metadata. They overlooked updating the organization name in metadata. Have provided some instructions for making this happen and offered to meet if needed.
#23 Updated by Monica Ihli about 7 years ago
Jim has completed updates to metadata generation script and new metadata record versions have been harvested. No failures observed in the CN logs.
#24 Updated by Dave Vieglais almost 7 years ago
- Sprint set to Production and Upcoming
#25 Updated by Dave Vieglais almost 7 years ago
- Sprint changed from Production and Upcoming to Ongoing Operational
#26 Updated by Amy Forrester about 6 years ago
- Target version changed from Deploy by end of Y4Q1 to Operational
#27 Updated by Amy Forrester over 5 years ago
From Jim Duncan James.Duncan@uvm.edu: Mike has moved on to a new position
#28 Updated by Amy Forrester over 5 years ago
Working with Jim to get a service agreement signed by our two institutions...DataONE doesn't have such a doc...Jim sending us one to sign.
8/27/19: sent Jim reminder email