Project

General

Profile

MNDeployment #7969

FEMC (Forest Ecosystem Monitoring Cooperative) - was VMC Vermont Monitoring Cooperative

Added by Laura Moyers almost 5 years ago. Updated over 2 years ago.

Status:
Operational
Priority:
Normal
Assignee:
Target version:
Start date:
2018-08-06
Due date:
% Done:

0%

Latitude:
44.48
Longitude:
-73.21
MN Description:
Base URL:
http://www.uvm.edu/vmc
NodeIdentifier:
urn:node:FEMC
MN Tier:
Tier 1
Software stack:
GMN
MN_Date_Online:
2017-10-02
Name:
Logo URL:
Date Upcoming:
Date Deprecated:
Information URL:
Version:

Description

Jim Duncan from VMC contacted us via the "Contact Us" page inquiring about participation with DataONE. See meeting notes from 11 Jan 2017 for summary: https://epad.dataone.org/pad/p/VMC_and_DataONE

From http://www.uvm.edu/vmc/about/

"The mission of the Vermont Monitoring Cooperative is to serve Vermont through improved understanding of long-term trends, annual conditions, and interdisciplinary relationships of the physical, chemical, and biological components of forested ecosystems in Vermont.

The VMC also promotes the efficient coordination of multi-disciplinary environmental monitoring and research activities among federal, state, university, and private-sector agencies with common interests in the long-term health, management, and protection of Vermont's forested ecosystems."

FEMC_logo_fulltitle_bottom.png (56.5 KB) Laura Moyers, 2017-06-30 12:22

FEMC_logo_fulltitle_right.png (88.5 KB) Laura Moyers, 2017-06-30 12:22

logo-icon-femc.png (57 KB) Laura Moyers, 2017-06-30 12:22

Member_Node_Description_Form_FEMC_JD.pdf - Updated FEMC MNDD (532 KB) Laura Moyers, 2017-08-22 20:12

GMN-Reinstall over existing.pdf (98.2 KB) Monica Ihli, 2017-09-18 20:05

586
587
588

Subtasks

Task #8670: FEMC - upgrade GMN 2.3.7 to 3.0.4NewRoger Dahl


Related issues

Related to Member Nodes - Task #8134: VMC/FEMC - Test Pip Upgrade Closed 2017-07-21
Related to Member Nodes - Task #8140: VMC/FEMC - Convert adapter Closed 2017-07-28
Related to Member Nodes - Task #8141: VMC/FEMC - Register in Sandbox Closed 2017-08-04

History

#1 Updated by Laura Moyers almost 5 years ago

  • % Done changed from 0 to 10
  • Target version set to Deploy by end of Y3Q2
  • Status changed from New to Planning

#2 Updated by Monica Ihli over 4 years ago

The server VMC is using is managed by the institution's central IT department. As of now, GMN has successfully been installed on CentOS 7: https://dataone.uvm.edu/mn/v2. This is presently a standalone installation, and will remain so until the adapter integration between their repository and GMN has been fully developed and tested.

#3 Updated by Laura Moyers over 4 years ago

  • Longitude set to -73.21
  • Base URL set to http://www.uvm.edu/vmc
  • Latitude set to 44.48

#4 Updated by Laura Moyers over 4 years ago

  • NodeIdentifier changed from urn:node:VMC to urn:node:FEMC
  • Assignee changed from Laura Moyers to Monica Ihli
  • Software stack set to GMN

VMC has broadened its scope to include state forestry resources/managers from neighboring states and will be known as the Forest Ecosystem Monitoring Cooperative (FEMC). Note node ID change from urn:node:VMC to urn:node:FEMC

FEMC uses EML as their metadata standard.

Jim and Mike are beginning work on the GMN adapter; next meeting 1 May.

#6 Updated by Laura Moyers over 4 years ago

  • Target version changed from Deploy by end of Y3Q2 to Deploy by end of Y3Q4
  • Subject changed from VMC - Vermont Monitoring Cooperative to FEMC (Forest Ecosystem Monitoring Cooperative) - was VMC Vermont Monitoring Cooperative

#7 Updated by Laura Moyers over 4 years ago

Jim provided this status update 22 June:

I just wanted to provide a status update to the whole group. Mike and Monica have been making headway on the URL endpoints and setting up our adapter, and will be having another round of work very soon. I don’t think we need to set up call immediately, I am hopeful that we will have something workable to show before the July 4 holiday, after which it would be great to talk again.


There was a bit of a problem getting access to the FEMC endpoints earlier in the month, but Monica and Mike/Jim were able to sort this out.

#8 Updated by Laura Moyers over 4 years ago

586
587
588

Jim passed on some edits to their MNDD and new logos (attached).

#9 Updated by Laura Moyers over 4 years ago

  • Related to Task #8134: VMC/FEMC - Test Pip Upgrade added

#10 Updated by Laura Moyers over 4 years ago

  • Related to Task #8140: VMC/FEMC - Convert adapter added

#11 Updated by Laura Moyers over 4 years ago

  • Related to Task #8141: VMC/FEMC - Register in Sandbox added

#12 Updated by Laura Moyers over 4 years ago

  • File deleted (Member_Node_Description_Form_FEMC_JD.pdf)

#13 Updated by Laura Moyers over 4 years ago

Removed poorly formatted MNDD, added updated MNDD.

Notes from UTK DataONE meeting 8/22/17:

Scheduled working meeting for 8/23 of this week to upgrade GMN software and review the status of their data exposure endpoint. FEMC has an internal deadline of mid-September to have a DataONE MN in production. This is possible, but iffy. The plan is still to upgrade the FEMC GMC to 2.3.5 before integrated testing.

#14 Updated by Monica Ihli over 4 years ago

ABOUT RESTFUL ENDPOINTS
This operates in two parts. First there is an endpoint which presents the identifiers which have changed since a point in time. Then these identifiers are fed into a 2nd endpoint which can be used to retrieve the metadata object for that specific set of identifiers.

The first endpoint for records changed is of the following format:

https://www.uvm.edu/femc/dataone/changedDatasets/changedSince//X-API-KEY/

For example:

https://www.uvm.edu/femc/dataone/changedDatasets/changedSince/1502668800/X-API-KEY/gumboandcrawfish

will return values such as:

12345
67890


12345
09876

Which may then be used in the following way as:

/femc/dataone/datasetEML/projectID//datasetID//X-API-KEY/

Such as for the above example:

/femc/dataone/datasetEML/projectID/1065/datasetID/2294/X-API-KEY/gumboandcrawfish

The result of the specific record request will be a science metadata xml file returned.

#15 Updated by Laura Moyers over 4 years ago

  • % Done changed from 10 to 50
  • Status changed from Planning to Testing

#16 Updated by Laura Moyers over 4 years ago

  • Start date set to 2017-01-11

setting start date based on initial meeting with VMC/FEMC in January 2017

#17 Updated by Laura Moyers over 4 years ago

  • Target version changed from Deploy by end of Y3Q4 to Deploy by end of Y4Q1

#18 Updated by Monica Ihli about 4 years ago

The build of metadata records from data in the native system was using an outdated algorithm which, among other issues, prefixed the eml namespace to all elements. This invalidated the xml document. An updated script for generating eml documents has been applied. There may still be a few invalid documents to look out for here or there to look out for with small errors.

Additional Requirement:

FEMC system permits researchers to temporarily set an existing data set to be private. They are concerned as to how this will be handled in DataONE. They don't want to archive it yet because something can't be unarchived once that is done.

Proposed solution:

Use .update() to remove public access rule from system metadata to create outcome of privacy. Add a status to the FEMC endpoint which lists altered records to identify appropriate action. This could also provide a tidy way of identifying if a record has been deleted /archived in your system so that DataONE knows to archive it on our copy as well. For example:

@

20
229
ACTIVE


20
229
HIDDEN


20
229
DELETED

@

An important consideration is to ensure that, if a metadata record changes between the time it is set to private through removal of access rules and the time the author wants it to go public again, we must make sure to execute an update() instead of just .updateSystemMetadata()

#19 Updated by Monica Ihli about 4 years ago

FEMC has been upgraded to 2.3.7 using the attached process.

#20 Updated by Monica Ihli about 4 years ago

  • FEMC has acquired their Stage DataONE test certificate/key.
  • New certificate are installed and CRUD tests done.
  • Adapter has been developed ( https://github.com/DataONEorg/SlenderNodes/tree/master/femc)
  • Adapter is installed on FEMC MN server and cron setup for 3 hour intervals. Testing is in progress. The node operator will wait for initial load to complete and then will manipulate some test records so we can confirm updates are cascading to GMN.
  • Will convene with Mike in 1-2 days to check progress and testing. If all looks good will complete registration at that time.

#21 Updated by Monica Ihli about 4 years ago

  • Status changed from Testing to Operational

#22 Updated by Monica Ihli about 4 years ago

  • MN_Date_Online set to 2017-10-02

FEMC requests support updating metadata. They overlooked updating the organization name in metadata. Have provided some instructions for making this happen and offered to meet if needed.

#23 Updated by Monica Ihli about 4 years ago

Jim has completed updates to metadata generation script and new metadata record versions have been harvested. No failures observed in the CN logs.

#24 Updated by Dave Vieglais almost 4 years ago

  • Sprint set to Production and Upcoming

#25 Updated by Dave Vieglais almost 4 years ago

  • Sprint changed from Production and Upcoming to Ongoing Operational

#26 Updated by Amy Forrester almost 3 years ago

  • Target version changed from Deploy by end of Y4Q1 to Operational

#27 Updated by Amy Forrester over 2 years ago

From Jim Duncan James.Duncan@uvm.edu: Mike has moved on to a new position

#28 Updated by Amy Forrester over 2 years ago

Working with Jim to get a service agreement signed by our two institutions...DataONE doesn't have such a doc...Jim sending us one to sign.

8/27/19: sent Jim reminder email

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)