Support #8017

NCEI - Implement GMN for replicating Arctic Data Center Data

Added by Monica Ihli over 6 years ago. Updated over 5 years ago.

Target version:
Start date:
Due date:
% Done:


Story Points:


New Additional Technical Contact:
Xiaoyan - Working at NOAA NCEI facility with Sheekela, Ken.

* Upgrade NCEI from Tier 1 to Tier 4, in order to fulfill replication partnshership with Arctic Data Center.
* Moving GMN hosted by DataONE to hosting by NCEI.
* Interested in further exploring implementation of DataONE API in current NCEI system.


Support #8019: Follow up on Preferred Replication Target TestingNewMonica Ihli

Related issues

Related to Member Nodes - MNDeployment #3228: NCEI - National Centers for Environmental Information Operational 2012-09-06


#1 Updated by Monica Ihli over 6 years ago

  • Related to MNDeployment #3228: NCEI - National Centers for Environmental Information added

#2 Updated by Monica Ihli over 6 years ago

+From Jan 17, 2017:+

Xiaoyan Li is installing GMN v2.

Once testing server is complete, and functions as a Tier 4 node in the test environment, and NCEI is ready to upgrade the production version, the Node document in DataONE would be changed to point to the new Tier 4 GMN rather than the older Tier 1 slender node URL.

Node operator will need to re-register the node in the test environment and then in production, pointing to the new baseUrl and advertising the new capabilities. For this document:, the baseURl would change from the current to the new one.

Some Notes about Existing Implementation:
* DataONE is not minting identifiers for NCEI.
* The slender-node implementation is just using what is available in the metadata provided.
* The MN implementation is simple in that it simply exposes the ISO metadata presented from NCEI.

Some Notes about New Implementation:
* Xiaoyan is submitting a request for GMN Security review internally to seek running GMN software on a standalone CentOS6 server at CNEI-MD.
* Anticipates pip install not being permitted due to NOAA5010 (NCEI-MD) security rules. Seeking to build/install GMN from source.

Summary of recommendations for source-based install:

_"Installing with pip is pretty much the same as installing from source code. Since it's all script, the packages downloaded by pip contain the source and can be examined. Since the machine must be visible from the web, pip can probably reach PyPI, but if there is a requirement to more tightly control available packages, you can copy the packages to a local pip repo and install from there with pip.

If pip is still restricted after that, it could be because the old type of packages allowed the code in the package to run on the local machine at install time. But that is not the case with the new(er) Wheel format, which is what GMN uses.

The main issue with installing GMN manually is that you'll have to handle installing all the dependencies manually as well."_

+From Feb 1, 2017:+

List of GMN Dependencies Provided:
* appdirs==1.4.0
* CacheControl==0.11.6
* cffi==1.9.1
* cryptography==1.5.2
* dataone.cli==1.2.5
* dataone.common==2.0.0
* dataone.gmn==2.0.1
* dataone.libclient==2.0.0
* Django==1.10.1
* enum34==1.1.6
* google.foresite-toolkit==1.3
* html5lib==0.999999999
* idna==2.2
* ipaddress==1.0.18
* iso8601==0.1.11
* isodate==0.5.4
* lxml==3.7.2
* packaging==16.8
* psycopg2==2.5.2
* pyasn1==0.1.9
* pycparser==2.17
* PyJWT==1.4.2
* pyOpenSSL==16.2.0
* pyparsing==1.5.7
* python-dateutil==2.1
* PyXB==1.2.5
* rdflib==4.0.1
* requests==2.12.4
* six==1.10.0
* SPARQLWrapper==1.8.0
* webencodings==0.5

#3 Updated by Monica Ihli over 6 years ago

More implementation notes:

NCEI is going to take GMN & Slender node code and adjust to enable replication support so that Arctic Data Center will replicate to NCEI. Therefore NCEI content is separate, but NCEI is in the role of node operator.

#4 Updated by Monica Ihli over 6 years ago

  • Subject changed from NCEI - Upgrade Node from Tier 1 to Tier 4 & Transfer Hosting to NCEI - Implement GMN for replicating Arctic Data Center Data

#5 Updated by Monica Ihli over 5 years ago

Contacted by Xioyan for help resolving what appears to be certificate errors preventing apache from starting in CentOS after attempting to generate self-signed certificate for testing.

Some clues in var/log/httpd/error.log

[Wed Apr 04 16:04:31.355966 2018] [ssl:info] [pid 129351] AH02200: Loading certificate & private key of SSL-aware server ''
[Wed Apr 04 16:04:31.356143 2018] [ssl:info] [pid 129351] AH01914: Configuring server for SSL protocol
[Wed Apr 04 16:04:31.356494 2018] [ssl:warn] [pid 129351] AH01906: RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)

The bit about RSA server certificate is a CA certificate may indicate that there's been some mix-up in which cert was referenced/placed where.

#6 Updated by Amy Forrester over 5 years ago

  • Assignee changed from Monica Ihli to Roger Dahl

#7 Updated by Amy Forrester about 5 years ago

  • Related to Task #8681: NCEI - install new GMN added

#8 Updated by Amy Forrester about 5 years ago

  • Related to deleted (Task #8681: NCEI - install new GMN )

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)