h1. Production Status Log
Log entries below are used to communicate the current technical status of the production Coordinating Node environment. If appropriate, log entries can be typed with 'Problem', 'Acknowledgement', and 'Recovery' as is done with Nagios alerts. If the production environment is in a 'problem' state, describe the problem. If you are handling the problem, add an acknowledgement entry, describing what you are doing. If the problem is fixed, add a recovery entry with what you did to clear the issue. Please organize log entries by date, with most recent entries at the top of the log.
CCI-2.0.1
* 2015-12-08
- all CNs are now in cluster with cn-ucsb-1 as the active CN
- d1-processing is running on cn-ucsb-1
- indexing is running on cn-orc-1
2014-08-20
- cn-ucsb-1 has been upgraded to the 1.3.0 release level.
- all CNs are now in cluster with cn-ucsb-1 as the active CN
- d1-processing is running on cn-ucsb-1
- indexing is running on all three CNs
2014-08-15
- cn-unm-1 and cn-orc-1 have been upgraded to the 1.3.0 release level.
- cn-unm-1 and cn-orc-1 both are in read-only mode (node.properties has cn.storage.readOnly=TRUE)
- cn-unm-1 is currently re-indexing the scimeta collection
- cn-orc-1 is re-aggregating all log records.
- The indexing of both collections will take several days.
- While the machines are indexing, all hazelcast ports, and the ldap port and postgresql port have been closed.
2014-07-25
- The CNs are running in a single node state, with cn-ucsb-1 as the active node
- Robert Waltz is evaluating the consistency of the CNs
- See https://redmine.dataone.org/issues/5523
2014-07-22
- Type: Recovery
- LDAP communication is fixed among the CNs
- See https://redmine.dataone.org/issues/5988
2014-07-22
- Type: Acknowledgement
- David Doyle is looking into the LDAP issue
- See https://redmine.dataone.org/issues/5988
2014-07-18
- Type: Problem
- LDAP communication is failing among the CNs, as reported by Nagios
