Task #3141
Increase production CN Metacat replication timeout
100%
Description
There are ~14k extra documents on UCSB that have yet to be replicated to UNM and ORC (see related bugs for more analysis).
When manually triggering Metacat-Metacat replication, I see that retrieving the list of documents to replicate (from UCSB to ORC and UNM) times out. It is currently set to do this in 60 seconds.
Using curl to retrieve this same list, shows that it takes ~1:02 minutes! So we just miss the window. I'd like to increase this timeout to 5 minutes which will require editing the metacat.properties file and restarting the Tomcat instances. The proposed procedure follows:
0. Stop daemons on all nodes by executing /usr/local/bin/stopDaemons.sh
1. Set DNS RR to UCSB-only
2. Edit metacat.properties file on ORC and UNM, setting the timeout to 5 minutes:
replication.client.timeout=300000
3. Restart Tomcat on ORC and UNM
4. restart indexing daemons on ORC and UNM by executing /usr/local/bin/startDaemons.sh
5. Set DNS RR to point to ORC and UNM
6. Edit metacat.properties file on UCSB
7. Restart Tomcat on UCSB
8. restart indexing daemons & d1-processing on UCSB by executing /usr/local/bin/startDaemons.sh
Related issues
History
#1 Updated by Ben Leinfelder over 11 years ago
I've changed this in the cn-buildout, so it will be included for subsequent releases of the dataone-cn-metacat package.
#2 Updated by Ben Leinfelder over 11 years ago
- Status changed from New to In Progress
- Assignee set to Ben Leinfelder
We have made the change on UNM and ORC. Content is still replicating from UCSB to those servers. When that completes, we will add UNM and ORC to the RR.
#3 Updated by Ben Leinfelder over 11 years ago
- Parent task deleted (
#3136)
#4 Updated by Ben Leinfelder over 11 years ago
- Status changed from In Progress to Closed
I believe UCSB has been restarted at this point (though it's a moot point with the pending 1.0.4 updates)
