Story #8158
Synchronization appears to fail under high load
Status:
Closed
Priority:
Normal
Assignee:
Category:
d1_synchronization
Target version:
Start date:
2017-08-04
Due date:
% Done:
100%
Story Points:
Sprint:
Description
When the number of objects that needs to be processed by synchronization is high, the process fails / crashes. For example, the Pangaea node with 325k objects fails on initial sync. The cause of this needs to be evaluated and the synchronization process refactored to allow arbitrarily large numbers of entries to be synchronized.
History
#1 Updated by Dave Vieglais over 6 years ago
- Assignee changed from Robert Waltz to Dave Vieglais
#2 Updated by Dave Vieglais over 6 years ago
- Sprint set to Infrastructure backlog
#3 Updated by Dave Vieglais over 6 years ago
- Target version changed from CCI-2.4.0 to CCI-2.3.7
#4 Updated by Rob Nahf about 6 years ago
harvest was refactored to better keep the queue under 50k items. (It doesn't harvest everything if the queue is too long). Synchronizing in stage only shows sync failing due to Hz errors (connection issues, CONCURRENT_MAP_REMOVES).
So, I believe the sync code is robust with respect to high load.
#5 Updated by Rob Nahf about 6 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 30
#6 Updated by Dave Vieglais almost 6 years ago
- % Done changed from 30 to 100
- Status changed from In Progress to Closed
Appears resolved with Rob's changes to synchronization process.
