Synchronization appears to fail under high load
When the number of objects that needs to be processed by synchronization is high, the process fails / crashes. For example, the Pangaea node with 325k objects fails on initial sync. The cause of this needs to be evaluated and the synchronization process refactored to allow arbitrarily large numbers of entries to be synchronized.
#4 Updated by Rob Nahf over 4 years ago
harvest was refactored to better keep the queue under 50k items. (It doesn't harvest everything if the queue is too long). Synchronizing in stage only shows sync failing due to Hz errors (connection issues, CONCURRENT_MAP_REMOVES).
So, I believe the sync code is robust with respect to high load.