Project

General

Profile

Task #2209

Story #2093: Implement Log Aggregation

Design recovery mechanism for revived CN

Added by Robert Waltz about 12 years ago. Updated about 12 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Robert Waltz
Category:
d1_log_aggregation
Start date:
2012-01-11
Due date:
% Done:

100%

Milestone:
CCI-1.0.0
Product Version:
*
Story Points:
Sprint:

Description

If a CN joins a cluster, it will need to build &/or rebuild its index.

I think i should look at the last identifier that the CN has indexed. And then start up and start receiving entries. It should not immediately processing MNs for log entries. It should only listen until the first entries received after startup. After the first new entries are received then we know the gap that needs to be recovered.

To assist ordering, the unique identifier should be a Double composed of the system's datetime as a long + . + Hazelcast.getAtomicNumber.incrementAndGet()

This should guarantee uniqueness, and a certain degree of sequentiality (depending upon system clock skew it can not be guaranteed). because sequentiality may not be absolute, any recovery must start at a sequences before the start of the gap and end after the gap. Thus, the unique identifier field should in solr terms overwritable.

Of course clock skew between the CN's can not be too great anyhow or else other facets of CN operation will be affected.

History

#1 Updated by Robert Waltz about 12 years ago

  • Status changed from New to Closed

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 14.8 MB)