Class KetchLeader
- java.lang.Object
-
- org.eclipse.jgit.internal.ketch.KetchLeader
-
public abstract class KetchLeader extends java.lang.Object
A leader managing consensus across remote followers.A leader instance starts up in
KetchLeader.State.CANDIDATE
and tries to begin a new term by sending anElectionRound
to all replicas. Its term starts if a majority of replicas have accepted this leader instance for the term.Once elected by a majority the instance enters
KetchLeader.State.LEADER
and runs proposals offered toqueueProposal(Proposal)
. This continues until the leader is timed out for inactivity, or is deposed by a competing leader gaining its own majority.Once timed out or deposed this
KetchLeader
instance should be discarded, and a new instance takes over.Each leader instance coordinates a group of
KetchReplica
s. Replica instances are owned by the leader instance and must be discarded when the leader is discarded.In Ketch all push requests are issued through the leader. The steps are as follows (see
KetchPreReceive
for an example):- Create a
Proposal
with theReceiveCommand
s that represent the push. - Invoke
queueProposal(Proposal)
on the leader instance. - Wait for consensus with
Proposal.await()
. - To examine the status of the push, check
Proposal.getCommands()
, looking atCommand.getResult()
.
The leader gains consensus by first pushing the needed objects and a
RefTree
representing the desired target repository state to therefs/txn/accepted
branch on each of the replicas. Once a majority has succeeded, the leader commits the state by either pushing therefs/txn/accepted
value torefs/txn/committed
(for Ketch-aware replicas) or by pushing updates torefs/heads/master
, etc. for stock Git replicas.Internally, the actual transport to replicas is performed on background threads via the
KetchSystem
's executor service. For performance, theKetchLeader
,KetchReplica
andProposal
objects share some state, and may invoke each other's methods on different threads. This access is protected by the leader'slock
object. Care must be taken to prevent concurrent access by correctly obtaining the leader's lock. - Create a
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
KetchLeader.State
Current state of the leader instance.
-
Field Summary
Fields Modifier and Type Field Description private LogIndex
committedIndex
Leader knows this (and all prior) states are committed.private KetchReplica[]
followers
private LogIndex
headIndex
End of the leader's log.private boolean
idle
Is the leader idle with no work pending? Iftrue
there is no work for the leader (normal state).(package private) java.util.concurrent.locks.Lock
lock
Lock protecting all data within this leader instance.private static org.slf4j.Logger
log
private java.util.List<Proposal>
queued
Pending proposals accepted into the queue in FIFO order.private RefTree
refTree
State of the repository's RefTree after applying all entries inqueued
.(package private) boolean
roundHoldsReferenceToRefTree
Iftrue
refTree
must be duplicated before queuing the next proposal.private Round
runningRound
Current round the leader is preparing and waiting for a vote on.private LocalReplica
self
private KetchLeader.State
state
private KetchSystem
system
private long
term
Term of this leader, once elected.private KetchReplica[]
voters
Leader's knowledge of replicas for this repository.
-
Constructor Summary
Constructors Modifier Constructor Description protected
KetchLeader(KetchSystem system)
Construct a leader for a Ketch instance.
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description private void
commitAsync(KetchReplica caller)
private static LocalReplica
findLocal(java.util.Collection<KetchReplica> voters)
(package private) LogIndex
getCommitted()
(package private) LogIndex
getHead()
(package private) KetchSystem
getSystem()
(package private) long
getTerm()
private void
initialize()
(package private) boolean
isIdle()
private ProposalRound
newProposalRound()
(package private) void
nextRound()
Schedule the next round; invoked whilelock
is held.private void
notifySuccess(Round round)
(package private) void
onReplicaUpdate(KetchReplica replica)
Asynchronous signal from a replica after completion.protected abstract Repository
openRepository()
Get an instance of the repository for use by a leader thread.void
queueProposal(Proposal proposal)
Queue a reference update proposal for consensus.(package private) void
runAsync(Round round)
private void
runLeader()
private void
scheduleLeader()
void
setReplicas(java.util.Collection<KetchReplica> replicas)
Configure the replicas used by this Ketch instance.void
shutdown()
Gracefully shutdown this leader and cancel outstanding operations.LeaderSnapshot
snapshot()
Snapshot this leaderjava.lang.String
toString()
private static java.util.Collection<java.lang.Integer>
validVoterCounts()
-
-
-
Field Detail
-
log
private static final org.slf4j.Logger log
-
system
private final KetchSystem system
-
voters
private KetchReplica[] voters
Leader's knowledge of replicas for this repository.
-
followers
private KetchReplica[] followers
-
self
private LocalReplica self
-
lock
final java.util.concurrent.locks.Lock lock
Lock protecting all data within this leader instance.This lock extends into the
KetchReplica
instances used by the leader. They share the same lock instance to simplify concurrency.
-
state
private KetchLeader.State state
-
term
private long term
Term of this leader, once elected.
-
queued
private final java.util.List<Proposal> queued
Pending proposals accepted into the queue in FIFO order.These proposals were preflighted and do not contain any conflicts with each other and their expectations matched the leader's local view of the agreed upon
refs/txn/accepted
tree.
-
refTree
private RefTree refTree
State of the repository's RefTree after applying all entries inqueued
. New proposals must be consistent with this tree to be appended to the end ofqueued
.Must be deep-copied with
RefTree.copy()
ifroundHoldsReferenceToRefTree
istrue
.
-
roundHoldsReferenceToRefTree
volatile boolean roundHoldsReferenceToRefTree
Iftrue
refTree
must be duplicated before queuing the next proposal. TherefTree
was passed into the constructor of aProposalRound
, and that external reference to theRefTree
object is held by the proposal until it materializes the tree object in the object store. This field is settrue
when the proposal begins execution and setfalse
once tree objects are persisted in the local repository's object store orrefTree
is replaced with a copy to isolate it from any running rounds.If proposals arrive less frequently than the
RefTree
is written out to the repository theroundHoldsReferenceToRefTree
behavior avoids duplicatingrefTree
, reducing both time and memory used. However if proposals arrive more frequentlyrefTree
must be duplicated to prevent newly queued proposals from corrupting therunningRound
.
-
headIndex
private LogIndex headIndex
End of the leader's log.
-
committedIndex
private LogIndex committedIndex
Leader knows this (and all prior) states are committed.
-
idle
private boolean idle
Is the leader idle with no work pending? Iftrue
there is no work for the leader (normal state). This field isfalse
when the leader thread is scheduled for execution, or whilerunningRound
defines a round in progress.
-
runningRound
private Round runningRound
Current round the leader is preparing and waiting for a vote on.
-
-
Constructor Detail
-
KetchLeader
protected KetchLeader(KetchSystem system)
Construct a leader for a Ketch instance.- Parameters:
system
- Ketch system configuration the leader must adhere to.
-
-
Method Detail
-
getSystem
KetchSystem getSystem()
- Returns:
- system configuration.
-
setReplicas
public void setReplicas(java.util.Collection<KetchReplica> replicas)
Configure the replicas used by this Ketch instance.Replicas should be configured once at creation before any proposals are executed. Once elections happen, reconfiguration is a complicated concept that is not currently supported.
- Parameters:
replicas
- members participating with the same repository.
-
validVoterCounts
private static java.util.Collection<java.lang.Integer> validVoterCounts()
-
findLocal
private static LocalReplica findLocal(java.util.Collection<KetchReplica> voters)
-
openRepository
protected abstract Repository openRepository() throws java.io.IOException
Get an instance of the repository for use by a leader thread.The caller will close the repository.
- Returns:
- opened repository for use by the leader thread.
- Throws:
java.io.IOException
- cannot reopen the repository for the leader.
-
queueProposal
public void queueProposal(Proposal proposal) throws java.lang.InterruptedException, java.io.IOException
Queue a reference update proposal for consensus.This method does not wait for consensus to be reached. The proposal is checked to look for risks of conflicts, and then submitted into the queue for distribution as soon as possible.
Callers must use
Proposal.await()
to see if the proposal is done.- Parameters:
proposal
- the proposed reference updates to queue for consideration. Once execution is complete the individual reference result fields will be populated with the outcome.- Throws:
java.lang.InterruptedException
- current thread was interrupted. The proposal may have been aborted if it was not yet queued for execution.java.io.IOException
- unrecoverable error preventing proposals from being attempted by this leader.
-
initialize
private void initialize() throws java.io.IOException
- Throws:
java.io.IOException
-
scheduleLeader
private void scheduleLeader()
-
runLeader
private void runLeader()
-
newProposalRound
private ProposalRound newProposalRound()
-
getTerm
long getTerm()
- Returns:
- term of this leader's reign.
-
getHead
LogIndex getHead()
- Returns:
- end of the leader's log.
-
getCommitted
LogIndex getCommitted()
- Returns:
- state leader knows it has committed across a quorum of replicas.
-
isIdle
boolean isIdle()
-
runAsync
void runAsync(Round round)
-
onReplicaUpdate
void onReplicaUpdate(KetchReplica replica)
Asynchronous signal from a replica after completion.Must be called while
lock
is held by the replica.- Parameters:
replica
- replica posting a completion event.
-
notifySuccess
private void notifySuccess(Round round)
-
commitAsync
private void commitAsync(KetchReplica caller)
-
nextRound
void nextRound()
Schedule the next round; invoked whilelock
is held.
-
snapshot
public LeaderSnapshot snapshot()
Snapshot this leader- Returns:
- snapshot of this leader
-
shutdown
public void shutdown()
Gracefully shutdown this leader and cancel outstanding operations.
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
-