This section describes what the Application can expect from CometBFT.
The Tendermint consensus algorithm, currently adopted in CometBFT, is designed to protect safety under any network conditions, as long as less than 1/3 of validators’ voting power is byzantine. Most of the time, though, the network will behave synchronously, no process will fall behind, and there will be no byzantine process. The following describes what will happen during a block height h in these frequent, benign conditions:
PrepareProposalwill be called exactly once at the proposer process of round 0, height h;
ProcessProposalwill be called exactly once at all processes, and will return accept in its
ExtendVotewill be called exactly once at all processes;
VerifyVoteExtensionwill be called exactly n-1 times at each validator process, where n is the number of validators, and will always return accept in its
BeginBlockwill be called exactly once at all processes, conveying the same prepared block header that all calls to
ProcessProposalhad previously reported for height h; and
DeliverTxwill be called exactly once for each transaction within the block.
EndBlockwill be called exactly once after
DeliverTxhas been executed for all transactions and marks the end of processing for the block.
Commitwill finally be called exactly once at all processes at the end of height h.
However, the Application logic must be ready to cope with any possible run of Tendermint for a given height, including bad periods (byzantine proposers, network being asynchronous). In these cases, the sequence of calls to ABCI++ methods may not be so straighforward, but the Application should still be able to handle them, e.g., without crashing. The purpose of this section is to define what these sequences look like in a precise way.
As mentioned in the Basic Concepts section, CometBFT acts as a client of ABCI++ and the Application acts as a server. Thus, it is up to CometBFT to determine when and in which order the different ABCI++ methods will be called. A well-written Application design should consider any of these possible sequences.
The following grammar, written in case-sensitive Augmented Backus–Naur form (ABNF, specified in IETF rfc7405), specifies all possible sequences of calls to ABCI++, taken by a correct process, across all heights from the genesis block, including recovery runs, from the point of view of the Application.
start = clean-start / recovery clean-start = init-chain [state-sync] consensus-exec state-sync = *state-sync-attempt success-sync info state-sync-attempt = offer-snapshot *apply-chunk success-sync = offer-snapshot 1*apply-chunk recovery = info consensus-exec consensus-exec = (inf)consensus-height consensus-height = *consensus-round decide commit consensus-round = proposer / non-proposer proposer = [prepare-proposal [process-proposal]] non-proposer = [process-proposal] decide = begin-block *deliver-txs end-block init-chain = %s"<InitChain>" offer-snapshot = %s"<OfferSnapshot>" apply-chunk = %s"<ApplySnapshotChunk>" info = %s"<Info>" prepare-proposal = %s"<PrepareProposal>" process-proposal = %s"<ProcessProposal>" begin-block = %s"<BeginBlock>" deliver-txs = %s"<DeliverTx>" end-block = %s"<EndBlock>" commit = %s"<Commit>"
We have kept some ABCI methods out of the grammar, in order to keep it as clear and concise as possible. A common reason for keeping all these methods out is that they all can be called at any point in a sequence defined by the grammar above. Other reasons depend on the method in question:
Flushare only used for debugging purposes. Further, their handling by the Application should be trivial.
CheckTxis detached from the main method call sequence that drives block execution.
Queryprovides read-only access to the current Application state, so handling it should also be independent from block execution.
LoadSnapshotChunkprovide read-only access to the Application’s previously created snapshots (if any), and help populate the parameters of
ApplySnapshotChunkat a process performing state-sync while bootstrapping. Unlike
ApplySnapshotChunkare included in the grammar.
Info is a special case. The method’s purpose is three-fold, it can be used
We have left
Info’s first purpose out of the grammar for the same reasons as all the others: it can happen
at any time, and has nothing to do with the block execution sequence. The second and third purposes, on the other
hand, are present in the grammar.
Let us now examine the grammar line by line, providing further details.
start = clean-start / recovery
InitChain, then it may optionally start a state-sync mechanism to catch up with other processes. Finally, it enters normal consensus execution.
clean-start = init-chain [state-sync] consensus-exec
ApplySnapshotChunkmethod follow to provide the Application with all the snapshots needed, in order to reconstruct the state locally. A successful attempt must provide at least one chunk via
ApplySnapshotChunk. At the end of a successful attempt, CometBFT calls
Infoto make sure the recontructed state’s AppHash matches the one in the block header at the corresponding height.
state-sync = *state-sync-attempt success-sync info state-sync-attempt = offer-snapshot *apply-chunk success-sync = offer-snapshot 1*apply-chunk
Infoto know from which height it needs to replay decisions to the Application. After this, CometBFT enters consensus execution, first in replay mode and then in normal mode.
recovery = info consensus-exec
consensus-execis a key point in this grammar. It is an infinite sequence of consensus heights. The grammar is thus an omega-grammar, since it produces infinite sequences of terminals (i.e., the API calls).
consensus-exec = (inf)consensus-height
BeginBlock-DeliverTx-EndBlock, followed by a call to
Commit. In each round, the sequence of method calls depends on whether the local process is the proposer or not. Note that, if a height contains zero rounds, this means the process is replaying an already decided value (catch-up mode).
consensus-height = *consensus-round decide commit consensus-round = proposer / non-proposer
For every round, if the local process is the proposer of the current round, CometBFT calls
A successful execution of
PrepareProposal implies in a proposal block being (i)signed and (ii)stored
(e.g., in stable storage).
A crash during this step will direct how the node proceeds the next time it is executed, for the same round, after restarted.
If it crashed before (i), then, during the recovery,
PrepareProposal will execute as if for the first time.
Following a crash between (i) and (ii) and in (the likely) case
PrepareProposal produces a different block,
the signing of this block will fail, which means that the new block will not be stored or broadcast.
If the crash happened after (ii), then signing fails but nothing happens to the stored block.
If a block was stored, it is sent to all validators, including the proposer.
Receiving a proposal block triggers
ProcessProposal with such a block.
proposer = [prepare-proposal [process-proposal]]
Also for every round, if the local process is not the proposer of the current round, CometBFT
ProcessProposal at most once.
At most one call to
ExtendVote may occur only after
ProcessProposal is called. A number of calls to
VerifyVoteExtension can occur in any order
with respect to
ExtendVote throughout the round. The reasons are the same
as above, namely, the process running slightly late in the current round, or votes from future
rounds of this height received.
non-proposer = [process-proposal]
init-chain = %s"<InitChain>" offer-snapshot = %s"<OfferSnapshot>" apply-chunk = %s"<ApplySnapshotChunk>" info = %s"<Info>" prepare-proposal = %s"<PrepareProposal>" process-proposal = %s"<ProcessProposal>" begin-block = %s"<BeginBlock>" deliver-txs = %s"<DeliverTx>" end-block = %s"<EndBlock>" commit = %s"<Commit>"
In some cases, an existing Application using the legacy ABCI may need to be adapted to work with ABCI++ with as minimal changes as possible. In this case, of course, ABCI++ will not provide any advange with respect to the existing implementation, but will keep the same guarantees already provided by ABCI. Here is how ABCI++ methods should be implemented.
First of all, all the methods that did not change from ABCI to ABCI++, namely
ApplySnapshotChunk, do not need to undergo any changes in their implementation.
As for the new methods:
PrepareProposal must create a list of transactions
by copying over the transaction list passed in
RequestPrepareProposal.txs, in the same order.
The Application must check whether the size of all transactions exceeds the byte limit
RequestPrepareProposal.max_tx_bytes). If so, the Application must remove transactions at the
end of the list until the total byte size is at or below the limit.
ResponseProcessProposal.statusto accept and return. <!–
ExtendVoteis to set
ResponseExtendVote.extensionto an empty byte array and return.
ResponseVerifyVoteExtension.acceptto true if the extension is an empty byte array and false otherwise, then return. –> <!–
FinalizeBlockis to coalesce the implementation of methods
EndBlock. Legacy applications looking to reuse old code that implemented
DeliverTxshould wrap the legacy
DeliverTxlogic in a loop that executes one transaction iteration per transaction in