Methods existing in ABCI
Echo
- Request:
Message (string): A string to echo back
- Response:
Message (string): The input string
- Usage:
- Echo a string to test an ABCI client/server implementation
Flush
- Usage:
- Signals that messages queued on the client should be flushed to the server. It is called periodically by the client implementation to ensure asynchronous requests are actually sent, and is called immediately to make a synchronous request, which returns when the Flush response comes back.
Info
-
Request:
-
Response:
-
Usage:
- Return information about the application state.
- Used to sync CometBFT with the application during a handshake that happens on startup or on recovery.
- The returned
app_versionwill be included in the Header of every block. - CometBFT expects
last_block_app_hashandlast_block_heightto be updated and persisted duringCommit.
Note: Semantic version is a reference to semantic versioning. Semantic versions in info will be displayed as X.X.x.
InitChain
-
Request:
-
Response:
-
Usage:
- Called once upon genesis.
- If
ResponseInitChain.Validatorsis empty, the initial validator set will be theRequestInitChain.Validators - If
ResponseInitChain.Validatorsis not empty, it will be the initial validator set (regardless of what is inRequestInitChain.Validators). - This allows the app to decide if it wants to accept the initial validator set proposed by CometBFT (ie. in the genesis file), or if it wants to use a different one (perhaps computed based on some application specific information in the genesis file).
- Both
RequestInitChain.ValidatorsandResponseInitChain.Validatorsare ValidatorUpdate structs. So, technically, they both are updating the set of validators from the empty set.
Query
-
Request:
-
Response:
-
Usage:
- Query for data from the application at current or past height.
- Optionally return Merkle proof.
- Merkle proof includes self-describing
typefield to support many types of Merkle trees and encoding formats.
CheckTx
-
Request:
-
Response:
-
Usage:
- Technically optional - not involved in processing blocks.
- Guardian of the mempool: every node runs
CheckTxbefore letting a transaction into its local mempool. - The transaction may come from an external user or another node
CheckTxvalidates the transaction against the current state of the application, for example, checking signatures and account balances, but does not apply any of the state changes described in the transaction.- Transactions where
ResponseCheckTx.Code != 0will be rejected - they will not be broadcast to other nodes or included in a proposal block. CometBFT attributes no other value to the response code.
Commit
Parameters and Types
- Request: Commit signals the application to persist application state. It takes no parameters.
-
Response:
-
Usage:
- Signal the Application to persist the application state.
Application is expected to persist its state at the end of this call, before calling
ResponseCommit. - Use
ResponseCommit.retain_heightwith caution! If all nodes in the network remove historical blocks then this data is permanently lost, and no new nodes will be able to join the network and bootstrap, unless state sync is enabled on the chain. Historical blocks may also be required for other purposes, e.g. auditing, replay of non-persisted heights, light client verification, and so on.
- Signal the Application to persist the application state.
Application is expected to persist its state at the end of this call, before calling
ListSnapshots
- Request: Empty request asking the application for a list of snapshots.
-
Response:
-
Usage:
- Used during state sync to discover available snapshots on peers.
- See
Snapshotdata type for details.
LoadSnapshotChunk
-
Request:
-
Response:
-
Usage:
- Used during state sync to retrieve snapshot chunks from peers.
OfferSnapshot
-
Request:
-
Response:
Result
- Usage:
OfferSnapshotis called when bootstrapping a node using state sync. The application may accept or reject snapshots as appropriate. Upon accepting, CometBFT will retrieve and apply snapshot chunks viaApplySnapshotChunk. The application may also choose to reject a snapshot in the chunk response, in which case it should be prepared to accept furtherOfferSnapshotcalls.- Only
AppHashcan be trusted, as it has been verified by the light client. Any other data can be spoofed by adversaries, so applications should employ additional verification schemes to avoid denial-of-service attacks. The verifiedAppHashis automatically checked against the restored application at the end of snapshot restoration. - For more information, see the
Snapshotdata type or the state sync section.
ApplySnapshotChunk
-
Request:
-
Response:
- Usage:
- The application can choose to refetch chunks and/or ban P2P peers as appropriate. CometBFT will not do this unless instructed by the application.
- The application may want to verify each chunk, e.g. by attaching chunk hashes in
Snapshot.Metadataand/or incrementally verifying contents againstAppHash. - When all chunks have been accepted, CometBFT will make an ABCI
Infocall to verify thatLastBlockAppHashandLastBlockHeightmatches the expected values, and record theAppVersionin the node state. It then switches to block sync or consensus and joins the network. - If CometBFT is unable to retrieve the next chunk after some time (e.g. because no suitable
peers are available), it will reject the snapshot and try a different one via
OfferSnapshot. The application should be prepared to reset and accept it or abort as appropriate.
New methods introduced in ABCI 2.0
PrepareProposal
Parameters and Types
-
Request:
-
Response:
-
Usage:
RequestPrepareProposal’s parameterstxs,misbehavior,height,time,next_validators_hash, andproposer_addressare the same as inRequestProcessProposalandRequestFinalizeBlock.RequestPrepareProposal.local_last_commitis a set of the precommit votes for the previous height, including the ones that led to the decision of the previous block, together with their corresponding vote extensions.- The
height,time, andproposer_addressvalues match the values from the header of the proposed block. RequestPrepareProposalcontains a preliminary set of transactionstxsthat CometBFT retrieved from the mempool, called raw proposal. The Application can modify this set and return a modified set of transactions viaResponsePrepareProposal.txs.- The Application can modify the raw proposal: it can reorder, remove or add transactions.
Let
txbe a transaction intxs(set of transactions withinRequestPrepareProposal):- If the Application considers that
txshould not be proposed in this block, e.g., there are other transactions with higher priority, then it should not include it inResponsePrepareProposal.txs. However, this will not removetxfrom the mempool. - If the Application wants to add a new transaction to the proposed block, then the
Application includes it in
ResponsePrepareProposal.txs. CometBFT will not add the transaction to the mempool.
- If the Application considers that
- The Application should be aware that removing and adding transactions may compromise
traceability.
Consider the following example: the Application transforms a client-submitted transaction
t1into a second transactiont2, i.e., the Application asks CometBFT to removet1from the block and addt2to the block. If a client wants to eventually check what happened tot1, it will discover thatt1is not in a committed block (assuming a re-CheckTx evicted it from the mempool), getting the wrong idea thatt1did not make it into a block. Note thatt2will be in a committed block, but unless the Application tracks this information, no component will be aware of it. Thus, if the Application wants traceability, it is its responsibility’s to support it. For instance, the Application could attach to a transformed transaction a list with the hashes of the transactions it derives from.
- The Application can modify the raw proposal: it can reorder, remove or add transactions.
Let
- The Application MAY configure CometBFT to include a list of transactions in
RequestPrepareProposal.txswhose total size in bytes exceedsRequestPrepareProposal.max_tx_bytes. If the Application setsConsensusParams.Block.MaxBytesto -1, CometBFT will include all transactions currently in the mempool inRequestPrepareProposal.txs, which may not fit inRequestPrepareProposal.max_tx_bytes. Therefore, if the size ofRequestPrepareProposal.txsis greater thanRequestPrepareProposal.max_tx_bytes, the Application MUST remove transactions to ensure that theRequestPrepareProposal.max_tx_byteslimit is respected by those transactions returned inResponsePrepareProposal.txs. This is specified in Requirement 2. - As a result of executing the prepared proposal, the Application may produce block events or transaction events.
The Application must keep those events until a block is decided and then pass them on to CometBFT via
ResponseFinalizeBlock. - CometBFT does NOT provide any additional validity checks (such as checking for duplicate transactions).
- If CometBFT fails to validate the
ResponsePrepareProposal, CometBFT will assume the Application is faulty and crash. - The implementation of
PrepareProposalMAY be non-deterministic.
When does CometBFT call “PrepareProposal” ?
When a validator p enters consensus round r, height h, in which p is the proposer, and p’s validValue isnil:
- CometBFT collects outstanding transactions from p’s mempool
- the transactions will be collected in order of priority
- p’s CometBFT creates a block header.
- p’s CometBFT calls
RequestPrepareProposalwith the newly generated block, the local commit of the previous height (with vote extensions), and any outstanding evidence of misbehavior. The call is synchronous: CometBFT’s execution will block until the Application returns from the call. - The Application uses the information received (transactions, commit info, misbehavior, time) to
(potentially) modify the proposal.
- the Application MAY fully execute the block and produce a candidate state (immediate execution)
- the Application can manipulate transactions:
- leave transactions untouched
- add new transactions (not present initially) to the proposal
- remove transactions from the proposal (but not from the mempool thus effectively delaying them) - the
Application does not include the transaction in
ResponsePrepareProposal.txs. - modify transactions (e.g. aggregate them). As explained above, this compromises client traceability, unless it is implemented at the Application level.
- reorder transactions - the Application reorders transactions in the list
- the Application MAY use the vote extensions in the commit info to modify the proposal, in which case it is suggested
that extensions be validated in the same maner as done in
VerifyVoteExtension, since extensions of votes included in the commit info after the minimum of +2/3 had been reached are not verified.
- The Application includes the transaction list (whether modified or not) in the return parameters (see the rules in section Usage), and returns from the call.
- p uses the (possibly) modified block as p’s proposal in round r, height h.
nil validValue in round r, height h,
the consensus algorithm will use it as proposal and will not call RequestPrepareProposal.
ProcessProposal
Parameters and Types
-
Request:
-
Response:
-
Usage:
- Contains all information on the proposed block needed to fully execute it.
- The Application may fully execute the block as though it was handling
RequestFinalizeBlock. - However, any resulting state changes must be kept as candidate state, and the Application should be ready to discard it in case another block is decided.
- The Application may fully execute the block as though it was handling
RequestProcessProposalis also called at the proposer of a round. Normally the call toRequestProcessProposaloccurs right after the call toRequestPrepareProposalandRequestProcessProposalmatches the block produced based onResponsePrepareProposal(i.e.,RequestPrepareProposal.txsequalsRequestProcessProposal.txs). However, no such guarantee is made since, in the presence of failures,RequestProcessProposalmay matchResponsePrepareProposalfrom an earlier invocation orProcessProposalmay not be invoked at all.- The height and time values match the values from the header of the proposed block.
- If
ResponseProcessProposal.statusisREJECT, consensus assumes the proposal received is not valid. - The Application MAY fully execute the block (immediate execution)
- The implementation of
ProcessProposalMUST be deterministic. Moreover, the value ofResponseProcessProposal.statusMUST exclusively depend on the parameters passed in the call toRequestProcessProposal, and the last committed Application state (see Requirements section). - Moreover, application implementors SHOULD always set
ResponseProcessProposal.statustoACCEPT, unless they really know what the potential liveness implications of returningREJECTare.
- Contains all information on the proposed block needed to fully execute it.
When does CometBFT call “ProcessProposal” ?
When a node p enters consensus round r, height h, in which q is the proposer (possibly p = q):- p sets up timer
ProposeTimeout. - If p is the proposer, p executes steps 1-6 in PrepareProposal.
- Upon reception of Proposal message (which contains the header) for round r, height h from q, p verifies the block header.
- Upon reception of Proposal message, along with all the block parts, for round r, height h
from q, p follows the validators’ algorithm to check whether it should prevote for the
proposed block, or
nil. - If the validators’ consensus algorithm indicates p should prevote non-nil:
- CometBFT calls
RequestProcessProposalwith the block. The call is synchronous. - The Application checks/processes the proposed block, which is read-only, and returns
ACCEPTorREJECTin theResponseProcessProposal.statusfield.- The Application, depending on its needs, may call
ResponseProcessProposal- either after it has completely processed the block (immediate execution),
- or after doing some basic checks, and process the block asynchronously. In this case the
Application will not be able to reject the block, or force prevote/precommit
nilafterwards. - or immediately, returning
ACCEPT, if p is not a validator and the Application does not want non-validating nodes to handleProcessProposal
- The Application, depending on its needs, may call
- If p is a validator and the returned value is
ACCEPT: p prevotes on this proposal for round r, height h.REJECT: p prevotesnil.
- CometBFT calls
ExtendVote
Parameters and Types
-
Request:
-
Response:
-
Usage:
ResponseExtendVote.vote_extensionis application-generated information that will be signed by CometBFT and attached to the Precommit message.- The Application may choose to use an empty vote extension (0 length).
- The contents of
RequestExtendVotecorrespond to the proposed block on which the consensus algorithm will send the Precommit message. ResponseExtendVote.vote_extensionwill only be attached to a non-nilPrecommit message. If the consensus algorithm is to precommitnil, it will not callRequestExtendVote.- The Application logic that creates the extension can be non-deterministic.
When does CometBFT call ExtendVote?
When a validator p is in consensus state prevote of round r, height h, in which q is the proposer; and p has received
- the Proposal message v for round r, height h, along with all the block parts, from q,
Prevotemessages from 2f + 1 validators’ voting power for round r, height h, prevoting for the same block id(v),
- p sets lockedValue and validValue to v, and sets lockedRound and validRound to r
- p’s CometBFT calls
RequestExtendVotewith v (RequestExtendVote). The call is synchronous. - The Application returns an array of bytes,
ResponseExtendVote.extension, which is not interpreted by the consensus algorithm. - p sets
ResponseExtendVote.extensionas the value of theextensionfield of type CanonicalVoteExtension, populates the other fields in CanonicalVoteExtension, and signs the populated data structure. - p constructs and signs the CanonicalVote structure.
- p constructs the Precommit message (i.e. Vote structure) using CanonicalVoteExtension and CanonicalVote.
- p broadcasts the Precommit message.
precommit nil messages (either 2f+1 prevote nil messages received,
or timeoutPrevote triggered), p’s CometBFT does not call RequestExtendVote and will not include
a CanonicalVoteExtension field in the precommit nil message.
VerifyVoteExtension
Parameters and Types
-
Request:
-
Response:
-
Usage:
RequestVerifyVoteExtension.vote_extensioncan be an empty byte array. The Application’s interpretation of it should be that the Application running at the process that sent the vote chose not to extend it. CometBFT will always callRequestVerifyVoteExtension, even for 0 length vote extensions.RequestVerifyVoteExtensionis not called for precommit votes sent by the local process.RequestVerifyVoteExtension.hashrefers to a proposed block. There is not guarantee that this proposed block has previously been exposed to the Application viaProcessProposal.- If
ResponseVerifyVoteExtension.statusisREJECT, the consensus algorithm will reject the whole received vote. See the Requirements section to understand the potential liveness implications of this. - The implementation of
VerifyVoteExtensionMUST be deterministic. Moreover, the value ofResponseVerifyVoteExtension.statusMUST exclusively depend on the parameters passed in the call toRequestVerifyVoteExtension, and the last committed Application state (see Requirements section). - Moreover, application implementers SHOULD always set
ResponseVerifyVoteExtension.statustoACCEPT, unless they really know what the potential liveness implications of returningREJECTare.
When does CometBFT call VerifyVoteExtension?
When a node p is in consensus round r, height h, and p receives a Precommit
message for round r, height h from validator q (q ≠ p):
- If the Precommit message does not contain a vote extension with a valid signature, p
discards the Precommit message as invalid.
- a 0-length vote extension is valid as long as its accompanying signature is also valid.
- Else, p’s CometBFT calls
RequestVerifyVoteExtension. - The Application returns
ACCEPTorREJECTviaResponseVerifyVoteExtension.status. - If the Application returns
ACCEPT, p will keep the received vote, together with its corresponding vote extension in its internal data structures. It will be used to populate the ExtendedCommitInfo structure in calls toRequestPrepareProposal, in rounds of height h + 1 where p is the proposer.REJECT, p will deem the Precommit message invalid and discard it.
RequestVerifyVoteExtension to verify it.
FinalizeBlock
Parameters and Types
-
Request:
-
Response:
-
Usage:
- Contains the fields of the newly decided block.
- This method is equivalent to the call sequence
BeginBlock, [DeliverTx], andEndBlockin ABCI 1.0. - The height and time values match the values from the header of the proposed block.
- The Application can use
RequestFinalizeBlock.decided_last_commitandRequestFinalizeBlock.misbehaviorto determine rewards and punishments for the validators. - The Application executes the transactions in
RequestFinalizeBlock.txsdeterministically, according to the rules set up by the Application, before returning control to CometBFT. Alternatively, it can apply the candidate state corresponding to the same block previously executed viaPrepareProposalorProcessProposal. ResponseFinalizeBlock.tx_results[i].Code == 0only if the i-th transaction is fully valid.- The Application must provide values for
ResponseFinalizeBlock.app_hash,ResponseFinalizeBlock.tx_results,ResponseFinalizeBlock.validator_updates, andResponseFinalizeBlock.consensus_param_updatesas a result of executing the block.- The values for
ResponseFinalizeBlock.validator_updates, orResponseFinalizeBlock.consensus_param_updatesmay be empty. In this case, CometBFT will keep the current values. ResponseFinalizeBlock.validator_updates, triggered by blockH, affect validation for blocksH+1,H+2, andH+3. Heights following a validator update are affected in the following way:- Height
H+1:NextValidatorsHashincludes the newvalidator_updatesvalue. - Height
H+2: The validator set change takes effect andValidatorsHashis updated. - Height
H+3:*_last_commitfields inPrepareProposal,ProcessProposal, andFinalizeBlocknow include the altered validator set.
- Height
ResponseFinalizeBlock.consensus_param_updatesreturned for blockHapply to the consensus params for blockH+1. For more information on the consensus parameters, see the consensus parameters section.
- The values for
ResponseFinalizeBlock.app_hashcontains an (optional) Merkle root hash of the application state.ResponseFinalizeBlock.app_hashis included as theHeader.AppHashin the next block.ResponseFinalizeBlock.app_hashmay also be empty or hard-coded, but MUST be deterministic - it must not be a function of anything that did not come from the parameters ofRequestFinalizeBlockand the previous committed state.
- Later calls to
Querycan return proofs about the application state anchored in this Merkle root hash. - The implementation of
FinalizeBlockMUST be deterministic, since it is making the Application’s state evolve in the context of state machine replication. - Currently, CometBFT will fill up all fields in
RequestFinalizeBlock, even if they were already passed on to the Application viaRequestPrepareProposalorRequestProcessProposal. - When calling
FinalizeBlockwith a block, the consensus algorithm run by CometBFT guarantees that at least one non-byzantine validator has runProcessProposalon that block.
When does CometBFT call FinalizeBlock?
When a node p is in consensus height h, and p receives
- the Proposal message with block v for a round r, along with all its block parts, from q, which is the proposer of round r, height h,
Precommitmessages from 2f + 1 validators’ voting power for round r, height h, precommitting the same block id(v),
- p persists v as the decision for height h.
- p’s CometBFT calls
RequestFinalizeBlockwith v’s data. The call is synchronous. - p’s Application executes block v.
- p’s Application calculates and returns the AppHash, along with a list containing the outputs of each of the transactions executed.
- p’s CometBFT hashes all the transaction outputs and stores it in ResultHash.
- p’s CometBFT persists the transaction outputs, AppHash, and ResultsHash.
- p’s CometBFT locks the mempool — no calls to
CheckTxon new transactions. - p’s CometBFT calls
RequestCommitto instruct the Application to persist its state. - p’s CometBFT, optionally, re-checks all outstanding transactions in the mempool against the newly persisted Application state.
- p’s CometBFT unlocks the mempool — newly received transactions can now be checked.
- p starts consensus for height h+1, round 0
Data Types existing in ABCI
Most of the data structures used in ABCI are shared common data structures. In certain cases, ABCI uses different data structures which are documented here:Validator
-
Fields:
-
Usage:
- Validator identified by address
- Used as part of
VoteInfowithinCommitInfo(used inProcessProposalandFinalizeBlock), andExtendedCommitInfo(used inPrepareProposal). - Does not include PubKey to avoid sending potentially large quantum pubkeys over the ABCI
ValidatorUpdate
-
Fields:
-
Usage:
- Validator identified by PubKey
- Used to tell CometBFT to update the validator set
Misbehavior
-
Fields:
MisbehaviorType
-
Fields
MisbehaviorType is an enum with the listed fields:
ConsensusParams
-
Fields:
ProofOps
-
Fields:
ProofOp
-
Fields:
Snapshot
-
Fields:
-
Usage:
- Used for state sync snapshots, see the state sync section for details.
- A snapshot is considered identical across nodes only if all fields are equal (including
Metadata). Chunks may be retrieved from all nodes that have the same snapshot. - When sent across the network, a snapshot message can be at most 4 MB.
Data types introduced or modified in ABCI++
VoteInfo
-
Fields:
-
Usage:
- Indicates whether a validator signed the last block, allowing for rewards based on validator availability.
- This information is typically extracted from a proposed or decided block.
ExtendedVoteInfo
-
Fields:
-
Usage:
- Indicates whether a validator signed the last block, allowing for rewards based on validator availability.
- This information is extracted from CometBFT’s data structures in the local process.
vote_extensioncontains the sending validator’s vote extension, whose signature was verified by CometBFT. It can be empty.extension_signatureis the signature of the vote extension, which was verified verified by CometBFT. This way, we expose the signature to the application for further processing or verification.
CommitInfo
-
Fields:
-
Notes
- The
VoteInfoinvotesare ordered by the voting power of the validators (descending order, highest to lowest voting power). - CometBFT guarantees the
votesordering through its logic to update the validator set in which, in the end, the validators are sorted (descending) by their voting power. - The ordering is also persisted when a validator set is saved in the store.
- The validator set is loaded from the store when building the
CommitInfo, ensuring order is maintained from the persisted validator set.
- The
ExtendedCommitInfo
-
Fields:
-
Notes
- The
ExtendedVoteInfoinvotesare ordered by the voting power of the validators (descending order, highest to lowest voting power). - CometBFT guarantees the
votesordering through its logic to update the validator set in which, in the end, the validators are sorted (descending) by their voting power. - The ordering is also persisted when a validator set is saved in the store.
- The validator set is loaded from the store when building the
ExtendedCommitInfo, ensuring order is maintained from the persisted validator set.
- The
ExecTxResult
-
Fields:
ProposalStatus
- Usage:
- Used within the ProcessProposal response.
- If
StatusisUNKNOWN, a problem happened in the Application. CometBFT will assume the application is faulty and crash. - If
StatusisACCEPT, the consensus algorithm accepts the proposal and will issue a Prevote message for it. - If
StatusisREJECT, the consensus algorithm rejects the proposal and will issue a Prevote fornilinstead.
- If
- Used within the ProcessProposal response.
VerifyStatus
- Usage:
- Used within the VerifyVoteExtension response.
- If
StatusisUNKNOWN, a problem happened in the Application. CometBFT will assume the application is faulty and crash. - If
StatusisACCEPT, the consensus algorithm will accept the vote as valid. - If
StatusisREJECT, the consensus algorithm will reject the vote as invalid.
- If
- Used within the VerifyVoteExtension response.