single

Single Sequencer

The single sequencer is a component of the Evolve framework that handles transaction ordering and batch submission to a Data Availability (DA) layer. It provides a reliable way to sequence transactions for applications via a designed node called the sequencer.

Overview

The sequencer receives transactions from clients, batches them together, and submits these batches to a Data Availability layer. It maintains transaction and batch queues, handles recovery from crashes, and provides verification mechanisms for batches.

flowchart LR
    Client["Client"] --> Sequencer
    Sequencer --> DA["DA Layer"]
    Sequencer <--> Database

Components

Sequencer

The main component that orchestrates the entire sequencing process. It:

• Receives transactions from clients
• Maintains transaction and batch queues
• Periodically creates and submits batches to the DA layer
• Handles recovery from crashes
• Provides verification mechanisms for batches

TransactionQueue

Manages the queue of pending transactions:

• Stores transactions in memory and in the database
• Provides methods to add transactions and extract batches
• Handles recovery of transactions from the database after a crash

BatchQueue

Manages the queue of batches:

• Stores batches in memory and in the database
• Provides methods to add and retrieve batches
• Handles recovery of batches from the database after a crash

DAClient

Handles communication with the Data Availability layer:

• Submits batches to the DA layer
• Retrieves batch status from the DA layer

Flow of Calls

Initialization Flow

flowchart TD
    A["NewSequencer()"] --> B["LoadLastBatchHashFromDB()"]
    B --> C["LoadSeenBatchesFromDB()"]
    C --> D["Load Transaction Queue from DB"]
    D --> E["Load BatchQueue from DB"]
    E --> F["Start batch submission loop"]

Transaction Submission Flow

flowchart TD
    A["SubmitBatchTxs()"] --> B["Validate ID"]
    B --> C["AddTransaction to Queue"]
    C --> D["Store in DB"]

Batch Creation and Submission Flow

flowchart TD
    A["batchSubmissionLoop()"] --> B["publishBatch()"]
    B --> C["GetNextBatch from TxQueue"]
    C --> D["If batch not empty"]
    D --> E["submitBatchToDA"]
    E --> F["Add to BatchQueue"]

Batch Retrieval Flow

flowchart TD
    A["GetNextBatch()"] --> B["Validate ID"]
    B --> C["Check batch hash match"]
    C --> D["If match or both nil"]
    D --> E["Get batch from BatchQueue"]
    E --> F["Update last batch hash"]

Batch Verification Flow

flowchart TD
    A["VerifyBatch()"] --> B["Validate ID"]
    B --> C["Check if batch hash in seen batches map"]
    C --> D["Return status"]

Database Layout

The single sequencer uses a key-value database to store transactions, batches, and metadata. Here's the layout of the database:

Keys

Key Details

Last Batch Hash Key (l)

• Stores the hash of the last processed batch
• Used for recovery after a crash
• Value: Raw bytes of the hash

Seen Batch Hash Keys (seen:<hex_encoded_hash>)

• Marks batches that have been seen and processed
• Used for batch verification
• Value: 1 (presence indicates the batch has been seen)

Batch Keys (<hex_encoded_hash>)

• Stores the actual batch data
• Key is the hex-encoded hash of the batch
• Value: Protobuf-encoded batch data

Transaction Keys (tx:<tx_hash>)

• Stores individual transactions
• Key is prefixed with tx: followed by the SHA-256 hash of the transaction bytes
• Value: Raw transaction bytes

Recovery Mechanism

The single sequencer implements a robust recovery mechanism to handle crashes:

1. On startup, it loads the last batch hash from the database
2. It loads all seen batch hashes into memory
3. It loads all pending transactions from the database into the transaction queue
4. It loads all pending batches from the database into the batch queue
5. It resumes normal operation, continuing from where it left off

This ensures that no transactions are lost in case of a crash, and the sequencer can continue operating seamlessly.

Metrics

The sequencer exposes the following metrics:

These metrics can be used to monitor the health and performance of the sequencer.

Usage

To create a new single sequencer:

seq, err := NewSequencer(
    context.Background(),
    logger,
    database,
    daLayer,
    namespace,
    Id,
    batchTime,
    metrics,
)

To submit transactions:

response, err := seq.SubmitBatchTxs(
    context.Background(),
    coresequencer.SubmitBatchTxsRequest{
        Id: Id,
        Batch: &coresequencer.Batch{
            Transactions: [][]byte{transaction},
        },
    },
)

To get the next batch:

response, err := seq.GetNextBatch(
    context.Background(),
    coresequencer.GetNextBatchRequest{
        Id: Id,
        LastBatchHash: lastHash,
    },
)

To verify a batch:

response, err := seq.VerifyBatch(
    context.Background(),
    coresequencer.VerifyBatchRequest{
        Id: Id,
        BatchHash: batchHash,
    },
)