feat(specs): Dencun P2P & Engine API changes (#8852)

* Dencun P2P & Engine API changes

* Reword pre/post Ecotone to be explicit

* Update specs/derivation.md
Co-authored-by: protolambda <proto@protolambda.com>

* Update specs/derivation.md
Co-authored-by: protolambda <proto@protolambda.com>

* Update specs to keep pre-Ectone logic

* Update specs/rollup-node-p2p.md
Co-authored-by: protolambda <proto@protolambda.com>

* Update specs/rollup-node-p2p.md
Co-authored-by: protolambda <proto@protolambda.com>

* Update specs/rollup-node-p2p.md
Co-authored-by: protolambda <proto@protolambda.com>

* Fix topic withdrawal rejections

---------
Co-authored-by: protolambda <proto@protolambda.com>

specs/derivation.md

@@ -77,6 +77,8 @@
     - [Payload Attributes Derivation](#payload-attributes-derivation)
     - [Engine Queue](#engine-queue)
       - [Engine API usage](#engine-api-usage)
+        - [Bedrock, Canyon, Delta: API Usage](#bedrock-canyon-delta-api-usage)
+        - [Ecotone: API Usage](#ecotone-api-usage)
       - [Forkchoice synchronization](#forkchoice-synchronization)
       - [L1-consolidation: payload attributes matching](#l1-consolidation-payload-attributes-matching)
       - [L1-sync: payload attributes processing](#l1-sync-payload-attributes-processing)
@@ -798,24 +800,55 @@ To interact with the engine, the [execution engine API][exec-engine] is used, wi
 
 [exec-engine]: exec-engine.md
 
+##### Bedrock, Canyon, Delta: API Usage
+
 - [`engine_forkchoiceUpdatedV2`] — updates the forkchoice (i.e. the chain head) to `headBlockHash` if different, and
   instructs the engine to start building an execution payload if the payload attributes parameter is not `null`.
 - [`engine_getPayloadV2`] — retrieves a previously requested execution payload build.
 - [`engine_newPayloadV2`] — executes an execution payload to create a block.
 
-The current version of `op-node` uses the `v2` RPC methods from the engine API, whereas prior versions used the `v1`
-equivalents. The `v2` methods are backwards compatible with `v1` payloads but support Shanghai.
+##### Ecotone: API Usage
+
+- [`engine_forkchoiceUpdatedV3`] — updates the forkchoice (i.e. the chain head) to `headBlockHash` if different, and
+  instructs the engine to start building an execution payload if the payload attributes parameter is not `null`.
+- [`engine_getPayloadV3`] — retrieves a previously requested execution payload build.
+- `engine_newPayload`
+  - [`engine_newPayloadV2`] — executes a Bedrock/Canyon/Delta execution payload to create a block.
+  - [`engine_newPayloadV3`] — executes an Ecotone execution payload to create a block.
+
+The current version of `op-node` uses the `v3` Engine API RPC methods as well as `engine_newPayloadV2`, due to
+`engine_newPayloadV3` only supporting Ecotone execution payloads. Both `engine_forkchoiceUpdatedV3` and
+`engine_getPayloadV3` are backwards compatible with Bedrock, Canyon & Delta payloads.
+
+Prior versions of `op-node` used `v2` and `v1` methods.
 
 [`engine_forkchoiceUpdatedV2`]: exec-engine.md#engine_forkchoiceupdatedv2
+[`engine_forkchoiceUpdatedV3`]: exec-engine.md#engine_forkchoiceupdatedv3
 [`engine_getPayloadV2`]: exec-engine.md#engine_getpayloadv2
+[`engine_getPayloadV3`]: exec-engine.md#engine_getpayloadv3
 [`engine_newPayloadV2`]: exec-engine.md#engine_newpayloadv2
+[`engine_newPayloadV3`]: exec-engine.md#engine_newpayloadv3
+
+The execution payload is an object of type [`ExecutionPayloadV3`][eth-payload].
 
-The execution payload is an object of type [`ExecutionPayloadV2`][eth-payload].
+[eth-payload]: https://github.com/ethereum/execution-apis/blob/main/src/engine/cancun.md
 
-[eth-payload]: https://github.com/ethereum/execution-apis/blob/main/src/engine/shanghai.md#payloadattributesv2
+The `ExecutionPayload` has the following requirements:
 
-With V2 of the execution payload, before Canyon the withdrawals field is required to be nil. After Canyon the
-withdrawals field is required to be non-nil. The op-node should set the withdrawals field to be an empty list.
+- Bedrock
+  - The withdrawals field MUST be nil
+  - The blob gas used field MUST be nil
+  - The blob gas limit field MUST be nil
+- Canyon, Delta
+  - The withdrawals field MUST be non-nil
+  - The withdrawals field MUST be an empty list
+  - The blob gas used field MUST be nil
+  - The blob gas limit field MUST be nil
+- Ecotone
+  - The withdrawals field MUST be non-nil
+  - The withdrawals field MUST be an empty list
+  - The blob gas used field MUST be 0
+  - The blob gas limit field MUST be 0
 
 #### Forkchoice synchronization
 
@@ -828,7 +861,7 @@ This synchronization may happen when:
 - A successful consolidation of unsafe L2 blocks: updating the "safe" L2 block.
 - The first thing after a derivation pipeline reset, to ensure a consistent execution engine forkchoice state.
 
-The new forkchoice state is applied with `engine_forkchoiceUpdatedV2`.
+The new forkchoice state is applied by calling [fork choice updated](#engine-api-usage) on the engine API.
 On forkchoice-state validity errors the derivation pipeline must be reset to recover to consistent state.
 
 #### L1-consolidation: payload attributes matching
@@ -874,11 +907,13 @@ Engine][exec-engine-comm] section.
 
 The payload attributes are then processed with a sequence of:
 
-- `engine_forkchoiceUpdatedV2` with current forkchoice state of the stage, and the attributes to start block building.
+- [Engine: Fork choice updated](#engine-api-usage) with current forkchoice state of the stage, and the attributes to
+start block building.
   - Non-deterministic sources, like the tx-pool, must be disabled to reconstruct the expected block.
-- `engine_getPayload` to retrieve the payload, by the payload-ID in the result of the previous step.
-- `engine_newPayload` to import the new payload into the execution engine.
-- `engine_forkchoiceUpdatedV2` to make the new payload canonical,
+- [Engine: Get Payload](#engine-api-usage) to retrieve the payload, by the payload-ID in the result of the previous
+step.
+- [Engine: New Payload](#engine-api-usage) to import the new payload into the execution engine.
+- [Engine: Fork Choice Updated](#engine-api-usage) to make the new payload canonical,
    now with a change of both `safe` and `unsafe` fields to refer to the payload, and no payload attributes.
 
 Engine API Error handling:
@@ -907,8 +942,12 @@ To process unsafe payloads, the payload must:
 
 The payload is then processed with a sequence of:
 
-- `engine_newPayloadV2`: process the payload. It does not become canonical yet.
-- `engine_forkchoiceUpdatedV2`: make the payload the canonical unsafe L2 head, and keep the safe/finalized L2 heads.
+- Bedrock/Canyon/Delta Payloads
+  - `engine_newPayloadV2`: process the payload. It does not become canonical yet.
+  - `engine_forkchoiceUpdatedV2`: make the payload the canonical unsafe L2 head, and keep the safe/finalized L2 heads.
+- Ecotone Payloads
+  - `engine_newPayloadV3`: process the payload. It does not become canonical yet.
+  - `engine_forkchoiceUpdatedV3`: make the payload the canonical unsafe L2 head, and keep the safe/finalized L2 heads.
 
 Engine API Error handling:
 

specs/exec-engine.md

@@ -17,8 +17,13 @@
 - [Engine API](#engine-api)
   - [`engine_forkchoiceUpdatedV2`](#engine_forkchoiceupdatedv2)
     - [Extended PayloadAttributesV2](#extended-payloadattributesv2)
+  - [`engine_forkchoiceUpdatedV3`](#engine_forkchoiceupdatedv3)
+    - [Extended PayloadAttributesV3](#extended-payloadattributesv3)
   - [`engine_newPayloadV2`](#engine_newpayloadv2)
+  - [`engine_newPayloadV3`](#engine_newpayloadv3)
   - [`engine_getPayloadV2`](#engine_getpayloadv2)
+  - [`engine_getPayloadV3`](#engine_getpayloadv3)
+    - [Extended Response](#extended-response)
   - [`engine_signalSuperchainV1`](#engine_signalsuperchainv1)
 - [Networking](#networking)
 - [Sync](#sync)
@@ -184,11 +189,6 @@ ways:
 
 ## Engine API
 
-<!--
-*Note: the [Engine API][l1-api-spec] is in alpha, `v1.0.0-alpha.5`.
-There may be subtle tweaks, beta starts in a few weeks*
--->
-
 ### `engine_forkchoiceUpdatedV2`
 
 This updates which L2 blocks the engine considers to be canonical (`forkchoiceState` argument),
@@ -252,16 +252,78 @@ If not specified as rollup, a `STATUS_INVALID` is returned.
 
 [rollup-driver]: rollup-node.md
 
+### `engine_forkchoiceUpdatedV3`
+
+See [`engine_forkchoiceUpdatedV2`](#engine_forkchoiceUpdatedV2) for a description of the forkchoice updated method.
+
+To support rollup functionality, one backwards-compatible change is introduced
+to [`engine_forkchoiceUpdatedV3`][engine_forkchoiceUpdatedV3]: the extended `PayloadAttributesV3`
+
+#### Extended PayloadAttributesV3
+
+[`PayloadAttributesV3`][PayloadAttributesV3] is extended to:
+
+```js
+PayloadAttributesV3: {
+    timestamp: QUANTITY
+    random: DATA (32 bytes)
+    suggestedFeeRecipient: DATA (20 bytes)
+    withdrawals: array of WithdrawalV1
+    parentBeaconBlockRoot: DATA (32 bytes)
+    transactions: array of DATA
+    noTxPool: bool
+    gasLimit: QUANTITY or null
+}
+```
+
+The requirements of this object are the same as extended [`PayloadAttributesV2`][#extended-payloadattributesv2] with
+the addition of `parentBeaconBlockRoot` which is the parent beacon block root from the L1 origin block of the L2 block.
+
+`parentBeaconBlockRoot` must be nil for Bedrock/Canyon/Delta payloads, but must be set to the L1
+origin `parentBeaconBlockRoot`.
+
 ### `engine_newPayloadV2`
 
 No modifications to [`engine_newPayloadV2`][engine_newPayloadV2].
 Applies a L2 block to the engine state.
 
+### `engine_newPayloadV3`
+
+[`engine_newPayloadV3`][engine_newPayloadV3] applies an Ecotone L2 block to the engine state. There are no
+modifications to this API. The additional parameters should be set as follows:
+
+- `expectedBlobVersionedHashes` MUST be an empty array.
+- `parentBeaconBlockRoot` MUST be the parent beacon block root from the L1 origin block of the L2 block.
+
 ### `engine_getPayloadV2`
 
 No modifications to [`engine_getPayloadV2`][engine_getPayloadV2].
 Retrieves a payload by ID, prepared by `engine_forkchoiceUpdatedV2` when called with `payloadAttributes`.
 
+### `engine_getPayloadV3`
+
+[`engine_getPayloadV3`][engine_getPayloadV3] retrieves a payload by ID, prepared by `engine_forkchoiceUpdatedV3`
+when called with `payloadAttributes`.
+
+#### Extended Response
+
+The [response][GetPayloadV3Response] is extended to:
+
+```js
+{
+    executionPayload: ExecutionPayload
+    blockValue: QUANTITY
+    blobsBundle: BlobsBundle
+    shouldOverrideBuilder: BOOLEAN
+    parentBeaconBlockRoot: DATA (32 bytes)
+}
+```
+
+[GetPayloadV3Response]: (https://github.com/ethereum/execution-apis/blob/main/src/engine/cancun.md#response-2)
+
+For Bedrock and Canyon `parentBeaconBlockRoot` MUST be nil and in Ecotone it MUST be set to the parentBeaconBlockRoot
+from the L1 Origin block of the L2 block.
+
 ### `engine_signalSuperchainV1`
 
 Optional extension to the Engine API. Signals superchain information to the Engine:
@@ -337,9 +399,14 @@ as the engine implementation can sync state faster through methods like [snap-sy
 ### Happy-path sync
 
 1. The rollup node informs the engine of the L2 chain head, unconditionally (part of regular node operation):
-   - [`engine_newPayloadV2`][engine_newPayloadV2] is called with latest L2 block received from P2P.
-   - [`engine_forkchoiceUpdatedV2`][engine_forkchoiceUpdatedV2] is called with the current
-     `unsafe`/`safe`/`finalized` L2 block hashes.
+   - Bedrock / Canyon / Delta Payloads
+      - [`engine_newPayloadV2`][engine_newPayloadV2] is called with latest L2 block received from P2P.
+      - [`engine_forkchoiceUpdatedV2`][engine_forkchoiceUpdatedV2] is called with the current
+        `unsafe`/`safe`/`finalized` L2 block hashes.
+   - Ecotone Payloads
+      - [`engine_newPayloadV3`][engine_newPayloadV3] is called with latest L2 block received from P2P.
+      - [`engine_forkchoiceUpdatedV3`][engine_forkchoiceUpdatedV3] is called with the current
+        `unsafe`/`safe`/`finalized` L2 block hashes.
 2. The engine requests headers from peers, in reverse till the parent hash matches the local chain
 3. The engine catches up:
     a) A form of state sync is activated towards the finalized or head block hash
@@ -354,7 +421,7 @@ the operation within the engine is the exact same as with L1 (although with an E
 2. The rollup node maintains latest head from engine (poll `eth_getBlockByNumber` and/or maintain a header subscription)
 3. The rollup node activates sync if the engine is out of sync but not syncing through P2P (`eth_syncing`)
 4. The rollup node inserts blocks, derived from L1, one by one, potentially adapting to L1 reorg(s),
-   as outlined in the [rollup node spec] (`engine_forkchoiceUpdatedV2`, `engine_newPayloadV2`)
+   as outlined in the [rollup node spec].
 
 [rollup node spec]: rollup-node.md
 
@@ -397,10 +464,14 @@ For the Ecotone upgrade, this entails that:
 [eip-2718-transactions]: https://eips.ethereum.org/EIPS/eip-2718#transactions
 [exec-api-data]: https://github.com/ethereum/execution-apis/blob/769c53c94c4e487337ad0edea9ee0dce49c79bfa/src/engine/specification.md#structures
 [l1-api-spec]: https://github.com/ethereum/execution-apis/blob/769c53c94c4e487337ad0edea9ee0dce49c79bfa/src/engine/specification.md
+[PayloadAttributesV3]: https://github.com/ethereum/execution-apis/blob/cea7eeb642052f4c2e03449dc48296def4aafc24/src/engine/cancun.md#payloadattributesv3
 [PayloadAttributesV2]: https://github.com/ethereum/execution-apis/blob/584905270d8ad665718058060267061ecfd79ca5/src/engine/shanghai.md#PayloadAttributesV2
 [ExecutionPayloadV1]: https://github.com/ethereum/execution-apis/blob/769c53c94c4e487337ad0edea9ee0dce49c79bfa/src/engine/specification.md#ExecutionPayloadV1
+[engine_forkchoiceUpdatedV3]: https://github.com/ethereum/execution-apis/blob/cea7eeb642052f4c2e03449dc48296def4aafc24/src/engine/cancun.md#engine_forkchoiceupdatedv3
 [engine_forkchoiceUpdatedV2]: https://github.com/ethereum/execution-apis/blob/584905270d8ad665718058060267061ecfd79ca5/src/engine/shanghai.md#engine_forkchoiceupdatedv2
 [engine_newPayloadV2]: https://github.com/ethereum/execution-apis/blob/584905270d8ad665718058060267061ecfd79ca5/src/engine/shanghai.md#engine_newpayloadv2
+[engine_newPayloadV3]: https://github.com/ethereum/execution-apis/blob/cea7eeb642052f4c2e03449dc48296def4aafc24/src/engine/cancun.md#engine_newpayloadv3
 [engine_getPayloadV2]: https://github.com/ethereum/execution-apis/blob/584905270d8ad665718058060267061ecfd79ca5/src/engine/shanghai.md#engine_getpayloadv2
+[engine_getPayloadV3]: https://github.com/ethereum/execution-apis/blob/a0d03086564ab1838b462befbc083f873dcf0c0f/src/engine/cancun.md#engine_getpayloadv3
 [HEX value encoding]: https://eth.wiki/json-rpc/API#hex-value-encoding
 [JSON-RPC-API]: https://github.com/ethereum/execution-apis

specs/overview.md

@@ -185,14 +185,17 @@ For each iteration of the block derivation loop described above, the rollup driv
 object and send it to the execution engine. The execution engine will then convert the payload attributes object into a
 block, and add it to the chain. The basic sequence of the rollup driver is as follows:
 
-1. Call `engine_forkchoiceUpdatedV2` with the payload attributes object. We'll skip over the details of the fork choice
-state parameter for now - just know that one of its fields is the L2 chain's `headBlockHash`, and that it is set to the
-block hash of the tip of the L2 chain. The Engine API returns a payload ID.
-2. Call `engine_getPayloadV2` with the payload ID returned in step 1. The engine API returns a payload object that
-includes a block hash as one of its fields.
-3. Call `engine_newPayloadV2` with the payload returned in step 2.
-4. Call `engine_forkchoiceUpdatedV2` with the fork choice parameter's `headBlockHash` set to the block hash returned in
-step 2. The tip of the L2 chain is now the block created in step 1.
+1. Call [fork choice updated][EngineAPIVersion] with the payload attributes object. We'll skip over the details of the
+fork choice state parameter for now - just know that one of its fields is the L2 chain's `headBlockHash`, and that it
+is set to the block hash of the tip of the L2 chain. The Engine API returns a payload ID.
+2. Call [get payload][EngineAPIVersion] with the payload ID returned in step 1. The engine API returns a payload object
+that includes a block hash as one of its fields.
+3. Call [new payload][EngineAPIVersion] with the payload returned in step 2. (Ectone blocks, must use V3, pre-Ecotone
+blocks MUST use the V2 version)
+4. Call [fork choice updated][EngineAPIVersion] with the fork choice parameter's `headBlockHash` set to the block hash
+returned in step 2. The tip of the L2 chain is now the block created in step 1.
+
+[EngineAPIVersion]: derivation.md#engine-api-usage
 
 The swimlane diagram below visualizes the process:
 

specs/rollup-node-p2p.md

@@ -53,11 +53,12 @@ and are adopted by several other blockchains, most notably the [L1 consensus lay
 - [Gossip Topics](#gossip-topics)
   - [`blocksv1`](#blocksv1)
   - [`blocksv2`](#blocksv2)
-    - [Block encoding](#block-encoding)
-    - [Block signatures](#block-signatures)
-    - [Block validation](#block-validation)
-      - [Block processing](#block-processing)
-      - [Block topic scoring parameters](#block-topic-scoring-parameters)
+  - [`blocksv3`](#blocksv3)
+  - [Block encoding](#block-encoding)
+  - [Block signatures](#block-signatures)
+  - [Block validation](#block-validation)
+    - [Block processing](#block-processing)
+    - [Block topic scoring parameters](#block-topic-scoring-parameters)
 - [Req-Resp](#req-resp)
   - [`payload_by_number`](#payload_by_number)
 
@@ -248,7 +249,7 @@ The extended validator emits one of the following validation signals:
 
 ## Gossip Topics
 
-There are two topics for distributing blocks to other nodes faster than proxying through L1 would. These are:
+There are three topics for distributing blocks to other nodes faster than proxying through L1 would. These are:
 
 ### `blocksv1`
 
@@ -256,30 +257,39 @@ Pre-Canyon/Shanghai blocks are broadcast on `/optimism/<chainId>/0/blocks`.
 
 ### `blocksv2`
 
-Post-Canyon/Shanghai blocks are broadcast on `/optimism/<chainId>/1/blocks`.
+Canyon/Delta blocks are broadcast on `/optimism/<chainId>/1/blocks`.
 
-#### Block encoding
+### `blocksv3`
+
+Ecotone blocks are broadcast on `/optimism/<chainId>/2/blocks`.
+
+### Block encoding
 
 A block is structured as the concatenation of:
 
-- `signature`: A `secp256k1` signature, always 65 bytes, `r (uint256), s (uint256), y_parity (uint8)`
-- `payload`: A SSZ-encoded `ExecutionPayload`, always the remaining bytes.
+- V1 and V2 topics
+  - `signature`: A `secp256k1` signature, always 65 bytes, `r (uint256), s (uint256), y_parity (uint8)`
+  - `payload`: A SSZ-encoded `ExecutionPayload`, always the remaining bytes.
+- V3 topic
+  - `signature`: A `secp256k1` signature, always 65 bytes, `r (uint256), s (uint256), y_parity (uint8)`
+  - `parentBeaconBlockRoot`: L1 origin parent beacon block root, always 32 bytes
+  - `payload`: A SSZ-encoded `ExecutionPayload`, always the remaining bytes.
 
-The topic uses Snappy block-compression (i.e. no snappy frames):
+All topics use Snappy block-compression (i.e. no snappy frames):
 the above needs to be compressed after encoding, and decompressed before decoding.
 
-#### Block signatures
+### Block signatures
 
 The `signature` is a `secp256k1` signature, and signs over a message:
 `keccak256(domain ++ chain_id ++ payload_hash)`, where:
 
 - `domain` is 32 bytes, reserved for message types and versioning info. All zero for this signature.
 - `chain_id` is a big-endian encoded `uint256`.
-- `payload_hash` is `keccak256(payload)`, where `payload` is the SSZ-encoded `ExecutionPayload`
+- `payload_hash` is `keccak256(payload)`, where `payload` is the remaining bytes of the payload.
 
 The `secp256k1` signature must have `y_parity = 1 or 0`, the `chain_id` is already signed over.
 
-#### Block validation
+### Block validation
 
 An [extended-validator] checks the incoming messages as follows, in order of operation:
 
@@ -290,8 +300,14 @@ An [extended-validator] checks the incoming messages as follows, in order of ope
 - `[REJECT]` if the `payload.timestamp` is more than 5 seconds into the future
 - `[REJECT]` if the `block_hash` in the `payload` is not valid
 - `[REJECT]` if the block is on the V1 topic and has withdrawals
-- `[REJECT]` if the block is on the V2 topic and does not have withdrawals
-- `[REJECT]` if the block is on the V2 topic and has a non-zero amount of withdrawals
+- `[REJECT]` if the block is on the V1 topic and has a withdrawals list
+- `[REJECT]` if the block is on a topic >= V2 and does not have an empty withdrawals list
+- `[REJECT]` if the block is on a topic <= V2 and has a blob gas-used value set
+- `[REJECT]` if the block is on a topic <= V2 and has an excess blob gas value set
+- `[REJECT]` if the block is on a topic >= V3 and has a blob gas-used value that is not zero
+- `[REJECT]` if the block is on a topic >= V3 and has an excess blob gas value that is not zero
+- `[REJECT]` if the block is on a topic <= V2 and the parent beacon block root is not nil
+- `[REJECT]` if the block is on a topic >= V3 and the parent beacon block root is nil
 - `[REJECT]` if more than 5 different blocks have been seen with the same block height
 - `[IGNORE]` if the block has already been seen
 - `[REJECT]` if the signature by the sequencer is not valid
@@ -305,15 +321,15 @@ Note that blocks that a block may still be propagated even if the L1 already con
 The local L1 view of the node may be wrong, and the time and signature validation will prevent spam.
 Hence, calling into the execution engine with a block lookup every propagation step is not worth the added delay.
 
-##### Block processing
+#### Block processing
 
 A node may apply the block to their local engine ahead of L1 availability, if it ensures that:
 
 - The application of the block is reversible, in case of a conflict with delayed L1 information
 - The subsequent forkchoice-update ensures this block is recognized as "unsafe"
-  (see [`engine_forkchoiceUpdatedV2`](./exec-engine.md#engine_forkchoiceupdatedv2))
+  (see [fork choice updated](derivation.md#engine-api-usage))
 
-##### Block topic scoring parameters
+#### Block topic scoring parameters
 
 TODO: GossipSub per-topic scoring to fine-tune incentives for ideal propagation delay and bandwidth usage.