1. Launch [a consensus monitor](https://github.com/ralexstokes/ethereum_consensus_monitor) instance attached to the network
-[x] do a wait for epoch finalization
1. Optionally block until the Beacon nodes finalize an epoch (i.e. finalized_epoch > 0)
-[] assert that finalization epoch > 0
-[x] wait for CL genesis to occur before adding forkmon - DESCOPED as I don't see the bug
For much more detailed information about how the merge works in Ethereum testnets, see [this document](https://notes.ethereum.org/@ExXcnR0-SJGthjz1dwkA1A/H1MSKgm3F).
-[x] Module IO (this is blocked on Startosis Args working)
-[x] forkmon (this is blocked on CL clients running)
Quickstart
-[x] prometheus (this is blocked on CL clients running)
----------
-[x] grafana (this is blocked on prometheus running)
-[x] grafana needs an upload files endpoint in Startosis
1.[Install Docker if you haven't done so already][docker-installation]
-[x] static_files package
1.[Install the Kurtosis CLI, or upgrade it to the latest version if it's already installed][kurtosis-cli-installation]
-[x] testnet_verifier (this is blocked on CL/EL clients running)
1. Ensure your Docker engine is running:
-[x] transaction_spammer (this is blocked on EL clients running)
```bash
-[x] participant_network/participant_network DEMO
docker image ls
-[x] has most data generation things, needs to start EL/CL clients
```
-[x] needs upload files to be implemented
1. Create a file in your home directory `eth2-module-params.json` with the following contents:
-[x] need to fill in the dictionary with all el / cl types
-[x] participant_network/participant
```yaml
-[x] pure POJO should be quick to implement NO BLOCKERS
global_client_log_level:"info"
-[x] mev_boost participant_network/mev_boost NO BLOCKERS - removed some attributes that aren't used
```
-[x] mev_boost_context pure POJO NO BLOCKERS
-[x] mev_boost_launcher NO BLOCKERS
1. Execute the module, passing in the params from the file:
-[x] participant_network/pre_launch_data_generator (the only missing piece here is remove_service)
-[x] participant_network/el (requires facts and waits)
-[x] besu
Management
-[x] facts and waits + private_ip_address_placeholder
----------
-[x] framework
-[x] facts could use more waiting
Kurtosis will create a new enclave to house the services of the Ethereum network. [This page][using-the-cli] contains documentation for managing the created enclave & viewing detailed information about it.
-[x] erigon
-[x] facts and waits + private_ip_address_placeholder
Configuration
-[x] framework
-------------
-[x] geth DEMO
-[x] facts and waits + private_ip_address_placeholder
To configure the module behaviour, you can modify your `eth2-module-params.yaml` file. The full YAML schema that can be passed in is as follows with the defaults ([from here](https://github.com/kurtosis-tech/eth2-module/blob/master/types.proto) provided:
-[x] framework TESTED
-[x] nethermind
Note: Following an update starting the network post-merge, `erigon`, `nimbus` and `prysm` clients don't work anymore. Fixes are tracked in the following Github issues:
-[x] facts and waits + private_ip_address_placeholder
- Prysm: [#11508][prysm-issue]
-[x] framework
- Nimbus: [#4193][nimbus-issue]
-[x] facts could use more waiting
- Erigon: [#154][erigon-issue]
-[x] el_client_context pure POJO NO BLOCKERS
-[x] el_client_launcher interface not necessary
<details>
-[x] el_availability_waiter - facts and waits - DESCOPED facts and waits will do this
<summary>Click to show all configuration options</summary>
-[x] el_rest_client/api_response_objects.go DESCOPED as facts will do this
-[x] el_rest_client/el_rest_client - facts and waits DESCOPED as facts will do this
<!-- Yes, it's weird that none of this is indented but it's intentional - indenting anything inside this "details" expandable will cause it to render weird" -->
-[x] participant_network/cl (requires facts and waits)
"preregistered_validator_keys_mnemonic":"giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete"
1. Rebuild and re-execute the module by running the following from the root of the repo:
```bash
kurtosis exec.--args"{}"
```
NOTE 1: You can change the value of the `--args` flag to pass in extra configuration to the module per the "Configuration" section above!
NOTE 2: The `--args` flag accepts JSON.
To get detailed information about the structure of the module, visit [the architecture docs](./docs/architecture.md).
When you're happy with your changes:
- Add an entry to `docs/changelog.md` under the `# TBD` header describing your changes (this is required for CI checks to pass!)
- Create a PR
- Add one of the maintainers of the repo as a "Review Request":
-`parithosh` (Ethereum)
-`gbouv` (Kurtosis)
-`h4ck3rk3y` (Kurtosis)
-`mieubrisse` (Kurtosis)
- Once everything works, merge!
## Known Bugs
`wait_for_epoch_finalization` - doesn't work as expected, as Starlark doesn't have ways to do assertions on facts just yet. The [issue](https://github.com/kurtosis-tech/eth2-module/issues/15) tracks this.
<!------------------------ Only links below here -------------------------------->
This repo is a Kurtosis module. To get general information on what a Kurtosis module is and how it works, visit [the Starlark documentation](https://docs.kurtosis.com/reference/starlark-introduction).
The overview of this particular module's operation is as follows:
1. Parse user parameters
1. Launch a network of Ethereum participants
1. Generate execution layer (EL) client config data
1. Launch EL clients
1. Wait for EL clients to start mining, such that all EL clients have a nonzero block number
1. Generate consensus layer (CL) client config data
The module has six main components, in accordance with the above operation:
1.[Main Function][main-function]
1.[Module I/O][module-io]
1.[Static Files][static-files]
1.[Participant Network][participant-network]
1.[Auxiliary Services][auxiliary-services]
1.[Merge Verification Logic][testnet-verifier]
[Main][main-function]
------------------------------------
The main function is the module's entrypoint, where parameters are received from the user, lower-level calls are made, and a response is returned.
[Module I/O][module-io]
-----------------------
This particular module has many configuration options (see the "Configuration" section in the README for the full list of values). These are passed in as a JSON-serialized string, and arrive to the module's main function via the `input_args` variable. The process of setting defaults, overriding them with the user's desired options, and validating the resulting config object requires some space in the codebase. All this logic happens inside the `module_io` directory, so you'll want to visit this directory if you want to:
- View or change parameters that the module can receive
- Change the default values of module parameters
- View or change the validation logic that the module applies to configuration parameters
- View or change the properties that the module passes back to the user after execution is complete
[Static Files][static-files]
----------------------------
Kurtosis modules can have static files that are made available inside the container during the module's operation. For this module, [the static files included][static-files] are various key files and config templates which get used during participant network operation.
[Participant Network][participant-network]
------------------------------------------
The participant network is the beating heart at the center of the module. The participant network code is responsible for:
1. Generating EL client config data
1. Starting the EL clients
1. Waiting until the EL clients have started mining
1. Generating CL client config data
1. Starting the CL clients
We'll explain these phases one by one.
### Generating EL client data
All EL clients require both a genesis file and a JWT secret. The exact format of the genesis file differs per client, so we first leverage [a Docker image containing tools for generating this genesis data][ethereum-genesis-generator] to create the actual files that the EL clients-to-be will need. This is accomplished by filling in several genesis generation config files found in [the `static_files` directory][static-files].
The generated output files then get stored in the Kurtosis enclave, ready for use when we start the EL clients. The information about these stored files is tracked in [the `el_genesis_data` struct](https://github.com/kurtosis-tech/eth2-module/blob/master/src/participant_network/prelaunch_data_generator/el_genesis/el_genesis_data.star).
### Starting EL clients
Next, we plug the generated genesis data [into EL client "launchers"](https://github.com/kurtosis-tech/eth2-module/tree/master/src/participant_network/el) to start a mining network of EL nodes. The launchers come with a `launch` function that consumes EL genesis data and produces information about the running EL client node. Running EL node information is represented by [an `el_client_context` struct](https://github.com/kurtosis-tech/eth2-module/blob/master/src/participant_network/el/el_client_context.star). Each EL client type has its own launcher (e.g. [Geth](https://github.com/kurtosis-tech/eth2-module/tree/master/src/participant_network/el/geth), [Besu](https://github.com/kurtosis-tech/eth2-module/tree/master/src/participant_network/el/besu)) because each EL client will require different environment variables and flags to be set when launching the client's container.
### Waiting until EL clients have started mining
Once we have a network of EL nodes started, we block until all EL clients have a block number > 0 to ensure that they are in fact working. After the nodes have started mining, we're ready to move on to adding the CL client network.
### Generating CL client data
CL clients, like EL clients, also have genesis and config files that they need. We use [the same Docker image with tools for generating genesis data][ethereum-genesis-generator] to create the necessary CL files, we provide the genesis generation config using templates in [the `static_files` directory][static-files], and we store the generated output files in the Kurtosis enclave in the same way as the EL client genesis files. Like with EL nodes, CL genesis data information is tracked in [the `cl_client_context` struct](https://github.com/kurtosis-tech/eth2-module/blob/master/src/participant_network/el/el_client_context.star).
The major difference with CL clients is that CL clients have validator keys, so we need to generate keys for the CL clients to use during validation. The generation happens using [the same genesis-generating Docker image][ethereum-genesis-generator], but via a call to [a different tool bundled inside the Docker image](https://github.com/protolambda/eth2-val-tools).
### Starting CL clients
Once CL genesis data and keys have been created, the CL client nodes are started via [the CL client launchers](https://github.com/kurtosis-tech/eth2-module/tree/master/src/participant_network/cl). Just as with EL clients:
- CL client launchers implement come with a `launch` method
- One CL client launcher exists per client type (e.g. [Nimbus](https://github.com/kurtosis-tech/eth2-module/tree/master/src/participant_network/cl/nimbus), [Lighthouse](https://github.com/kurtosis-tech/eth2-module/tree/master/src/participant_network/cl/lighthouse))
- Launched CL node information is tracked in [a `cl_client_context` struct](https://github.com/kurtosis-tech/eth2-module/blob/master/src/participant_network/cl/cl_client_context.star)
There are only two major difference between CL client and EL client launchers. First, the `cl_client_launcher.launch` method also consumes an `el_client_context`, because each CL client is connected in a 1:1 relationship with an EL client. Second, because CL clients have keys, the keystore files are passed in to the `launch` function as well.
Auxiliary Services
------------------
After the Ethereum network is up and running, this module starts several auxiliary containers to make it easier to work with the Ethereum network. At time of writing, these are:
-[Forkmon](https://github.com/kurtosis-tech/eth2-module/tree/master/src/forkmon), a "fork monitor" web UI for visualizing the CL clients' forks
-[Prometheus](https://github.com/kurtosis-tech/eth2-module/tree/master/src/prometheus) for collecting client node metrics
-[Grafana](https://github.com/kurtosis-tech/eth2-module/tree/master/src/grafana) for visualizing client node metrics
-[An ETH transaction spammer](https://github.com/kurtosis-tech/eth2-module/tree/master/src/transaction_spammer), which [has been forked off](https://github.com/kurtosis-tech/tx-fuzz) of [Marius' transaction spammer code](https://github.com/MariusVanDerWijden/tx-fuzz) so that it can run as a container
[Testnet Verifier][testnet-verifier]
------------------------------------
Once the Ethereum network is up and running, verification logic will be run to ensure that the Merge has happened successfully. This happens via [a testnet-verifying Docker image](https://github.com/ethereum/merge-testnet-verifier) that periodically polls the network to check the state of the merge. If the merge doesn't occur, the testnet-verifying image returns an unsuccessful exit code which in turn signals the Kurtosis module to exit with an error. This merge verification can be disabled in the module's configuration (see the "Configuration" section in the README).
<!------------------------ Only links below here -------------------------------->
if required_num_validtors > actual_num_validators:
if required_num_validtors > actual_num_validators:
fail("required_num_validtors - {0} is greater than actual_num_validators - {1}".format(required_num_validtors, actual_num_validators))
fail("required_num_validtors - {0} is greater than actual_num_validators - {1}".format(required_num_validtors, actual_num_validators))
...
@@ -149,7 +149,7 @@ def default_network_params():
...
@@ -149,7 +149,7 @@ def default_network_params():
# this is temporary till we get params working
# this is temporary till we get params working
return {
return {
"preregistered_validator_keys_mnemonic" : "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete",
"preregistered_validator_keys_mnemonic" : "giant issue aisle success illegal bike spike question tent bar rely arctic volcano long crawl hungry vocal artwork sniff fantasy very lucky have athlete",
// Number of slots in an epoch on the Beacon chain
// Number of slots in an epoch on the Beacon chain
optionaluint32slots_per_epoch=4;
optionaluint32slots_per_epoch=4;
// The number of validator keys that each CL validator node should get
// The number of validator keys that each CL validator node should get
optionaluint32num_validators_per_keynode=5;
optionaluint32num_validator_keys_per_node=5;
// This menmonic will a) be used to create keystores for all the types of validators that we have and b) be used to generate a CL genesis.ssz that has the children
// This menmonic will a) be used to create keystores for all the types of validators that we have and b) be used to generate a CL genesis.ssz that has the children
// validator keys already preregistered as validators
// validator keys already preregistered as validators
