Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
E
ethereum-package
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
vicotor
ethereum-package
Commits
8513c060
Unverified
Commit
8513c060
authored
Sep 27, 2024
by
Barnabas Busa
Committed by
GitHub
Sep 27, 2024
1
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
feat: enable one parameter supernode, refactor el,cl,vc (#778)
parent
cb87a9a8
Changes
29
Hide whitespace changes
Inline
Side-by-side
Showing
29 changed files
with
945 additions
and
1328 deletions
+945
-1328
peerdas-deneb.yaml
.github/tests/peerdas-deneb.yaml
+165
-0
peerdas-deneb.yaml.norun
.github/tests/peerdas-deneb.yaml.norun
+0
-55
peerdas-devnet-2.yaml
.github/tests/peerdas-devnet-2.yaml
+38
-0
peerdas-devnet-2.yaml.norun
.github/tests/peerdas-devnet-2.yaml.norun
+0
-10
README.md
README.md
+6
-0
network_params.yaml
network_params.yaml
+1
-0
cl_launcher.star
src/cl/cl_launcher.star
+2
-48
grandine_launcher.star
src/cl/grandine/grandine_launcher.star
+64
-97
lighthouse_launcher.star
src/cl/lighthouse/lighthouse_launcher.star
+63
-90
lodestar_launcher.star
src/cl/lodestar/lodestar_launcher.star
+63
-88
nimbus_launcher.star
src/cl/nimbus/nimbus_launcher.star
+51
-88
prysm_launcher.star
src/cl/prysm/prysm_launcher.star
+61
-89
teku_launcher.star
src/cl/teku/teku_launcher.star
+63
-100
besu_launcher.star
src/el/besu/besu_launcher.star
+38
-61
el_launcher.star
src/el/el_launcher.star
+1
-26
erigon_launcher.star
src/el/erigon/erigon_launcher.star
+45
-67
ethereumjs_launcher.star
src/el/ethereumjs/ethereumjs_launcher.star
+36
-61
geth_launcher.star
src/el/geth/geth_launcher.star
+58
-81
nethermind_launcher.star
src/el/nethermind/nethermind_launcher.star
+38
-63
nimbus_launcher.star
src/el/nimbus-eth1/nimbus_launcher.star
+36
-62
reth_launcher.star
src/el/reth/reth_launcher.star
+39
-66
input_parser.star
src/package_io/input_parser.star
+2
-0
sanity_check.star
src/package_io/sanity_check.star
+1
-0
lighthouse.star
src/vc/lighthouse.star
+14
-25
lodestar.star
src/vc/lodestar.star
+16
-26
nimbus.star
src/vc/nimbus.star
+13
-23
prysm.star
src/vc/prysm.star
+13
-23
teku.star
src/vc/teku.star
+13
-23
vc_launcher.star
src/vc/vc_launcher.star
+5
-56
No files found.
.github/tests/peerdas-deneb.yaml
0 → 100644
View file @
8513c060
participants
:
# Super nodes
-
cl_type
:
prysm
cl_image
:
ethpandaops/prysm-beacon-chain:peerDAS
supernode
:
true
cl_extra_params
:
-
--minimum-peers-per-subnet=1
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
lighthouse
cl_image
:
ethpandaops/lighthouse:unstable
supernode
:
true
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
teku
cl_image
:
ethpandaops/teku:nashatyrev-das
supernode
:
true
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
nimbus
cl_image
:
ethpandaops/nimbus-eth2:pdsync
supernode
:
true
cl_extra_params
:
-
--sync-light-client=no
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
lodestar
cl_image
:
ethpandaops/lodestar:peerDAS
supernode
:
true
cl_extra_params
:
-
--persistNetworkIdentity
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
grandine
cl_image
:
ethpandaops/grandine:hangleang-fix-negotiation-timeout
supernode
:
true
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
# Full nodes
-
cl_type
:
prysm
cl_image
:
ethpandaops/prysm-beacon-chain:peerDAS
cl_extra_params
:
-
--minimum-peers-per-subnet=1
-
--data-columns-withhold-count=64
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
lighthouse
cl_image
:
ethpandaops/lighthouse:unstable
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
lodestar
cl_image
:
ethpandaops/lodestar:peerDAS
cl_extra_params
:
[
--persistNetworkIdentity
]
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
nimbus
cl_image
:
ethpandaops/nimbus-eth2:pdsync
cl_extra_params
:
-
--sync-light-client=no
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
teku
cl_image
:
ethpandaops/teku:nashatyrev-das
count
:
3
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
-
cl_type
:
grandine
cl_image
:
ethpandaops/grandine:hangleang-fix-negotiation-timeout
cl_min_cpu
:
1000
cl_min_mem
:
1024
cl_max_cpu
:
4000
cl_max_mem
:
8096
el_min_cpu
:
1000
el_min_mem
:
1024
el_max_cpu
:
4000
el_max_mem
:
8096
network_params
:
eip7594_fork_epoch
:
0
eip7594_fork_version
:
"
0x50000038"
genesis_delay
:
1200
snooper_enabled
:
true
global_log_level
:
debug
additional_services
:
-
dora
-
goomy_blob
-
prometheus_grafana
dora_params
:
image
:
ethpandaops/dora:peerdas-as-deneb-column-view-latest
env
:
{
"
FRONTEND_SHOW_PEER_DAS_INFOS"
:
"
true"
,
"
FRONTEND_SHOW_SENSITIVE_PEER_INFOS"
:
"
true"
}
ethereum_metrics_exporter_enabled
:
true
.github/tests/peerdas-deneb.yaml.norun
deleted
100644 → 0
View file @
cb87a9a8
participants:
# Supernodes
- cl_type: prysm
cl_image: ethpandaops/prysm-beacon-chain:peerDAS
cl_extra_params: [--subscribe-all-subnets, --pprof]
cl_max_mem: 4096
count: 2
# - cl_type: lighthouse
# cl_image: ethpandaops/lighthouse:unstable
# cl_extra_params: [--subscribe-all-data-column-subnets]
#- cl_type: teku
# cl_image: ethpandaops/teku:nashatyrev-das
# cl_extra_params: [--p2p-subscribe-all-custody-subnets-enabled]
# - cl_type: nimbus
# cl_image: ethpandaops/nimbus-eth2:pdsync-minimal
# cl_extra_params: [--subscribe-all-subnets=true, --sync-light-client=no ]
# count: 2
# - cl_type: lodestar
# cl_image: ethpandaops/lodestar:peerDAS
# cl_extra_params: [--persistNetworkIdentity, --supernode]
# # Non supernodes
- cl_type: prysm
cl_image: ethpandaops/prysm-beacon-chain:peerDAS
cl_max_mem: 4096
# - cl_type: lighthouse
# cl_image: ethpandaops/lighthouse:unstable
#- cl_type: teku
# cl_image: ethpandaops/teku:nashatyrev-das
# - cl_type: nimbus
# cl_image: ethpandaops/nimbus-eth2:pdsync-minimal
# cl_extra_params: [--sync-light-client=no]
# - cl_type: lodestar
# cl_image: ethpandaops/lodestar:peerDAS
# cl_extra_params: [--persistNetworkIdentity]
network_params:
eip7594_fork_epoch: 0
eip7594_fork_version: "0x50000038"
#preset: minimal
snooper_enabled: true
global_log_level: debug
additional_services:
- dora
- goomy_blob
- prometheus_grafana
- assertoor
dora_params:
image: ethpandaops/dora:peerdas-as-deneb
ethereum_metrics_exporter_enabled: true
assertoor_params:
run_stability_check: false
run_block_proposal_check: false
tests:
- { file: "https://raw.githubusercontent.com/ethpandaops/assertoor-test/master/assertoor-tests/block-proposal-check.yaml" }
- { file: "https://raw.githubusercontent.com/ethpandaops/assertoor-test/master/assertoor-tests/blob-transactions-test.yaml" }
- { file: "https://raw.githubusercontent.com/ethpandaops/assertoor-test/master/assertoor-tests/stability-check.yaml" }
.github/tests/peerdas-devnet-2.yaml
0 → 100644
View file @
8513c060
participants
:
# Supernodes
-
cl_type
:
prysm
cl_image
:
ethpandaops/prysm-beacon-chain:peerDAS-df42f37
supernode
:
true
-
cl_type
:
lighthouse
cl_image
:
ethpandaops/lighthouse:unstable-2792705
supernode
:
true
-
cl_type
:
teku
cl_image
:
ethpandaops/teku:nashatyrev-das-458f7e7
supernode
:
true
-
cl_type
:
nimbus
cl_image
:
ethpandaops/nimbus-eth2:pdsync-30f8ef9
supernode
:
true
-
cl_type
:
lodestar
cl_image
:
ethpandaops/lodestar:peerDAS-a3de70f
supernode
:
true
-
cl_type
:
grandine
cl_image
:
ethpandaops/grandine:hangleang-feature-das-2c2f74b
supernode
:
true
# Fullnodes
-
cl_type
:
prysm
cl_image
:
ethpandaops/prysm-beacon-chain:peerDAS-df42f37
-
cl_type
:
lighthouse
cl_image
:
ethpandaops/lighthouse:unstable-2792705
-
cl_type
:
teku
cl_image
:
ethpandaops/teku:nashatyrev-das-458f7e7
-
cl_type
:
nimbus
cl_image
:
ethpandaops/nimbus-eth2:pdsync-30f8ef9
-
cl_type
:
lodestar
cl_image
:
ethpandaops/lodestar:peerDAS-a3de70f
-
cl_type
:
grandine
cl_image
:
ethpandaops/grandine:hangleang-feature-das-2c2f74b
network_params
:
network
:
peerdas-devnet-2
snooper_enabled
:
true
additional_services
:
-
dora
.github/tests/peerdas-devnet-2.yaml.norun
deleted
100644 → 0
View file @
cb87a9a8
participants:
- cl_type: prysm
cl_image: ethpandaops/prysm-beacon-chain:peerDAS
- cl_type: lighthouse
cl_image: ethpandaops/lighthouse:das
- cl_type: teku
cl_image: ethpandaops/teku:nashatyrev-das
network_params:
network: peerdas-devnet-2
snooper_enabled: true
README.md
View file @
8513c060
...
...
@@ -284,6 +284,12 @@ participants:
cl_min_mem
:
0
cl_max_mem
:
0
# Whether to act as a supernode for the network
# Supernodes will subscribe to all subnet topics
# This flag should only be used with peerdas
# Defaults to false
supernode
:
false
# Whether to use a separate validator client attached to the CL client.
# Defaults to false for clients that can run both in one process (Teku, Nimbus)
use_separate_vc
:
true
...
...
network_params.yaml
View file @
8513c060
...
...
@@ -25,6 +25,7 @@ participants:
cl_max_cpu
:
0
cl_min_mem
:
0
cl_max_mem
:
0
supernode
:
false
use_separate_vc
:
true
# Validator
vc_type
:
lighthouse
...
...
src/cl/cl_launcher.star
View file @
8513c060
...
...
@@ -63,8 +63,6 @@ def launch(
el_cl_data,
jwt_file,
network_params,
prysm_password_relative_filepath,
prysm_password_artifact_uuid,
),
"launch_method": prysm.launch,
},
...
...
@@ -108,22 +106,6 @@ def launch(
participant.cl_tolerations, participant.tolerations, global_tolerations
)
(
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
cl_volume_size,
) = shared_utils.get_cpu_mem_resource_limits(
participant.cl_min_cpu,
participant.cl_max_cpu,
participant.cl_min_mem,
participant.cl_max_mem,
participant.cl_volume_size,
network_name,
participant.cl_type,
)
if cl_type not in cl_launchers:
fail(
"Unsupported launcher '{0}', need one of '{1}'".format(
...
...
@@ -171,30 +153,16 @@ def launch(
plan,
cl_launcher,
cl_service_name,
participant.cl_image,
participant.cl_log_level,
participant,
global_log_level,
cl_context_BOOTNODE,
el_context,
full_name,
new_cl_node_validator_keystores,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
participant.snooper_enabled,
snooper_engine_context,
participant.blobber_enabled,
participant.blobber_extra_params,
participant.cl_extra_params,
participant.cl_extra_env_vars,
participant.cl_extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
participant.use_separate_vc,
participant.keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
...
...
@@ -206,30 +174,16 @@ def launch(
plan,
cl_launcher,
cl_service_name,
participant.cl_image,
participant.cl_log_level,
participant,
global_log_level,
boot_cl_client_ctx,
el_context,
full_name,
new_cl_node_validator_keystores,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
participant.snooper_enabled,
snooper_engine_context,
participant.blobber_enabled,
participant.blobber_extra_params,
participant.cl_extra_params,
participant.cl_extra_env_vars,
participant.cl_extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
participant.use_separate_vc,
participant.keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
...
...
src/cl/grandine/grandine_launcher.star
View file @
8513c060
...
...
@@ -33,68 +33,38 @@ VERBOSITY_LEVELS = {
def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
beacon_service_name,
participant,
global_log_level,
bootnode_context,
bootnode_context
s
,
el_context,
full_name,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
blobber_enabled,
blobber_extra_params,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
use_separate_vc,
keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
participant_index,
):
beacon_service_name = "{0}".format(service_name)
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.cl
_log_level, global_log_level, VERBOSITY_LEVELS
)
extra_params = [param for param in extra_params]
config = get_beacon_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
keymanager_enabled,
image,
launcher,
beacon_service_name,
bootnode_context,
participant,
log_level,
bootnode_contexts,
el_context,
full_name,
log_level,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
use_separate_vc,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -103,7 +73,7 @@ def launch(
participant_index,
)
beacon_service = plan.add_service(service_name, config)
beacon_service = plan.add_service(
beacon_
service_name, config)
beacon_http_port = beacon_service.ports[constants.HTTP_PORT_ID]
beacon_http_url = "http://{0}:{1}".format(
...
...
@@ -125,14 +95,14 @@ def launch(
},
)
response = plan.request(
recipe=beacon_node_identity_recipe, service_name=service_name
recipe=beacon_node_identity_recipe, service_name=
beacon_
service_name
)
beacon_node_enr = response["extract.enr"]
beacon_multiaddr = response["extract.multiaddr"]
beacon_peer_id = response["extract.peer_id"]
beacon_node_metrics_info = node_metrics.new_node_metrics_info(
service_name, BEACON_METRICS_PATH, beacon_metrics_url
beacon_
service_name, BEACON_METRICS_PATH, beacon_metrics_url
)
nodes_metrics_info = [beacon_node_metrics_info]
return cl_context.new_cl_context(
...
...
@@ -145,7 +115,7 @@ def launch(
beacon_service_name=beacon_service_name,
multiaddr=beacon_multiaddr,
peer_id=beacon_peer_id,
snooper_enabled=snooper_enabled,
snooper_enabled=
participant.
snooper_enabled,
snooper_engine_context=snooper_engine_context,
validator_keystore_files_artifact_uuid=node_keystore_files.files_artifact_uuid
if node_keystore_files
...
...
@@ -155,29 +125,16 @@ def launch(
def get_beacon_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
keymanager_enabled,
image,
service_name,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
full_name,
log_level,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
use_separate_vc,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -197,7 +154,7 @@ def get_beacon_config(
node_keystore_files.teku_secrets_relative_dirpath,
)
# If snooper is enabled use the snooper engine context, otherwise use the execution client context
if snooper_enabled:
if
participant.
snooper_enabled:
EXECUTION_ENGINE_ENDPOINT = "http://{0}:{1}".format(
snooper_engine_context.ip_addr,
snooper_engine_context.engine_rpc_port_num,
...
...
@@ -232,7 +189,9 @@ def get_beacon_config(
cmd = [
"--network={0}".format(
network if network in constants.PUBLIC_NETWORKS else "custom"
launcher.network
if launcher.network in constants.PUBLIC_NETWORKS
else "custom"
),
"--data-dir=" + BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER,
"--http-address=0.0.0.0",
...
...
@@ -270,31 +229,39 @@ def get_beacon_config(
# "--validator-api-bearer-file=" + constants.KEYMANAGER_MOUNT_PATH_ON_CONTAINER, Not yet supported
]
supernode_cmd = [
"--subscribe-all-data-column-subnets",
]
if participant.supernode:
cmd.extend(supernode_cmd)
# If checkpoint sync is enabled, add the checkpoint sync url
if checkpoint_sync_enabled:
if checkpoint_sync_url:
cmd.append("--checkpoint-sync-url=" + checkpoint_sync_url)
else:
if (
network in constants.PUBLIC_NETWORKS
or network == constants.NETWORK_NAME.ephemery
launcher.
network in constants.PUBLIC_NETWORKS
or
launcher.
network == constants.NETWORK_NAME.ephemery
):
cmd.append(
"--checkpoint-sync-url=" + constants.CHECKPOINT_SYNC_URL[network]
"--checkpoint-sync-url="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
else:
fail(
"Checkpoint sync URL is required if you enabled checkpoint_sync for custom networks. Please provide a valid URL."
)
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append(
"--configuration-directory="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
)
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if bootnode_contexts != None:
cmd.append(
...
...
@@ -306,44 +273,44 @@ def get_beacon_config(
]
)
)
elif network == constants.NETWORK_NAME.ephemery:
elif
launcher.
network == constants.NETWORK_NAME.ephemery:
cmd.append(
"--boot-nodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
elif constants.NETWORK_NAME.shadowfork in network:
elif constants.NETWORK_NAME.shadowfork in
launcher.
network:
cmd.append(
"--boot-nodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
else: # Devnets
cmd.append(
"--boot-nodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
# we do the list comprehension as the default extra_params is a proto repeated string
cmd.extend([param for param in extra_params])
if len(
participant.cl_
extra_params) > 0:
# we do the list comprehension as the default
participant.
extra_params is a proto repeated string
cmd.extend([param for param in
participant.cl_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if node_keystore_files != None and not use_separate_vc:
if node_keystore_files != None and not
participant.
use_separate_vc:
cmd.extend(validator_default_cmd)
files[
constants.VALIDATOR_KEYS_DIRPATH_ON_SERVICE_CONTAINER
] = node_keystore_files.files_artifact_uuid
if keymanager_enabled:
if
participant.
keymanager_enabled:
cmd.extend(keymanager_api_cmd)
used_ports.update(vc_shared.VALIDATOR_KEYMANAGER_USED_PORTS)
public_ports.update(
...
...
@@ -352,16 +319,20 @@ def get_beacon_config(
if persistent:
files[BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=cl_volume_size,
persistent_key="data-{0}".format(beacon_service_name),
size=int(participant.cl_volume_size)
if int(participant.cl_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.CL_TYPE.grandine + "_volume_size"
],
)
config_args = {
"image": image,
"image":
participant.cl_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": extra_env_vars,
"env_vars":
participant.cl_
extra_env_vars,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"ready_conditions": cl_node_ready_conditions.get_ready_conditions(
constants.HTTP_PORT_ID
...
...
@@ -369,27 +340,23 @@ def get_beacon_config(
"labels": shared_utils.label_maker(
constants.CL_TYPE.grandine,
constants.CLIENT_TYPES.cl,
image,
participant.cl_
image,
el_context.client_name,
extra_labels,
participant.cl_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
"user": User(uid=0, gid=0),
}
if cl_min_cpu > 0:
config_args["min_cpu"] = cl_min_cpu
if cl_max_cpu > 0:
config_args["max_cpu"] = cl_max_cpu
if cl_min_mem > 0:
config_args["min_memory"] = cl_min_mem
if cl_max_mem > 0:
config_args["max_memory"] = cl_max_mem
if int(participant.cl_min_cpu) > 0:
config_args["min_cpu"] = int(participant.cl_min_cpu)
if int(participant.cl_max_cpu) > 0:
config_args["max_cpu"] = int(participant.cl_max_cpu)
if int(participant.cl_min_mem) > 0:
config_args["min_memory"] = int(participant.cl_min_mem)
if int(participant.cl_max_mem) > 0:
config_args["max_memory"] = int(participant.cl_max_mem)
return ServiceConfig(**config_args)
...
...
src/cl/lighthouse/lighthouse_launcher.star
View file @
8513c060
...
...
@@ -39,64 +39,38 @@ VERBOSITY_LEVELS = {
def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
beacon_service_name,
participant,
global_log_level,
bootnode_contexts,
el_context,
full_name,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
blobber_enabled,
blobber_extra_params,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
use_separate_vc,
keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
participant_index,
):
beacon_service_name = "{0}".format(service_name)
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.cl
_log_level, global_log_level, VERBOSITY_LEVELS
)
# Launch Beacon node
beacon_config = get_beacon_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
log_level,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
full_name,
node_keystore_files,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -112,13 +86,13 @@ def launch(
)
# Blobber config
if blobber_enabled:
if
participant.
blobber_enabled:
blobber_service_name = "{0}-{1}".format("blobber", beacon_service_name)
blobber_config = blobber_launcher.get_config(
blobber_service_name,
node_keystore_files,
beacon_http_url,
blobber_extra_params,
participant.
blobber_extra_params,
node_selectors,
)
...
...
@@ -166,7 +140,7 @@ def launch(
beacon_service_name=beacon_service_name,
multiaddr=beacon_multiaddr,
peer_id=beacon_peer_id,
snooper_enabled=snooper_enabled,
snooper_enabled=
participant.
snooper_enabled,
snooper_engine_context=snooper_engine_context,
validator_keystore_files_artifact_uuid=node_keystore_files.files_artifact_uuid
if node_keystore_files
...
...
@@ -176,25 +150,16 @@ def launch(
def get_beacon_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
image,
service_name,
boot_cl_client_ctxs,
el_context,
launcher,
beacon_service_name,
participant,
log_level,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
bootnode_contexts,
el_context,
full_name,
node_keystore_files,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -203,7 +168,7 @@ def get_beacon_config(
participant_index,
):
# If snooper is enabled use the snooper engine context, otherwise use the execution client context
if snooper_enabled:
if
participant.
snooper_enabled:
EXECUTION_ENGINE_ENDPOINT = "http://{0}:{1}".format(
snooper_engine_context.ip_addr,
snooper_engine_context.engine_rpc_port_num,
...
...
@@ -278,82 +243,94 @@ def get_beacon_config(
"--enable-private-discovery",
]
supernode_cmd = [
"--subscribe-all-data-column-subnets",
]
if participant.supernode:
cmd.extend(supernode_cmd)
# If checkpoint sync is enabled, add the checkpoint sync url
if checkpoint_sync_enabled:
if checkpoint_sync_url:
cmd.append("--checkpoint-sync-url=" + checkpoint_sync_url)
else:
if (
network in constants.PUBLIC_NETWORKS
or network == constants.NETWORK_NAME.ephemery
launcher.
network in constants.PUBLIC_NETWORKS
or
launcher.
network == constants.NETWORK_NAME.ephemery
):
cmd.append(
"--checkpoint-sync-url=" + constants.CHECKPOINT_SYNC_URL[network]
"--checkpoint-sync-url="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
else:
fail(
"Checkpoint sync URL is required if you enabled checkpoint_sync for custom networks. Please provide a valid URL."
)
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append("--testnet-dir=" + constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER)
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if boot
_cl_client_ctx
s != None:
if boot
node_context
s != None:
cmd.append(
"--boot-nodes="
+ ",".join(
[
ctx.enr
for ctx in boot
_cl_client_ctx
s[: constants.MAX_ENR_ENTRIES]
for ctx in boot
node_context
s[: constants.MAX_ENR_ENTRIES]
]
)
)
elif network == constants.NETWORK_NAME.ephemery:
elif
launcher.
network == constants.NETWORK_NAME.ephemery:
cmd.append(
"--boot-nodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
else: # Devnets
cmd.append(
"--boot-nodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
else: # Public networks
cmd.append("--network=" + network)
cmd.append("--network=" +
launcher.
network)
if len(extra_params) > 0:
if len(
participant.cl_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.cl_
extra_params])
recipe = GetHttpRequestRecipe(
endpoint="/eth/v1/node/identity", port_id=constants.HTTP_PORT_ID
)
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[BEACON_DATA_DIRPATH_ON_BEACON_SERVICE_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=cl_volume_size,
persistent_key="data-{0}".format(beacon_service_name),
size=int(participant.cl_volume_size)
if int(participant.cl_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.CL_TYPE.lighthouse + "_volume_size"
],
)
env = {RUST_BACKTRACE_ENVVAR_NAME: RUST_FULL_BACKTRACE_KEYWORD}
env
.update(
extra_env_vars)
env
_vars
= {RUST_BACKTRACE_ENVVAR_NAME: RUST_FULL_BACKTRACE_KEYWORD}
env
_vars.update(participant.cl_
extra_env_vars)
config_args = {
"image": image,
"image":
participant.cl_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": env,
"env_vars": env
_vars
,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"ready_conditions": cl_node_ready_conditions.get_ready_conditions(
constants.HTTP_PORT_ID
...
...
@@ -361,26 +338,22 @@ def get_beacon_config(
"labels": shared_utils.label_maker(
constants.CL_TYPE.lighthouse,
constants.CLIENT_TYPES.cl,
image,
participant.cl_
image,
el_context.client_name,
extra_labels,
participant.cl_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if cl_min_cpu > 0:
config_args["min_cpu"] = cl_min_cpu
if cl_max_cpu > 0:
config_args["max_cpu"] = cl_max_cpu
if cl_min_mem > 0:
config_args["min_memory"] = cl_min_mem
if cl_max_mem > 0:
config_args["max_memory"] = cl_max_mem
if int(participant.cl_min_cpu) > 0:
config_args["min_cpu"] = int(participant.cl_min_cpu)
if int(participant.cl_max_cpu) > 0:
config_args["max_cpu"] = int(participant.cl_max_cpu)
if int(participant.cl_min_mem) > 0:
config_args["min_memory"] = int(participant.cl_min_mem)
if int(participant.cl_max_mem) > 0:
config_args["max_memory"] = int(participant.cl_max_mem)
return ServiceConfig(**config_args)
...
...
src/cl/lodestar/lodestar_launcher.star
View file @
8513c060
...
...
@@ -29,69 +29,44 @@ VERBOSITY_LEVELS = {
def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
beacon_service_name,
participant,
global_log_level,
bootnode_contexts,
el_context,
full_name,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
blobber_enabled,
blobber_extra_params,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
use_separate_vc,
keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
participant_index,
):
beacon_service_name = "{0}".format(service_name)
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.cl
_log_level, global_log_level, VERBOSITY_LEVELS
)
# Launch Beacon node
beacon_config = get_beacon_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
log_level,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
full_name,
node_keystore_files,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
launcher.preset,
participant_index,
)
...
...
@@ -104,13 +79,13 @@ def launch(
)
# Blobber config
if blobber_enabled:
if
participant.
blobber_enabled:
blobber_service_name = "{0}-{1}".format("blobber", beacon_service_name)
blobber_config = blobber_launcher.get_config(
blobber_service_name,
node_keystore_files,
beacon_http_url,
blobber_extra_params,
participant.
blobber_extra_params,
)
blobber_service = plan.add_service(blobber_service_name, blobber_config)
...
...
@@ -146,7 +121,7 @@ def launch(
)
beacon_node_metrics_info = node_metrics.new_node_metrics_info(
service_name, METRICS_PATH, beacon_metrics_url
beacon_
service_name, METRICS_PATH, beacon_metrics_url
)
nodes_metrics_info = [beacon_node_metrics_info]
...
...
@@ -160,7 +135,7 @@ def launch(
beacon_service_name=beacon_service_name,
multiaddr=beacon_multiaddr,
peer_id=beacon_peer_id,
snooper_enabled=snooper_enabled,
snooper_enabled=
participant.
snooper_enabled,
snooper_engine_context=snooper_engine_context,
validator_keystore_files_artifact_uuid=node_keystore_files.files_artifact_uuid
if node_keystore_files
...
...
@@ -170,31 +145,21 @@ def launch(
def get_beacon_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
image,
service_name,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
log_level,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
full_name,
node_keystore_files,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
preset,
participant_index,
):
el_client_rpc_url_str = "http://{0}:{1}".format(
...
...
@@ -203,7 +168,7 @@ def get_beacon_config(
)
# If snooper is enabled use the snooper engine context, otherwise use the execution client context
if snooper_enabled:
if
participant.
snooper_enabled:
EXECUTION_ENGINE_ENDPOINT = "http://{0}:{1}".format(
snooper_engine_context.ip_addr,
snooper_engine_context.engine_rpc_port_num,
...
...
@@ -261,24 +226,32 @@ def get_beacon_config(
# ^^^^^^^^^^^^^^^^^^^ METRICS CONFIG ^^^^^^^^^^^^^^^^^^^^^
]
supernode_cmd = [
"--supernode",
]
if participant.supernode:
cmd.extend(supernode_cmd)
# If checkpoint sync is enabled, add the checkpoint sync url
if checkpoint_sync_enabled:
if checkpoint_sync_url:
cmd.append("--checkpointSyncUrl=" + checkpoint_sync_url)
else:
if (
network in constants.PUBLIC_NETWORKS
or network == constants.NETWORK_NAME.ephemery
launcher.
network in constants.PUBLIC_NETWORKS
or
launcher.
network == constants.NETWORK_NAME.ephemery
):
cmd.append(
"--checkpointSyncUrl=" + constants.CHECKPOINT_SYNC_URL[network]
"--checkpointSyncUrl="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
else:
fail(
"Checkpoint sync URL is required if you enabled checkpoint_sync for custom networks. Please provide a valid URL."
)
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append(
"--paramsFile="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
...
...
@@ -290,8 +263,8 @@ def get_beacon_config(
+ "/genesis.ssz"
)
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if bootnode_contexts != None:
cmd.append(
...
...
@@ -303,47 +276,53 @@ def get_beacon_config(
]
)
)
elif network == constants.NETWORK_NAME.ephemery:
elif
launcher.
network == constants.NETWORK_NAME.ephemery:
cmd.append(
"--bootnodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
else: # Devnets
cmd.append(
"--bootnodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
else: # Public testnet
cmd.append("--network=" + network)
cmd.append("--network=" +
launcher.
network)
if len(extra_params) > 0:
if len(
participant.cl_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.cl_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=cl_volume_size,
persistent_key="data-{0}".format(beacon_service_name),
size=int(participant.cl_volume_size)
if int(participant.cl_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.CL_TYPE.lodestar + "_volume_size"
],
)
if preset == "minimal":
extra_env_vars["LODESTAR_PRESET"] = "minimal"
env_vars = participant.cl_extra_env_vars
if launcher.preset == "minimal":
env_vars["LODESTAR_PRESET"] = "minimal"
config_args = {
"image": image,
"image":
participant.cl_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"ready_conditions": cl_node_ready_conditions.get_ready_conditions(
constants.HTTP_PORT_ID
...
...
@@ -351,26 +330,22 @@ def get_beacon_config(
"labels": shared_utils.label_maker(
constants.CL_TYPE.lodestar,
constants.CLIENT_TYPES.cl,
image,
participant.cl_
image,
el_context.client_name,
extra_labels,
participant.cl_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if cl_min_cpu > 0:
config_args["min_cpu"] = cl_min_cpu
if cl_max_cpu > 0:
config_args["max_cpu"] = cl_max_cpu
if cl_min_mem > 0:
config_args["min_memory"] = cl_min_mem
if cl_max_mem > 0:
config_args["max_memory"] = cl_max_mem
if int(participant.cl_min_cpu) > 0:
config_args["min_cpu"] = int(participant.cl_min_cpu)
if int(participant.cl_max_cpu) > 0:
config_args["max_cpu"] = int(participant.cl_max_cpu)
if int(participant.cl_min_mem) > 0:
config_args["min_memory"] = int(participant.cl_min_mem)
if int(participant.cl_max_mem) > 0:
config_args["max_memory"] = int(participant.cl_max_mem)
return ServiceConfig(**config_args)
...
...
src/cl/nimbus/nimbus_launcher.star
View file @
8513c060
...
...
@@ -49,68 +49,38 @@ ENTRYPOINT_ARGS = ["sh", "-c"]
def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
beacon_service_name,
participant,
global_log_level,
bootnode_contexts,
el_context,
full_name,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
blobber_enabled,
blobber_extra_params,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
use_separate_vc,
keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
participant_index,
):
beacon_service_name = "{0}".format(service_name)
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.cl
_log_level, global_log_level, VERBOSITY_LEVELS
)
beacon_config = get_beacon_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
keymanager_enabled,
launcher.keymanager_file,
launcher.network,
image,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
full_name,
log_level,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
use_separate_vc,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -139,14 +109,14 @@ def launch(
},
)
response = plan.request(
recipe=beacon_node_identity_recipe, service_name=service_name
recipe=beacon_node_identity_recipe, service_name=
beacon_
service_name
)
beacon_node_enr = response["extract.enr"]
beacon_multiaddr = response["extract.multiaddr"]
beacon_peer_id = response["extract.peer_id"]
nimbus_node_metrics_info = node_metrics.new_node_metrics_info(
service_name, BEACON_METRICS_PATH, beacon_metrics_url
beacon_
service_name, BEACON_METRICS_PATH, beacon_metrics_url
)
nodes_metrics_info = [nimbus_node_metrics_info]
...
...
@@ -160,7 +130,7 @@ def launch(
beacon_service_name=beacon_service_name,
multiaddr=beacon_multiaddr,
peer_id=beacon_peer_id,
snooper_enabled=snooper_enabled,
snooper_enabled=
participant.
snooper_enabled,
snooper_engine_context=snooper_engine_context,
validator_keystore_files_artifact_uuid=node_keystore_files.files_artifact_uuid
if node_keystore_files
...
...
@@ -170,30 +140,16 @@ def launch(
def get_beacon_config(
plan,
el_cl_genesis_data,
jwt_file,
keymanager_enabled,
keymanager_file,
network,
image,
service_name,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
full_name,
log_level,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
use_separate_vc,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -213,7 +169,7 @@ def get_beacon_config(
node_keystore_files.raw_secrets_relative_dirpath,
)
# If snooper is enabled use the snooper engine context, otherwise use the execution client context
if snooper_enabled:
if
participant.
snooper_enabled:
EXECUTION_ENGINE_ENDPOINT = "http://{0}:{1}".format(
snooper_engine_context.ip_addr,
snooper_engine_context.engine_rpc_port_num,
...
...
@@ -252,8 +208,8 @@ def get_beacon_config(
"--udp-port={0}".format(discovery_port),
"--tcp-port={0}".format(discovery_port),
"--network={0}".format(
network
if network in constants.PUBLIC_NETWORKS
launcher.
network
if
launcher.
network in constants.PUBLIC_NETWORKS
else constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
),
"--data-dir=" + BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER,
...
...
@@ -294,15 +250,22 @@ def get_beacon_config(
"--keymanager-token-file=" + constants.KEYMANAGER_MOUNT_PATH_ON_CONTAINER,
]
if network not in constants.PUBLIC_NETWORKS:
supernode_cmd = [
"--subscribe-all-subnets",
]
if participant.supernode:
cmd.extend(supernode_cmd)
if launcher.network not in constants.PUBLIC_NETWORKS:
cmd.append(
"--bootstrap-file="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
+ "/bootstrap_nodes.txt"
)
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if bootnode_contexts == None:
cmd.append("--subscribe-all-subnets")
...
...
@@ -310,22 +273,22 @@ def get_beacon_config(
for ctx in bootnode_contexts[: constants.MAX_ENR_ENTRIES]:
cmd.append("--bootstrap-node=" + ctx.enr)
if len(extra_params) > 0:
cmd.extend([param for param in extra_params])
if len(
participant.cl_
extra_params) > 0:
cmd.extend([param for param in
participant.cl_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if node_keystore_files != None and not use_separate_vc:
if node_keystore_files != None and not
participant.
use_separate_vc:
cmd.extend(validator_default_cmd)
files[
VALIDATOR_KEYS_MOUNTPOINT_ON_CLIENTS
] = node_keystore_files.files_artifact_uuid
files[constants.KEYMANAGER_MOUNT_PATH_ON_CLIENTS] = keymanager_file
files[constants.KEYMANAGER_MOUNT_PATH_ON_CLIENTS] =
launcher.
keymanager_file
if keymanager_enabled:
if
participant.
keymanager_enabled:
cmd.extend(keymanager_api_cmd)
used_ports.update(vc_shared.VALIDATOR_KEYMANAGER_USED_PORTS)
public_ports.update(
...
...
@@ -334,17 +297,21 @@ def get_beacon_config(
if persistent:
files[BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=cl_volume_size,
persistent_key="data-{0}".format(beacon_service_name),
size=int(participant.cl_volume_size)
if int(participant.cl_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.CL_TYPE.nimbus + "_volume_size"
],
)
config_args = {
"image": image,
"image":
participant.cl_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": extra_env_vars,
"env_vars":
participant.cl_
extra_env_vars,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"ready_conditions": cl_node_ready_conditions.get_ready_conditions(
constants.HTTP_PORT_ID
...
...
@@ -352,27 +319,23 @@ def get_beacon_config(
"labels": shared_utils.label_maker(
constants.CL_TYPE.nimbus,
constants.CLIENT_TYPES.cl,
image,
participant.cl_
image,
el_context.client_name,
extra_labels,
participant.cl_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
"user": User(uid=0, gid=0),
}
if cl_min_cpu > 0:
config_args["min_cpu"] = cl_min_cpu
if cl_max_cpu > 0:
config_args["max_cpu"] = cl_max_cpu
if cl_min_mem > 0:
config_args["min_memory"] = cl_min_mem
if cl_max_mem > 0:
config_args["max_memory"] = cl_max_mem
if int(participant.cl_min_cpu) > 0:
config_args["min_cpu"] = int(participant.cl_min_cpu)
if int(participant.cl_max_cpu) > 0:
config_args["max_cpu"] = int(participant.cl_max_cpu)
if int(participant.cl_min_mem) > 0:
config_args["min_memory"] = int(participant.cl_min_mem)
if int(participant.cl_max_mem) > 0:
config_args["max_memory"] = int(participant.cl_max_mem)
return ServiceConfig(**config_args)
...
...
src/cl/prysm/prysm_launcher.star
View file @
8513c060
...
...
@@ -33,68 +33,43 @@ VERBOSITY_LEVELS = {
def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
beacon_service_name,
participant,
global_log_level,
bootnode_contexts,
el_context,
full_name,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
blobber_enabled,
blobber_extra_params,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
use_separate_vc,
keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
participant_index,
):
beacon_service_name = "{0}".format(service_name)
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.cl
_log_level, global_log_level, VERBOSITY_LEVELS
)
beacon_config = get_beacon_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
log_level,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
full_name,
node_keystore_files,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
launcher.preset,
participant_index,
)
...
...
@@ -142,7 +117,7 @@ def launch(
beacon_grpc_url=beacon_grpc_url,
multiaddr=beacon_multiaddr,
peer_id=beacon_peer_id,
snooper_enabled=snooper_enabled,
snooper_enabled=
participant.
snooper_enabled,
snooper_engine_context=snooper_engine_context,
validator_keystore_files_artifact_uuid=node_keystore_files.files_artifact_uuid
if node_keystore_files
...
...
@@ -152,35 +127,25 @@ def launch(
def get_beacon_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
beacon_image,
service_name,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
log_level,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
full_name,
node_keystore_files,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
preset,
participant_index,
):
# If snooper is enabled use the snooper engine context, otherwise use the execution client context
if snooper_enabled:
if
participant.
snooper_enabled:
EXECUTION_ENGINE_ENDPOINT = "http://{0}:{1}".format(
snooper_engine_context.ip_addr,
snooper_engine_context.engine_rpc_port_num,
...
...
@@ -249,33 +214,43 @@ def get_beacon_config(
"--pprofport={0}".format(PROFILING_PORT_NUM),
]
supernode_cmd = [
"--subscribe-all-subnets=true",
]
if participant.supernode:
cmd.extend(supernode_cmd)
# If checkpoint sync is enabled, add the checkpoint sync url
if checkpoint_sync_enabled:
if checkpoint_sync_url:
cmd.append("--checkpoint-sync-url=" + checkpoint_sync_url)
cmd.append(
"--genesis-beacon-api-url=" + constants.CHECKPOINT_SYNC_URL[network]
"--genesis-beacon-api-url="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
else:
if (
network in constants.PUBLIC_NETWORKS
or network == constants.NETWORK_NAME.ephemery
launcher.
network in constants.PUBLIC_NETWORKS
or
launcher.
network == constants.NETWORK_NAME.ephemery
):
cmd.append(
"--checkpoint-sync-url=" + constants.CHECKPOINT_SYNC_URL[network]
"--checkpoint-sync-url="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
cmd.append(
"--genesis-beacon-api-url=" + constants.CHECKPOINT_SYNC_URL[network]
"--genesis-beacon-api-url="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
else:
fail(
"Checkpoint sync URL is required if you enabled checkpoint_sync for custom networks. Please provide a valid URL."
)
if preset == "minimal":
if
launcher.
preset == "minimal":
cmd.append("--minimal-config=true")
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append("--p2p-static-id=true")
cmd.append(
"--chain-config-file="
...
...
@@ -289,15 +264,16 @@ def get_beacon_config(
)
cmd.append("--contract-deployment-block=0")
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if bootnode_contexts != None:
for ctx in bootnode_contexts[: constants.MAX_ENR_ENTRIES]:
cmd.append("--bootstrap-node=" + ctx.enr)
elif network == constants.NETWORK_NAME.ephemery:
elif
launcher.
network == constants.NETWORK_NAME.ephemery:
cmd.append(
"--genesis-beacon-api-url=" + constants.CHECKPOINT_SYNC_URL[network]
"--genesis-beacon-api-url="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
cmd.append(
"--bootstrap-node="
...
...
@@ -311,30 +287,34 @@ def get_beacon_config(
+ "/bootstrap_nodes.yaml"
)
else: # Public network
cmd.append("--{}".format(network))
cmd.append("--{}".format(
launcher.
network))
if len(extra_params) > 0:
if len(
participant.cl_
extra_params) > 0:
# we do the for loop as otherwise its a proto repeated array
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.cl_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=cl_volume_size,
persistent_key="data-{0}".format(beacon_service_name),
size=int(participant.cl_volume_size)
if int(participant.cl_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.CL_TYPE.prysm + "_volume_size"
],
)
config_args = {
"image":
beacon
_image,
"image":
participant.cl
_image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": extra_env_vars,
"env_vars":
participant.cl_
extra_env_vars,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"ready_conditions": cl_node_ready_conditions.get_ready_conditions(
constants.HTTP_PORT_ID
...
...
@@ -342,26 +322,22 @@ def get_beacon_config(
"labels": shared_utils.label_maker(
constants.CL_TYPE.prysm,
constants.CLIENT_TYPES.cl,
beacon
_image,
participant.cl
_image,
el_context.client_name,
extra_labels,
participant.cl_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if cl_min_cpu > 0:
config_args["min_cpu"] = cl_min_cpu
if cl_max_cpu > 0:
config_args["max_cpu"] = cl_max_cpu
if cl_min_mem > 0:
config_args["min_memory"] = cl_min_mem
if cl_max_mem > 0:
config_args["max_memory"] = cl_max_mem
if int(participant.cl_min_cpu) > 0:
config_args["min_cpu"] = int(participant.cl_min_cpu)
if int(participant.cl_max_cpu) > 0:
config_args["max_cpu"] = int(participant.cl_max_cpu)
if int(participant.cl_min_mem) > 0:
config_args["min_memory"] = int(participant.cl_min_mem)
if int(participant.cl_max_mem) > 0:
config_args["max_memory"] = int(participant.cl_max_mem)
return ServiceConfig(**config_args)
...
...
@@ -369,14 +345,10 @@ def new_prysm_launcher(
el_cl_genesis_data,
jwt_file,
network_params,
prysm_password_relative_filepath,
prysm_password_artifact_uuid,
):
return struct(
el_cl_genesis_data=el_cl_genesis_data,
jwt_file=jwt_file,
network=network_params.network,
preset=network_params.preset,
prysm_password_artifact_uuid=prysm_password_artifact_uuid,
prysm_password_relative_filepath=prysm_password_relative_filepath,
)
src/cl/teku/teku_launcher.star
View file @
8513c060
...
...
@@ -36,69 +36,38 @@ VERBOSITY_LEVELS = {
def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
beacon_service_name,
participant,
global_log_level,
bootnode_context,
bootnode_context
s
,
el_context,
full_name,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
blobber_enabled,
blobber_extra_params,
extra_params,
extra_env_vars,
extra_labels,
persistent,
cl_volume_size,
tolerations,
node_selectors,
use_separate_vc,
keymanager_enabled,
checkpoint_sync_enabled,
checkpoint_sync_url,
port_publisher,
participant_index,
):
beacon_service_name = "{0}".format(service_name)
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.cl
_log_level, global_log_level, VERBOSITY_LEVELS
)
extra_params = [param for param in extra_params]
config = get_beacon_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
keymanager_enabled,
launcher.keymanager_file,
launcher.network,
image,
launcher,
beacon_service_name,
bootnode_context,
participant,
log_level,
bootnode_contexts,
el_context,
full_name,
log_level,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
use_separate_vc,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -107,7 +76,7 @@ def launch(
participant_index,
)
beacon_service = plan.add_service(service_name, config)
beacon_service = plan.add_service(
beacon_
service_name, config)
beacon_http_port = beacon_service.ports[constants.HTTP_PORT_ID]
beacon_http_url = "http://{0}:{1}".format(
...
...
@@ -129,14 +98,14 @@ def launch(
},
)
response = plan.request(
recipe=beacon_node_identity_recipe, service_name=service_name
recipe=beacon_node_identity_recipe, service_name=
beacon_
service_name
)
beacon_node_enr = response["extract.enr"]
beacon_multiaddr = response["extract.multiaddr"]
beacon_peer_id = response["extract.peer_id"]
beacon_node_metrics_info = node_metrics.new_node_metrics_info(
service_name, BEACON_METRICS_PATH, beacon_metrics_url
beacon_
service_name, BEACON_METRICS_PATH, beacon_metrics_url
)
nodes_metrics_info = [beacon_node_metrics_info]
...
...
@@ -150,7 +119,7 @@ def launch(
beacon_service_name=beacon_service_name,
multiaddr=beacon_multiaddr,
peer_id=beacon_peer_id,
snooper_enabled=snooper_enabled,
snooper_enabled=
participant.
snooper_enabled,
snooper_engine_context=snooper_engine_context,
validator_keystore_files_artifact_uuid=node_keystore_files.files_artifact_uuid
if node_keystore_files
...
...
@@ -160,30 +129,16 @@ def launch(
def get_beacon_config(
plan,
el_cl_genesis_data,
jwt_file,
keymanager_enabled,
keymanager_file,
network,
image,
service_name,
launcher,
beacon_service_name,
participant,
log_level,
bootnode_contexts,
el_context,
full_name,
log_level,
node_keystore_files,
cl_min_cpu,
cl_max_cpu,
cl_min_mem,
cl_max_mem,
snooper_enabled,
snooper_engine_context,
extra_params,
extra_env_vars,
extra_labels,
use_separate_vc,
persistent,
cl_volume_size,
tolerations,
node_selectors,
checkpoint_sync_enabled,
...
...
@@ -203,7 +158,7 @@ def get_beacon_config(
node_keystore_files.teku_secrets_relative_dirpath,
)
# If snooper is enabled use the snooper engine context, otherwise use the execution client context
if snooper_enabled:
if
participant.
snooper_enabled:
EXECUTION_ENGINE_ENDPOINT = "http://{0}:{1}".format(
snooper_engine_context.ip_addr,
snooper_engine_context.engine_rpc_port_num,
...
...
@@ -240,8 +195,8 @@ def get_beacon_config(
"--logging=" + log_level,
"--log-destination=CONSOLE",
"--network={0}".format(
network
if network in constants.PUBLIC_NETWORKS
launcher.
network
if
launcher.
network in constants.PUBLIC_NETWORKS
else constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER + "/config.yaml"
),
"--data-path=" + BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER,
...
...
@@ -291,32 +246,40 @@ def get_beacon_config(
"--Xvalidator-api-unsafe-hosts-enabled=true",
]
supernode_cmd = [
"--p2p-subscribe-all-custody-subnets-enabled=true",
]
if participant.supernode:
cmd.extend(supernode_cmd)
# If checkpoint sync is enabled, add the checkpoint sync url
if checkpoint_sync_enabled:
if checkpoint_sync_url:
cmd.append("--checkpoint-sync-url=" + checkpoint_sync_url)
else:
if (
network in constants.PUBLIC_NETWORKS
or network == constants.NETWORK_NAME.ephemery
launcher.
network in constants.PUBLIC_NETWORKS
or
launcher.
network == constants.NETWORK_NAME.ephemery
):
cmd.append(
"--checkpoint-sync-url=" + constants.CHECKPOINT_SYNC_URL[network]
"--checkpoint-sync-url="
+ constants.CHECKPOINT_SYNC_URL[launcher.network]
)
else:
fail(
"Checkpoint sync URL is required if you enabled checkpoint_sync for custom networks. Please provide a valid URL."
)
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append(
"--initial-state="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
+ "/genesis.ssz"
)
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if bootnode_contexts != None:
cmd.append(
...
...
@@ -328,45 +291,45 @@ def get_beacon_config(
]
)
)
elif network == constants.NETWORK_NAME.ephemery:
elif
launcher.
network == constants.NETWORK_NAME.ephemery:
cmd.append(
"--p2p-discovery-bootnodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
elif constants.NETWORK_NAME.shadowfork in network:
elif constants.NETWORK_NAME.shadowfork in
launcher.
network:
cmd.append(
"--p2p-discovery-bootnodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
else: # Devnets
cmd.append(
"--p2p-discovery-bootnodes="
+ shared_utils.get_devnet_enrs_list(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
if len(
participant.cl_
extra_params) > 0:
# we do the list comprehension as the default extra_params is a proto repeated string
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.cl_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if node_keystore_files != None and not use_separate_vc:
if node_keystore_files != None and not
participant.
use_separate_vc:
cmd.extend(validator_default_cmd)
files[
constants.VALIDATOR_KEYS_DIRPATH_ON_SERVICE_CONTAINER
] = node_keystore_files.files_artifact_uuid
if keymanager_enabled:
files[constants.KEYMANAGER_MOUNT_PATH_ON_CLIENTS] = keymanager_file
if
participant.
keymanager_enabled:
files[constants.KEYMANAGER_MOUNT_PATH_ON_CLIENTS] =
launcher.
keymanager_file
cmd.extend(keymanager_api_cmd)
used_ports.update(vc_shared.VALIDATOR_KEYMANAGER_USED_PORTS)
public_ports.update(
...
...
@@ -375,17 +338,21 @@ def get_beacon_config(
if persistent:
files[BEACON_DATA_DIRPATH_ON_SERVICE_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=cl_volume_size,
persistent_key="data-{0}".format(beacon_service_name),
size=int(participant.cl_volume_size)
if int(participant.cl_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.CL_TYPE.teku + "_volume_size"
],
)
config_args = {
"image": image,
"image":
participant.cl_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": extra_env_vars,
"env_vars":
participant.cl_
extra_env_vars,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"ready_conditions": cl_node_ready_conditions.get_ready_conditions(
constants.HTTP_PORT_ID
...
...
@@ -393,27 +360,23 @@ def get_beacon_config(
"labels": shared_utils.label_maker(
constants.CL_TYPE.teku,
constants.CLIENT_TYPES.cl,
image,
participant.cl_
image,
el_context.client_name,
extra_labels,
participant.cl_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
"user": User(uid=0, gid=0),
}
if cl_min_cpu > 0:
config_args["min_cpu"] = cl_min_cpu
if cl_max_cpu > 0:
config_args["max_cpu"] = cl_max_cpu
if cl_min_mem > 0:
config_args["min_memory"] = cl_min_mem
if cl_max_mem > 0:
config_args["max_memory"] = cl_max_mem
if int(participant.cl_min_cpu) > 0:
config_args["min_cpu"] = int(participant.cl_min_cpu)
if int(participant.cl_max_cpu) > 0:
config_args["max_cpu"] = int(participant.cl_max_cpu)
if int(participant.cl_min_mem) > 0:
config_args["min_memory"] = int(participant.cl_min_mem)
if int(participant.cl_max_mem) > 0:
config_args["max_memory"] = int(participant.cl_max_mem)
return ServiceConfig(**config_args)
...
...
src/el/besu/besu_launcher.star
View file @
8513c060
...
...
@@ -34,49 +34,30 @@ def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
participant,
global_log_level,
existing_el_clients,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.el
_log_level, global_log_level, VERBOSITY_LEVELS
)
cl_client_name = service_name.split("-")[3]
config = get_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -112,23 +93,13 @@ def launch(
def get_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -186,24 +157,26 @@ def get_config(
"--engine-rpc-port={0}".format(ENGINE_HTTP_RPC_PORT_NUM),
"--sync-mode=FULL",
"--data-storage-format={0}".format(
"VERKLE" if "verkle-gen" in network else "BONSAI"
"VERKLE" if "verkle-gen" in
launcher.
network else "BONSAI"
),
"--metrics-enabled=true",
"--metrics-host=0.0.0.0",
"--metrics-port={0}".format(METRICS_PORT_NUM),
"--min-gas-price=1000000000",
"--bonsai-limit-trie-logs-enabled=false" if "verkle" not in network else "",
"--bonsai-limit-trie-logs-enabled=false"
if "verkle" not in launcher.network
else "",
]
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append(
"--genesis-file="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
+ "/besu.json"
)
else:
cmd.append("--network=" + network)
cmd.append("--network=" +
launcher.
network)
if network == constants.NETWORK_NAME.kurtosis:
if
launcher.
network == constants.NETWORK_NAME.kurtosis:
if len(existing_el_clients) > 0:
cmd.append(
"--bootnodes="
...
...
@@ -215,64 +188,68 @@ def get_config(
)
)
elif (
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in network
launcher.
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in
launcher.
network
):
cmd.append(
"--bootnodes="
+ shared_utils.get_devnet_enodes(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
if len(
participant.el_
extra_params) > 0:
# we do this as extra_params isn't a normal [] but a proto repeated array
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.el_
extra_params])
cmd_str = " ".join(cmd)
e
xtra_env_vars =
extra_env_vars | JAVA_OPTS
e
nv_vars = participant.el_
extra_env_vars | JAVA_OPTS
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=el_volume_size,
size=int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
)
config_args = {
"image": image,
"image":
participant.el_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": [cmd_str],
"files": files,
"entrypoint": ENTRYPOINT_ARGS,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.EL_TYPE.besu,
constants.CLIENT_TYPES.el,
image,
participant.el_
image,
cl_client_name,
extra_labels,
participant.el_
extra_labels,
),
"user": User(uid=0, gid=0),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if el_min_cpu > 0:
config_args["min_cpu"] = el_min_cpu
if el_max_cpu > 0:
config_args["max_cpu"] = el_max_cpu
if el_min_mem > 0:
config_args["min_memory"] = el_min_mem
if el_max_mem > 0:
config_args["max_memory"] = el_max_mem
if
participant.
el_min_cpu > 0:
config_args["min_cpu"] =
participant.
el_min_cpu
if
participant.
el_max_cpu > 0:
config_args["max_cpu"] =
participant.
el_max_cpu
if
participant.
el_min_mem > 0:
config_args["min_memory"] =
participant.
el_min_mem
if
participant.
el_max_mem > 0:
config_args["max_memory"] =
participant.
el_max_mem
return ServiceConfig(**config_args)
...
...
src/el/el_launcher.star
View file @
8513c060
...
...
@@ -110,22 +110,6 @@ def launch(
participant.el_tolerations, participant.tolerations, global_tolerations
)
(
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
el_volume_size,
) = shared_utils.get_cpu_mem_resource_limits(
participant.el_min_cpu,
participant.el_max_cpu,
participant.el_min_mem,
participant.el_max_mem,
participant.el_volume_size,
network_name,
participant.el_type,
)
if el_type not in el_launchers:
fail(
"Unsupported launcher '{0}', need one of '{1}'".format(
...
...
@@ -147,19 +131,10 @@ def launch(
plan,
el_launcher,
el_service_name,
participant.el_image,
participant.el_log_level,
participant,
global_log_level,
all_el_contexts,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
participant.el_extra_params,
participant.el_extra_env_vars,
participant.el_extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
src/el/erigon/erigon_launcher.star
View file @
8513c060
...
...
@@ -31,51 +31,30 @@ def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
participant,
global_log_level,
existing_el_clients,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.el
_log_level, global_log_level, VERBOSITY_LEVELS
)
cl_client_name = service_name.split("-")[3]
config = get_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
launcher.networkid,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
launcher.prague_time,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -113,25 +92,13 @@ def launch(
def get_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
networkid,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
verbosity_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
prague_time,
log_level,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -171,12 +138,12 @@ def get_config(
cmd = [
"erigon",
"{0}".format(
"--override.prague=" + str(prague_time)
if constants.NETWORK_NAME.shadowfork in network
"--override.prague=" + str(
launcher.
prague_time)
if constants.NETWORK_NAME.shadowfork in
launcher.
network
else ""
),
"--networkid={0}".format(networkid),
"--log.console.verbosity=" +
verbosity
_level,
"--networkid={0}".format(
launcher.
networkid),
"--log.console.verbosity=" +
log
_level,
"--datadir=" + EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER,
"--port={0}".format(discovery_port),
"--http.api=eth,erigon,engine,web3,net,debug,trace,txpool,admin",
...
...
@@ -195,21 +162,31 @@ def get_config(
"--metrics",
"--metrics.addr=0.0.0.0",
"--metrics.port={0}".format(METRICS_PORT_NUM),
"--db.size.limit={0}MB".format(el_volume_size),
"--db.size.limit={0}MB".format(
int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
),
]
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=el_volume_size,
size=int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
)
if network == constants.NETWORK_NAME.kurtosis:
if
launcher.
network == constants.NETWORK_NAME.kurtosis:
if len(existing_el_clients) > 0:
cmd.append(
"--bootnodes="
...
...
@@ -221,54 +198,55 @@ def get_config(
)
)
elif (
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in network
launcher.
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in
launcher.
network
):
cmd.append(
"--bootnodes="
+ shared_utils.get_devnet_enodes(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
cmd.extend([param for param in extra_params])
if len(
participant.el_
extra_params) > 0:
cmd.extend([param for param in
participant.el_
extra_params])
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
command_arg = [init_datadir_cmd_str, " ".join(cmd)]
command_arg_str = " && ".join(command_arg)
else:
command_arg_str = " ".join(cmd)
env_vars = participant.el_extra_env_vars
config_args = {
"image": image,
"image":
participant.el_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": [command_arg_str],
"files": files,
"entrypoint": ENTRYPOINT_ARGS,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.EL_TYPE.erigon,
constants.CLIENT_TYPES.el,
image,
participant.el_
image,
cl_client_name,
extra_labels,
participant.el_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
"user": User(uid=0, gid=0),
}
if el_min_cpu > 0:
config_args["min_cpu"] = el_min_cpu
if el_max_cpu > 0:
config_args["max_cpu"] = el_max_cpu
if el_min_mem > 0:
config_args["min_memory"] = el_min_mem
if el_max_mem > 0:
config_args["max_memory"] = el_max_mem
if
participant.
el_min_cpu > 0:
config_args["min_cpu"] =
participant.
el_min_cpu
if
participant.
el_max_cpu > 0:
config_args["max_cpu"] =
participant.
el_max_cpu
if
participant.
el_min_mem > 0:
config_args["min_memory"] =
participant.
el_min_mem
if
participant.
el_max_mem > 0:
config_args["max_memory"] =
participant.
el_max_mem
return ServiceConfig(**config_args)
...
...
src/el/ethereumjs/ethereumjs_launcher.star
View file @
8513c060
...
...
@@ -33,49 +33,30 @@ def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
participant,
global_log_level,
existing_el_clients,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.el
_log_level, global_log_level, VERBOSITY_LEVELS
)
cl_client_name = service_name.split("-")[3]
config = get_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -111,23 +92,13 @@ def launch(
def get_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
verbosity_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
log_level,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -180,19 +151,19 @@ def get_config(
"--extIP={0}".format(port_publisher.nat_exit_ip),
"--sync=full",
"--isSingleNode=true",
"--logLevel={0}".format(
verbosity
_level),
"--logLevel={0}".format(
log
_level),
]
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append(
"--gethGenesis="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
+ "/genesis.json",
)
else:
cmd.append("--network=" + network)
cmd.append("--network=" +
launcher.
network)
if network == constants.NETWORK_NAME.kurtosis:
if
launcher.
network == constants.NETWORK_NAME.kurtosis:
if len(existing_el_clients) > 0:
cmd.append(
"--bootnodes="
...
...
@@ -204,59 +175,63 @@ def get_config(
)
)
elif (
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in network
launcher.
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in
launcher.
network
):
cmd.append(
"--bootnodes="
+ shared_utils.get_devnet_enodes(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
if len(
participant.el_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.el_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=el_volume_size,
size=int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
)
env_vars = participant.el_extra_env_vars
config_args = {
"image": image,
"image":
participant.el_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"entrypoint": ENTRYPOINT_ARGS,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.EL_TYPE.ethereumjs,
constants.CLIENT_TYPES.el,
image,
participant.el_
image,
cl_client_name,
extra_labels,
participant.el_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if el_min_cpu > 0:
config_args["min_cpu"] = el_min_cpu
if el_max_cpu > 0:
config_args["max_cpu"] = el_max_cpu
if el_min_mem > 0:
config_args["min_memory"] = el_min_mem
if el_max_mem > 0:
config_args["max_memory"] = el_max_mem
if
participant.
el_min_cpu > 0:
config_args["min_cpu"] =
participant.
el_min_cpu
if
participant.
el_max_cpu > 0:
config_args["max_cpu"] =
participant.
el_max_cpu
if
participant.
el_min_mem > 0:
config_args["min_memory"] =
participant.
el_min_mem
if
participant.
el_max_mem > 0:
config_args["max_memory"] =
participant.
el_max_mem
return ServiceConfig(**config_args)
...
...
src/el/geth/geth_launcher.star
View file @
8513c060
...
...
@@ -41,52 +41,30 @@ def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
participant,
global_log_level,
# If empty then the node will be launched as a bootnode
existing_el_clients,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.el
_log_level, global_log_level, VERBOSITY_LEVELS
)
cl_client_name = service_name.split("-")[3]
config = get_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
launcher.networkid,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
launcher.prague_time,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -124,42 +102,33 @@ def launch(
def get_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
networkid,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
verbosity_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
prague_time,
log_level,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
if "--gcmode=archive" in extra_params or "--gcmode archive" in extra_params:
if (
"--gcmode=archive" in participant.el_extra_params
or "--gcmode archive" in participant.el_extra_params
):
gcmode_archive = True
else:
gcmode_archive = False
# TODO: Remove this once electra fork has path based storage scheme implemented
if (
constants.NETWORK_NAME.verkle in network
) and constants.NETWORK_NAME.shadowfork not in network:
if constants.NETWORK_NAME.verkle + "-gen" in network: # verkle-gen
constants.NETWORK_NAME.verkle in
launcher.
network
) and constants.NETWORK_NAME.shadowfork not in
launcher.
network:
if constants.NETWORK_NAME.verkle + "-gen" in
launcher.
network: # verkle-gen
init_datadir_cmd_str = "geth --datadir={0} --cache.preimages --override.prague={1} init {2}".format(
EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER,
prague_time,
launcher.
prague_time,
constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER + "/genesis.json",
)
else: # verkle
...
...
@@ -169,7 +138,7 @@ def get_config(
constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER + "/genesis.json",
)
)
elif constants.NETWORK_NAME.shadowfork in network: # shadowfork
elif constants.NETWORK_NAME.shadowfork in
launcher.
network: # shadowfork
init_datadir_cmd_str = "echo shadowfork"
elif gcmode_archive: # Disable path based storage scheme archive mode
...
...
@@ -216,15 +185,19 @@ def get_config(
# Disable path based storage scheme for electra fork and verkle
# TODO: REMOVE Once geth default db is path based, and builder rebased
"{0}".format(
"--state.scheme=hash" if "verkle" in network or gcmode_archive else ""
"--state.scheme=hash"
if "verkle" in launcher.network or gcmode_archive
else ""
),
# Override prague fork timestamp for electra fork
"{0}".format("--cache.preimages" if "verkle" in network else ""),
"{0}".format("--cache.preimages" if "verkle" in
launcher.
network else ""),
"{0}".format(
"--{}".format(network) if network in constants.PUBLIC_NETWORKS else ""
"--{}".format(launcher.network)
if launcher.network in constants.PUBLIC_NETWORKS
else ""
),
"--networkid={0}".format(networkid),
"--verbosity=" +
verbosity
_level,
"--networkid={0}".format(
launcher.
networkid),
"--verbosity=" +
log
_level,
"--datadir=" + EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER,
"--http",
"--http.addr=0.0.0.0",
...
...
@@ -240,7 +213,6 @@ def get_config(
"--ws.origins=*",
"--allow-insecure-unlock",
"--nat=extip:" + port_publisher.nat_exit_ip,
"--verbosity=" + verbosity_level,
"--authrpc.port={0}".format(ENGINE_RPC_PORT_NUM),
"--authrpc.addr=0.0.0.0",
"--authrpc.vhosts=*",
...
...
@@ -254,14 +226,14 @@ def get_config(
"--port={0}".format(discovery_port),
]
if BUILDER_IMAGE_STR in image:
if BUILDER_IMAGE_STR in
participant.el_
image:
for index, arg in enumerate(cmd):
if "--http.api" in arg:
cmd[index] = "--http.api=admin,engine,net,eth,web3,debug,mev,flashbots"
if "--ws.api" in arg:
cmd[index] = "--ws.api=admin,engine,net,eth,web3,debug,mev,flashbots"
if SUAVE_ENABLED_GETH_IMAGE_STR in image:
if SUAVE_ENABLED_GETH_IMAGE_STR in
participant.el_
image:
for index, arg in enumerate(cmd):
if "--http.api" in arg:
cmd[index] = "--http.api=admin,engine,net,eth,web3,debug,suavex"
...
...
@@ -269,8 +241,8 @@ def get_config(
cmd[index] = "--ws.api=admin,engine,net,eth,web3,debug,suavex"
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if len(existing_el_clients) > 0:
cmd.append(
...
...
@@ -282,30 +254,30 @@ def get_config(
]
)
)
if constants.NETWORK_NAME.shadowfork in network: # shadowfork
cmd.append("--override.prague=" + str(prague_time))
if "verkle" in network: # verkle-shadowfork
if constants.NETWORK_NAME.shadowfork in
launcher.
network: # shadowfork
cmd.append("--override.prague=" + str(
launcher.
prague_time))
if "verkle" in
launcher.
network: # verkle-shadowfork
cmd.append("--override.overlay-stride=10000")
cmd.append("--override.blockproof=true")
cmd.append("--clear.verkle.costs=true")
elif (
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in network
launcher.
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in
launcher.
network
):
cmd.append(
"--bootnodes="
+ shared_utils.get_devnet_enodes(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
if len(
participant.el_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.el_
extra_params])
cmd_str = " ".join(cmd)
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
subcommand_strs = [
init_datadir_cmd_str,
cmd_str,
...
...
@@ -315,42 +287,47 @@ def get_config(
command_str = cmd_str
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=el_volume_size,
size=int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
)
env_vars = participant.el_extra_env_vars
config_args = {
"image": image,
"image":
participant.el_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": [command_str],
"files": files,
"entrypoint": ENTRYPOINT_ARGS,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.EL_TYPE.geth,
constants.CLIENT_TYPES.el,
image,
participant.el_
image,
cl_client_name,
extra_labels,
participant.el_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if el_min_cpu > 0:
config_args["min_cpu"] = el_min_cpu
if el_max_cpu > 0:
config_args["max_cpu"] = el_max_cpu
if el_min_mem > 0:
config_args["min_memory"] = el_min_mem
if el_max_mem > 0:
config_args["max_memory"] = el_max_mem
if
participant.
el_min_cpu > 0:
config_args["min_cpu"] =
participant.
el_min_cpu
if
participant.
el_max_cpu > 0:
config_args["max_cpu"] =
participant.
el_max_cpu
if
participant.
el_min_mem > 0:
config_args["min_memory"] =
participant.
el_min_mem
if
participant.
el_max_mem > 0:
config_args["max_memory"] =
participant.
el_max_mem
return ServiceConfig(**config_args)
...
...
src/el/nethermind/nethermind_launcher.star
View file @
8513c060
...
...
@@ -30,49 +30,30 @@ def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
participant,
global_log_level,
existing_el_clients,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.el
_log_level, global_log_level, VERBOSITY_LEVELS
)
cl_client_name = service_name.split("-")[3]
config = get_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -109,23 +90,13 @@ def launch(
def get_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -179,15 +150,15 @@ def get_config(
"--Metrics.ExposeHost=0.0.0.0",
]
if constants.NETWORK_NAME.shadowfork in network:
if constants.NETWORK_NAME.shadowfork in
launcher.
network:
cmd.append(
"--Init.ChainSpecPath="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
+ "/chainspec.json"
)
cmd.append("--config=" + network.split("-")[0])
cmd.append("--Init.BaseDbPath=" + network.split("-")[0])
elif network not in constants.PUBLIC_NETWORKS:
cmd.append("--config=" +
launcher.
network.split("-")[0])
cmd.append("--Init.BaseDbPath=" +
launcher.
network.split("-")[0])
elif
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append("--config=none.cfg")
cmd.append(
"--Init.ChainSpecPath="
...
...
@@ -195,11 +166,11 @@ def get_config(
+ "/chainspec.json"
)
else:
cmd.append("--config=" + network)
cmd.append("--config=" +
launcher.
network)
if (
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in network
launcher.
network == constants.NETWORK_NAME.kurtosis
or constants.NETWORK_NAME.shadowfork in
launcher.
network
):
if len(existing_el_clients) > 0:
cmd.append(
...
...
@@ -212,58 +183,62 @@ def get_config(
)
)
elif (
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in network
launcher.
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in
launcher.
network
):
cmd.append(
"--Discovery.Bootnodes="
+ shared_utils.get_devnet_enodes(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
if len(
participant.el_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.el_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=el_volume_size,
size=int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
)
env_vars = participant.el_extra_env_vars
config_args = {
"image": image,
"image":
participant.el_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.EL_TYPE.nethermind,
constants.CLIENT_TYPES.el,
image,
participant.el_
image,
cl_client_name,
extra_labels,
participant.el_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if el_min_cpu > 0:
config_args["min_cpu"] = el_min_cpu
if el_max_cpu > 0:
config_args["max_cpu"] = el_max_cpu
if el_min_mem > 0:
config_args["min_memory"] = el_min_mem
if el_max_mem > 0:
config_args["max_memory"] = el_max_mem
if
participant.
el_min_cpu > 0:
config_args["min_cpu"] =
participant.
el_min_cpu
if
participant.
el_max_cpu > 0:
config_args["max_cpu"] =
participant.
el_max_cpu
if
participant.
el_min_mem > 0:
config_args["min_memory"] =
participant.
el_min_mem
if
participant.
el_max_mem > 0:
config_args["max_memory"] =
participant.
el_max_mem
return ServiceConfig(**config_args)
...
...
src/el/nimbus-eth1/nimbus_launcher.star
View file @
8513c060
...
...
@@ -30,50 +30,30 @@ def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
participant,
global_log_level,
# If empty then the node will be launched as a bootnode
existing_el_clients,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.el
_log_level, global_log_level, VERBOSITY_LEVELS
)
cl_client_name = service_name.split("-")[3]
config = get_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -110,23 +90,13 @@ def launch(
def get_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
verbosity_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
log_level,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
...
...
@@ -159,7 +129,7 @@ def get_config(
used_ports = shared_utils.get_port_specs(used_port_assignments)
cmd = [
"--log-level={0}".format(
verbosity
_level),
"--log-level={0}".format(
log
_level),
"--data-dir=" + EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER,
"--http-port={0}".format(WS_RPC_PORT_NUM),
"--http-address=0.0.0.0",
...
...
@@ -177,16 +147,16 @@ def get_config(
"--nat=extip:{0}".format(port_publisher.nat_exit_ip),
"--tcp-port={0}".format(discovery_port),
]
if network not in constants.PUBLIC_NETWORKS:
if
launcher.
network not in constants.PUBLIC_NETWORKS:
cmd.append(
"--custom-network="
+ constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER
+ "/genesis.json"
)
else:
cmd.append("--network=" + network)
cmd.append("--network=" +
launcher.
network)
if network == constants.NETWORK_NAME.kurtosis:
if
launcher.
network == constants.NETWORK_NAME.kurtosis:
if len(existing_el_clients) > 0:
cmd.append(
"--bootstrap-node="
...
...
@@ -198,58 +168,62 @@ def get_config(
)
)
elif (
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in network
launcher.
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in
launcher.
network
):
cmd.append(
"--bootstrap-node="
+ shared_utils.get_devnet_enodes(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
if len(
participant.el_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.el_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=el_volume_size,
size=int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
)
env_vars = participant.el_extra_env_vars
config_args = {
"image": image,
"image":
participant.el_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.EL_TYPE.nimbus,
constants.CLIENT_TYPES.el,
image,
participant.el_
image,
cl_client_name,
extra_labels,
participant.el_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if el_min_cpu > 0:
config_args["min_cpu"] = el_min_cpu
if el_max_cpu > 0:
config_args["max_cpu"] = el_max_cpu
if el_min_mem > 0:
config_args["min_memory"] = el_min_mem
if el_max_mem > 0:
config_args["max_memory"] = el_max_mem
if
participant.
el_min_cpu > 0:
config_args["min_cpu"] =
participant.
el_min_cpu
if
participant.
el_max_cpu > 0:
config_args["max_cpu"] =
participant.
el_max_cpu
if
participant.
el_min_mem > 0:
config_args["min_memory"] =
participant.
el_min_mem
if
participant.
el_max_mem > 0:
config_args["max_memory"] =
participant.
el_max_mem
return ServiceConfig(**config_args)
...
...
src/el/reth/reth_launcher.star
View file @
8513c060
...
...
@@ -34,53 +34,32 @@ def launch(
plan,
launcher,
service_name,
image,
participant_log_level,
participant,
global_log_level,
# If empty then the node will be launched as a bootnode
existing_el_clients,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
port_publisher,
participant_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.el
_log_level, global_log_level, VERBOSITY_LEVELS
)
cl_client_name = service_name.split("-")[3]
config = get_config(
plan,
launcher.el_cl_genesis_data,
launcher.jwt_file,
launcher.network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
log_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
persistent,
el_volume_size,
tolerations,
node_selectors,
launcher.builder,
port_publisher,
participant_index,
)
...
...
@@ -115,26 +94,15 @@ def launch(
def get_config(
plan,
el_cl_genesis_data,
jwt_file,
network,
image,
launcher,
participant,
service_name,
existing_el_clients,
cl_client_name,
verbosity_level,
el_min_cpu,
el_max_cpu,
el_min_mem,
el_max_mem,
extra_params,
extra_env_vars,
extra_labels,
log_level,
persistent,
el_volume_size,
tolerations,
node_selectors,
builder,
port_publisher,
participant_index,
):
...
...
@@ -167,13 +135,13 @@ def get_config(
used_ports = shared_utils.get_port_specs(used_port_assignments)
cmd = [
"/usr/local/bin/mev build" if builder else "reth",
"/usr/local/bin/mev build" if
launcher.
builder else "reth",
"node",
"-{0}".format(
verbosity
_level),
"-{0}".format(
log
_level),
"--datadir=" + EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER,
"--chain={0}".format(
network
if network in constants.PUBLIC_NETWORKS
launcher.
network
if
launcher.
network in constants.PUBLIC_NETWORKS
else constants.GENESIS_CONFIG_MOUNT_PATH_ON_CONTAINER + "/genesis.json"
),
"--http",
...
...
@@ -197,7 +165,7 @@ def get_config(
"--port={0}".format(discovery_port),
]
if network == constants.NETWORK_NAME.kurtosis:
if
launcher.
network == constants.NETWORK_NAME.kurtosis:
if len(existing_el_clients) > 0:
cmd.append(
"--bootnodes="
...
...
@@ -209,66 +177,71 @@ def get_config(
)
)
elif (
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in network
launcher.
network not in constants.PUBLIC_NETWORKS
and constants.NETWORK_NAME.shadowfork not in
launcher.
network
):
cmd.append(
"--bootnodes="
+ shared_utils.get_devnet_enodes(
plan, el_cl_genesis_data.files_artifact_uuid
plan,
launcher.
el_cl_genesis_data.files_artifact_uuid
)
)
if len(extra_params) > 0:
if len(
participant.el_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.el_
extra_params])
cmd_str = " ".join(cmd)
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS: jwt_file,
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS:
launcher.
el_cl_genesis_data.files_artifact_uuid,
constants.JWT_MOUNTPOINT_ON_CLIENTS:
launcher.
jwt_file,
}
if persistent:
files[EXECUTION_DATA_DIRPATH_ON_CLIENT_CONTAINER] = Directory(
persistent_key="data-{0}".format(service_name),
size=el_volume_size,
size=int(participant.el_volume_size)
if int(participant.el_volume_size) > 0
else constants.VOLUME_SIZE[launcher.network][
constants.EL_TYPE.besu + "_volume_size"
],
)
if builder:
if
launcher.
builder:
files[
mev_rs_builder.MEV_BUILDER_MOUNT_DIRPATH_ON_SERVICE
] = mev_rs_builder.MEV_BUILDER_FILES_ARTIFACT_NAME
env_vars = participant.el_extra_env_vars
config_args = {
"image": image,
"image":
participant.el_
image,
"ports": used_ports,
"public_ports": public_ports,
"cmd": [cmd_str],
"files": files,
"entrypoint": ENTRYPOINT_ARGS,
"private_ip_address_placeholder": constants.PRIVATE_IP_ADDRESS_PLACEHOLDER,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.EL_TYPE.
g
eth,
constants.EL_TYPE.
r
eth,
constants.CLIENT_TYPES.el,
image,
participant.el_
image,
cl_client_name,
extra_labels,
participant.el_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if el_min_cpu > 0:
config_args["min_cpu"] = el_min_cpu
if el_max_cpu > 0:
config_args["max_cpu"] = el_max_cpu
if el_min_mem > 0:
config_args["min_memory"] = el_min_mem
if el_max_mem > 0:
config_args["max_memory"] = el_max_mem
if
participant.
el_min_cpu > 0:
config_args["min_cpu"] =
participant.
el_min_cpu
if
participant.
el_max_cpu > 0:
config_args["max_cpu"] =
participant.
el_max_cpu
if
participant.
el_min_mem > 0:
config_args["min_memory"] =
participant.
el_min_mem
if
participant.
el_max_mem > 0:
config_args["max_memory"] =
participant.
el_max_mem
return ServiceConfig(**config_args)
...
...
src/package_io/input_parser.star
View file @
8513c060
...
...
@@ -214,6 +214,7 @@ def input_parser(plan, input_args):
vc_extra_env_vars=participant["vc_extra_env_vars"],
vc_extra_labels=participant["vc_extra_labels"],
builder_network_params=participant["builder_network_params"],
supernode=participant["supernode"],
el_min_cpu=participant["el_min_cpu"],
el_max_cpu=participant["el_max_cpu"],
el_min_mem=participant["el_min_mem"],
...
...
@@ -887,6 +888,7 @@ def default_participant():
"cl_max_cpu": 0,
"cl_min_mem": 0,
"cl_max_mem": 0,
"supernode": False,
"use_separate_vc": None,
"vc_type": "",
"vc_image": "",
...
...
src/package_io/sanity_check.star
View file @
8513c060
...
...
@@ -24,6 +24,7 @@ PARTICIPANT_CATEGORIES = {
"cl_max_cpu",
"cl_min_mem",
"cl_max_mem",
"supernode",
"use_separate_vc",
"vc_type",
"vc_image",
...
...
src/vc/lighthouse.star
View file @
8513c060
...
...
@@ -16,22 +16,15 @@ VERBOSITY_LEVELS = {
def get_config(
participant,
el_cl_genesis_data,
image,
participant_log_level,
global_log_level,
beacon_http_url,
cl_context,
el_context,
full_name,
node_keystore_files,
vc_min_cpu,
vc_max_cpu,
vc_min_mem,
vc_max_mem,
extra_params,
extra_env_vars,
extra_labels,
tolerations,
node_selectors,
keymanager_enabled,
...
...
@@ -41,7 +34,7 @@ def get_config(
vc_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.vc
_log_level, global_log_level, VERBOSITY_LEVELS
)
validator_keys_dirpath = shared_utils.path_join(
...
...
@@ -84,15 +77,15 @@ def get_config(
"--unencrypted-http-transport",
]
if len(extra_params):
cmd.extend([param for param in extra_params])
if len(
participant.vc_
extra_params):
cmd.extend([param for param in
participant.vc_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
constants.VALIDATOR_KEYS_DIRPATH_ON_SERVICE_CONTAINER: node_keystore_files.files_artifact_uuid,
}
env = {RUST_BACKTRACE_ENVVAR_NAME: RUST_FULL_BACKTRACE_KEYWORD}
env.update(extra_env_vars)
env.update(
participant.vc_
extra_env_vars)
public_ports = {}
public_keymanager_port_assignment = {}
...
...
@@ -130,22 +123,18 @@ def get_config(
constants.CLIENT_TYPES.validator,
image,
cl_context.client_name,
extra_labels,
participant.vc_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if vc_min_cpu > 0:
config_args["min_cpu"] = vc_min_cpu
if vc_max_cpu > 0:
config_args["max_cpu"] = vc_max_cpu
if vc_min_mem > 0:
config_args["min_memory"] = vc_min_mem
if vc_max_mem > 0:
config_args["max_memory"] = vc_max_mem
if participant.vc_min_cpu > 0:
config_args["min_cpu"] = participant.vc_min_cpu
if participant.vc_max_cpu > 0:
config_args["max_cpu"] = participant.vc_max_cpu
if participant.vc_min_mem > 0:
config_args["min_memory"] = participant.vc_min_mem
if participant.vc_max_mem > 0:
config_args["max_memory"] = participant.vc_max_mem
return ServiceConfig(**config_args)
src/vc/lodestar.star
View file @
8513c060
...
...
@@ -13,23 +13,16 @@ VERBOSITY_LEVELS = {
def get_config(
participant,
el_cl_genesis_data,
keymanager_file,
image,
participant_log_level,
global_log_level,
beacon_http_url,
cl_context,
el_context,
full_name,
node_keystore_files,
vc_min_cpu,
vc_max_cpu,
vc_min_mem,
vc_max_mem,
extra_params,
extra_env_vars,
extra_labels,
tolerations,
node_selectors,
keymanager_enabled,
...
...
@@ -38,7 +31,7 @@ def get_config(
vc_index,
):
log_level = input_parser.get_client_log_level_or_default(
participant_log_level, global_log_level, VERBOSITY_LEVELS
participant
.vc
_log_level, global_log_level, VERBOSITY_LEVELS
)
validator_keys_dirpath = shared_utils.path_join(
...
...
@@ -80,9 +73,9 @@ def get_config(
"--keymanager.tokenFile=" + constants.KEYMANAGER_MOUNT_PATH_ON_CONTAINER,
]
if len(extra_params) > 0:
if len(
participant.vc_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.vc_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
...
...
@@ -114,8 +107,9 @@ def get_config(
shared_utils.get_port_specs(public_keymanager_port_assignment)
)
env_vars = participant.vc_extra_env_vars
if preset == "minimal":
e
xtra_e
nv_vars["LODESTAR_PRESET"] = "minimal"
env_vars["LODESTAR_PRESET"] = "minimal"
config_args = {
"image": image,
...
...
@@ -123,28 +117,24 @@ def get_config(
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": e
xtra_e
nv_vars,
"env_vars": env_vars,
"labels": shared_utils.label_maker(
constants.CL_TYPE.lodestar,
constants.CLIENT_TYPES.validator,
image,
cl_context.client_name,
extra_labels,
participant.vc_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if vc_min_cpu > 0:
config_args["min_cpu"] = vc_min_cpu
if vc_max_cpu > 0:
config_args["max_cpu"] = vc_max_cpu
if vc_min_mem > 0:
config_args["min_memory"] = vc_min_mem
if vc_max_mem > 0:
config_args["max_memory"] = vc_max_mem
if participant.vc_min_cpu > 0:
config_args["min_cpu"] = participant.vc_min_cpu
if participant.vc_max_cpu > 0:
config_args["max_cpu"] = participant.vc_max_cpu
if participant.vc_min_mem > 0:
config_args["min_memory"] = participant.vc_min_mem
if participant.vc_max_mem > 0:
config_args["max_memory"] = participant.vc_max_mem
return ServiceConfig(**config_args)
src/vc/nimbus.star
View file @
8513c060
...
...
@@ -4,6 +4,7 @@ vc_shared = import_module("./shared.star")
def get_config(
participant,
el_cl_genesis_data,
image,
keymanager_file,
...
...
@@ -12,13 +13,6 @@ def get_config(
el_context,
full_name,
node_keystore_files,
vc_min_cpu,
vc_max_cpu,
vc_min_mem,
vc_max_mem,
extra_params,
extra_env_vars,
extra_labels,
tolerations,
node_selectors,
keymanager_enabled,
...
...
@@ -57,9 +51,9 @@ def get_config(
"--keymanager-token-file=" + constants.KEYMANAGER_MOUNT_PATH_ON_CONTAINER,
]
if len(extra_params) > 0:
if len(
participant.vc_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.vc_
extra_params])
files = {
constants.VALIDATOR_KEYS_DIRPATH_ON_SERVICE_CONTAINER: node_keystore_files.files_artifact_uuid,
...
...
@@ -96,29 +90,25 @@ def get_config(
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": extra_env_vars,
"env_vars":
participant.vc_
extra_env_vars,
"labels": shared_utils.label_maker(
constants.CL_TYPE.nimbus,
constants.CLIENT_TYPES.validator,
image,
cl_context.client_name,
extra_labels,
participant.vc_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
"user": User(uid=0, gid=0),
}
if vc_min_cpu > 0:
config_args["min_cpu"] = vc_min_cpu
if vc_max_cpu > 0:
config_args["max_cpu"] = vc_max_cpu
if vc_min_mem > 0:
config_args["min_memory"] = vc_min_mem
if vc_max_mem > 0:
config_args["max_memory"] = vc_max_mem
if participant.vc_min_cpu > 0:
config_args["min_cpu"] = participant.vc_min_cpu
if participant.vc_max_cpu > 0:
config_args["max_cpu"] = participant.vc_max_cpu
if participant.vc_min_mem > 0:
config_args["min_memory"] = participant.vc_min_mem
if participant.vc_max_mem > 0:
config_args["max_memory"] = participant.vc_max_mem
return ServiceConfig(**config_args)
src/vc/prysm.star
View file @
8513c060
...
...
@@ -7,6 +7,7 @@ PRYSM_BEACON_RPC_PORT = 4000
def get_config(
participant,
el_cl_genesis_data,
keymanager_file,
image,
...
...
@@ -15,13 +16,6 @@ def get_config(
el_context,
full_name,
node_keystore_files,
vc_min_cpu,
vc_max_cpu,
vc_min_mem,
vc_max_mem,
extra_params,
extra_env_vars,
extra_labels,
prysm_password_relative_filepath,
prysm_password_artifact_uuid,
tolerations,
...
...
@@ -70,9 +64,9 @@ def get_config(
cmd.append("--beacon-rpc-provider=" + cl_context.beacon_grpc_url)
cmd.append("--beacon-rest-api-provider=" + cl_context.beacon_grpc_url)
if len(extra_params) > 0:
if len(
participant.vc_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.vc_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
...
...
@@ -116,28 +110,24 @@ def get_config(
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": extra_env_vars,
"env_vars":
participant.vc_
extra_env_vars,
"labels": shared_utils.label_maker(
constants.CL_TYPE.prysm,
constants.CLIENT_TYPES.validator,
image,
cl_context.client_name,
extra_labels,
participant.vc_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if vc_min_cpu > 0:
config_args["min_cpu"] = vc_min_cpu
if vc_max_cpu > 0:
config_args["max_cpu"] = vc_max_cpu
if vc_min_mem > 0:
config_args["min_memory"] = vc_min_mem
if vc_max_mem > 0:
config_args["max_memory"] = vc_max_mem
if participant.vc_min_cpu > 0:
config_args["min_cpu"] = participant.vc_min_cpu
if participant.vc_max_cpu > 0:
config_args["max_cpu"] = participant.vc_max_cpu
if participant.vc_min_mem > 0:
config_args["min_memory"] = participant.vc_min_mem
if participant.vc_max_mem > 0:
config_args["max_memory"] = participant.vc_max_mem
return ServiceConfig(**config_args)
src/vc/teku.star
View file @
8513c060
...
...
@@ -4,6 +4,7 @@ vc_shared = import_module("./shared.star")
def get_config(
participant,
el_cl_genesis_data,
keymanager_file,
image,
...
...
@@ -12,13 +13,6 @@ def get_config(
el_context,
full_name,
node_keystore_files,
vc_min_cpu,
vc_max_cpu,
vc_min_mem,
vc_max_mem,
extra_params,
extra_env_vars,
extra_labels,
tolerations,
node_selectors,
keymanager_enabled,
...
...
@@ -67,9 +61,9 @@ def get_config(
"--Xvalidator-api-unsafe-hosts-enabled=true",
]
if len(extra_params) > 0:
if len(
participant.vc_
extra_params) > 0:
# this is a repeated<proto type>, we convert it into Starlark
cmd.extend([param for param in extra_params])
cmd.extend([param for param in
participant.vc_
extra_params])
files = {
constants.GENESIS_DATA_MOUNTPOINT_ON_CLIENTS: el_cl_genesis_data.files_artifact_uuid,
...
...
@@ -107,28 +101,24 @@ def get_config(
"public_ports": public_ports,
"cmd": cmd,
"files": files,
"env_vars": extra_env_vars,
"env_vars":
participant.vc_
extra_env_vars,
"labels": shared_utils.label_maker(
constants.CL_TYPE.teku,
constants.CLIENT_TYPES.validator,
image,
cl_context.client_name,
extra_labels,
participant.vc_
extra_labels,
),
"tolerations": tolerations,
"node_selectors": node_selectors,
}
if vc_min_cpu > 0:
config_args["min_cpu"] = vc_min_cpu
if vc_max_cpu > 0:
config_args["max_cpu"] = vc_max_cpu
if vc_min_mem > 0:
config_args["min_memory"] = vc_min_mem
if vc_max_mem > 0:
config_args["max_memory"] = vc_max_mem
if participant.vc_min_cpu > 0:
config_args["min_cpu"] = participant.vc_min_cpu
if participant.vc_max_cpu > 0:
config_args["max_cpu"] = participant.vc_max_cpu
if participant.vc_min_mem > 0:
config_args["min_memory"] = participant.vc_min_mem
if participant.vc_max_mem > 0:
config_args["max_memory"] = participant.vc_max_mem
return ServiceConfig(**config_args)
src/vc/vc_launcher.star
View file @
8513c060
...
...
@@ -54,44 +54,18 @@ def launch(
cl_context.beacon_http_url,
)
(
vc_min_cpu,
vc_max_cpu,
vc_min_mem,
vc_max_mem,
_,
) = shared_utils.get_cpu_mem_resource_limits(
participant.vc_min_cpu,
participant.vc_max_cpu,
participant.vc_min_mem,
participant.vc_max_mem,
0,
network,
vc_type,
)
extra_params = participant.vc_extra_params
extra_env_vars = participant.vc_extra_env_vars
extra_labels = participant.vc_extra_labels
participant_log_level = participant.vc_log_level
keymanager_enabled = participant.keymanager_enabled
if vc_type == constants.VC_TYPE.lighthouse:
config = lighthouse.get_config(
participant=participant,
el_cl_genesis_data=launcher.el_cl_genesis_data,
image=image,
participant_log_level=participant_log_level,
global_log_level=global_log_level,
beacon_http_url=beacon_http_url,
cl_context=cl_context,
el_context=el_context,
full_name=full_name,
node_keystore_files=node_keystore_files,
vc_min_cpu=vc_min_cpu,
vc_max_cpu=vc_max_cpu,
vc_min_mem=vc_min_mem,
vc_max_mem=vc_max_mem,
extra_params=extra_params,
extra_env_vars=extra_env_vars,
extra_labels=extra_labels,
tolerations=tolerations,
node_selectors=node_selectors,
keymanager_enabled=keymanager_enabled,
...
...
@@ -102,23 +76,16 @@ def launch(
)
elif vc_type == constants.VC_TYPE.lodestar:
config = lodestar.get_config(
participant=participant,
el_cl_genesis_data=launcher.el_cl_genesis_data,
keymanager_file=keymanager_file,
image=image,
participant_log_level=participant_log_level,
global_log_level=global_log_level,
beacon_http_url=beacon_http_url,
cl_context=cl_context,
el_context=el_context,
full_name=full_name,
node_keystore_files=node_keystore_files,
vc_min_cpu=vc_min_cpu,
vc_max_cpu=vc_max_cpu,
vc_min_mem=vc_min_mem,
vc_max_mem=vc_max_mem,
extra_params=extra_params,
extra_env_vars=extra_env_vars,
extra_labels=extra_labels,
tolerations=tolerations,
node_selectors=node_selectors,
keymanager_enabled=keymanager_enabled,
...
...
@@ -128,6 +95,7 @@ def launch(
)
elif vc_type == constants.VC_TYPE.teku:
config = teku.get_config(
participant=participant,
el_cl_genesis_data=launcher.el_cl_genesis_data,
keymanager_file=keymanager_file,
image=image,
...
...
@@ -136,13 +104,6 @@ def launch(
el_context=el_context,
full_name=full_name,
node_keystore_files=node_keystore_files,
vc_min_cpu=vc_min_cpu,
vc_max_cpu=vc_max_cpu,
vc_min_mem=vc_min_mem,
vc_max_mem=vc_max_mem,
extra_params=extra_params,
extra_env_vars=extra_env_vars,
extra_labels=extra_labels,
tolerations=tolerations,
node_selectors=node_selectors,
keymanager_enabled=keymanager_enabled,
...
...
@@ -151,6 +112,7 @@ def launch(
)
elif vc_type == constants.VC_TYPE.nimbus:
config = nimbus.get_config(
participant=participant,
el_cl_genesis_data=launcher.el_cl_genesis_data,
keymanager_file=keymanager_file,
image=image,
...
...
@@ -159,13 +121,6 @@ def launch(
el_context=el_context,
full_name=full_name,
node_keystore_files=node_keystore_files,
vc_min_cpu=vc_min_cpu,
vc_max_cpu=vc_max_cpu,
vc_min_mem=vc_min_mem,
vc_max_mem=vc_max_mem,
extra_params=extra_params,
extra_env_vars=extra_env_vars,
extra_labels=extra_labels,
tolerations=tolerations,
node_selectors=node_selectors,
keymanager_enabled=keymanager_enabled,
...
...
@@ -174,6 +129,7 @@ def launch(
)
elif vc_type == constants.VC_TYPE.prysm:
config = prysm.get_config(
participant=participant,
el_cl_genesis_data=launcher.el_cl_genesis_data,
keymanager_file=keymanager_file,
image=image,
...
...
@@ -182,13 +138,6 @@ def launch(
el_context=el_context,
full_name=full_name,
node_keystore_files=node_keystore_files,
vc_min_cpu=vc_min_cpu,
vc_max_cpu=vc_max_cpu,
vc_min_mem=vc_min_mem,
vc_max_mem=vc_max_mem,
extra_params=extra_params,
extra_env_vars=extra_env_vars,
extra_labels=extra_labels,
prysm_password_relative_filepath=prysm_password_relative_filepath,
prysm_password_artifact_uuid=prysm_password_artifact_uuid,
tolerations=tolerations,
...
...
vicotor
@luxueqian
mentioned in commit
861cf2ec
·
Apr 13, 2025
mentioned in commit
861cf2ec
mentioned in commit 861cf2ec532efe6cc2496f53af192826607e7524
Toggle commit list
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment