Commit 770cbc62 authored by Mark Tyneway's avatar Mark Tyneway Committed by GitHub

repo: migrate away from pnpm (#11395)

Migrate to just from pnpm. This is continued tech debt
cleanup away from `pnpm` and typescript that is no longer
used.
parent bf4693e7
......@@ -188,38 +188,18 @@ jobs:
name: "Check L1 geth version"
command: ./ops/scripts/geth-version-checker.sh || (echo "geth version is wrong, update ci-builder"; false)
- install-contracts-dependencies
- restore_cache:
name: Restore PNPM Package Cache
keys:
- pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
- restore_cache:
name: Restore Go modules cache
key: gomod-{{ checksum "go.sum" }}
# Fetch node_modules into the pnpm store
# This will cache node_modules based on pnpm-lock so other steps can instantly install them with `pnpm install --prefer-offline`
# --prefer-offline installs node_modules instantly by just reading from cache if it exists rather than fetching from network
# when installing node_modules pnpm simply adds symlinks instead of copying the files which is why it is pretty much instant to run --prefer-offline
# this allows a caching strategy of only checking pnpm-lockfile so we don't have to keep it in sync with our packages
# For more information see https://pnpm.io/cli/fetch
- run:
name: Fetch dependencies
command: pnpm fetch --frozen-lockfile --prefer-offline
- save_cache:
name: Save PNPM Package Cache
key: pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
paths:
- "node_modules"
- run:
name: Install dependencies
command: pnpm install
- run:
name: print forge version
command: forge --version
- run:
name: Build monorepo
name: Build contracts
environment:
FOUNDRY_PROFILE: ci
command: pnpm build
command: just build
working_directory: packages/contracts-bedrock
- run:
name: Generate L2OO allocs
command: DEVNET_L2OO="true" make devnet-allocs
......@@ -244,12 +224,9 @@ jobs:
- persist_to_workspace:
root: "."
paths:
- "packages/**/dist"
- "packages/contracts-bedrock/cache"
- "packages/contracts-bedrock/artifacts"
- "packages/contracts-bedrock/forge-artifacts"
- "packages/contracts-bedrock/tsconfig.tsbuildinfo"
- "packages/contracts-bedrock/tsconfig.build.tsbuildinfo"
- ".devnet/allocs-l1.json"
- ".devnet/allocs-l2-delta.json"
- ".devnet/allocs-l2-ecotone.json"
......@@ -580,21 +557,12 @@ jobs:
resource_class: xlarge
steps:
- checkout
- install-contracts-dependencies
- restore_cache:
name: Restore PNPM Package Cache
keys:
- pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
- attach_workspace: { at: "." }
- install-contracts-dependencies
- check-changed:
patterns: contracts-bedrock,op-node
- setup_remote_docker:
docker_layer_caching: true
# populate node modules from the cache
- run:
name: Install dependencies
command: pnpm install
working_directory: packages/contracts-bedrock
- run:
name: forge version
command: forge --version
......@@ -692,14 +660,7 @@ jobs:
resource_class: medium
steps:
- checkout
- restore_cache:
name: Restore PNPM Package Cache
keys:
- pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
- attach_workspace: { at: "." }
- run:
name: Install dependencies
command: pnpm install
- check-changed:
patterns: contracts-bedrock
- run:
......@@ -1156,26 +1117,11 @@ jobs:
echo 'export PATH=$HOME/.foundry/bin:$PATH' >> $BASH_ENV
source $HOME/.bashrc
forge --version
- run:
name: Install NVM
command: |
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm --version
- run:
name: Install Node
command: |
nvm install
nvm use && node --version && npm --version
- run:
name: Install Just
command: |
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to $HOME/bin
echo 'export PATH="${PATH}:$HOME/bin"' >> $BASH_ENV
- run:
name: Install pnpm
command: |
npm i pnpm --global
- install-contracts-dependencies
- attach_workspace:
at: "."
......@@ -1204,9 +1150,9 @@ jobs:
docker tag "$IMAGE_BASE_PREFIX/op-challenger:<<pipeline.git.revision>>" "$IMAGE_BASE_PREFIX/op-challenger:devnet"
docker tag "$IMAGE_BASE_PREFIX/da-server:<<pipeline.git.revision>>" "$IMAGE_BASE_PREFIX/da-server:devnet"
- run:
name: pnpm install and build
command: |
pnpm install && pnpm build
name: Build contracts
working_directory: packages/contracts-bedrock
command: just build
- run:
name: Bring up the stack
command: |
......
......@@ -3,29 +3,5 @@ description: Common setup steps used by our workflows
runs:
using: composite
steps:
- name: Setup pnpm
uses: pnpm/action-setup@v3
with:
version: 9
- name: Setup node
uses: actions/setup-node@v3
with:
node-version-file: .nvmrc
registry-url: https://registry.npmjs.org
cache: pnpm
- name: Setup foundry
uses: foundry-rs/foundry-toolchain@v1
- name: Install node dependencies
shell: bash
run: pnpm install --frozen-lockfile
- name: Derive appropriate SHAs for base and head for `nx affected` commands
uses: nrwl/nx-set-shas@v3
with:
main-branch-name: "develop"
- run: |
echo "nx using following shas:"
echo "BASE: ${{ env.NX_BASE }}"
echo "HEAD: ${{ env.NX_HEAD }}"
shell: bash
......@@ -27,7 +27,7 @@ Interactions within this repository are subject to a [Code of Conduct](https://g
| [go](https://go.dev/) | `^1.21` | `go version` |
| [node](https://nodejs.org/en/) | `^20` | `node --version` |
| [nvm](https://github.com/nvm-sh/nvm) | `^0.39` | `nvm --version` |
| [pnpm](https://pnpm.io/installation) | `^8` | `pnpm --version` |
| [just](https://github.com/casey/just) | `^1.34.0`| `just --version` |
| [foundry](https://github.com/foundry-rs/foundry#installation) | `^0.2.0` | `forge --version` |
| [make](https://linux.die.net/man/1/make) | `^3` | `make --version` |
| [jq](https://github.com/jqlang/jq) | `^1.6` | `jq --version` |
......@@ -46,7 +46,7 @@ You can use [`nvm`](https://github.com/nvm-sh/nvm) to manage multiple versions o
`foundry` is updated frequently and occasionally contains breaking changes.
This repository pins a specific version of `foundry` inside of [`versions.json`](./versions.json).
Use the command `pnpm update:foundry` at the root of the monorepo to make sure that your version of `foundry` is the same as the one currently being used in CI.
Use the command `just update-foundry` at the root of the monorepo to make sure that your version of `foundry` is the same as the one currently being used in CI.
#### `direnv`
......@@ -89,19 +89,11 @@ Use the above command to rebuild the monorepo.
Before running tests: **follow the above instructions to get everything built.**
#### Running unit tests (typescript)
Run unit tests for all packages in parallel via:
```bash
pnpm test
```
To run unit tests for a specific package:
#### Running unit tests (solidity)
```bash
cd packages/package-to-test
pnpm test
cd packages/contracts-bedrock
just test
```
#### Running unit tests (Go)
......@@ -125,7 +117,7 @@ To run `slither` locally, do:
```bash
cd packages/contracts-bedrock
pip3 install slither-analyzer
pnpm slither
just slither
```
## Labels
......
......@@ -23,14 +23,6 @@ lint-go-fix: ## Lints Go code with specific linters and fixes reported issues
golangci-lint run -E goimports,sqlclosecheck,bodyclose,asciicheck,misspell,errorlint --timeout 5m -e "errors.As" -e "errors.Is" ./... --fix
.PHONY: lint-go-fix
build-ts: submodules ## Builds TypeScript components
if [ -f "$$NVM_DIR/nvm.sh" ]; then \
. $$NVM_DIR/nvm.sh && nvm use; \
fi
pnpm install:ci
pnpm build
.PHONY: build-ts
ci-builder: ## Builds the CI builder Docker image
docker build -t ci-builder -f ops/docker/ci-builder/Dockerfile .
.PHONY: ci-builder
......@@ -207,7 +199,7 @@ test-unit: ## Runs unit tests for all components
make -C ./op-proposer test
make -C ./op-batcher test
make -C ./op-e2e test
pnpm test
(cd packages/contracts-bedrock && just test)
.PHONY: test-unit
# Remove the baseline-commit to generate a base reading & show all issues
......
issues:
./ops/scripts/todo-checker.sh
lint-shellcheck:
find . -type f -name '*.sh' -not -path '*/node_modules/*' -not -path './packages/contracts-bedrock/lib/*' -not -path './packages/contracts-bedrock/kout*/*' -exec sh -c 'echo \"Checking $1\"; shellcheck \"$1\"' _ {} \\;
install-foundry:
curl -L https://foundry.paradigm.xyz | bash && just update-foundry
update-foundry:
bash ./ops/scripts/install-foundry.sh
check-foundry:
bash ./packages/contracts-bedrock/scripts/checks/check-foundry-install.sh
install-kontrol:
curl -L https://kframework.org/install | bash && just update-kontrol
update-kontrol:
kup install kontrol --version v$(jq -r .kontrol < versions.json)
install-abigen:
go install github.com/ethereum/go-ethereum/cmd/abigen@$(jq -r .abigen < versions.json)
print-abigen:
abigen --version | sed -e 's/[^0-9]/ /g' -e 's/^ *//g' -e 's/ *$//g' -e 's/ /./g' -e 's/^/v/'
check-abigen:
[[ $(just print-abigen) = $(cat versions.json | jq -r '.abigen') ]] && echo '✓ abigen versions match' || (echo '✗ abigen version mismatch. Run `just upgrade:abigen` to upgrade.' && exit 1)
upgrade-abigen:
jq '.abigen = $v' --arg v $(just print:abigen) <<<$(cat versions.json) > versions.json
install-slither:
pip3 install slither-analyzer==$(jq -r .slither < versions.json)
print-slither:
slither --version
check-slither:
[[ $(just print-slither) = $(jq -r .slither < versions.json) ]] && echo '✓ slither versions match' || (echo '✗ slither version mismatch. Run `just upgrade-slither` to upgrade.' && exit 1)
upgrade-slither:
jq '.slither = $v' --arg v $(just print-slither) <<<$(cat versions.json) > versions.json
......@@ -29,6 +29,6 @@ make test-http
### Troubleshooting
If you encounter errors:
* ensure you have the latest version of foundry installed: `pnpm update:foundry`
* ensure you have the latest version of foundry installed: `just update-foundry`
* try deleting the `packages/contracts-bedrock/forge-artifacts` directory
* if the above step doesn't fix the error, try `pnpm clean`
......@@ -92,33 +92,25 @@ COPY --from=rust-build /root/.foundry/bin/anvil /usr/local/bin/anvil
COPY --from=rust-build /root/.cargo/bin/svm /usr/local/bin/svm
COPY --from=rust-build /root/.cargo/bin/just /usr/local/bin/just
COPY .nvmrc .nvmrc
COPY ./versions.json ./versions.json
ENV NODE_MAJOR=20
RUN /bin/sh -c set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends bash curl openssh-client git build-essential ca-certificates jq gnupg binutils-mips-linux-gnu python3 python3-pip python3-setuptools; \
mkdir -p /etc/apt/keyrings; \
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg; \
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list; \
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg; \
chmod a+r /etc/apt/keyrings/docker.gpg; \
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null; \
apt-get update; \
apt-get install -y nodejs docker-ce-cli; \
apt-get install -y docker-ce-cli; \
ln -s /usr/local/go/bin/gofmt /usr/local/bin/gofmt; \
pip install capstone pyelftools; \
curl -fLSs https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh | bash; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
rm -rf /var/lib/apt/lists/*; \
rm -rf /root/.cache/pip; \
rm -rf /root/.cache/npm;
RUN npm i -g pnpm && npm i -g yarn@1 && pnpm --version && yarn --version
rm -rf /root/.cache/pip;
RUN svm install 0.5.17 && \
RUN svm install 0.8.25 && \
svm install 0.8.15 && \
svm install 0.8.19
......
{
"name": "optimism",
"version": "1.0.0",
"author": "OP Labs PBC",
"license": "MIT",
"scripts": {
"clean": "rm -rf node_modules packages/**/node_modules",
"build": "cd packages/contracts-bedrock && just build",
"test": "cd packages/contracts-bedrock && just test",
"issues": "./ops/scripts/todo-checker.sh",
"lint:shellcheck": "find . -type f -name '*.sh' -not -path '*/node_modules/*' -not -path './packages/contracts-bedrock/lib/*' -not -path './packages/contracts-bedrock/kout*/*' -exec sh -c 'echo \"Checking $1\"; shellcheck \"$1\"' _ {} \\;",
"install:foundry": "curl -L https://foundry.paradigm.xyz | bash && pnpm update:foundry",
"update:foundry": "bash ./ops/scripts/install-foundry.sh",
"check:foundry": "bash ./packages/contracts-bedrock/scripts/checks/check-foundry-install.sh",
"install:kontrol": "curl -L https://kframework.org/install | bash && pnpm update:kontrol",
"update:kontrol": "kup install kontrol --version v$(jq -r .kontrol < versions.json)",
"install:abigen": "go install github.com/ethereum/go-ethereum/cmd/abigen@$(jq -r .abigen < versions.json)",
"print:abigen": "abigen --version | sed -e 's/[^0-9]/ /g' -e 's/^ *//g' -e 's/ *$//g' -e 's/ /./g' -e 's/^/v/'",
"check:abigen": "[[ $(pnpm -s print:abigen) = $(cat versions.json | jq -r '.abigen') ]] && echo '✓ abigen versions match' || (echo '✗ abigen version mismatch. Run `pnpm upgrade:abigen` to upgrade.' && exit 1)",
"upgrade:abigen": "jq '.abigen = $v' --arg v $(pnpm -s print:abigen) <<<$(cat versions.json) > versions.json",
"install:slither": "pip3 install slither-analyzer==$(jq -r .slither < versions.json)",
"print:slither": "slither --version",
"check:slither": "[[ $(pnpm -s print:slither) = $(jq -r .slither < versions.json) ]] && echo '✓ slither versions match' || (echo '✗ slither version mismatch. Run `pnpm upgrade:slither` to upgrade.' && exit 1)",
"upgrade:slither": "jq '.slither = $v' --arg v $(pnpm -s print:slither) <<<$(cat versions.json) > versions.json"
}
}
......@@ -51,12 +51,11 @@ Enhancement suggestions are tracked as [GitHub issues](/issues).
The best place to begin contributing is by looking through the issues with the `good first issue` label. These are issues that are relatively easy to implement and are a great way to get familiar with the codebase.
Optimism's smart contracts are written in Solidity and we use [foundry](https://github.com/foundry-rs/foundry) as our development framework. To get started, you'll need to install several dependencies:
1. [pnpm](https://pnpm.io)
1. [just](https://github.com/casey/just)
1. Make sure to `just install`
1. [foundry](https://getfoundry.sh)
1. Foundry is built with [rust](https://www.rust-lang.org/tools/install), and this project uses a pinned version of foundry. Install the rust toolchain with `rustup`.
1. Make sure to install the version of foundry used by `ci-builder`, defined in the `versions.json` file in the root of this repo under the `foundry` key. Once you have `foundryup` installed, there is a helper to do this: `pnpm install:foundry`
1. Make sure to install the version of foundry used by `ci-builder`, defined in the `versions.json` file in the root of this repo under the `foundry` key. Once you have `foundryup` installed, there is a helper to do this: `just install-foundry`
1. [golang](https://golang.org/doc/install)
1. [python](https://www.python.org/downloads/)
......
......@@ -26,7 +26,7 @@ This directory contains documentation for all defined invariant tests within `co
## Usage
To auto-generate documentation for invariant tests, run `pnpm autogen:invariant-docs`.
To auto-generate documentation for invariant tests, run `just autogen-invariant-docs`.
## Documentation Standard
......
......@@ -8,7 +8,7 @@ VERSIONS_FILE="${MONOREPO_BASE}/versions.json"
if ! command -v forge &> /dev/null
then
# shellcheck disable=SC2006
echo "Is Foundry not installed? Consider installing via pnpm install:foundry" >&2
echo "Is Foundry not installed? Consider installing via just install-foundry" >&2
exit 1
fi
......@@ -34,5 +34,5 @@ if [ "$INSTALLED_VERSION" = "$EXPECTED_VERSION" ]; then
else
echo "Mismatch between installed Foundry version ($INSTALLED_VERSION) and expected version ($EXPECTED_VERSION)."
echo "Your version of Foundry may either not be up to date, or it could be a later version."
echo "Running pnpm update:foundry will install the expected version."
echo "Running just update-foundry will install the expected version."
fi
......@@ -32,7 +32,6 @@ versionFoundry() {
ver_git=$(version "$(git --version)")
ver_go=$(version "$(go version)")
ver_node=$(version "$(node --version)")
ver_pnpm=$(version "$(pnpm --version)")
ver_foundry=$(versionFoundry "$(forge --version)")
ver_make=$(version "$(make --version)")
ver_jq=$(version "$(jq --version)")
......@@ -44,7 +43,6 @@ echo "Dependency | Minimum | Actual"
echo "git 2 $ver_git"
echo "go 1.21 $ver_go"
echo "node 20 $ver_node"
echo "pnpm 9 $ver_pnpm"
echo "foundry 0.2.0 (a5efe4f) $ver_foundry"
echo "make 3 $ver_make"
echo "jq 1.6 $ver_jq"
......
......@@ -11,7 +11,7 @@ A lightweight binary for utilities accessed via `forge`'s `ffi` cheatcode in the
## Usage
To build, run `pnpm build:go-ffi` from this directory or the `contract-bedrock` package.
To build, run `just build-go-ffi` from this directory or the `contract-bedrock` package.
### In a Forge Test
......
......@@ -52,8 +52,8 @@ The directory is structured as follows
### Installation
1. `cd` to the root of this repo.
2. Install Foundry by running `pnpm install:foundry`. This installs `foundryup`, the foundry toolchain installer, then installs the required foundry version.
3. Install Kontrol by running `pnpm install:kontrol`. This installs `kup`, the package manager for RV tools, then installs the required kontrol version.
2. Install Foundry by running `just install-foundry`. This installs `foundryup`, the foundry toolchain installer, then installs the required foundry version.
3. Install Kontrol by running `just install-kontrol`. This installs `kup`, the package manager for RV tools, then installs the required kontrol version.
4. Install Docker.
## Usage
......
This source diff could not be displayed because it is too large. You can view the blob instead.
packages:
- 'packages/*'
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment