Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(a3p-integration): Document and clean up core-eval generation #10466

Merged
merged 2 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions a3p-integration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,12 +153,12 @@ make -C ../packages/deployment docker-build-sdk

In a3p-integration, many core-eval proposals' `submission` content has to be
generated from the local `agoric-sdk`, and must be rebuilt every time there is a
change. The `scripts/build-all-submissions.sh` script contains commands to
generate the core-eval content and move it to the expected proposal package's
submission directory. The generation is executed as part of `a3p-integration`'s
`build:submissions` step. Each proposal that requires such a build step should
add an entry to the `sdk-generate` array in the `agoricProposal` section of
`package.json`.
change. This package's package.json `build:submissions` script runs
`scripts/build-all-submissions.sh` to generate that content and move it into
submission subdirectories under the corresponding [proposal
directories](#package-layering). Each proposal that requires such a build step
should have a corresponding entry in the `sdk-generate` array of its proposal
directory's package.json `agoricProposal` section.

Submissions that don't need to pass in options or generate references to source
bundles can be written directly in a `foo-submission` subdirectory of the
Expand All @@ -172,24 +172,25 @@ provided, it should be written as two parts: a core eval (in one of the
`.../proposals` directories) and a builder for it (under `.../builders/scripts`).

The `build-all-submissions.sh` script reads instructions from
`agoricProposal.sdk-generate` in `package.json`. That field contains a list of
strings, each of which describes a single submission. If there is only one
submission, it can use the default directory name `submission` by specifying
only the name of the script file in `builders/scripts/vars`:
`agoricProposal.sdk-generate` in package.json. That field contains a list of
strings, each of which describes a single submission as a list of
space-separated fields. If there is only one submission, it can use the default
directory name `submission` by specifying just one field, containing the path of
a builder script file relative to
[packages/builders/scripts](../packages/builders/scripts):
```json
"sdk-generate": ["test-localchain"],
```
If there are multiple submissions, each entry has to specify the script name and
a distinct directory name.
If there are multiple submissions, each entry has to specify a distinct
directory name in the second field:
```json
"sdk-generate": [
"probe-zcf-bundle probe-submission",
"updatePriceFeeds priceFeed-submission",
"add-auction newAuction-submission"
],
```
A third argument can be used to provide additional parameters to the
build script.
Any remaining fields provide additional arguments to the build script:
```json
"sdk-generate": [
"inter-protocol/updatePriceFeeds.js submission/main main",
Expand Down
10 changes: 7 additions & 3 deletions a3p-integration/debug-current.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
#!/bin/sh
#!/bin/bash
set -ueo pipefail

# Convenience script to debug the current proposal being worked on.

scripts/build-submission.sh proposals/z:acceptance testing/start-valueVow.js start-valueVow
scripts/build-submission.sh proposals/z:acceptance testing/restart-valueVow.js restart-valueVow
(
cd 'proposals/z:acceptance'
../../scripts/build-submission.sh testing/start-valueVow.js start-valueVow
../../scripts/build-submission.sh testing/restart-valueVow.js restart-valueVow
)

yarn test -m acceptance --debug
15 changes: 11 additions & 4 deletions a3p-integration/scripts/build-all-submissions.sh
Original file line number Diff line number Diff line change
@@ -1,17 +1,24 @@
#!/bin/bash
set -ueo pipefail

# cd prints its target, so without the redirect, we get two copies
SCRIPT_DIR=$(cd ${0%/*} > /dev/null && pwd -P)
# Look in the "proposals" subdirectory of the working directory for
# "$char:$name" subdirectories (cf. ../README.md#package-layering), and for each
# one, extract from its package.json "agoricProposal" section a list of
# "sdk-generate" entries corresponding to core-eval submission content that must
# be generated (cf ../README.md#generating-core-eval-submissions) and then use
# ./build-submission.sh to do so.

SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &> /dev/null && pwd)

IFS=$'\n'

for proposal in ./proposals/?:*; do
echo >&2 "Building $proposal ..."
cd $proposal
# build submission if proposal specifies an sdk-generate
while read -r line; do
IFS=' ' parts=($line)
$SCRIPT_DIR/build-submission.sh $proposal ${parts[@]}
"$SCRIPT_DIR"/build-submission.sh ${parts[@]}
done < <(jq -r '.agoricProposal["sdk-generate"][]?' < package.json)
cd -
echo >&2 "Built $proposal"
done
37 changes: 24 additions & 13 deletions a3p-integration/scripts/build-submission.sh
Original file line number Diff line number Diff line change
@@ -1,23 +1,34 @@
#!/bin/bash
set -ueo pipefail

sdkroot=$(git rev-parse --show-toplevel)
# Usage: $0 <builder script> [submission directory name] [builder script arg]...
# Run the specified builder script from packages/builders/scripts and move the
# output into the specified submission directory (defaulting to "submission")
# relative to the working directory.
# Must be run from inside agoric-sdk.

cd "$sdkroot"
builderScript=$1
shift
submissionDir=${1:-submission}
shift || true

a3pProposalDir=$1
builderScript=$2
submissionDirName=${3:-submission}
submissionDir="./a3p-integration/$a3pProposalDir/$submissionDirName"
extraParams=${4:-}
# Run the builder script in a subshell at agoric-sdk.
sdkroot=$(git rev-parse --show-toplevel)
(
cd "$sdkroot"
yarn agoric run "packages/builders/scripts/$builderScript" "$@"
)

yarn agoric run "packages/builders/scripts/$builderScript" $extraParams
# Create and populate the submission directory.
mkdir -p "$submissionDir"
echo >&2 "Populating $(basename -- "$(pwd -P)")/$submissionDir ..."
ls "$sdkroot"/*-plan.json | while read plan; do
# Copy from the bundle cache.
cp $(grep -oh '/.*b1-.*.json' "$plan") "$submissionDir"

# Move from the root directory.
prefix=${plan%-plan.json}
mv "$prefix"* "$submissionDir"

plans=*-plan.json
for plan in $plans; do
base=${plan%-plan.json}
cp $(grep -oh '/.*b1-.*.json' "$base"-plan.json) "$submissionDir"
mv "$base"* "$submissionDir"
ls -oS "$submissionDir"
done
Loading