specs: fix spec markdown linting (#3061)

* style(specs): fix toc/lint in specs, update CI

style(specs): lint/toc/link fixes

style(readme): fix bedrock contracts link

chore(specs): specs lint scripts

chore(specs): update CI jobs to use new lint scripts and unsilence links linter

style(specs): fix toc, fix quotes in markdown lint script

* yarn: update linting dependencies

* specs: add minimal batch submitter spec, fix broken links

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Author: protolambda

.circleci/config.yml

@@ -277,14 +277,19 @@ jobs:
       image: ubuntu-2004:202111-02
     steps:
       - checkout
+      - run:
+          name: yarn dev deps  # todo: what's the best way to pull in the dependencies for linting? yarn install above is using production env without dev dependencies
+          command: yarn install --production=false
+      - run:
+          name: specs toc
+          command: yarn lint:specs:toc && git diff --exit-code
       - run:
           name: markdown lint
-          command: |
-            docker run -v `pwd`:/workdir davidanson/markdownlint-cli2:0.4.0 "op-node/README.md" "./specs/**/*.md" "#**/node_modules"
+          command: yarn lint:specs:check
       - run:
           name: link lint
           command: |
-            docker run --init -it -v `pwd`:/input lycheeverse/lychee --verbose --no-progress --exclude-loopback --exclude twitter.com --exclude-mail /input/README.md "/input/specs/**/*.md" "/input/meta/**/*.md" "/input/op-node/**/*.md" || exit 0
+            docker run --init -it -v `pwd`:/input lycheeverse/lychee --verbose --no-progress --exclude-loopback --exclude twitter.com --exclude-mail /input/README.md "/input/specs/**/*.md"
 
   fuzz-op-node:
     docker:

README.md

@@ -58,7 +58,7 @@ root
 ├── <a href="./teleportr">teleportr</a>: Bridge for teleporting ETH between L1 and L2 at low cost
 
 ~~ BEDROCK upgrade - Not production-ready yet, part of next major upgrade ~~
-├── <a href="./contracts-bedrock">contracts-bedrock</a>: Bedrock smart contracts. To be merged with ./packages/contracts.
+├── <a href="./packages/contracts-bedrock">packages/contracts-bedrock</a>: Bedrock smart contracts. To be merged with ./packages/contracts.
 ├── <a href="./op-bindings">op-bindings</a>: Go bindings for Bedrock smart contracts.
 ├── <a href="./op-batcher">op-batcher</a>: L2-Batch Submitter, submits bundles of batches to L1
 ├── <a href="./op-e2e">op-e2e</a>: End-to-End testing of all bedrock components in Go

package.json

@@ -46,6 +46,9 @@
     "lint:ts:check": "yarn lerna run lint:ts:check",
     "lint:check": "yarn lerna run lint:check",
     "lint:fix": "yarn lerna run lint:fix --parallel",
+    "lint:specs:fix": "yarn run markdownlint-cli2-fix \"./specs/**/*.md\"",
+    "lint:specs:check": "yarn run markdownlint-cli2  \"./specs/**/*.md\"",
+    "lint:specs:toc": "yarn run doctoc '--title=**Table of Contents**' ./specs",
     "postinstall": "patch-package",
     "ready": "yarn lint && yarn test",
     "prepare": "husky install",
@@ -63,7 +66,7 @@
     "chai": "^4.2.0",
     "copyfiles": "^2.3.0",
     "depcheck": "^1.4.3",
-    "doctoc": "2.1.0",
+    "doctoc": "^2.2.0",
     "eslint": "^8.16.0",
     "eslint-config-prettier": "^8.3.0",
     "eslint-config-standard": "^16.0.3",
@@ -79,7 +82,7 @@
     "lerna": "^4.0.0",
     "lint-staged": "11.0.0",
     "markdownlint": "^0.24.0",
-    "markdownlint-cli2": "^0.3.2",
+    "markdownlint-cli2": "0.4.0",
     "mkdirp": "^1.0.4",
     "mocha": "^8.4.0",
     "nyc": "^15.1.0",

specs/README.md

@@ -1,7 +1,7 @@
 <!-- DOCTOC SKIP -->
 # Optimism Bedrock specs
 
-This directory contains the plain english specs for Optimism, an minimal optimistic rollup protocol
+This directory contains the plain english specs for Optimism, a minimal optimistic rollup protocol
 that maintains 1:1 compatibility with Ethereum.
 
 ## Specification Contents

specs/batcher.md

@@ -0,0 +1,25 @@
+<!-- DOCTOC SKIP -->
+# Batch Submitter
+
+The batch submitter, also referred to as the batcher, is the entity submitting the L2 sequencer data to L1,
+to make it available for verifiers.
+
+[derivation-spec]: ./derivation.md
+
+The format of the data transactions is defined in the [derivation spec]: the data is constructed from L2 blocks
+in the reverse order as it is derived from data into L2 blocks.
+
+The timing, operation and transaction signing is implementation-specific: any data can be submitted at any time,
+but only the data that matches the [derivation spec] rules will be valid from the verifier perspective.
+
+The most minimal batcher implementation can be defined as a loop of the following operations:
+
+1. See if the `unsafe` L2 block number is past the `safe` block number: `unsafe` data needs to be submitted.
+2. Iterate over all unsafe L2 blocks, skip any that were previously submitted.
+3. Open a channel, buffer all the L2 block data to be submitted,
+   while applying the encoding and compression as defined in the [derivation spec].
+4. Pull frames from the channel to fill data transactions with, until the channel is empty.
+5. Submit the data transactions to L1
+
+The L2 view of safe/unsafe does not instantly update after data is submitted, nor when it gets confirmed on L1,
+so special care may have to be taken to not duplicate data submissions.

specs/deposits.md

@@ -36,7 +36,7 @@ with the authorization and validation conditions on L2.
 - [User-Deposited Transactions](#user-deposited-transactions)
   - [Deposit Contract](#deposit-contract)
     - [Address Aliasing](#address-aliasing)
-    - [Deposit Feed Contract: Reference Implementation](#deposit-feed-contract-reference-implementation)
+    - [Deposit Contract Implementation: Optimism Portal](#deposit-contract-implementation-optimism-portal)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
@@ -221,7 +221,7 @@ The contract has the following solidity interface, and can be interacted with ac
 
 A reference implementation of the L1 Attributes predeploy contract can be found in [L1Block.sol].
 
-[L1Block.sol]: ../packages/contracts/contracts/L2/L1Block.sol
+[L1Block.sol]: ../packages/contracts-bedrock/contracts/L2/L1Block.sol
 
 After running `yarn build` in the `packages/contracts` directory, the bytecode to add to the genesis
 file will be located in the `deployedBytecode` field of the build artifacts file at
@@ -233,8 +233,8 @@ file will be located in the `deployedBytecode` field of the build artifacts file
 
 [User-deposited transactions][g-user-deposited] are [deposited transactions][deposited-tx-type]
 generated by the [L2 Chain Derivation][g-derivation] process. The content of each user-deposited
-transaction are determined by the corresponding `TransactionDeposited` event emitted by the [deposit
-feed contract][deposit-feed-contract] on L1.
+transaction are determined by the corresponding `TransactionDeposited` event emitted by the
+[deposit contract][deposit-contract] on L1.
 
 1. `from` is unchanged from the emitted value (though it may have been transformed to an alias in
    the deposit feed contract).
@@ -275,8 +275,8 @@ contract on L1 has the same address as a contract on L2 but doesn't have the sam
 for EOAs because they're guaranteed to have the same "code" (i.e. no code at all). This also makes
 it possible for users to interact with contracts on L2 even when the Sequencer is down.
 
-#### Deposit Feed Contract: Reference Implementation
+#### Deposit Contract Implementation: Optimism Portal
 
-A reference implementation of the Deposit Feed contract can be found in [DepositFeed.sol].
+A reference implementation of the deposit contract can be found in [OptimismPortal.sol].
 
-[DepositFeed.sol]: ../packages/contracts/contracts/L1/abstracts/DepositFeed.sol
+[OptimismPortal.sol]: ../packages/contracts-bedrock/contracts/L1/OptimismPortal.sol

specs/derivation.md

@@ -28,7 +28,7 @@
 [g-avail-provider]: glossary.md#data-availability-provider
 [g-batcher]: glossary.md#batcher
 [g-l2-output]: glossary.md#l2-output
-[g-fault-proof]: glosary.md#fault-proof
+[g-fault-proof]: glossary.md#fault-proof
 [g-channel]: glossary.md#channel
 [g-channel-frame]: glossary.md#channel-frame
 [g-rollup-node]: glossary.md#rollup-node
@@ -181,7 +181,7 @@ by a correct output root **for the existing sequencer batches.**
 
 Refer to the [Batch Submission specification][batcher-spec] for more information.
 
-[batcher-spec]: batching.md
+[batcher-spec]: batcher.md
 
 > **TODO** rewrite the batch submission specification
 >
@@ -303,7 +303,7 @@ where:
 >   can influence things down the line!!
 
 [sqlite-uvarint]: https://www.sqlite.org/src4/doc/trunk/www/varint.wiki
-[batcher-spec]: batching.md
+[batcher-spec]: batcher.md
 
 ### Channel Format
 

specs/introduction.md

@@ -138,7 +138,7 @@ Links to components mentioned in this diagram:
 - [Execution Engine](./exec-engine.md)
 - Sequencer Batch Submitter (WIP)
 - [L2 Output Oracle](./proposals.md#l2-output-oracle-smart-contract)
-- [L2 Output Submitter](./proposals#proposing-l2-output-commitments)
+- [L2 Output Submitter](./proposals.md#proposing-l2-output-commitments)
 - Fault Proof VM (WIP)
 
 ### Withdrawing

specs/meta/devnet.md

@@ -1,3 +1,4 @@
+<!-- DOCTOC SKIP -->
 # Bedrock Local Devnet Setup
 
 <!-- START doctoc generated TOC please keep comment here to allow auto update -->

specs/meta/linting.md

@@ -24,10 +24,13 @@ Justification for linting rules in [.markdownlint.json](/.markdownlint.json):
 - *no-emphasis-as-heading*: enable emphasized paragraphs
 
 ```shell
-yarn             # Install dependencies
-yarn lint        # Run linter
-yarn lint:links  # Check links
-yarn lint:toc    # Update TOC docs
+yarn                   # Install dependencies
+yarn lint:specs:check  # Run linter
+yarn lint:specs:fix    # Fix lint issues
+yarn lint:specs:toc    # Update TOC docs
+
+# Check links
+docker run --init -it -v `pwd`:/input lycheeverse/lychee --verbose --no-progress --exclude-loopback --exclude twitter.com --exclude-mail /input/README.md "/input/specs/**/*.md"
 ```
 
 To check links, you'll need to install [lychee]. The [version ran in CI][lychee-ci] is 0.8.1, but

specs/meta/markdown-style.md

@@ -43,7 +43,7 @@ In general:
   - e.g. `[g-block]: glossary.md#block`
 - Example: [Rollup Node Specification source][rollup-node]
 
-[glossary]: ../specs/glossary.md
+[glossary]: ../glossary.md
 [rollup-node]: https://raw.githubusercontent.com/ethereum-optimism/optimistic-specs/main/specs/rollup-node.md
 
 ## Internal (In-File) Links

specs/overview.md

@@ -85,7 +85,7 @@ and fault proofs.
 
 **Spec links:**
 
-- [Execution Engine](specs/exec-engine.md)
+- [Execution Engine](./exec-engine.md)
 
 Since the EE uses Geth under the hood, Optimism uses Geth's built-in peer-to-peer network and transaction pool to
 propagate transactions. The same network can also be used to propagate submitted blocks and support snap-sync.

specs/predeploys.md

@@ -333,9 +333,9 @@ Reserved for future use.
 
 ## L1Block
 
-[l1-block-predeploy]: glossary.md#l1-block-predeployed-contract
+[l1-block-predeploy]: glossary.md#l1-attributes-predeployed-contract
 
-The [L1Block](l1-block-predeploy) was introduced in Bedrock and is responsible for
+The [L1Block][l1-block-predeploy] was introduced in Bedrock and is responsible for
 maintaining L1 context in L2. This allows for L1 state to be accessed in L2.
 
 ```solidity
@@ -370,10 +370,11 @@ interface L1Block {
      * @dev sets the latest L1 block attributes
      */
     function setL1BlockValues(
-        uint256 _number,
-        uint256 _timestamp,
+        uint64 _number,
+        uint64 _timestamp,
         uint256 _basefee,
-        bytes32 _hash
+        bytes32 _hash,
+        uint64 _sequenceNumber
     ) external;
 }
 ```

specs/rollup-node-p2p.md

@@ -171,7 +171,7 @@ For async communication over different channels over the same connection, multip
 
 #### GossipSub
 
-[GossipSub 1.1](gossipsub) (`/meshsub/1.1.0`, i.e. with peer-scoring extension) is a pubsub protocol for mesh-networks,
+[GossipSub 1.1][gossipsub] (`/meshsub/1.1.0`, i.e. with peer-scoring extension) is a pubsub protocol for mesh-networks,
 deployed on L1 consensus (Eth2) and other protocols such as Filecoin, offering lots of customization options.
 
 ##### Content-based message identification

specs/rollup-node.md

@@ -24,7 +24,6 @@ currently only concerned with the specification of the rollup driver.
   - [Derivation](#derivation)
 - [L2 Output RPC method](#l2-output-rpc-method)
   - [Output Method API](#output-method-api)
-- [L2 Batch creation RPC method](#l2-batch-creation-rpc-method)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
@@ -74,7 +73,3 @@ The input and return types here are as defined by the [engine API specs][engine-
 - returns:
   1. `version`: `DATA`, 32 Bytes - the output root version number, beginning with 0.
   1. `l2OutputRoot`: `DATA`, 32 Bytes - the output root
-
-## L2 Batch creation RPC method
-
-See [Batch derivation](./batching.md) for an RPC specification for batch data submission.