Celestia mainnet config #1038

mzabaluev · 2025-02-06T16:52:45Z

Resolves #1030

Summary

Categories: protocol-units, scripts, util.

Add "mainnet" configuration and setup infrastructure for Celestia DA.

Redo the celestia DA configuration: transpose the network enum and the parameter struct, so that network is a field. This makes for a lot less code duplication.

l-monninger · 2025-02-06T18:08:27Z

Next Steps

Testing e2e

Test against Celestia mainnet. Use an arbitrary namespace. You may request TIA from myself.

Key Info

The keys used should be available into the celestia-appd/keys under validator.info and <matching-address>.address. Per #1030 , I would like this to be populated from some value in our config containing serialized or unserialized key material. This makes it easier than brittle file redirection for the moment.

Keyring Naming

It is fine to leave as test, but you can also rename this as you need. I do not believe using a keyring with a name other than "test" actually changes the encoding used.

Providers

consensus.lunaroasis.net
public-celestia-rpc.numia.xyz
rpc.celestia.pops.one

mzabaluev · 2025-02-07T13:34:26Z

Key Info

The keys used should be available into the celestia-appd/keys under validator.info and <matching-address>.address. Per #1030 , I would like this to be populated from some value in our config containing serialized or unserialized key material. This makes it easier than brittle file redirection for the moment.

The way it can work with the cel-key utility is, we export the key in PGP ASCII armor, protected by a random passphrase generated by setup. Then the cel-key import command can import the key to be consumable by celestia node(s).

Keyring Naming

It is fine to leave as test, but you can also rename this as you need. I do not believe using a keyring with a name other than "test" actually changes the encoding used.

There are no named keyrings, but you can select backends. I think the only backend available on Linux is test (there is also memory, but it's for in-process use).

mzabaluev · 2025-02-10T16:40:12Z

Per #1030 , I would like this to be populated from some value in our config containing serialized or unserialized key material. This makes it easier than brittle file redirection for the moment.

I have looked in detail into what's available via the provided cel-key tool and cosmos-sdk keyrings.

The import/export format used by cel-key is not readily available in Rust libraries for Tendermint or Cosmos. It's possible to recreate with some extra implementation cost.
The format of the file-based key store is defined by a Go codec, so it would be fragile to try to import directly into this from Rust unless the format is well understood and stable.

The most feasible integration option to me is to use the key name from the keyring as the configuration value, and rely on setup crates (for local setups) or the infrastructure to provision the key into the keyring as expected by celestia light node.
I have added the light node configuration with the requisite key name in this PR.

Redo the celestia DA configuration: transpose the network enum and the parameter struct, so that network is a field. This makes for a lot less code duplication.

The crate name has been changed recently.

When "celestia light init ..." is run in setup, the options need to include --keyring.backend.

The crate is not currently listed in the workspace, its inclusion impeded by a dependency problem.

To avoid a costly sync from height 0, make the initial height a config parameter settable by config.

Make the block height fetching routines do the real thing.

If the Celestia network is not local, don't generate a random chain ID.

Harmonize the format of the namespace string with the one expected by the celestia CLI: 0x prefix for hex, otherwise base64. No more expecting quoted base64-encoded string as if it were JSON.

mzabaluev · 2025-02-20T19:33:44Z

Relevant configuration via environment variables given to the setup binary that creates config for movement-celestia-da-light-node and movement-celestia-light runner:

CELESTIA_NETWORK, with values mainnet, mocha, arabica, or local (the default).
CELESTIA_NAMESPACE: hexadecimal namespace suffix for a v0 ID (up to 10 bytes) if the value starts with "0x", otherwise decoded as the base64 encoding of the full 29-byte namespace ID.
- WARNING: the = padding character of base64 is known to be misinterpreted when used in the environment specification in process-compose; use an .env file.
- If you are testing with public Celestia networks, be sure to pick a randomly generated namespace.
CELESTIA_LIGHT_KEYNAME: name of the key in the test cosmos-sdk keyring, that can be provisioned with cel-key. The default key name is "movement_celestia_light".

The command to run the celestia light node (unless using the Movement runner) can be obtained from the runner source code for the respective network.

For Docker, see this guide, including the section on mounting the keyring.

I may have time to add this to the docker-compose files in the repository.

…movementlabsxyz/movement into mikhail/celestia-mainnet-config

Also rename the .celestia-mainnet-trusted docker-compose overlay to .celestia-mainnet because we're running the local node.

A string starting with 0x needs quotes, otherwise it's treated as a number in YAML.

…movementlabsxyz/movement into mikhail/celestia-mainnet-config

musitdev

I have a practical question. For local execution is it still the same as before?

mzabaluev · 2025-02-26T19:30:30Z

I have a practical question. For local execution is it still the same as before?

Yes, it should work the same way. I have tested this while working on the branch, but not on the latest commits.

Co-authored-by: Liam Monninger <l.mak.monninger@gmail.com>

mzabaluev added 11 commits February 13, 2025 16:30

feat!: Celestia mainnet config

196c1fb

Redo the celestia DA configuration: transpose the network enum and the parameter struct, so that network is a field. This makes for a lot less code duplication.

fix(CI): movement-da-util in unit-tests

39b8d73

The crate name has been changed recently.

fix: celestia mainnet URLs

180cdbd

feat!(da): celestia light node keyname configuration

29c0428

feat: process-compose overlay for celestia-mainnet

e7d94fd

feat(da): set Celestia light node key name from env

fc7fb79

chore(da): clean up old celestia files

e60f2e5

fix(da): --keyring.backend to celestia light init

4806157

When "celestia light init ..." is run in setup, the options need to include --keyring.backend.

feat(da): output namespace in da light node log

e1c73f7

fix(da): --keyring.backend for celestia light start

5c46929

fix(da): don't ignore celestia RPC errors

fbe04d1

mzabaluev force-pushed the mikhail/celestia-mainnet-config branch from f367a49 to fbe04d1 Compare February 13, 2025 14:32

mzabaluev changed the base branch from l-monninger/secure-signing-e2e-integration to main February 13, 2025 14:32

mzabaluev marked this pull request as ready for review February 13, 2025 15:43

mzabaluev requested review from l-monninger, 0xmovses and musitdev as code owners February 13, 2025 15:43

ci: cut out movement-signer-test

b955c7c

The crate is not currently listed in the workspace, its inclusion impeded by a dependency problem.

mzabaluev force-pushed the mikhail/celestia-mainnet-config branch from 4046471 to b955c7c Compare February 14, 2025 10:23

mzabaluev added 7 commits February 17, 2025 13:43

fix(da): streamline random celestia ID generation

0d8d851

feat!: set initial celestia synced_height

95e1d16

To avoid a costly sync from height 0, make the initial height a config parameter settable by config.

fix(da): fetch current block height in setup

2ef3ca9

Make the block height fetching routines do the real thing.

fix(da): force_new_chain to false when well-known

90a4418

If the Celestia network is not local, don't generate a random chain ID.

fix(da): saner parsing of CELESTIA_NAMESPACE env

186730f

Harmonize the format of the namespace string with the one expected by the celestia CLI: 0x prefix for hex, otherwise base64. No more expecting quoted base64-encoded string as if it were JSON.

fix: redundant clone

c23a46d

Merge branch 'main' into mikhail/celestia-mainnet-config

c586b80

mzabaluev added the cicd:movement-containers label Feb 20, 2025

l-monninger and others added 19 commits February 20, 2025 18:49

feat: celestia-appd loop.

d5b88aa

fix: increase to 16cpvu.

51308b5

fix(process-compose): pre-build tests

c894644

feat: move into docker compose.

19f0c45

Merge branch 'mikhail/celestia-mainnet-config' of https://github.com/…

4a954bd

…movementlabsxyz/movement into mikhail/celestia-mainnet-config

fix: strip out eth.

94f7f0d

fix: movement celestia still needed.

427b651

fix: udpate websocket connection.

d837e68

fix(docker): run light node for celestia-mainnet

e0392e3

Also rename the .celestia-mainnet-trusted docker-compose overlay to .celestia-mainnet because we're running the local node.

fix(docker): workable celestia light node check

bc90825

fix(docker): workable CELESTIA_NAMESPACE

3e7ff9d

A string starting with 0x needs quotes, otherwise it's treated as a number in YAML.

feat: Run celestia light node with --node.store

5a07c0e

fix(docker): use --node.store with celestia light node

796fc4c

fix: lumina version.

742e884

Merge branch 'mikhail/celestia-mainnet-config' of https://github.com/…

fbc3175

…movementlabsxyz/movement into mikhail/celestia-mainnet-config

fix: debug add blogb logging.

4064b6b

fix: additional logging for da blob.

209bf3a

Config parameter for celestia light node store directory (#1066)

e82cbf6

deps: switch lumina back to upstream

c2c40ad

mzabaluev mentioned this pull request Feb 25, 2025

[Node] Add retry on DA connection #1054

Merged

musitdev approved these changes Feb 26, 2025

View reviewed changes

musitdev approved these changes Feb 27, 2025

View reviewed changes

Merge branch 'main' into mikhail/celestia-mainnet-config

04c5590

mzabaluev merged commit 7b6f1f7 into main Feb 28, 2025
121 of 125 checks passed

mcmillennick pushed a commit that referenced this pull request Mar 9, 2025

fix: Celestia mainnet config (#1038)

338d56b

Co-authored-by: Liam Monninger <l.mak.monninger@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Celestia mainnet config #1038

Celestia mainnet config #1038

mzabaluev commented Feb 6, 2025 •

edited

Loading

l-monninger commented Feb 6, 2025

mzabaluev commented Feb 7, 2025

Key Info

Keyring Naming

mzabaluev commented Feb 10, 2025

mzabaluev commented Feb 20, 2025

musitdev left a comment

mzabaluev commented Feb 26, 2025

Celestia mainnet config #1038

Celestia mainnet config #1038

Conversation

mzabaluev commented Feb 6, 2025 • edited Loading

Summary

l-monninger commented Feb 6, 2025

Next Steps

Testing e2e

Key Info

Keyring Naming

Providers

mzabaluev commented Feb 7, 2025

Key Info

Keyring Naming

mzabaluev commented Feb 10, 2025

mzabaluev commented Feb 20, 2025

musitdev left a comment

Choose a reason for hiding this comment

mzabaluev commented Feb 26, 2025

mzabaluev commented Feb 6, 2025 •

edited

Loading