M1 DA Light Node #33

l-monninger · 2024-04-17T16:03:01Z

Summary

Ticket: https://movement.atlassian.net/browse/ENG-181
Related RFCs:
- https://github.com/movementlabsxyz/org/blob/main/rfc/rfc-m2-da-light-node/rfc-m2-da-light-node.md

Note: many of the subdirs have READMEs. If you are confused, double-check for one.

Implements RFC M2 DA Light Node [sic].

Core Software Components

M1 DA Light Node modules including a light node server, a light node client, and verifier.
Run and test script targets for light node orchestration.

Ancillary Software Components

move-rocks mempool used for provisional sequencing.
buildtime helper crate for tonic gRPC ergonomics with grander ambitions for our organization.
memseq a sequencer built on move-rocks.
movement scripting standard
Nixified Celestia and releated components

Testing

e2e and integration

Because of the difficulty of dependency management, particularly for Celestia, I recommend running:

nix develop
just m1-da-light-node test.local

This will run e2e and integration tests for the light-node implementation.

If you want to play around with tests yourself, you can use

. ./scripts/celestia-env

to populate an environment which will be amenable to your testing.

You will still likely want to simply run the m1-da-light-node services the background or from another process.

nix develop
just m1-da-light-node local

unit

cargo tests should run successfully without additional orchestration for all crates besides m1-da-light-node-verifier, m1-da-light-node, and m1-da-light-client.

Outstanding Issues

celestia-node struggles to deal with multiple attempted ws connections, particularly from the same process. Until this is resolved, cargo-based e2e and integration tests should be run with -- --test-threads=1.
Integration work with ENG-187 Extract Aptos execution module for Monza Partial #30 is likely to alter this PR slightly.
cel-key may benefit from being Nixified separately from celestia-node.
The CI has not been updated.
The DA verification logic will occasionally (1/20x) fail. This logic is pulled directly from the Lumia implementation and likely simply reflects network latency or issues with celestia-node. A retry period may be worth implementation.
anyhow::Error is used liberally. Custom error types may better serve Typed Error Propagation in the future.
Documentation will be enhanced as requested.
6,000 LOC, sorry.

… l-monninger/celestia-testing

l-monninger · 2024-04-19T03:29:40Z

@0xmovses it ended up being quite the little odyssey to fix the Nix Flake. In the end, I just abandoned the Makefile and built the go modules directly. It should work now.

mzabaluev · 2024-04-19T12:27:18Z

proto/movementlabs/protocol_units/da/m1/light_node/v1beta1.proto

+service LightNodeService {
+  // Stream blobs from a specified height or from the latest height.
+  rpc StreamReadFromHeight (StreamReadFromHeightRequest) returns (stream StreamReadFromHeightResponse);
+  rpc StreamReadLatest (Empty) returns (stream StreamReadLatestResponse);


More compliance needed:

Suggested change

rpc StreamReadLatest (Empty) returns (stream StreamReadLatestResponse);

rpc StreamReadLatest (StreamReadLatestRequest) returns (stream StreamReadLatestResponse);

and StreamReadLatestRequest defined as an empty message.

Oh, for some reason I thought that was canonical. Will change.

mzabaluev · 2024-04-19T12:28:44Z

proto/movementlabs/protocol_units/da/m1/light_node/v1beta1.proto

@@ -0,0 +1,102 @@
+syntax = "proto3";
+package movementlabs.protocol_units.da.m1.light_node.v1beta1;


This looks sufficiently ramified now. The proto looks good in general.

mzabaluev · 2024-04-19T12:29:33Z

proto/movementlabs/protocol_units/da/m1/light_node/v1beta1.proto

+message BlobWrite {
+    bytes data = 1;
+}


This leaves room for future extensions, I assume.

Yeah, some changes may already be rolling in.

0xmovses · 2024-04-19T12:43:28Z

nix develop successfully builds the shell env now. When I run cargo test after nix develop I get:

running 2 tests
test test::e2e::test_submit_and_read ... FAILED
test test::e2e::test_light_node_submits_blob_over_stream ... FAILED

failures:

---- test::e2e::test_submit_and_read stdout ----
Error: transport error

Caused by:
    0: error trying to connect: tcp connect error: Connection refused (os error 111)
    1: tcp connect error: Connection refused (os error 111)
    2: Connection refused (os error 111)

---- test::e2e::test_light_node_submits_blob_over_stream stdout ----
Error: transport error

Caused by:
    0: error trying to connect: tcp connect error: Connection refused (os error 111)
    1: tcp connect error: Connection refused (os error 111)
    2: Connection refused (os error 111)


failures:
    test::e2e::test_light_node_submits_blob_over_stream
    test::e2e::test_submit_and_read

test result: FAILED. 0 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

l-monninger · 2024-04-19T14:31:05Z

nix develop successfully builds the shell env now. When I run cargo test after nix develop I get:

running 2 tests
test test::e2e::test_submit_and_read ... FAILED
test test::e2e::test_light_node_submits_blob_over_stream ... FAILED

failures:

---- test::e2e::test_submit_and_read stdout ----
Error: transport error

Caused by:
    0: error trying to connect: tcp connect error: Connection refused (os error 111)
    1: tcp connect error: Connection refused (os error 111)
    2: Connection refused (os error 111)

---- test::e2e::test_light_node_submits_blob_over_stream stdout ----
Error: transport error

Caused by:
    0: error trying to connect: tcp connect error: Connection refused (os error 111)
    1: tcp connect error: Connection refused (os error 111)
    2: Connection refused (os error 111)


failures:
    test::e2e::test_light_node_submits_blob_over_stream
    test::e2e::test_submit_and_read

test result: FAILED. 0 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

You can't run all the tests directly rn. Hence...

nix develop
just m1-da-light-node-test-native

If we want to make this a requirement for the org, I will look into add startup hooks for the e2e and integration tests.

l-monninger · 2024-04-19T14:33:57Z

If there's any funny business with a Celestia account not being available, the fund-account process is supposed to handle that. I will need to fix any issue for CI/CD. But, for testing, you can also just manually fund your account here: https://faucet.celestia-arabica-11.com/ (it should error with the account address).

0xmovses · 2024-04-19T14:37:04Z

Hmm still get an error

just m1-da-light-node-test-native

scripts/movement/test-native m1-da-light-node
Using sequencer storage path: /tmp/nix-shell.oKl3sA/m2-UBri4hRF
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  6704    0  6704    0     0  64076      0 --:--:-- --:--:-- --:--:-- 64461
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  6704    0  6704    0     0  70384      0 --:--:-- --:--:-- --:--:-- 70568
{"level":"warn","error":"open /home/movses/.config/process-compose/settings.yaml: no such file or directory","time":"2024-04-19T15:35:27+01:00","message":"Error reading settings file /home/movses/.config/process-compose/settings.yaml"}
Cleaning up storage: /tmp/nix-shell.oKl3sA/m2-UBri4hRF
removed directory '/tmp/nix-shell.oKl3sA/m
2-UBri4hRF'
error: Recipe `m1-da-light-node-test-native` failed on line 2 with exit code 255

l-monninger · 2024-04-19T14:48:17Z

That's an error in the fund account step. Though it's a little weird that it reports null as the value for you.

Anyways, you can drop that step by commenting out here for the moment:

movement/process-compose/m1-da-light-node/test/process-compose.yml

Line 23 in 1ff12eb

celestia-arabica-fund-account:

Then you'll get another error complaining about your Celestia account not existing. You should be able to fix that with the manual step mentioned above.

I will see if I can reproduce.

l-monninger · 2024-04-19T14:49:33Z

Actually, the later processes I think are only dependent on the "completed" status for now. So, you should be able to do this without commenting out that step and just doing the manual funding.

0xmovses · 2024-04-19T14:53:34Z

Totally happy to manually test this with steps. But if its not massive amounts of trouble / effort (I'll let you decide), I think it would be beneficial to automate these steps, so that anyone committing code to the repo can run these tests and get some assurance about regressions.

l-monninger · 2024-04-19T14:55:24Z

That's the idea. I'm suggesting this for now, because I don't know what's happening on your machine.

l-monninger · 2024-04-19T14:56:34Z

It's just the fund account step that should be manual for you will that issue with the fund-account process persists.

0xmovses · 2024-04-19T15:28:33Z

I ran celestia light init, used the account address to fund my account here https://faucet.celestia-arabica-11.com/

Then when running the test again I get this

movses@movses-ubuntu:~/mvmt/sdk$ just m1-da-light-node-test-native
scripts/movement/test-native m1-da-light-node
Using sequencer storage path: /tmp/nix-shell.VyjDnl/m2-mMPJ1Rod
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2271    0  2271    0     0  16066      0 --:--:-- --:--:-- --:--:-- 16106
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2271    0  2271    0     0  24795      0 --:--:-- --:--:-- --:--:-- 24956
{"level":"warn","error":"open /home/movses/.config/process-compose/settings.yaml: no such file or directory","time":"2024-04-19T16:13:34+01:00","message":"Error reading settings file /home/movses/.config/proces
s-compose/settings.yaml"}
24-04-19 16:13:34.272 FTL Failed to load project error="circular dependency found in 'm1-da-light-node-verifier-tests'"
Cleaning up storage: /tmp/nix-shell.VyjDnl/m2-mMPJ1Rod
removed directory '/tmp/nix-shell.VyjDnl/m2-mMPJ1Rod'
error: Recipe `m1-da-light-node-test-native` failed on line 2 with exit code 1

Am I missing another step?

l-monninger · 2024-04-19T21:34:04Z

I think I've stabilized this. As I mentioned in Slack, circular dependency isn't something I had seen. However, when I refactored my script, I saw it pop up once when the the .yaml was just totally wack haha. So, maybe there's some whitespace issue here.

Otherwise, this should work. I am going to add different test network modes though because state syncing takes a while. Generally, in CI/CD, we I think we should run the following.

Test against localized Celestia. This is the fastest. If tests don't pass, consider incorrect.
Test against Mocha after localized passes. If tests don't pass, consider incorrect.
Test against Arabica after localized passes. If tests don't pass, consider warning for future implementations. (Arabica is a preview.)

l-monninger · 2024-04-20T22:12:12Z

The latest commit provides a local runner for Celestia. Check the latest docs. Will add CI in a bit. @0xmovses

0xmovses · 2024-04-22T17:26:57Z

l-monninger added 30 commits April 2, 2024 15:48

feat: nix flake.

c89ae3c

feat: separate celestia-node package.

d25d934

feat: switch to actix.

b76a162

feat: separte celestia-node.nix.

8c47fe7

chore: refactor.

5ffccaf

feat: extractors.

5e509e7

fix: celestia installation.

2a13244

fix: drop check.

bd60e3a

feat: add process compose.

3729abe

feat: add proxy.

943e759

feat: actix pattern matcher.

00da8c8

chore: switching to async.

ce57399

Merge remote-tracking branch 'origin/0xmovses/execution-updates' into…

5db9f47

… l-monninger/celestia-testing

feat: extractor.

9b4325a

feat: aptos proxy breakout.

14d878b

feat: including http method.

c2969da

feat: path matching pre reorg.

be5d95c

feat: rpc breakout.

e16a92a

feat: preparing to merge.

4c4e2a1

fix: merge.

8dc7ef9

fix: merge.

b3d11b3

feat: rpc updates.

be15a91

fix: lots of compile issues with vendors.

f6837b2

feat: file localization.

d6d9243

fix: devendoring.

0cca93c

fix: monza-testing.

5af1e90

feat: localize vendors.

b746820

fix: resolve merge.

6b44533

feat: rpc updates.

309038c

chore: tidy up.

84970a9

l-monninger added 2 commits April 18, 2024 20:20

fix: better error reporting in funding step.

2f871fe

fix: dropping sleeps, unecessary now.

1ca96c0

mzabaluev reviewed Apr 19, 2024

View reviewed changes

fix: protobuf request name.

1ff12eb

fix: celestia startup.

e8331db

l-monninger added 2 commits April 20, 2024 14:58

feat: localized running and testing, reorg of testing scripts.

3920c3b

fix: m1-dalight-node-client-tests bad dep.

1fc93ef

fix: cold start issue.

e116494

fix: suppress warning by adding process-compose/settings.yaml.

aa5ce55

l-monninger mentioned this pull request Apr 22, 2024

ENG-190: Partial Monza #34

Merged

l-monninger removed the request for review from andyjsbell April 22, 2024 17:45

l-monninger added 4 commits April 22, 2024 17:14

cicd: m1-da-light-node ci.

c57b2aa

cicd: try update nix.

42e4c6a

cicd: testing determinate systems.

a0c06d2

cicd: testing determinate systems.

d81c06b

l-monninger merged commit f54f0c5 into monza Apr 25, 2024
1 of 2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

M1 DA Light Node #33

M1 DA Light Node #33

l-monninger commented Apr 17, 2024 •

edited

Loading

l-monninger commented Apr 19, 2024

mzabaluev Apr 19, 2024

l-monninger Apr 19, 2024

mzabaluev Apr 19, 2024

mzabaluev Apr 19, 2024

l-monninger Apr 19, 2024

0xmovses commented Apr 19, 2024 •

edited

Loading

l-monninger commented Apr 19, 2024

l-monninger commented Apr 19, 2024

0xmovses commented Apr 19, 2024

l-monninger commented Apr 19, 2024

l-monninger commented Apr 19, 2024

0xmovses commented Apr 19, 2024

l-monninger commented Apr 19, 2024

l-monninger commented Apr 19, 2024

0xmovses commented Apr 19, 2024

l-monninger commented Apr 19, 2024

l-monninger commented Apr 20, 2024 •

edited

Loading

0xmovses commented Apr 22, 2024

	rpc StreamReadLatest (Empty) returns (stream StreamReadLatestResponse);
	rpc StreamReadLatest (StreamReadLatestRequest) returns (stream StreamReadLatestResponse);

		@@ -0,0 +1,102 @@
		syntax = "proto3";
		package movementlabs.protocol_units.da.m1.light_node.v1beta1;

M1 DA Light Node #33

M1 DA Light Node #33

Conversation

l-monninger commented Apr 17, 2024 • edited Loading

Summary

Core Software Components

Ancillary Software Components

Testing

e2e and integration

unit

Outstanding Issues

l-monninger commented Apr 19, 2024

mzabaluev Apr 19, 2024

Choose a reason for hiding this comment

l-monninger Apr 19, 2024

Choose a reason for hiding this comment

mzabaluev Apr 19, 2024

Choose a reason for hiding this comment

mzabaluev Apr 19, 2024

Choose a reason for hiding this comment

l-monninger Apr 19, 2024

Choose a reason for hiding this comment

0xmovses commented Apr 19, 2024 • edited Loading

l-monninger commented Apr 19, 2024

l-monninger commented Apr 19, 2024

0xmovses commented Apr 19, 2024

l-monninger commented Apr 19, 2024

l-monninger commented Apr 19, 2024

0xmovses commented Apr 19, 2024

l-monninger commented Apr 19, 2024

l-monninger commented Apr 19, 2024

0xmovses commented Apr 19, 2024

l-monninger commented Apr 19, 2024

l-monninger commented Apr 20, 2024 • edited Loading

0xmovses commented Apr 22, 2024

l-monninger commented Apr 17, 2024 •

edited

Loading

0xmovses commented Apr 19, 2024 •

edited

Loading

l-monninger commented Apr 20, 2024 •

edited

Loading