Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Identity #1621

Merged
merged 169 commits into from
Mar 6, 2025
Merged
Show file tree
Hide file tree
Changes from 136 commits
Commits
Show all changes
169 commits
Select commit Hold shift + click to select a range
b26f010
passkey member identifier
codabrink Feb 11, 2025
d482c21
nit
codabrink Feb 11, 2025
6d0d691
incremental progress
codabrink Feb 12, 2025
3e3385f
move public identifiers into its own file
codabrink Feb 12, 2025
173f60d
adapt association log
codabrink Feb 12, 2025
594554b
external identifier
codabrink Feb 12, 2025
05071f7
enforce permissions via data structures
codabrink Feb 13, 2025
3acb143
wip
codabrink Feb 14, 2025
3aa7974
wip
codabrink Feb 15, 2025
5169f60
cleaning up
codabrink Feb 18, 2025
7d3db42
more cleanup
codabrink Feb 18, 2025
42f03f0
wip
codabrink Feb 18, 2025
2510ee7
cleanup
codabrink Feb 18, 2025
9249ac7
more cleanup
codabrink Feb 18, 2025
2667f77
cleanup verified signature
codabrink Feb 18, 2025
c144ad3
cleanup id/mod.rs
codabrink Feb 18, 2025
8cf1056
cleanup serialization, update protos
codabrink Feb 19, 2025
296e39b
more cleanup
codabrink Feb 19, 2025
501eb15
more cleanup
codabrink Feb 19, 2025
c63c486
continue making improvements
codabrink Feb 19, 2025
93f3e60
cleanup
codabrink Feb 19, 2025
1d6ab2e
checking in progress
codabrink Feb 19, 2025
067d636
continue cleanup
codabrink Feb 19, 2025
3b5a904
cleaner approach
codabrink Feb 19, 2025
23408ac
more cleanup
codabrink Feb 19, 2025
e9fc089
cleanup
codabrink Feb 19, 2025
756e535
fix the query
codabrink Feb 19, 2025
5c13280
cleanup
codabrink Feb 19, 2025
942969d
database adjustments
codabrink Feb 20, 2025
136d577
rename wallet entry to identity cache
codabrink Feb 20, 2025
a1ab4ed
consent record to root identifier
codabrink Feb 20, 2025
35993bb
cleanup builder.rs
codabrink Feb 20, 2025
b56185c
update consent
codabrink Feb 20, 2025
ab514d5
update consent save
codabrink Feb 20, 2025
72d5dec
update a few more methods
codabrink Feb 20, 2025
d87fccc
two errors left
codabrink Feb 20, 2025
d9752a7
xmtp_mls is done, now for the bindings
codabrink Feb 20, 2025
250fa67
continue cleanup
codabrink Feb 20, 2025
74ae126
work on the ffi more
codabrink Feb 20, 2025
dc01ea4
Update the InboxOwner trait
codabrink Feb 21, 2025
aa8e1d8
rename to PublicIdentifier
codabrink Feb 21, 2025
72e37bb
wip
codabrink Feb 24, 2025
91b0ec4
refactor ffi progress
codabrink Feb 24, 2025
73f3806
mls cleanup
codabrink Feb 24, 2025
46ab7e5
cleanup
codabrink Feb 24, 2025
5ff28db
update the cli example
codabrink Feb 24, 2025
23307a7
continue cleanup
codabrink Feb 24, 2025
ad0fb61
continue updating
codabrink Feb 24, 2025
164605d
finish with node/client.rs
codabrink Feb 24, 2025
8095206
update node consent state
codabrink Feb 24, 2025
66e6d9d
consent state in node
codabrink Feb 25, 2025
cac7e3e
more cleaning up
codabrink Feb 25, 2025
bfac131
continue cleanup
codabrink Feb 25, 2025
9d88b1c
fix typo
codabrink Feb 25, 2025
da301d5
cleanup wasm client
codabrink Feb 25, 2025
be0f619
cleanup
codabrink Feb 25, 2025
48fada4
proto
codabrink Feb 25, 2025
e6cf357
Merge remote-tracking branch 'origin/main' into coda/passkeys
codabrink Feb 25, 2025
704ea54
fix a test
codabrink Feb 25, 2025
7876598
revert signature text
codabrink Feb 25, 2025
d82a5ed
update migration
codabrink Feb 25, 2025
1389448
update migration
codabrink Feb 25, 2025
8d6fafc
update migration one more time
codabrink Feb 25, 2025
6406790
fix
codabrink Feb 25, 2025
1dfb5a3
rework migration approach
codabrink Feb 25, 2025
7bdff33
schema gen
codabrink Feb 25, 2025
047f74b
fix
codabrink Feb 25, 2025
19bf598
Merge branch 'main' into coda/passkeys
codabrink Feb 25, 2025
7be3477
checking in
codabrink Feb 25, 2025
11849c6
Merge branch 'coda/passkeys' of github.com:xmtp/libxmtp into coda/pas…
codabrink Feb 25, 2025
27d9003
turn off for now
codabrink Feb 25, 2025
e3e6e2c
default ident kind to Ethereum
codabrink Feb 25, 2025
fd73292
update proto
codabrink Feb 25, 2025
a2ebd5a
update nomenclature
codabrink Feb 25, 2025
ae7d933
fix consent
codabrink Feb 25, 2025
8e21872
test fix
codabrink Feb 26, 2025
70f97e8
test fix
codabrink Feb 26, 2025
8336db6
cleanup
codabrink Feb 26, 2025
46b0820
Merge branch 'main' into coda/passkeys
codabrink Feb 26, 2025
1048298
tweak udl
codabrink Feb 26, 2025
c26a7cd
Merge branch 'coda/passkeys' of github.com:xmtp/libxmtp into coda/pas…
codabrink Feb 26, 2025
ab6ddbd
taplo
codabrink Feb 26, 2025
a4b452b
adjust ffi
codabrink Feb 26, 2025
3957383
update the udl
codabrink Feb 26, 2025
4d685e6
cleanup
codabrink Feb 26, 2025
564233c
sign
codabrink Feb 26, 2025
d96f39b
put the udl in the namespace
codabrink Feb 26, 2025
a72d9bc
relying partner
codabrink Feb 27, 2025
cd11dc0
ffi relying partner
codabrink Feb 27, 2025
d274965
node relying partner
codabrink Feb 27, 2025
45e6239
wasm relying partner
codabrink Feb 27, 2025
5f8f6b4
udl
codabrink Feb 27, 2025
8a44ec6
remove udl
codabrink Feb 27, 2025
b8e14a5
java kts file
codabrink Feb 27, 2025
6c92bd5
Merge branch 'main' into coda/passkeys
codabrink Feb 27, 2025
88c5702
api fix
codabrink Feb 27, 2025
e39dcdb
ignore faulty test
codabrink Feb 27, 2025
bcc4a14
cleanup
codabrink Feb 27, 2025
d09cc83
cleanup
codabrink Feb 27, 2025
743f906
test fix
codabrink Feb 27, 2025
03c0542
lint
codabrink Feb 27, 2025
da65020
test fix
codabrink Feb 27, 2025
1f26a9d
lint
codabrink Feb 27, 2025
dc0dd77
fix bench
codabrink Feb 28, 2025
0ae0538
prettier
codabrink Feb 28, 2025
616674d
lint
codabrink Feb 28, 2025
b1eb933
lint
codabrink Feb 28, 2025
c102494
fix bench
codabrink Feb 28, 2025
50b53ff
lint
codabrink Feb 28, 2025
3d674ab
lint
codabrink Feb 28, 2025
b802bf1
lint
codabrink Feb 28, 2025
cb0d286
cleanup
codabrink Feb 28, 2025
8b66cff
update kotlin gen script
codabrink Feb 28, 2025
a958bc9
todo done
codabrink Feb 28, 2025
4b8dc11
re-add udl file to get android build working (#1678)
cameronvoell Feb 28, 2025
ab63bbc
remove ffirootidentifier
codabrink Feb 28, 2025
5486087
Merge branch 'coda/passkeys' of github.com:xmtp/libxmtp into coda/pas…
codabrink Feb 28, 2025
35712ac
remove wasm root identifier
codabrink Feb 28, 2025
da35ada
remove node root identifier
codabrink Feb 28, 2025
337b58e
lint
codabrink Feb 28, 2025
cf115e8
cleanup
codabrink Mar 1, 2025
9d62b3b
Merge remote-tracking branch 'origin/main' into coda/passkeys
codabrink Mar 1, 2025
3bc9897
fixes for the new tests
codabrink Mar 1, 2025
0c08905
Merge branch 'main' of https://github.com/xmtp/libxmtp into coda/pass…
nplasterer Mar 2, 2025
28fb1b7
Merge branch 'coda/passkeys' of https://github.com/xmtp/libxmtp into …
nplasterer Mar 2, 2025
12ed025
cleanup
codabrink Mar 3, 2025
f429703
use an env var
codabrink Mar 3, 2025
629beeb
drop address from consent
codabrink Mar 3, 2025
1d62ccf
lint
codabrink Mar 3, 2025
65b1d4b
Merge branch 'main' into coda/passkeys
codabrink Mar 3, 2025
005ee3a
fix
codabrink Mar 3, 2025
8fd99e9
Merge branch 'coda/passkeys' of github.com:xmtp/libxmtp into coda/pas…
codabrink Mar 3, 2025
54a1463
lint
codabrink Mar 3, 2025
b11cf3e
Merge branch 'main' into coda/passkeys
codabrink Mar 3, 2025
c91209f
lint
codabrink Mar 3, 2025
10da327
Merge branch 'coda/passkeys' of github.com:xmtp/libxmtp into coda/pas…
codabrink Mar 3, 2025
897e58e
legacy interop tests (#1694)
codabrink Mar 4, 2025
f241dae
Update xmtp_mls/src/client.rs
codabrink Mar 4, 2025
e2bd468
Update xmtp_mls/src/identity_updates.rs
codabrink Mar 4, 2025
db9744a
Update xmtp_mls/src/client.rs
codabrink Mar 4, 2025
5842bfe
Update xmtp_mls/src/client.rs
codabrink Mar 4, 2025
7db8483
undo comment
codabrink Mar 4, 2025
ace1841
Merge remote-tracking branch 'origin/main' into coda/passkeys
codabrink Mar 4, 2025
21cb265
update can_message
codabrink Mar 4, 2025
b0524be
lint
codabrink Mar 4, 2025
dec9b10
fix node tests
codabrink Mar 4, 2025
ecab152
Merge branch 'main' into coda/passkeys
codabrink Mar 4, 2025
1598584
ffi public identifier -> ffi identifier
codabrink Mar 4, 2025
712cfa2
Merge branch 'coda/passkeys' of github.com:xmtp/libxmtp into coda/pas…
codabrink Mar 4, 2025
19d3569
cleanup
codabrink Mar 4, 2025
9775a42
Merge branch 'main' into coda/passkeys
codabrink Mar 4, 2025
26059fb
cleanup
codabrink Mar 4, 2025
c6e1fe3
Merge branch 'coda/passkeys' of github.com:xmtp/libxmtp into coda/pas…
codabrink Mar 4, 2025
3ccec12
just identifier
codabrink Mar 5, 2025
75cab8c
Merge remote-tracking branch 'origin/main' into coda/passkeys
codabrink Mar 5, 2025
853e358
Merge remote-tracking branch 'origin/main' into coda/passkeys
codabrink Mar 6, 2025
7a473a8
bump
codabrink Mar 6, 2025
49fcbf5
lint
codabrink Mar 6, 2025
d9ab7a4
remove disappearing messages test from node
codabrink Mar 6, 2025
caf59a2
disable another test
codabrink Mar 6, 2025
911761e
more cleanup
codabrink Mar 6, 2025
95cbc9c
Merge branch 'main' into coda/passkeys
nplasterer Mar 6, 2025
b89efbf
flakiness
codabrink Mar 6, 2025
e643bc2
Merge remote-tracking branch 'origin/main' into coda/passkeys
codabrink Mar 6, 2025
21e330f
lint
codabrink Mar 6, 2025
34c6893
proto fixes
codabrink Mar 6, 2025
dcfe5f9
lint
codabrink Mar 6, 2025
c3b2307
lint
codabrink Mar 6, 2025
85bff24
lint
codabrink Mar 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -166,3 +166,6 @@ ecies_bindings_wasm/
**/tracing-flamegraph.svg

chain_urls.json

# Mise Configs
.mise.toml
5 changes: 3 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions bindings_ffi/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ crate-type = ["lib", "cdylib", "staticlib"]

[dependencies]
futures.workspace = true
hex.workspace = true
parking_lot.workspace = true
prost.workspace = true
thiserror.workspace = true
Expand Down
27 changes: 16 additions & 11 deletions bindings_ffi/benches/create_client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ use xmtp_common::{
bench::{bench_async_setup, BENCH_ROOT_SPAN},
tmp_path,
};
use xmtp_id::InboxOwner;
use xmtp_id::associations::test_utils::WalletTestExt;
use xmtp_mls::utils::test::HISTORY_SYNC_URL;
use xmtpv3::generate_inbox_id;
use xmtpv3::identity::FfiPublicIdentifier;

#[macro_use]
extern crate tracing;
Expand Down Expand Up @@ -52,8 +52,9 @@ fn create_ffi_client(c: &mut Criterion) {
|| {
bench_async_setup(|| async {
let wallet = xmtp_cryptography::utils::generate_local_wallet();
let ident = wallet.public_identifier();
let nonce = 1;
let inbox_id = generate_inbox_id(wallet.get_address(), nonce).unwrap();
let inbox_id = ident.inbox_id(nonce).unwrap();
let path = tmp_path();
let (url, is_secure) = network_url();
let api = xmtpv3::mls::connect_to_backend(url, is_secure)
Expand All @@ -62,20 +63,21 @@ fn create_ffi_client(c: &mut Criterion) {
(
api,
inbox_id,
wallet.get_address(),
wallet.public_identifier(),
nonce,
path,
span.clone(),
)
})
},
|(api, inbox_id, address, nonce, path, span)| async move {
|(api, inbox_id, ident, nonce, path, span)| async move {
let ffi_ident: FfiPublicIdentifier = ident.into();
xmtpv3::mls::create_client(
api,
Some(path),
Some(vec![0u8; 32]),
&inbox_id,
address,
ffi_ident,
nonce,
None,
Some(HISTORY_SYNC_URL.to_string()),
Expand All @@ -99,9 +101,11 @@ fn cached_create_ffi_client(c: &mut Criterion) {
let _ = fdlimit::raise_fd_limit();
let mut benchmark_group = c.benchmark_group("create_client_from_cached");
let wallet = xmtp_cryptography::utils::generate_local_wallet();
let ident = wallet.public_identifier();
let nonce = 1;
let inbox_id = generate_inbox_id(wallet.get_address(), nonce).unwrap();
let address = wallet.get_address();
let inbox_id = ident.inbox_id(nonce).unwrap();
let ffi_ident: FfiPublicIdentifier = ident.into();
let address = wallet.public_identifier();
let path = tmp_path();
let (url, is_secure) = network_url();
let api = runtime.block_on(async {
Expand All @@ -113,7 +117,7 @@ fn cached_create_ffi_client(c: &mut Criterion) {
Some(path.clone()),
Some(vec![0u8; 32]),
&inbox_id.clone(),
address.clone(),
ffi_ident,
nonce,
None,
Some(HISTORY_SYNC_URL.to_string()),
Expand All @@ -138,13 +142,14 @@ fn cached_create_ffi_client(c: &mut Criterion) {
span.clone(),
)
},
|(api, inbox_id, address, nonce, path, history_sync, span)| async move {
|(api, inbox_id, ident, nonce, path, history_sync, span)| async move {
let ffi_ident: FfiPublicIdentifier = ident.into();
xmtpv3::mls::create_client(
api,
Some(path),
Some(vec![0u8; 32]),
&inbox_id,
address,
ffi_ident,
nonce,
None,
Some(history_sync),
Expand Down
1 change: 0 additions & 1 deletion bindings_ffi/build.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use std::process::Command;

fn main() {
uniffi::generate_scaffolding("./src/xmtpv3.udl").expect("Building the UDL file failed");
Command::new("make")
.args(["libxmtp-version"])
.status()
Expand Down
1 change: 0 additions & 1 deletion bindings_ffi/gen_kotlin.sh
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,3 @@ unzip -o LibXMTPKotlinFFI.zip
cd ../../..

cp -r $BINDINGS_PATH/src/uniffi/$PROJECT_NAME/jniLibs/* ~/XMTP/xmtp-android/library/src/main/jniLibs

105 changes: 105 additions & 0 deletions bindings_ffi/src/identity.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
use std::fmt::Display;
use xmtp_cryptography::signature::IdentifierValidationError;
use xmtp_id::associations::{ident, PublicIdentifier};

use crate::GenericError;

#[derive(uniffi::Record, Hash, PartialEq, Eq, Clone)]
pub struct FfiPublicIdentifier {
pub identifier: String,
pub identifier_kind: FfiPublicIdentifierKind,
pub relying_partner: Option<String>,
}

#[derive(uniffi::Enum, Hash, PartialEq, Eq, Clone)]
pub enum FfiPublicIdentifierKind {
Ethereum,
Passkey,
}

impl FfiPublicIdentifier {
pub fn inbox_id(&self, nonce: u64) -> Result<String, GenericError> {
let ident: PublicIdentifier = self.clone().try_into().map_err(GenericError::from_error)?;
Ok(ident.inbox_id(nonce)?)
}
}

impl Display for FfiPublicIdentifier {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self.identifier_kind {
FfiPublicIdentifierKind::Ethereum => write!(f, "{}", self.identifier),
FfiPublicIdentifierKind::Passkey => write!(f, "{}", hex::encode(&self.identifier)),
}
}
}

#[allow(unused)]
#[uniffi::export]
pub fn generate_inbox_id(
account_identifier: FfiPublicIdentifier,
nonce: u64,
) -> Result<String, GenericError> {
account_identifier.inbox_id(nonce)
}

impl From<PublicIdentifier> for FfiPublicIdentifier {
fn from(ident: PublicIdentifier) -> Self {
match ident {
PublicIdentifier::Ethereum(ident::Ethereum(addr)) => Self {
identifier: addr,
identifier_kind: FfiPublicIdentifierKind::Ethereum,
relying_partner: None,
},
PublicIdentifier::Passkey(ident::Passkey {
key,
relying_partner,
}) => Self {
identifier: hex::encode(key),
identifier_kind: FfiPublicIdentifierKind::Passkey,
relying_partner,
},
}
}
}

impl TryFrom<FfiPublicIdentifier> for PublicIdentifier {
type Error = IdentifierValidationError;
fn try_from(ident: FfiPublicIdentifier) -> Result<Self, Self::Error> {
let ident = match ident.identifier_kind {
FfiPublicIdentifierKind::Ethereum => Self::eth(ident.identifier)?,
FfiPublicIdentifierKind::Passkey => {
Self::passkey_str(&ident.identifier, ident.relying_partner)?
}
};
Ok(ident)
}
}

pub trait IdentityExt<T, U> {
fn to_internal(self) -> Result<Vec<U>, IdentifierValidationError>;
}

impl IdentityExt<FfiPublicIdentifier, PublicIdentifier> for Vec<FfiPublicIdentifier> {
fn to_internal(self) -> Result<Vec<PublicIdentifier>, IdentifierValidationError> {
let ident: Result<Vec<_>, IdentifierValidationError> =
self.into_iter().map(|ident| ident.try_into()).collect();
ident
}
}

pub trait FfiCollectionExt<T> {
fn to_ffi(self) -> Vec<T>;
}
impl FfiCollectionExt<FfiPublicIdentifier> for Vec<PublicIdentifier> {
fn to_ffi(self) -> Vec<FfiPublicIdentifier> {
self.into_iter().map(Into::into).collect()
}
}
pub trait FfiCollectionTryExt<T> {
fn to_internal(self) -> Result<Vec<T>, IdentifierValidationError>;
}
impl FfiCollectionTryExt<PublicIdentifier> for Vec<FfiPublicIdentifier> {
fn to_internal(self) -> Result<Vec<PublicIdentifier>, IdentifierValidationError> {
self.into_iter().map(|ident| ident.try_into()).collect()
}
}
25 changes: 20 additions & 5 deletions bindings_ffi/src/inbox_owner.rs
Original file line number Diff line number Diff line change
@@ -1,21 +1,32 @@
use xmtp_cryptography::signature::{RecoverableSignature, SignatureError};
use crate::identity::FfiPublicIdentifier;
use xmtp_cryptography::signature::{
IdentifierValidationError, RecoverableSignature, SignatureError,
};
use xmtp_id::associations::PublicIdentifier;

// TODO proper error handling
#[derive(Debug, thiserror::Error)]
#[derive(uniffi::Error, Debug, thiserror::Error)]
pub enum SigningError {
#[error("This is a generic error")]
Generic,
}

#[derive(uniffi::Error, Debug, thiserror::Error)]
pub enum IdentityValidationError {
#[error("Error: {0:?}")]
Generic(String),
}

impl From<uniffi::UnexpectedUniFFICallbackError> for SigningError {
fn from(_: uniffi::UnexpectedUniFFICallbackError) -> Self {
Self::Generic
}
}

// A simplified InboxOwner passed to Rust across the FFI boundary
#[uniffi::export(with_foreign)]
pub trait FfiInboxOwner: Send + Sync {
fn get_address(&self) -> String;
fn get_identifier(&self) -> Result<FfiPublicIdentifier, IdentityValidationError>;
fn sign(&self, text: String) -> Result<Vec<u8>, SigningError>;
}

Expand All @@ -30,8 +41,12 @@ impl RustInboxOwner {
}

impl xmtp_mls::InboxOwner for RustInboxOwner {
fn get_address(&self) -> String {
self.ffi_inbox_owner.get_address().to_lowercase()
fn get_identifier(&self) -> Result<PublicIdentifier, IdentifierValidationError> {
let ident = self
.ffi_inbox_owner
.get_identifier()
.map_err(|err| IdentifierValidationError::Generic(err.to_string()))?;
ident.try_into()
}

fn sign(&self, text: &str) -> Result<RecoverableSignature, SignatureError> {
Expand Down
12 changes: 5 additions & 7 deletions bindings_ffi/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,22 +1,18 @@
#![recursion_limit = "256"]
#![warn(clippy::unwrap_used)]
pub mod identity;
pub mod inbox_owner;
pub mod logger;
pub mod mls;

pub use crate::inbox_owner::SigningError;
use inbox_owner::FfiInboxOwner;
pub use mls::*;
use std::error::Error;
use xmtp_cryptography::signature::IdentifierValidationError;

extern crate tracing as log;

pub use ffi::*;
#[allow(clippy::all)]
mod ffi {
use super::*;
uniffi::include_scaffolding!("xmtpv3");
}
uniffi::setup_scaffolding!("xmtpv3");

#[derive(uniffi::Error, thiserror::Error, Debug)]
#[uniffi(flat_error)]
Expand Down Expand Up @@ -63,6 +59,8 @@ pub enum GenericError {
ApiClientBuild(#[from] xmtp_api_grpc::GrpcBuilderError),
#[error(transparent)]
Grpc(#[from] xmtp_api_grpc::GrpcError),
#[error(transparent)]
AddressValidation(#[from] IdentifierValidationError),
}

#[derive(uniffi::Error, thiserror::Error, Debug)]
Expand Down
Loading
Loading