Skip to content

Commit 94a0f67

Browse files
opendansorgus-opentensorbasfromancamfairchildCameron Fairchild
authored
Rust Wallet (#39)
* adds documentation * adds tests Signed-off-by: Gus <gus@opentensor.dev> * feat: mnemonic & hotkey pair * feat: wallet & hotkey file write * fmt * clean up and todos * testing sign * docs and signing working * password working - needs cleanup * adds coldkey gen * clean up * wallet integration with python wip * encrypt using secret box access * Test signed commit * wip restructure * update structure * update python package folder name * working * update structure * fix Keypair `#pyclass` macros * delete pyproject.toml * update structure * convert public_key to bytes * convert private_key to bytes * empty struct Keypair and getters * impl Default for Keypair * add pair * add getters ss58_address and seed_hex * all fields in Keypair work * add __repr__ and __str__ * Minimum functionality done * add print for test * update README with a test for Keypair * fix ci * add creating from public_key * update manual test script * remove comments from public_key getter * add creation with ss58_address * add the checker `If public_key is missing (ss58_address wasn't created), return an error` * add sign function + manual test case * BROKEN: Add SS58 address validation utility This commit introduces a new utility module for SS58 address validation. It includes a function `is_valid_ss58_address` and integrates it into the main library module. Additionally, a test for the validation function is provided. * add verify function + manual test case * fix utils::is_valid_ss58_address function * fix utils::is_valid_ss58_address function + test * add utils::get_ss58_format + test * keep support crypto_type 1 only (sr25519) * update README.md * utils done * update README.md * add funcs to the pymodule in the lib.rs * update Cargo.toml * GREEEEN BUILD * cargo fmt * cargo clippy GREEN * Add comments with keyfile.py usage across the wallet, btcli, sdk * frm * cargo files update * add Keyfile.rs structure * add Keyfile.rs structure * add Keyfile::exists_on_device * Add keyfile-related functions to library Integrated multiple keyfile-related Python functions into the library, including serialization, validation, and encryption functionalities. These functions currently contain unimplemented placeholders, awaiting further development. * add Keyfile::make_dirs * add Keyfile::_may_overwrite * add Keyfile::_write_keyfile_data_to_file * add Keyfile::exists_on_device update * add Keyfile::is_readable * add Keyfile::_read_keyfile_data_from_file, Keyfile::is_writable * Enable keyfile serialization and password validation This commit implements functions to serialize/deserialize keypair data to/from keyfiles in `src/keyfile.rs`, adds password strength validation, and introduces new dependencies for JSON handling and password analysis. Also, modifies visibility of `seed_hex` in `Keypair` to `pub(crate)`. * add Keyfile::is_encrypted * add TODOs * update deps in Cargo.toml * Refactor keyfile functions to use PyBytes instead of Vec<u8> Replaced Vec<u8> with PyBytes in keyfile serialization and deserialization functions for better compatibility with Python. Updated corresponding internal logic to handle PyBytes and ensured consistent error handling throughout. Adjusted test cases to reflect these changes. * add pyo3 - signature: green build * formatter * remove unused prefixes and pub sign * update README.md -> tests * update README.md -> tests * Enable encryption detection functions Uncomment and implement the functions to detect various encryption methods in keyfile data, including NaCl, Ansible, and legacy formats. Also, update the logic for retrieving the coldkey password from environment variables. * fix input logic and messages * add keyfile::keyfile_data_encryption_method * Add legacy keyfile encryption function Integrate ansible_vault for legacy keyfile encryption. Implement the legacy_encrypt_keyfile_data function and add necessary dependencies to Cargo.toml. * add keyfile::encrypt_keyfile_data * update docstring * update docstring + add pyo3 signature * Update error handling and getter methods in keyfile.rs Replaced `PyException` with more specific `pyo3::exceptions::PyUnicodeDecodeError` for error handling. Also, adjusted getter methods to return actual keyfile data by reading from the file instead of returning placeholder values. * add keypair::decrypt_keyfile_data * Refactor `set_keypair` to handle encryption and file writing Consolidated encryption logic and streamlined file writing process in `set_keypair` function. Now, the keypair data is serialized and optionally encrypted, then written to file with overwrite handling. This improves code clarity and functionality. * add keypair::Keypair.set_keypair, fix: keypair::encrypt_keyfile_data, add printing messages * move nacl_decrypt and legacy_decrypt into decrypt_keyfile_data * add keyfile::Keyfile.is_encrypted * commented keypair getter * add keypair::Keypair.get_keypair * add keypair::Keypair.check_and_update_encryption * Improve file encryption and decryption error handling Added comprehensive checks to ensure the keyfile exists, is readable, and writable before performing encryption and decryption. Enhanced the encryption and decryption methods to handle unencrypted data properly and ensured the results are written back correctly. * Refactor `decrypt` to handle errors with `PyResult` Refactor the `decrypt` method to return `PyResult<()>` and improve error handling by replacing `panic!` calls with appropriate `PyOSError` returns. Also, streamline the decryption process and key pair deserialization for better readability and maintainability. * Refactor keyfile encryption error handling Convert panic calls to proper error handling using PyResult for better stability and clarity. Modify the encrypt function to return detailed Python exceptions when the keyfile does not exist, is not readable, or not writable. This change ensures a more robust and informative error management. * activate GPG key * add wallet::Wallet.create_if_non_existent * update config * fix keyfile.rs * add list of functions into wallet.rs * make keypair::Keypair.create_from_uri public * add empty structure to wallet create, create some of the functions * add some empty functions into wallet::Wallet * fix cc compiler problem * Refactor wallet initialization to use config defaults * Add comprehensive tests for keyfile.rs Expanded the README.md to include detailed tests for keyfile.rs. These tests cover serialization, encryption, password validation, environment password retrieval, file operations, and various Keyfile methods. * add wallet::Wallet.regenerate_hotkey, wallet::Wallet.set_hotkey, wallet::Wallet.hotkey_file, update constants wallet path * fix keypair::Keypair.mnemonic() * add wallet::Wallet.regenerate_coldkeypub, coldkey_file, coldkeypub_file, set_coldkey, set_coldkeypub, get_coldkeypub, get_coldkey, get_hotkey, regenerate_coldkey * cleanup * fixes * add keypair::Keypair.create_from_encrypted_json * cls annotations fix * deps cleanup * import cleanup * keypair::Keypair.create_from_encrypted_json * temporarily remove incorrect private key usage in `create_from_encrypted_json` * create correct python package structure * add errors::KeyFileError, add its usage in the same way as in python wallet * update python package structure * README, refactoring * README, refactoring * README -> add test `create_from_encrypted_json` * lib.rs -> add classes on the package top level * Fix keypair::Keypair.create_from_encrypted_json * update README (import logic) * add #[pyclass(subclass)] ability for main classes, add mock subpackage * improve keyfile::Keyfile.__str__() * add export for wallet::display_mnemonic_msg * fix classic import logic * add errors::ConfigurationError * improve keyfile::Keyfile.new logic * add pyo3 signature to keypair::Keypair methods (functions) * add error into the module * add bittensor_wallet.keypair submodule * update ConfigurationError usage for Keypair::new * update errors * update types of errors in keypair * update types of errors in keypair * can't mock functions that are not implemented yet (rust doesn't accept this way). This limits testing from Python. temporarily commenting on mocking tests. * add `rpassword::prompt_password` instead of input for asking password. * Update version to 2.0.0 Upgrade the package version from 1.0.0 to 2.0.0 in Cargo.toml. This indicates significant changes, potentially including new features, improvements, or breaking changes. Ensure compatibility with the intended systems and dependencies. * use direct import for errors types in keyfile.rs * update .gitignore * use direct import for error types in keypair.rs * use direct import for error types in wallet.rs * use direct import for error types in utils.rs * update for wallets properties * add *args, **kwargs * use new instead of init * Refactor keypair decoding and wallet initialization Improve `keypair.rs` by adding public key conversion assertion. Refactor `wallet_mock.py` to streamline mock wallet initialization. Simplify wallet constructor signature in `wallet.rs` by removing unused arguments. * move assert into match arm * add back extension-module feature * add version * unused imports * fix create_from_seed (add union str, bytes) processing * add ScaleCodec to keypair::Keypair.sign and keypair::Keypair.verify * add tests to readme * Add getter methods for wallet name, path, and hotkey This commit introduces getter methods to retrieve the wallet's name, path, and hotkey string. These methods enhance the usability of the wallet by providing convenient access to these attributes. Each method returns a cloned value of the respective attribute. * use pythons stdout (#30) * Add getters for name and path in Keyfile Implement getter methods for retrieving the name and path from the keyfile object. This enhances the accessibility of these attributes, allowing for more efficient data manipulation and retrieval within the module. * Updated readme docs * add prompt and prompt password * chore: clippy * Adds extra line after print * try eval bound with import first * Add wallet::Wallet.add_args * update config::Config -> add getters * Refactor error handling in keyfile.rs Replaced the usage of `PyUserWarning` with a custom `KeyFileError` for more specific error handling. This change promotes better clarity and consistency in error reporting within the `keyfile.rs` module. * wrap with err * remove import * fix panic error to KeyFileError * delete double import * Fix typos in status messages Corrected the phrasing of status messages in keyfile.rs from "Encryption data..." and "Decrypt data..." to "Encrypting data..." and "Decrypting data...". This enhances clarity and consistency in the user interface. * wallet.coldkey.ss58_address doesn't ask typing pass after waller.unlock_coldkey() * wallet.hotkey.ss58_address doesn't ask typing pass after waller.unlock_hotkey() * Cam/no excepts (#32) * Remove except usage * use alias for now * use different constructor format * fix log * remove extra import * fix keypair.rs errors * chore: ruff * chore: clippy * chore: fmt * chore: ruff * use pypi trusted release * move `keypair::Keypair.private_key` outside the python scope * Add re-trying to type the password after wallet creation * update tests * remove utils * Feat/ci/add release (#38) * replace with pyproject.toml * fix release for manylinux builds * rename package and file * fix classifiers * remove btwallet dir * make non pre-release * try ignoring tests dir * try with maturin config * add bip39 * test with more platforms * remove other platforms * comment out other platforms * Bumps to a5 * Bumps to 2.0.0 * Updates Changelog --------- Signed-off-by: Gus <gus@opentensor.dev> Co-authored-by: Gus <gus@opentensor.dev> Co-authored-by: Roman <roman@opentensor.dev> Co-authored-by: Cameron Fairchild <cameron@opentensor.ai> Co-authored-by: Cameron Fairchild <cameron.fairchild@mail.utoronto.ca> Co-authored-by: Watchmaker <rajk@opentensor.dev> Co-authored-by: ibraheem-opentensor <ibraheem@opentensor.dev> Co-authored-by: Roman <167799377+roman-opentensor@users.noreply.github.com>
1 parent 8491a77 commit 94a0f67

34 files changed

+7030
-2952
lines changed

‎.github/workflows/release.yml

+140-57
Original file line numberDiff line numberDiff line change
@@ -1,72 +1,155 @@
1-
name: Build and Publish Python bittensor-wallet
1+
# This file is autogenerated by maturin v1.7.4
2+
# To update, run
3+
#
4+
# maturin generate-ci github
5+
#
6+
name: Build wheels
27

38
on:
9+
push:
10+
branches:
11+
- main
12+
tags:
13+
- '*'
14+
pull_request:
415
workflow_dispatch:
5-
inputs:
6-
version:
7-
description: 'Version to release'
8-
required: true
9-
type: string
16+
17+
permissions:
18+
contents: read
1019

1120
jobs:
12-
build:
13-
name: Build Python distribution
14-
runs-on: ubuntu-latest
21+
linux:
22+
runs-on: ${{ matrix.platform.runner }}
23+
strategy:
24+
matrix:
25+
platform:
26+
- runner: ubuntu-latest
27+
target: x86_64
28+
- runner: ubuntu-latest
29+
target: x86
30+
# - runner: ubuntu-latest
31+
# target: aarch64
32+
# - runner: ubuntu-latest
33+
# target: armv7
34+
# - runner: ubuntu-latest
35+
# target: s390x
36+
# - runner: ubuntu-latest
37+
# target: ppc64le
1538
steps:
16-
- uses: actions/checkout@v3
17-
18-
- name: Set up Python
19-
uses: actions/setup-python@v4
20-
with:
21-
python-version: '3.10'
39+
- uses: actions/checkout@v4
40+
- uses: actions/setup-python@v5
41+
with:
42+
python-version: 3.x
43+
- name: Build wheels
44+
uses: PyO3/maturin-action@v1.44.0
45+
with:
46+
target: ${{ matrix.platform.target }}
47+
args: --release --out dist --find-interpreter
48+
sccache: 'true'
49+
manylinux: auto
50+
before-script-linux: (apt-get update && apt-get install -y apt-utils && apt-get install -y pkg-config libssl-dev) || (yum install openssl openssl-devel -y)
51+
- name: Upload wheels
52+
uses: actions/upload-artifact@v4
53+
with:
54+
name: wheels-linux-${{ matrix.platform.target }}
55+
path: dist
2256

23-
- name: Install dependencies
24-
run: |
25-
python -m pip install --upgrade pip
26-
pip install build wheel twine
57+
# musllinux:
58+
# runs-on: ${{ matrix.platform.runner }}
59+
# strategy:
60+
# matrix:
61+
# platform:
62+
# - runner: ubuntu-latest
63+
# target: x86_64
64+
# # - runner: ubuntu-latest
65+
# # target: x86
66+
# # - runner: ubuntu-latest
67+
# # target: aarch64
68+
# # - runner: ubuntu-latest
69+
# # target: armv7
70+
# steps:
71+
# - uses: actions/checkout@v4
72+
# - uses: actions/setup-python@v5
73+
# with:
74+
# python-version: 3.x
75+
# - name: Build wheels
76+
# uses: PyO3/maturin-action@v1.44.0
77+
# with:
78+
# target: ${{ matrix.platform.target }}
79+
# args: --release --out dist --find-interpreter
80+
# sccache: 'true'
81+
# manylinux: musllinux_1_2
82+
# before-script-linux: (apt-get update && apt-get install -y apt-utils && apt-get install -y pkg-config libssl-dev) || (yum install openssl openssl-devel -y)
83+
# - name: Upload wheels
84+
# uses: actions/upload-artifact@v4
85+
# with:
86+
# name: wheels-musllinux-${{ matrix.platform.target }}
87+
# path: dist
2788

28-
- name: Build package
29-
run: python setup.py sdist bdist_wheel
30-
31-
- name: Check if package version already exists
32-
run: |
33-
PACKAGE_NAME=$(python setup.py --name)
34-
PACKAGE_VERSION=${{ github.event.inputs.version }}
35-
if twine check dist/*; then
36-
if pip install $PACKAGE_NAME==$PACKAGE_VERSION; then
37-
echo "Error: Version $PACKAGE_VERSION of $PACKAGE_NAME already exists on PyPI"
38-
exit 1
39-
else
40-
echo "Version $PACKAGE_VERSION of $PACKAGE_NAME does not exist on PyPI. Proceeding with upload."
41-
fi
42-
else
43-
echo "Error: Twine check failed."
44-
exit 1
45-
fi
89+
macos:
90+
runs-on: ${{ matrix.platform.runner }}
91+
strategy:
92+
matrix:
93+
platform:
94+
- runner: macos-12
95+
target: x86_64
96+
- runner: macos-14
97+
target: aarch64
98+
steps:
99+
- uses: actions/checkout@v4
100+
- uses: actions/setup-python@v5
101+
with:
102+
python-version: 3.x
103+
- name: Build wheels
104+
uses: PyO3/maturin-action@v1.44.0
105+
with:
106+
target: ${{ matrix.platform.target }}
107+
args: --release --out dist --find-interpreter
108+
sccache: 'true'
109+
- name: Upload wheels
110+
uses: actions/upload-artifact@v4
111+
with:
112+
name: wheels-macos-${{ matrix.platform.target }}
113+
path: dist
46114

47-
- name: Upload artifact
48-
uses: actions/upload-artifact@v3
49-
with:
50-
name: dist
51-
path: dist/
115+
sdist:
116+
runs-on: ubuntu-latest
117+
steps:
118+
- uses: actions/checkout@v4
119+
- name: Build sdist
120+
uses: PyO3/maturin-action@v1.44.0
121+
with:
122+
command: sdist
123+
args: --out dist
124+
- name: Upload sdist
125+
uses: actions/upload-artifact@v4
126+
with:
127+
name: wheels-sdist
128+
path: dist
52129

53-
approve-and-publish:
54-
needs: build
130+
release:
131+
name: Release
55132
runs-on: ubuntu-latest
56133
environment: release
134+
if: ${{ startsWith(github.ref, 'refs/tags/') || github.event_name == 'workflow_dispatch' }}
135+
#needs: [linux, macos, musllinux, sdist]
136+
needs: [linux, macos, sdist]
57137
permissions:
58-
contents: read
138+
# Use to sign the release artifacts
59139
id-token: write
60-
140+
# Used to upload release artifacts
141+
contents: write
142+
# Used to generate artifact attestation
143+
attestations: write
61144
steps:
62-
- name: Download artifact
63-
uses: actions/download-artifact@v3
64-
with:
65-
name: dist
66-
path: dist/
67-
68-
- name: Publish package distributions to PyPI
69-
uses: pypa/gh-action-pypi-publish@release/v1
70-
with:
71-
verbose: true
72-
print-hash: true
145+
- uses: actions/download-artifact@v4
146+
- name: Generate artifact attestation
147+
uses: actions/attest-build-provenance@v1
148+
with:
149+
subject-path: 'wheels-*/*'
150+
- name: Publish to PyPI
151+
if: "startsWith(github.ref, 'refs/tags/')"
152+
uses: PyO3/maturin-action@v1.44.0
153+
with:
154+
command: upload
155+
args: --non-interactive --skip-existing wheels-*/*

‎.gitignore

+3-1
Original file line numberDiff line numberDiff line change
@@ -213,4 +213,6 @@ replicate.yaml
213213
# Notebooks
214214
*.ipynb
215215

216-
tests/zombienet/bin/**/*
216+
tests/zombienet/bin/**/*
217+
218+
Cargo.lock

‎CHANGELOG.MD

+34
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,39 @@
11
# Changelog
22

3+
## 2.0.0 /2024-10-03
4+
5+
## What's Changed
6+
* Massive wallet change after copy-paste by @roman-opentensor in https://github.com/opentensor/btwallet/pull/1
7+
* Bump substrate-interface to 1.7.9 by @thewhaleking in https://github.com/opentensor/btwallet/pull/3
8+
* Fix flaky tests by @roman-opentensor in https://github.com/opentensor/btwallet/pull/5
9+
* Initial commit: circleci testing config by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/6
10+
* bug/roman/requirements by @roman-opentensor in https://github.com/opentensor/btwallet/pull/8
11+
* License fix by @roman-opentensor in https://github.com/opentensor/btwallet/pull/9
12+
* Add unlock method for wallet coldkey by @thewhaleking in https://github.com/opentensor/btwallet/pull/10
13+
* Add `unlock_hotkey` method to the wallet. Add unit tests. by @roman-opentensor in https://github.com/opentensor/btwallet/pull/11
14+
* Add `unlock_hotkeypub` method to the wallet. Add unit tests. by @roman-opentensor in https://github.com/opentensor/btwallet/pull/12
15+
* Updates mnemonic command output by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/17
16+
* Raises a KeyFileError when one is encountered when unlocking. by @thewhaleking in https://github.com/opentensor/btwallet/pull/19
17+
* bump up the version by @roman-opentensor in https://github.com/opentensor/btwallet/pull/20
18+
* Password unlocking changes by @thewhaleking in https://github.com/opentensor/btwallet/pull/18
19+
* bump up cryptography version until 43.0.1 by @roman-opentensor in https://github.com/opentensor/btwallet/pull/21
20+
* Deployment script + updates by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/23
21+
* Release 1.0.0 - Changelog by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/24
22+
* Pins py-bip39-bindings by @ibraheem-opentensor in https://github.com/opentensor/btwallet/pull/25
23+
* use pythons stdout by @camfairchild in https://github.com/opentensor/btwallet/pull/30
24+
* Cam/no excepts by @camfairchild in https://github.com/opentensor/btwallet/pull/32
25+
* Updated readme docs by @rajkaramchedu in https://github.com/opentensor/btwallet/pull/31
26+
* Feat/ci/add release by @camfairchild in https://github.com/opentensor/btwallet/pull/38
27+
28+
## New Contributors
29+
* @roman-opentensor made their first contribution in https://github.com/opentensor/btwallet/pull/1
30+
* @thewhaleking made their first contribution in https://github.com/opentensor/btwallet/pull/3
31+
* @ibraheem-opentensor made their first contribution in https://github.com/opentensor/btwallet/pull/6
32+
* @camfairchild made their first contribution in https://github.com/opentensor/btwallet/pull/30
33+
* @rajkaramchedu made their first contribution in https://github.com/opentensor/btwallet/pull/31
34+
35+
**Full Changelog**: https://github.com/opentensor/btwallet/commits/v2.0.0
36+
337
## 1.0.0 /2024-09-25
438

539
## What's Changed

0 commit comments

Comments
 (0)