No working documented method to create S3 client with self signed certs

[russfellows]

Describe the issue

According to AWZS developer / maintainer (aajtod), there is documentation in the developer guide that shows how to create an S3 client with CA cert overrides.

This is in fact, not accurate as there is not a complete example, only a small code snip that cannot stand alone, nor be used to build a client successfully. We need a complete example of all steps required to build an S3 client that performs these overrides, and can be used to perform basic operations, such as a list, or other S3 op.

See the links below for the discussion and issue.

Also, see link to a Github repo I am working on which is attempting to create the S3 client with the cert overrides, and has an example that is a good starting point, but at this point does not yet compile.

Assistance needed.

Links

#1277
#1278

Link to Github repo developing S3 Rust library for AI/ML, with issue discussing in ability to create the S3 client with CA cert overrides, and potential code to try, but needs work:
russfellows/dlio_s3_rust#4

[aajtodd]

The code (1, 2) you've linked does not match the developer guide nor does it take the suggestions/fixes outlined in the discussion thread.

The code in the developer guide is accurate and matches the integration tests that specifically test custom CA support.

[russfellows]

Aaron, I believe you are trying to be helpful, but the developer guide link you provided also does not provide a working example. Instead of telling me all the ways I am getting things wrong, it seems to be a better use of everyone's time to just provide a working example of standalone code. The test code buried deep in the repo in the "smithy-lang" repo does not appear to be code that is ever intended to work as stand alone code. The references to the code here: rust-runtime/aws-smithy-http-client/tests/tls.rs do not appear to work independently of this repo.

[russfellows]

I have been programming, on and off for 35 years and have often run into "gate keepers" who feel it is their sworn duty to keep out those whom they do not feel are worthy of their code. Developers who claim that a user guide they wrote is inherently readable are missing the obvious. If others say something isn't clear, they are probably correct. Note, when I finally get this working, I will post the solution here, so that other developers don't endure the pain I have...

[russfellows]

Note: IF this gets closed, I doubt it is because the issue was actually resolved, but rather the developers decided to close it to keep their bug count artificially low. Just documenting this fact here. We'll see, they could always prove me wrong by actually posting a working example here...

Happy to report that obviously I was wrong. I apologize.

[aajtodd]

AWS SDK for Rust Custom CA Example

End-to-end working example of configuring the AWS SDK for Rust with a custom root trust store.

The SDK runtime expects PEM encoded certificates. This example includes a bash script to download
the root Amazon Trust CA's into a bundle. You can provide
your own custom CA instead.

# download the aws-root-ca.pem bundle OR supply your own
./fetch-aws-root-ca.sh

# run the example driver program
AWS_PROFILE=my-profile cargo run -- --ca-bundle aws-root-ca.pem 

---

# file: Cargo.toml
[package]
name = "custom-ca-example"
version = "0.1.0"
edition = "2024"

[dependencies]
aws-config = "1.6.2"
aws-sdk-s3 = "1.84.0"
aws-smithy-http-client = { version = "1.0.1", features = ["rustls-aws-lc"] }
clap = { version = "4.5.37", features = ["derive"] }
tokio = { version = "1.45.0", features = ["full"] }

// src/main.rs
use std::path::Path;

use aws_config::BehaviorVersion;
use clap::Parser;
use aws_smithy_http_client::tls::{self, rustls_provider::CryptoMode};

#[derive(Parser, Debug)]
struct Args {
    #[arg(long, value_name = "CA_ROOT", value_hint = clap::ValueHint::FilePath)]
    ca_bundle: std::path::PathBuf
}

/// read the PEM encoded root CA (bundle) and return a custom TLS context
fn tls_context_from_pem(filename: impl AsRef<Path>) -> tls::TlsContext {
    let pem_contents = std::fs::read(filename).unwrap();
    
    // Create a new empty trust store (this will not load platform native certificates)
    let trust_store = tls::TrustStore::empty()
        .with_pem_certificate(pem_contents.as_slice());
        
    tls::TlsContext::builder()
        .with_trust_store(trust_store)
        .build()
        .expect("valid TLS config")
}

#[tokio::main]
async fn main() {
    let args = Args::parse();
    println!("{args:?}");

    let http_client = aws_smithy_http_client::Builder::new()
        .tls_provider(tls::Provider::Rustls(CryptoMode::AwsLc))
        .tls_context(tls_context_from_pem(args.ca_bundle))
        .build_https();

    let sdk_config = aws_config::defaults(BehaviorVersion::latest())
        .http_client(http_client)
        .load()
        .await;


    let s3_client = aws_sdk_s3::Client::new(&sdk_config);

    let result = s3_client.list_buckets().max_buckets(5).send().await;

    println!("list_buckets: {result:?}");
}

Bash script to download AWS root trust CA bundle

#!/bin/bash
#file: fetch-aws-root-ca.sh

# Download AWS trust CA's into a bundle from https://www.amazontrust.com/repository/

curl -L https://www.amazontrust.com/repository/AmazonRootCA1.pem >> aws-root-ca.pem
curl -L https://www.amazontrust.com/repository/AmazonRootCA2.pem >> aws-root-ca.pem
curl -L https://www.amazontrust.com/repository/AmazonRootCA3.pem >> aws-root-ca.pem
curl -L https://www.amazontrust.com/repository/AmazonRootCA4.pem >> aws-root-ca.pem
curl -L https://www.amazontrust.com/repository/SFSRootCAG2.pem >> aws-root-ca.pem

[russfellows]

THANK you! And not just from me, but also the other members of the MLCommons group that are working on an S3 data loader for AI workloads. The biggest issue has been the un ending array of Crates, including rustls, hyper-rustls, tokio-rustls, etc. etc. Plus the warnings on the aws-smithy crates that explicitly say “These are not expected to be used directly”. Collectively, we have spent more than a week of person hours trying to solve this issue. Regards, — Russ Fellows

…

On May 6, 2025, at 8:24 AM, Aaron Todd ***@***.***> wrote: aajtodd left a comment (awslabs/aws-sdk-rust#1286) <#1286 (comment)> AWS SDK for Rust Custom CA Example End-to-end working example of configuring the AWS SDK for Rust with a custom root trust store. The SDK runtime expects PEM encoded certificates. This example includes a bash script to download the root Amazon Trust CA's <https://www.amazontrust.com/repository/> into a bundle. You can provide your own custom CA instead. # download the aws-root-ca.pem bundle OR supply your own ./fetch-aws-root-ca.sh # run the example driver program AWS_PROFILE=my-profile cargo run -- --ca-bundle aws-root-ca.pem # file: Cargo.toml [package] name = "custom-ca-example" version = "0.1.0" edition = "2024" [dependencies] aws-config = "1.6.2" aws-sdk-s3 = "1.84.0" aws-smithy-http-client = { version = "1.0.1", features = ["rustls-aws-lc"] } clap = { version = "4.5.37", features = ["derive"] } tokio = { version = "1.45.0", features = ["full"] } // src/main.rs use std::path::Path; use aws_config::BehaviorVersion; use clap::Parser; use aws_smithy_http_client::tls::{self, rustls_provider::CryptoMode}; #[derive(Parser, Debug)] struct Args { #[arg(long, value_name = "CA_ROOT", value_hint = clap::ValueHint::FilePath)] ca_bundle: std::path::PathBuf } /// read the PEM encoded root CA (bundle) and return a custom TLS context fn tls_context_from_pem(filename: impl AsRef<Path>) -> tls::TlsContext { let pem_contents = std::fs::read(filename).unwrap(); // Create a new empty trust store (this will not load platform native certificates) let trust_store = tls::TrustStore::empty() .with_pem_certificate(pem_contents.as_slice()); tls::TlsContext::builder() .with_trust_store(trust_store) .build() .expect("valid TLS config") } #[tokio::main] async fn main() { let args = Args::parse(); println!("{args:?}"); let http_client = aws_smithy_http_client::Builder::new() .tls_provider(tls::Provider::Rustls(CryptoMode::AwsLc)) .tls_context(tls_context_from_pem(args.ca_bundle)) .build_https(); let sdk_config = aws_config::defaults(BehaviorVersion::latest()) .http_client(http_client) .load() .await; let s3_client = aws_sdk_s3::Client::new(&sdk_config); let result = s3_client.list_buckets().max_buckets(5).send().await; println!("list_buckets: {result:?}"); } — Reply to this email directly, view it on GitHub <#1286 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF64UJ7A3CE2JT7IVRQ4TFL25DA2BAVCNFSM6AAAAAB3ZPR26GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNJUG43TGNBSGI>. You are receiving this because you authored the thread.

[DavidSouther]

Due to some quirks in how we manage code examples at AWS, we'll track this in awsdocs/aws-doc-sdk-examples#7435.