Experiment implementation for catalog builder

[liurenjie1024]

What changes are included in this PR?

Demo implementation for catalog builder.

Are these changes tested?

UT.

[liurenjie1024]

I also want to remove builder for catalog configs.

[liurenjie1024]

cc @Xuanwo I've figured out a way to keep both traits without introducing an enum, see

iceberg-rust/crates/catalog/loader/src/lib.rs

Line 10 in f4a2efe

pub trait BoxedCatalogBuilder {

WDYT?

[liurenjie1024]

cc @Fokko @sdd @Xuanwo Could you take a look at this pr to see if this is the right direction to go?

[c-thiel]

@liurenjie1024 thanks for the initiative!
For my understanding I first need to answer this question:
Are we sure that we need the CatalogBuilder trait? What would be its purpose?

My initial idea would be to just have different typesafe builders for different catalogs. The current CatalogBuilder trait currently contains fields that are not required for some catalogs. For example in-memory or dynamo don't even need a uri.
On the other hand we have the rest catalog that needs significantly more configurations, and I don't think we want to make those key value pairs. I would be in favor to have the interfaces as typesafe as possible.

[liurenjie1024]

cc @Xuanwo

[liurenjie1024]

Are we sure that we need the CatalogBuilder trait? What would be its purpose?

Hi, @c-thiel Sorry for misclarification. For more background, please refer to #1228. In short, we are trying to develop a catalog loader so that it can be used by some applications such as iceberg-playground or data driven integration test framework.

My initial idea would be to just have different typesafe builders for different catalogs. The current CatalogBuilder trait currently contains fields that are not required for some catalogs. For example in-memory or dynamo don't even need a uri.

I've refined the trait definition as proposed by @Xuanwo in #1372 .

On the other hand we have the rest catalog that needs significantly more configurations, and I don't think we want to make those key value pairs. I would be in favor to have the interfaces as typesafe as possible.

I think a pure type safe approach may not be practical since we also need to pass file io's configurations through catalog, and the actual file io used is determined at runtime.

As a compromise, we could define a config struct for each catalog like following:

struct RestCatalogConfig {
   #[serde(name = "name")]
   name: String,
  #[serde(name = "uri")]
   uri: Url,
  #[serde(name = "credential")]
   credential: String
}

impl TryFrom<HashMap<String, String>> for RestCatalogConfig {
 .....
}

WDYT?

[sdd]

Makes sense to me