pyiceberg write an extra .db in the schema path

[djouallah]

Apache Iceberg version

0.9.1

Please describe the bug 🐞

pyiceberg add an extra .dot in the path, is there a way to turn it off.

from   pyiceberg.catalog.sql import SqlCatalog
catalog = SqlCatalog(
      "default",
      **{
          "uri"                : "sqlite:////tmp/pyiceberg.db",
          "legacy-current-snapshot-id": True ,
          "warehouse" : "/lakehouse/default/Files"
      },
                        )
schema         = "simple"
catalog.create_namespace_if_not_exists(schema)
catalog.list_tables(schema)

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[kevinjqliu]

Thanks for reporting this!

For context here is where the .db suffix is appended.

iceberg-python/pyiceberg/catalog/__init__.py

Lines 931 to 933 in 9185389

	if warehouse_path := self.properties.get(WAREHOUSE_LOCATION):
	warehouse_path = warehouse_path.rstrip("/")
	return f"{warehouse_path}/{database_name}.db/{table_name}"

I believe this is done to mirror the hive/HMS convention.
https://github.com/apache/hive/blob/de994e5c556930d1761601a61d6941c69fa9e153/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java#L74

I dont think we need/want to do this as part of the base catalog class.

[amitgilad3]

this is the behavior that also happens in spark if i'm not mistaken , for example glue - https://github.com/apache/iceberg/blob/a4b2a0dab092821d4843749b8abc30208622e164/aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java#L264

[kevinjqliu]

Thanks @amitgilad3 thats a good point.

defaultWarehouseLocation is an abstract function of BaseMetastoreCatalog

Referencing catalog implementations in Java:

• HiveCatalog, DynamoDbCatalog, and GlueCatalog all use .db suffix
• JdbcCatalog, HadoopCatalog, and InMemoryCatalog do not use .db suffix

I'll make the change in #2059 accordingly