Skip to content

subtask: Added Spark OceanBase catalog implementation for OceanBase MySQL mode #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Feb 12, 2025
Merged

subtask: Added Spark OceanBase catalog implementation for OceanBase MySQL mode #20

merged 6 commits into from
Feb 12, 2025

Conversation

yuanoOo
Copy link
Member

@yuanoOo yuanoOo commented Feb 10, 2025

Summary

close #16

Solution Description

feat:

  1. Only supports OceanBase's MySQL mode.
  2. Support spark to read OceanBase in parallel .
  3. Support direct-load implementation based on Spark DataSource V2 API.
  4. Support jdbc write implementation based on Spark DataSource V2 API.

bugfix

  1. Reduce the probability of direct-load failure in integration tests.
Caused by: com.alipay.oceanbase.rpc.direct_load.exception.ObDirectLoadNotSupportedException: direct load in ob version 0.0.0.0 is not supported, minimum version required is 4.2.1.7

tests

  1. com.oceanbase.spark.OBCatalogMySQLITCase#

@yuanoOo yuanoOo requested a review from whhe February 10, 2025 09:19
whhe
whhe approved these changes Feb 12, 2025
View reviewed changes
Copy link
Member

@whhe whhe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@whhe whhe merged commit ff656c2 into oceanbase:main Feb 12, 2025
8 checks passed
@yuanoOo yuanoOo deleted the catalog branch March 31, 2025 07:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@whhe whhe whhe approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[subtask]: Added Spark OceanBase catalog implementation for OceanBase MySQL mode.
2 participants
@yuanoOo @whhe