Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FSTORE-1008] add java client to hsfs #448

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open

[FSTORE-1008] add java client to hsfs #448

wants to merge 36 commits into from

Conversation

davitbzh
Copy link
Contributor

This PR adds/fixes/changes...

  • please summarize your changes to the code
  • and make sure to include all changes to user-facing APIs

JIRA Issue: -

Priority for Review: -

Related PRs: -

How Has This Been Tested?

  • Unit Tests
  • Integration Tests
  • Manual Tests on VM

Checklist For The Assigned Reviewer:

- [ ] Checked if merge conflicts with master exist
- [ ] Checked if stylechecks for Java and Python pass
- [ ] Checked if all docstrings were added and/or updated appropriately
- [ ] Ran spellcheck on docstring
- [ ] Checked if guides & concepts need to be updated
- [ ] Checked if naming conventions for parameters and variables were followed
- [ ] Checked if private methods are properly declared and used
- [ ] Checked if hard-to-understand areas of code are commented
- [ ] Checked if tests are effective
- [ ] Built and deployed changes on dev VM and tested manually
- [x] (Checked if all type annotations were added and/or updated appropriately)

@davitbzh davitbzh added the wip label Jan 14, 2025
@davitbzh davitbzh requested a review from SirOibaf January 14, 2025 20:50
@davitbzh davitbzh changed the title [FSTORE-1008] add java client to hsfs (fv read part) [FSTORE-1008] add java client to hsfs Jan 22, 2025
@davitbzh davitbzh removed the wip label Jan 22, 2025
@davitbzh davitbzh requested a review from bubriks January 23, 2025 09:43
bubriks
bubriks requested changes Jan 23, 2025
View reviewed changes
Copy link
Contributor

@bubriks bubriks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, most of my complaints are about duplication. I am not sure if it is crucial to be solved now, but if it isn't then the technical debt would continue accumulating.

@davitbzh davitbzh requested a review from bubriks January 26, 2025 18:19
bubriks
bubriks approved these changes Jan 27, 2025
View reviewed changes
Copy link
Contributor

@bubriks bubriks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this pr is good to be merged, there could be some changes that we can do in the future to further limit duplication.

SirOibaf and others added 14 commits March 3, 2025 11:20
@SirOibaf
Add Unit Tests for VectorServer.java
a6e0685
@SirOibaf
Add support for creating feature groups from Java client (#1418)
06c6f3e 
* Prepare for 3.9.0-RC14-SNAPSHOT development

* Add full methods to create feature groups

* * Make getStorageConnector methods available across all frameworks

* Add methods to use StreamFeatureGroup builders

* Fix JobConfiguration property names

* Add getOrCreate methods

* Add more documentations on the methods

* Bump version for releaes
cherry-pick
baa1936
rebase
c1a7a9c
Merge remote-tracking branch 'upstream/main' into java_engine
6f16243
docs
df3a6c5
numentries
d349d01
@SirOibaf
Add POJO and GenericRecords support when inserting data from the Java…
3d505c4 
… … (#1419)

* Add POJO and GenericObject support when inserting data from the Java client

* Remove leftover todo

* Bump UT to action/cache@v4

* Set SNAPSHOT version for Java client
ignore complexFieldSchema.getType() == Schema.Type.NULL
66caf4a
@SirOibaf
Initial work on supporting creating feature views
27d69d9
@SirOibaf
Add getOrCreateFeatureView methods
c37f249
@SirOibaf
Fix bug with deserialization of complex features with prefix
ffa786c
unused import
61c1d3b
revert pojo to generic and remove getTransformation
011c30e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bubriks bubriks bubriks approved these changes

@SirOibaf SirOibaf Awaiting requested review from SirOibaf

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@davitbzh @bubriks @SirOibaf