Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: patch job metadata endpoint #397

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ryuwd
Copy link
Contributor

@ryuwd ryuwd commented Feb 5, 2025

Todo:

  • Write PATCH /metadata
  • Tests
    • test patching attributes and parameters
    • test getting job status history in /search

Closes #162

@ryuwd ryuwd force-pushed the roneil-jobstateupdatehandler branch 2 times, most recently from 052332b to d2106a3 Compare February 7, 2025 14:08
@ryuwd ryuwd changed the title metadata patch endpoint feat: patch job metadata endpoint Feb 7, 2025
@ryuwd ryuwd force-pushed the roneil-jobstateupdatehandler branch 3 times, most recently from 328cf93 to 46559f7 Compare February 10, 2025 17:38
@ryuwd ryuwd force-pushed the roneil-jobstateupdatehandler branch 2 times, most recently from 46559f7 to dc8bee3 Compare February 20, 2025 17:28
@ryuwd ryuwd marked this pull request as ready for review February 20, 2025 17:57
@ryuwd
Copy link
Contributor Author

ryuwd commented Feb 20, 2025

Rebased and ready to go

@aldbr aldbr force-pushed the roneil-jobstateupdatehandler branch 2 times, most recently from a29bf3e to 41cbccf Compare March 6, 2025 12:30
@ryuwd
Copy link
Contributor Author

ryuwd commented Mar 6, 2025

@aldbr LGTM, please go ahead

ryuwd added 4 commits March 6, 2025 14:13
add policy

Missing awaits and return results

adjust returned output

Fixed mypy

mypy false positive?

and line 34
[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci
@aldbr aldbr force-pushed the roneil-jobstateupdatehandler branch from 06ec05d to ede0868 Compare March 6, 2025 13:16
Comment on lines 41 to +44
if body.parameters:
body.parameters.remove("LoggingInfo")
body.parameters = ["JobID"] + (body.parameters or [])
if not body.parameters:
body.parameters = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an impossible thing to happen

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This in essence forces body.parameters to None if removing LoggingInfo leaves an empty list, hence making it as if the default set of parameters (in addition to the logging info) are the ones should be queried when calling job_db.search. I ran into an unfortunate edge case in getting the unit tests to pass and this is what I came up with.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default parameter set includes JobID and a bunch of other things, but if you don't keep this then you end up with body.parameters = ["JobID"] only

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an edge case that can only happen in testing, but whatever let's keep the test.

Comment on lines +487 to +489
possible_attribute_columns = [
col.name.lower() for col in _get_columns(Jobs.__table__, None)
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this (lower() of all column names) and the subsequent check needed? Can't we compare without lowering?

Copy link
Contributor Author

@ryuwd ryuwd Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm inclined to agree. I just wonder what would happen if somebody passed a case-sensitive un-matching attribute column that then gets registered in the JobParameterDB instead of throwing an error. Maybe we should throw an exception if the case-sensitive vs. insensitive matches don't line up

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, nothing is perfect. Do as you feel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add routes for JobStateUpdateHandler
4 participants