Skip to content

AWQ -- Clean up forward passes with kwargs using inspect.bind #1385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
brian-dellabetta opened this issue Apr 25, 2025 · 1 comment · May be fixed by #1405
Open

AWQ -- Clean up forward passes with kwargs using inspect.bind #1385

brian-dellabetta opened this issue Apr 25, 2025 · 1 comment · May be fixed by #1405
Labels
enhancement New feature or request good first issue A good first issue for users wanting to contribute

Comments

@brian-dellabetta
Copy link
Collaborator

The current implementation of AWQ currently has a lot of helper code for passing kwargs into the .forward call of an arbitrary module. This can be cleaned up by using the inspect.bind method directly. Here's a nice github gist example -- https://gist.github.com/encukou/5060767
@brian-dellabetta brian-dellabetta added enhancement New feature or request good first issue A good first issue for users wanting to contribute labels Apr 25, 2025
@brian-dellabetta brian-dellabetta changed the title AWQ -- Cleanup forward passes with kwargs using inspect.bind AWQ -- Clean up forward passes with kwargs using inspect.bind Apr 25, 2025
@ved1beta
Copy link
Contributor

working on it

@ved1beta ved1beta mentioned this issue Apr 26, 2025
Closed
@brian-dellabetta brian-dellabetta mentioned this issue Apr 28, 2025
Merged
dsikka pushed a commit that referenced this issue Apr 28, 2025
@brian-dellabetta
awq -- hotfix to missing kwargs (#1395)
ef175d7 
SUMMARY:
This PR resolves the issues surrounding an Optional parameter passed
into a torch module's .forward method during AWQ. Previous attempts to
resolve in #1384 also added kwargs for parameters passed in positionally
later on. This will make the addition to kwargs more strict, only if the
annotation indicates if it is an optional field.

This hotfix will fail if optional fields are passed in positionally, if
typing annotation is `a: int | None` instead of `a:
typing.Optional[int]`, or if there is no typehint at all and the field
is not provided. It will be addressed with a more general solution soon,
see #1385


TEST PLAN:
New test was run with python 3.9 and passed --
https://github.com/neuralmagic/llm-compressor-testing/actions/runs/14713323963/job/41291028422

Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
kylesayrs pushed a commit that referenced this issue Apr 29, 2025
@brian-dellabetta @kylesayrs
awq -- hotfix to missing kwargs (#1395)
773fe7f 
SUMMARY:
This PR resolves the issues surrounding an Optional parameter passed
into a torch module's .forward method during AWQ. Previous attempts to
resolve in #1384 also added kwargs for parameters passed in positionally
later on. This will make the addition to kwargs more strict, only if the
annotation indicates if it is an optional field.

This hotfix will fail if optional fields are passed in positionally, if
typing annotation is `a: int | None` instead of `a:
typing.Optional[int]`, or if there is no typehint at all and the field
is not provided. It will be addressed with a more general solution soon,
see #1385


TEST PLAN:
New test was run with python 3.9 and passed --
https://github.com/neuralmagic/llm-compressor-testing/actions/runs/14713323963/job/41291028422

Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
@brian-dellabetta brian-dellabetta linked a pull request Apr 30, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request good first issue A good first issue for users wanting to contribute
Projects
None yet
Milestone
No milestone
2 participants
@brian-dellabetta @ved1beta