Skip to content

Fixes compute_output_shape for PaliGemmaVitEncoder and Gemma3VisionEncoderBlock #2210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 9, 2025

Conversation

JyotinderSingh
Copy link
Collaborator

@JyotinderSingh JyotinderSingh commented Apr 9, 2025

The logic for handling a Keras 3.1 edge case was missing a return statement, causing this code to run into the following error when running keras_hub/src/models/pali_gemma/pali_gemma_backbone_test.py:

self = <PaliGemmaVitEncoder name=image_encoder, built=True>
inputs_shape = [None, None, None]

    def compute_output_shape(self, inputs_shape):
        if inputs_shape is None:
            # Fix the compatibility issue with Keras 3.1 where
            # `compute_output_spec` fails to propagate `inputs_shape`
            # correctly, causing it to be `None`.
            inputs_shape = [None, None, None]
        return [
            inputs_shape[0],
>           (inputs_shape[1] // self.patch_size) ** 2,
            self.hidden_dim,
        ]
E       TypeError: unsupported operand type(s) for //: 'NoneType' and 'int'
Screenshot 2025-04-09 at 12 50 20 PM

Additionally, we replace the last dimension of the output shape with self.hidden_dim, since we know for sure that we will be projecting hidden_dim outputs to the next layer.

The failing test now passes with these changes:
Screenshot 2025-04-09 at 12 53 41 PM

@github-actions github-actions bot added the Gemma Gemma model specific issues label Apr 9, 2025
Copy link
Collaborator

@abheesht17 abheesht17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Is the UT which was failing earlier passing now?

@JyotinderSingh
Copy link
Collaborator Author

Thanks!

Is the UT which was failing earlier passing now?

Yes, verified locally with latest keras-team/keras master commit, which was failing earlier.

Attached failing and updated test results to the PR description

@abheesht17
Copy link
Collaborator

@JyotinderSingh
Copy link
Collaborator Author

Can you make the same change here too, please?

https://github.com/keras-team/keras-hub/blob/master/keras_hub/src/models/gemma3/gemma3_vision_encoder.py#L491

Done

@abheesht17 abheesht17 added the kokoro:force-run Runs Tests on GPU label Apr 9, 2025
@kokoro-team kokoro-team removed the kokoro:force-run Runs Tests on GPU label Apr 9, 2025
@JyotinderSingh JyotinderSingh changed the title Fixes compute_output_shape for PaliGemmaVitEncoder Fixes compute_output_shape for PaliGemmaVitEncoder and Gemma3VisionEncoderBlock Apr 9, 2025
@abheesht17 abheesht17 merged commit 24e4308 into keras-team:master Apr 9, 2025
10 of 11 checks passed
@JyotinderSingh JyotinderSingh deleted the pali_gemma_fix branch April 10, 2025 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Gemma Gemma model specific issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants