Skip to content

Add cpu accelerator fp16 dtype support #7207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add cpu accelerator fp16 dtype support #7207

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
935884d
Add cpu accelerator fp16 dtype support
Yejing-Lai Apr 8, 2025
43c3c2e
Merge branch 'master' into lyj/add_fp16
loadams Apr 9, 2025
1869846
Merge branch 'master' into lyj/add_fp16
loadams Apr 11, 2025
8510da4
Update cpu accelerator supported dtype
Yejing-Lai Apr 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion accelerator/cpu_accelerator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ def is_fp16_supported(self):
return False

def supported_dtypes(self):
return [torch.float, torch.bfloat16]
return [torch.float, torch.bfloat16, torch.float16]
Copy link
Collaborator

@loadams loadams Apr 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Yejing-Lai - perhaps this is the runner, but the default we have here does not have torch.float16 support (see unit test errors).

I think this may come back to the discussions we've had before on splitting the cpu accelerator into a cpu accelerator (generic cpu) and a xeon accelerator (PR here: #5126 )?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loadams, good point.

@Yejing-Lai, are you specifically targetting xeon with this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I test this PR on xeon and it works fine. We will discuss determining the datatype list based on the CPU flags. Please pending this PR temporarily, thanks~

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better to keep one CPU accelerator with different features. Xeon is not much different from perspective of Pytorch CPU support. Desktop or other non-Xeon CPU might also get fp16 support if they implemented proper instruction set and get supported in Pytorch. So it should be reasonable to have one CPU accelerator instead of seperate xeon accelerator.


# Graph operations
def create_graph(self):
Expand Down
Loading