Skip to content

[BUG]: Hybrid Parallel Plugin PP=1 TypeError: qwen2model.forward() got an unexpected keyword argument 'force_sp_output_gather' #6264

Open
@happynaruto

Description

@happynaruto

Is there an existing issue for this bug?

  • I have searched the existing issues

The bug has not been fixed in the latest main branch

  • I have checked the latest main branch

Do you feel comfortable sharing a concise (minimal) script that reproduces the error? :)

Yes, I will share a minimal reproducible script.

🐛 Describe the bug

colossal=0.4.9,Hybrid Parallel Plugin,zero_stage=1,zero_cpu_offload=true,在八张A100显卡上训练QWQ32B,当pp=2,tp=4时程序正常运行,但当pp=1,tp=8时报错:TypeError: qwen2model.forward() got an unexpected keyword argument 'force_sp_output_gather'

Environment

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions