Skip to content

“exact match” 评测传答案进行推理的问题 #550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
shiwanghua opened this issue May 9, 2025 · 2 comments
Closed

“exact match” 评测传答案进行推理的问题 #550

shiwanghua opened this issue May 9, 2025 · 2 comments
Labels
question Further information is requested

Comments

@shiwanghua
Copy link

Image

为什么不把标准答案 token 改为 padding 占位呢？

这样推理出来的内容就不是基于标准答案推出来的了
@HaomingX
Copy link
Collaborator

HaomingX commented May 9, 2025

目前EasyEdit评测的实现是token-level的teacher-forcing写法，所以实际上这部分代码是在逐token比较。

而你所说的不基于标准答案的比较方式已经在HERE 实现了。

@zxlzr
Copy link
Contributor

zxlzr commented May 9, 2025

请问您还有其他问题吗？

@zxlzr zxlzr added the question Further information is requested label May 9, 2025
@zxlzr zxlzr closed this as completed May 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zxlzr @shiwanghua @HaomingX