Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproduce parsing performance #12

Open
xgeric opened this issue May 9, 2021 · 0 comments
Open

Reproduce parsing performance #12

xgeric opened this issue May 9, 2021 · 0 comments

Comments

@xgeric
Copy link

xgeric commented May 9, 2021

Thank you for the awesome work!

We were trying to reproduce the performance of the parsing module with four different random seeds (1, 11, 111, 1111), while the average f1 score given by the default script is 47.6, with max f1 around 50.4.

For your reference, we are controlling random seeds using the following function:

def set_random_seed(seed=0):
    random.seed(seed)
    numpy.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)  # if you are using multi-GPU.
    torch.backends.cudnn.enabled = False 
    torch.backends.cudnn.benchmark = False
    torch.backends.cudnn.deterministic = True

If you may, could you please kindly share the random seeds or any specific settings aside from the default parameters for the Table 1 experiments for reproduction? Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant