Skip to content

Some entities in test.txt/valid.txt are not appear in train.txt for WN18RR and FB15K-237 #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
iieirkm opened this issue May 13, 2018 · 1 comment

Comments

@iieirkm
Copy link

iieirkm commented May 13, 2018

Hi, I have noticed that some entities in test.txt/valid.txt are not appear in train.txt, which means these entities' embedding will not update during training, how do you deal with these unseen entities?

@TimDettmers
Copy link
Owner

Thank you for noting this. I was unaware of this when I designed the dataset. Currently, it would be consistent with other work, if you just treat these entities as any other. This falsely inflates the error rates, but if everyone treats this in the same manner there are no problems — we just need to be able to compare scores between models.

Please also see #6 for an extended discussion on this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants