Knowledge Base Completion: Baselines Strike Back
Rudolf Kadlec, Ondřej Bajgar, Jan Kleindienst · Workshop on Representation Learning for NLP (RepL4NLP) · 2017
Many papers have been published on the knowledge base completion task in the past few years.Most of these introduce novel architectures for relation learning that are evaluated on standard datasets such as FB15k and WN18.This paper shows that the accuracy of almost all models published on the FB15k can be outperformed by an appropriately tuned baseline -our reimplementation of the Dist-Mult model.Our findings cast doubt on the claim that the performance improvements of recent models are due to architectural changes as opposed to hyperparameter tuning or different training objectives.This should prompt future research to re-consider how the performance of models is evaluated and reported.
Cited by 167 works. doi:10.18653/v1/w17-2609 Open-access PDF
Authors
References
- AutoResearch Lite: a reproducible paper-inspired experiment runner (Adam mechanism reproduction and convergence-speed comparison)
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- A direct adaptive method for faster backpropagation learning: the RPROP algorithm
- Freebase
- Knowledge Graph Embedding by Translating on Hyperplanes
- Learning Entity and Relation Embeddings for Knowledge Graph Completion
- Reasoning With Neural Tensor Networks for Knowledge Base Completion
- Learning Structured Embeddings of Knowledge Bases
- A Review of Relational Machine Learning for Knowledge Graphs
- A semantic matching energy function for learning with multi-relational data