Biomedical Knowledge Graph Completion with Efficient Contrastive Learning

Knowledge graphs (KGs) are typically incomplete, but the task of knowledge graph completion (KGC) can address this issue by using existing facts to deduce the missing links. Biomedical KGC aims to automatically predict the head or tail entity in KG triples from biomedical text, which enables data-driven tasks such as drug discovery and disease treatment. With the success of Transformer architecture, textual encoding methods have emerged that utilize pre-trained language models (PLMs) to learn entity and relation representations. Nevertheless, the performance of textual encoding methods still substantially falls behind graph embedding methods, primarily due to the efficient contrastive learning of the latter. This paper is based on SimKGC that employ three distinct types of negatives, respectively, in-batch negatives, pre-batch negatives, and hard negatives to enhance the performance of textual encoding methods in biomedical KGC. Furthermore, we adopt a two-tower model with biomedical PLMs to encode entities and relations, respectively. It is the first attempt to apply efficient contrastive learning in biomedical KGC. Extensive experiments reveal that the combination of InfoNCE loss from contrastive learning and biomedical PLMs can substantially outperform graph embedding methods on two biomedical KGs, UMLS and Hetionet, in terms of automatic evaluation metrics (MR, MRR, and Hits@{1,3,10}).