In this study, we address the Relation Extraction (RE) problem as follows: For a given sentence S, the RE problem is a classification problem, where the goal is to predict a semantic relation r between e₁ and e₂, both entities in S, following previous research, mainly ^[1], as our baseline model. RE plays a key role in information extraction from unstructured text, and it has a wide range of applications in many domains ^[2]-[4].

The rapid growth of unstructured text data and the valuable knowledge recorded in them has generated considerable interest in automatic detection and extraction of semantic relations ^[5].

Although many studies have been conducted to develop supervised relation extraction models ^[6]-[9], neural network-based approaches have been proposed for relation extraction, in particular with deep learning, e.g., Recursive Neural Networks ^[10]-[12], Recurrent Neural Networks (RNNs) ^[13]-[15], and Convolutional Neural Networks (CNNs) ^{[1], [2], [16]-[18]}.

Deep Learning (DL) has demonstrated its efficiency in improving the RE task.

Specifically regarding relations in English language, the deep learning (DL) models have been trained with little or no domain knowledge, and several studies have implemented DL methods for relation extraction from texts. However, depending on the language and domain of deep learning models for relation extraction, the following challenges may arise: (a) a lack of training samples in some languages and domains and (b) the generalization of model in a domain with different types of relations.

Against this backdrop, some researchers have been successful in performing RE for a specific domain. They have utilized large amounts of labelled data. However, there are insufficient labelled data for certain domains and languages. Therefore, domain adaptation, domain shift, domain bias, and domain transfer are used to perform relation extraction an unseen target domain or language. However, the factors and conditions that are appropriate for training and testing DL models with different types of datasets in a target domain or language should be explored. Therefore, transferring well-trained DL models to other domains remains a challenge.

This paper presents a baseline DL model ^[1] and an experiment conducted using multiple representations from the biomedical domain. To address both challenges mentioned above, we describe the impact of some biomedical datasets, the generalization capability of the deep learning model, and compare its performance when some representations of the baseline model are modified for the biomedical domain.

This paper is organized as follows.

Section 2 presents a DL model based on ^{[1], [16]} for relation extraction, considering the potential of DL for RE tasks in bioinformatics research (biology, biomedicine, and healthcare). Section 3 describes a suitable way to adapt a well-trained model to the biomedical domain.

Section 4 details the experimental setup and the evaluation of the baseline model on multiple public PPI, DDI and CPI corpora. The last two sections discuss and summarize the representations of the impacts and the behavior of the baseline model on the datasets.

Traditionally, Relation Extraction (RE) has been a classification problem that occurs between two or more named entities in the same sentence that have a semantic relationship. Depending on the number of semantic relation classes, RE tasks can be binary or multi-class. In this study, we considered a binary relation extraction task in the biomedical domain. ^[19]-[22].

Said task achieves a high performance with supervised approaches; however, it needs annotated data, which is time consuming and entails intensive human labor. Recently, models based on deep neural networks, such as CNNs and RNNs, have shown promising results for RE.

For example, in ^[1], the authors explored RE without exhaustive pre-processing. They employed a CNN and observed that any automatically learned features yielded promising results and could potentially replace the manually designed features.

In turn, in ^{[10], [23]} other authors proposed a DL approach with a RNN architecture and a matrix-vector word representation to explore the impact of the lack of explicit knowledge about the type of relation. Likewise, a study ^[11] compared the capabilities of CNN and RNN for the relation classification task. We reviewed other articles on relation extraction without specific domain. We also surveyed some models in the literature classified by network architecture and dataset. We hope this survey provides an overview to select a baseline model.

In Table 1, the baseline is marked in boldface, the CNN and RNN-based Models are learned on SemEval 2010, and the category Others is relevant for Deep Learning and RE tasks.

Table 1. Survey of studies into RE tasks using Deep Learning approaches.

Table 1 Table 1 Survey of studies into RE tasks using Deep Learning approaches

CNN-based Models
MODEL	F1
CNN [1]	82.7
CR-CNN [17]	84.1
CNN[16]	82.8
Attention- CNN [24]	85.4
depLCNN + NS [18]	85.6
Multi-Attention CNN [25]	88.0
Selection-Attention + CNN[26]	88.0
RNN-based Models
MV-RNN	82.4
Entity-Att BLSTM[10]	85.2
Hierarchical Attention BILSTM [27]	84.3
Attention BILSTM[28]	84.0
BILSTM [14]	82.7
Others
BRNN [29]	86.3
DRNN [30]	86.1
SDP-LSTM [13]	83.7
DepNN [31]	83.6
FCN [32]	83.0
RCNN [33]	96.5
PCNN [34]	84.0

Source: Created by the authors.

Most of the studies we reviewed are concerned with English language, and their models have not been extended to other languages or domains. There is a variety of possible relations between domains and languages, characterized by their own syntactic and lexical properties.

Nevertheless, the notion of a relation, what it “means”, is inherently ambiguous ^[35]. Many efforts have been devoted to biomedical relation extraction, whose goal is to discover valuable knowledge about proteins, drugs, diseases, genes, adverse effects, and other biological interactions from unstructured free text ^[36].

In this section, we introduce a model architecture based on Convolutional Neural Networks (CNN) and modifications that allow other representations.

These experiments are intended to show that DL models (baseline) for Relation Extraction (RE) can be adapted to another domain using multi-representation. First, we introduce the datasets and the metrics to evaluate precision: recall and f1-score. Next, we describe the parameters of the baseline model, the evaluation of the multi-representation, its effects, and performance on the data. Finally, we compare the performance of the baseline model with the modified model.

The performance of the datasets SemEval 2010, ADE, BioInfer, Chemprot, and reACE is presented in Table 6 with the F1-score. Overall, we can make two observations: (1) The baseline model is not stable for each dataset. Although the predictions of the baseline model are not sufficient to establish what happened, the performance is meant to provide a neutral benchmark to measure the effects of adaptation changes. (2) Our models did not produce, in terms of performance, results comparable to those reported with the baseline model when biomedical domain datasets were used. ADE and reACE contain an imbalanced class distribution, which is perhaps the reason behind their 99.56 and 25.24 performance, respectively.

Table 6 Table 6 Weighted F1scores of the baseline and the four dataset variations baseline score marked in boldface

Dataset	F1
SemEval2010	82.76
ADE	99.56
BioInfer	38.50
reACE	25.24
ChemProt	59.80

Source: Created by the authors.

Our hypothesis is that, as sample relations are extracted from an unbalanced corpus, our baseline model is more sensitive and a significant performance gap is produced. Our model, on the ADE corpus, achieved a high F1-score; however, the variation between the lowest and highest values of F1 in other datasets does not guarantee a superior performance.

This is perhaps not surprising since in-domain datasets contain short fragments of texts with scarce grammatical information, from which convolutions can capture relevant biomedical information and achieve a high F1. Moreover, after comparing CNN performance, we hypothesize that general and biomedical domain have equally or similarly difficult for RE task, when there is a difference across domain and domain models.

With supervised domain training and the model needs to capture knowledge and learn automatically features from the target domain. While we suspect there is still room for improvement, without utilizing domain specific information, the datasets may differ in ways we cannot account for with our reasoning.

Our baseline model exhibits two main aspects: First, unbalanced corpora have a negative impact on the F1-score.

More importantly, using a corpus with a balanced proportion of positive to negative relations can result in a better performance. Second, there are a number element in a deep learning model implementation, which makes exact replication of the results difficult, particularly performance results, but we compared our modified model from CNN base model (baseline), and the performance was quite similar. We believe our performance can be attributed to (1) vector representation and (2) class imbalance from the dataset. Thus, in-domain word embeddings and position embedding combinations are better for our model than out-of-domain word embeddings, although they cannot achieve results comparable to those of the baseline.

We presented a multi-step reasoning to train a model for other domains in cases in which data with other distribution and classes is available and the task is the same.

We also showed that our reasoning for model adaptation did not achieve a performance similar to that of the baseline model. Therefore, we carried out corpus-based exploration to address the adaptation of a deep learning model. We evaluated the DL model on different datasets. We also tested the deep learning model to extract semantic relations between entities implementing a similar experimental setup to that in the study by Zeng et. al ^[1].

After training and testing, our DL model should have learned how to extract semantic relationships due to the automatic learning of similar in-domain and out-of-domain features. However, our results confirm the need for a balanced dataset and additional information about the in-domain task. The proportion of positive and negative relations and the number of annotated data in the samples are different in each dataset (ADE, BioInfer, reACE, Chemprot, and SemEval).

the in-domain task. The proportion of positive and negative relations and the number of annotated data in the samples are different in each dataset (ADE, BioInfer, reACE, Chemprot, and SemEval).

Nevertheless, the problem of class imbalance between datasets has been reported in the literature ^[47].

We observed that the model is sensitive to word representations, which plays a significant role in model training. There are several embedding representations: position embedding (which represents the relative positions of entities and words in the sentence), medical and biological embedding (which contains specific information) in-domain word embedding (which includes methods that can generate domain-sensitive word embeddings).

Future studies can consider a similar reasoning, exploring, with combinations, different word representations (static, contextualized, with domain knowledge, and others).

In this paper, we proposed a DL model adapted to a new domain, more specifically, RE task for biomedical domain. We used an architecture to transfer the RE task from the generic domain to a biomedical one. After pre-processing the dataset, we obtained experimental results on several benchmark datasets. Nevertheless, we cannot confirm any advantage of the proposed model because it did not achieve a similar performance on different biomedical datasets or results comparable to those of SemEval 2010, which reached an F1-score of 82.76 (Baseline). Even though reACE, BioInfer, Chemprot, and ADE exhibited F1-scores of 25.24, 38.50, 59.80, and 99.56, respectively, these outputs cannot be rejected. We also analyzed the error and discuss the reasons behind our results.

Finally, our study explored different representations and results to avoid the duplicity of research efforts in the development of future systems.