Recognizing and linking named entities in Portuguese medieval texts

Abstract

Despite the continuous development of approaches and tools for Named Entity Recognition (NER), historical texts still face issues that modern ones do not have. These issues relate to the nature and type of the documents, the time period when they were produced, the diachronic difference in the language and how they were extracted from their source and preserved in their digital form. This paper addresses the challenges of identifying, recognizing, and categorizing named entities (NE) in Old and Middle Portuguese. We also briefly address the challenge of disambiguating Named Entities. The Portuguese Corpus of Ancient Texts consists of texts dating back to the 13th century up to 1525. All texts are transcribed and preserved with little editorial intervention, in XML format, using the web-based platform TEITOK. A part-of-speech (PoS) automatic annotation model was created and applied to six texts of the corpus, to enrich them and improve search queries. Following the automatic annotation task, a manual correction step ensued, so that more than half a million tokens have been lemmatized and annotated with their respective part-of-speech and inflection. For the NER task, the method’s pipeline is presented, following the annotation of NE of a corpus of +400k tokens. In addition, the criteria for determining the boundaries of the NE were established, namely for the identification of name-chains and composite entities that comprise more than one word. For this task, the immediate context of the words was considered. Concerning disambiguation, this paper presents how ambiguity is pervasive within the texts and across the corpus.