The corpus MODAL is a corpus of 160.000 tokens annotated with modality information. The corpus was extracted from the written sub-part of the Reference Corpus of Contemporary Portuguese (CRPC) (Généreux et al., 2012), restricted to European Portuguese and excluding documents from Politics and Law to avoid formal language usage. The following categories are marked as potential triggers when they denote an event’s modality: nouns, verbs, adverbs, adjectives that are part of a verbal phrase and the verb+prep combinations ter de ‘must’ and haver de ‘have to’. There are 3183 triggers in the annotated corpus, but a total of 3352 modal values attributed, due to ambiguous cases.
The corpus is under revision and will be soon available.
Annotation scheme: modal values
Our scheme covers a total of 13 modal values: epistemic and its su b-values (knowledge, belief, doubt, possibility and interrogative), deontic and sub-values (obligation, permission), participant-internal and sub-values (capacity, necessity), volition, evaluation, effort and success. The typology follows closely linguistic proposals such as Palmer (1986). The inclusion of participant-internal values follows directly from the typology of van der Auwera and Plungian (1998) (these values match what is called dynamic modality in other typologies (Palmer, 1986)). However, contrary to van der Auwera and Plungian (1998), the scheme doesn’t consider participant-external modality as an independent type, but rather as a sub-type of deontic modality. Other values, such as evaluation, effort and success, are inspired in other annotation schemes for modality (Baker et al., 2010).
Annotation scheme: components
The main components of the annotation scheme are:
- Trigger: the element conveying the modal value;
- Target: the expression in the scope of the trigger;
- Source of the event mention (speaker or writer);
- Source of the modality (agent or experiencer).
The annotation was performed over each trigger, and not over the global interpretation of the sentence. The trigger receives an attribute modal value, while both trigger and target are marked for polarity. This feature describes the positive or negative polarity of the trigger and not the polarity of full sentences. A trigger may be marked with negative polarity by a negation adverb, or the negative polarity can be expressed morphologically by an affix (improbable) or by the lexical verbal form itself (proibir ‘to forbid’).
Modal verbs may have more than one meaning and it is sometimes difficult to distinguish between those modal values, even when the annotator takes into consideration a larger context. To address this issue, the scheme includes an Ambiguity field, where the annotators can write down secondary meanings when present in a specific context. The annotator chooses the most salient meaning as the main modal value.
Tense is not annotated. So, there is no annotation of the past tense (although it provides certainty about the realization of an event), nor future (possibility) nor conditional (unless there is a conjunction introducing the conditional clause that we can consider a trigger). Declaratives are also not tagged, although they have an epistemic reading of belief (and even factuality) (Palmer, 1986). We consider, following Oliveira (1988), declaratives as representing the unmarked level of modality as they provide no evident trigger for the modal value. Evidentials are also not annotated as a separate value, and instead are marked as epistemic belief (supported by evidences).
The annotation was performed with the MMAX2 annotation software tool (Müller and Strube, 2006). The MMAX2 software is platform-independent, written in java and can freely be downloaded from http://mmax2.sourceforge.net/. MMAX2 offers a visual interface to annotate sentences by marking textual strings and creating links between the marked elements. MMAX2 enables the annotation of non-contiguous elements and produces stand-off XML annotation. The elements of the annotation consist of markables (namely the trigger, target, source of modality and source of event) that are linked to the same modal event.
The annotation was performed by one annotator and all difficult cases were discussed with a second annotator and included in the guidelines. A study to measure the inter-annotator agreement (IAA) for the task of modality annotation was conducted over 50 sentences by two linguists, targeting two specific features of our scheme: trigger identification and modal value attribution. We computed IAA using the kappa-statistic (Cohen, 1960) for each field in the annotation. For the Trigger the kappa value was .65 and for the accompanying Modal value a kappa of .85 was obtained, similar to the reported IAA for English (Matsuyoshi et al., 2010).
In recent years we see a clear trend in information extraction applications to go beyond the extraction of pure facts, to focus on personal opinions in sentiment analysis and opinion mining, and to distinguish between factual and probable information, to detect uncertainty, speculation and negation in biomedical text mining. This interest has resulted in some proposals for the annotation of modality mainly for the English language. The MODAL corpus contributes with a modality scheme for Portuguese, and its application over a 160.000-tokens corpus.
The corpus has been used to develop an automatic tool for modality identification and labeling, in partnership with the Department of Informatics of the University of Évora with Paulo Quaresma, Teresa Gonçalves and João Sequeira (see publications).
Hendrickx, Iris, Amália Mendes, Silvia Mencarelli, Agostinho Salgueiro (2012). Modality Annotation Manual, version 1.0. Centro de Linguística da Universidade de Lisboa, Lisboa, Portugal.
Hendrickx, Iris, Amália Mendes, Silvia Mencarelli (2012) Modality in Text: a proposal for corpus annotation. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation - LREC 2012, Istambul, 21-27 de Maio de 2012, pp. 1805-1812.
Mendes, Amália, Iris Hendrickx, Agostinho Salgueiro, Luciana Ávila (2013) Annotating the Interaction between Modality and Focus: the case of exclusive particles. In: Proceedings of the 7th Linguistic Annotation Workshop & Interoperability with Discourse (LAW VII), Association for Computational Linguistics, Sofia, Bulgaria, August 8-9 2013, pp. 228-237.
Quaresma, Paulo, Amália Mendes, Iris Hendrickx, Teresa Gonçalves (2014) Automatic tagging of modality: identifying triggers and modal values. In: Bunt, Harry (ed.) Proceedings of the 10th Joint ISO - ACL SIGSEM Workshop on Interoperable Semantic Annotation, May 26, 2014, Reykjavik, Iceland, pp. 95-101.
Ávila, Luciana Beatriz, Amália Mendes, Iris Hendrickx (2015) Towards a Unified Approach to Modality Annotation in Portuguese. In: Proceedings of the IWCS Workshop on Models for Modality Annotation, MOMA 2015, 14 April, 2015
Queen Mary University of London London, UK, Association for Computational Linguistics, pp. 1-8.
Quaresma, Paulo, Amália Mendes, Iris Hendrickx, Teresa Gonçalves (2014) Tagging and Labeling Portuguese Modal Verbs. In: Baptista, Jorge & Nuno Mamede, PROPOR 2014, LNAI 8775 – Lecture Notes in Artificial Intelligence. Springer, Heidelberg, pp. 70-81. ISSN 0302-9743; ISBN 978-3-319-09760-2
Mendes, Amália (2016) Modalidade e foco: uma análise baseada em dados de corpus, Diadorim – Revista de Estudos Linguísticos e Literários, 2016 Volume Especial, 127-141. ISSN: 1980-2552
Sequeira, João, Teresa Gonçalves, Paulo Quaresma, Amália Mendes, Iris Hendrickx (2016) Using syntactic and semantic features for classifying modal values in the Portuguese language. In: Proceedings of CICLing-16, 17th international Conference on Intelligent Text Processing and Computational Linguistics, Lecture Notes in Computer Science. Springer.
Mendes, Amália, Iris Hendrickx, Luciana Ávila, Paulo Quaresma, Teresa Gonçalves, João Sequeira (2016) Modality Annotation for Portuguese: from manual annotation to automatic labeling, LiLT - Linguistic Issues in Language Technology, 14 (2016), Special volume on Modality: Modes of Modality in NLP. ISSN: 1945-3604
João Sequeira, Teresa Gonçalves, Paulo Quaresma, Amália Mendes, Iris Hendrickx (2018) A Multi- versus a Single-classifier Approach for the Identification of Modality in the Portuguese Language. In Proceedings of the 11th Language Resources and Evaluation Conference - LREC’2018, 7-12 May 2018, Miyazaki, Japan, forthcoming.