Scholarly record
ON THE DIFFERENCES BETWEEN ASSOCIATION MEASURES FOR AUTOMATIC COLLOCATION EXTRACTION: EVALUATION AGAINST DICTIONARIES
Abstract
Collocation extraction is a prominent task in natural language processing, its results are important in various areas of applied linguistics. The research focuses on a comparison between seven statistical measures based on the large Russian ruTenTen corpus. The paper examines the bigrams with a number of Russian high-frequency nouns that were extracted by the given measures. The analysis is organized in two ways. First, we ranked the lists of bigrams obtained by each measure and evaluated the results against the Russian dictionaries identifying automatically extracted and manually collected collocations. The second experiment involved the comparison between each pair of measures in order to determine to what extent they produce the same results. The results show that dictionary collocations have higher rankings in the lists. Also the produced bigrams can be considered as collocations by experts and thus may enrich dictionaries. Several measures prove to demonstrate a relative interchangeability giving the overlapping results.
Publication Impact Profile
Publication details
References8
Jakubíček M., Kilgarriff A., Kovář V., Rychlý P., Suchomel V. The TenTen Corpus Family. In Proceedings of the International Conference on Corpus Linguistics, 2013, pp. 125–127.
Kilgariff A., Rychly P., Smrz P., Tugwell D. The Sketch Engine. In Proceedings of Euralex Lorient, France, July, 2004, pp. 105–116.
Evert S. The Statistics of Word Cooccurrences: Word Pairs and Collocations. Dissertation, Institut für maschinelle Sprachverarbeitung, University of Stuttgart, available at: http://purl.org/stefan.evert/PUB/Evert2004phd.pdf (accessed July 10, 2017).
Pecina P. Lexical Association Measures. Collocation Extraction. Prague: Institute of Formal and Applied Linguistics, 2009.
Evert S. Association measures, available at: http://collocations.de (accessed July 10, 2017).
Khokhlova M. V. Eksperimental’naja proverka metodov vydelelnija kollokacij [Evaluation of Methods for Collocation Extraction]. In Slavica Helsingiensia 34. Instrumentarij rusistiki: Korpusnye podhody. Eds. A. Mustajoki, M.V. Kopotev, L.A.Birjulin, J.J. Protasova. Helsinki, 2008, pp. 343–357.
Denisov P.N., Morkovkin V.V. (comp. and ed.-in-chief). Uchebnyi slovar' sochetaemosti slov russkogo iazyka [An Academic Collocation Dictionary of Russian]. Moscow: Russkij jazyk, 1983.
Yevgen’yeva A.P. (ed.-in-chief). Slovar’ russkogo jazyka [A Dictionary of the Russian Language] vol. 1–4, 2nd edition, revised and supplemented. Moscow: Russkij jazyk, 1981–1984.
View or Download full articleAccess options
SWS access login
Login as SWS Scientific CommitteeLogin as SWS Scientific PartnerLogin as SWS AuthorAuthors and approved SWS contributors will read and export their own linked papers after identity matching by SWS profile, email and SGEM GlobalID.
For librarian assistance: [email protected]
Purchase Instant Access
- Article can be downloaded after successful payment.
- Article may be used according to SWS library access terms.
- Article cannot be redistributed.
