Welcome to Research Repository UCD
Research Repository UCD is a digital collection of open access scholarly research publications from University College Dublin. Research Repository UCD collects, preserves and makes freely available publications including peer-reviewed articles, working papers and conference papers created by UCD researchers. Where material has already been published it is made available subject to the open-access policies of the original publishers. This service is maintained by UCD Library.
Diverging Divergences: Examining Variants of Jensen Shannon Divergence for Corpus Comparison Tasks
Files in This Item:
| File | Size | Format | |
|---|---|---|---|
| Download | LREC2020__Diverging_Divergences__Examining_Variants_of_Jensen_Shannon_Divergence_for_Corpus_Comparison_Tasks.pdf | 693.74 kB | Adobe PDF |
| Title: | Diverging Divergences: Examining Variants of Jensen Shannon Divergence for Corpus Comparison Tasks | Authors: | Lu, Jinghui; Henchion, Maeve; MacNamee, Brian | Permanent link: | http://hdl.handle.net/10197/12574 | Date: | 13-May-2020 | Online since: | 2021-10-26T11:13:30Z | Abstract: | Jensen-Shannon divergence (JSD) is a distribution similarity measurement widely used in natural language processing. In corpus comparison tasks, where keywords are extracted to reveal the divergence between different corpora (for example, social media posts from proponents of different views on a political issue), two variants of JSD have emerged in the literature. One of these uses a weighting based on the relative sizes of the corpora being compared. In this paper we argue that this weighting is unnecessary and, in fact, can lead to misleading results. We recommend that this weighted version is not used. We base this recommendation on an analysis of the JSD variants and experiments showing how they impact corpus comparison results as the relative sizes of the corpora being compared change. | Funding Details: | Science Foundation Ireland Teagasc |
Type of material: | Conference Publication | Keywords: | Corpus comparison; Jensen-Shannon divergence | Other versions: | https://lrec2020.lrec-conf.org/en/ https://aclanthology.org/2020.lrec-1.832/ |
Language: | en | Status of Item: | Peer reviewed | Conference Details: | The 12th Language Resources and Evaluation Conference (LREC 2020), Marseille, France, 11-16 May 2020 (cancelled due to coronavirus outbreak) | This item is made available under a Creative Commons License: | https://creativecommons.org/licenses/by/3.0/ie/ |
| Appears in Collections: | Computer Science Research Collection Insight Research Collection |
Show full item record
Page view(s)
230
Last Week
5
5
Last month
13
13
checked on May 21, 2022
Download(s)
15
checked on May 21, 2022
Google ScholarTM
Check
If you are a publisher or author and have copyright concerns for any item, please email research.repository@ucd.ie and the item will be withdrawn immediately. The author or person responsible for depositing the article will be contacted within one business day.