Thornley, Clare V.Clare V.ThornleyJohnson, Andrea C.Andrea C.JohnsonSmeaton, Alan F.Alan F.SmeatonLee, HyowonHyowonLee2011-02-242011-02-242011 Wiley2011-04Journal of the American Society for Information Science and Technologyhttp://hdl.handle.net/10197/2803This paper reports on an investigation into the scholarly impact of the TRECVid (TREC Video Retrieval Evaluation) benchmarking conferences between 2003 and 2009. The contribution of TRECVid to research in video retrieval is assessed by analyzing publication content to show the development of techniques and approaches over time and by analyzing publication impact through publication numbers and citation analysis. Popular conference and journal venues for TRECVid publications are identified in terms of number of citations received. For a selection of participants at different career stages, the relative importance of TRECVid publications in terms of citations vis a vis their other publications is investigated. TRECVid, as an evaluation conference, provides data on which research teams ‘scored’ highly against the evaluation criteria and the relationship between ‘top scoring’ teams at TRECVid and the ‘top scoring’ papers in terms of citations is analysed. A strong relationship was found between ‘success’ at TRECVid and ‘success’ at citations both for high scoring and low scoring teams. The implications of the study in terms of the value of TRECVid as a research activity, and the value of bibliometric analysis as a research evaluation tool, are discussed.2779952 bytesapplication/pdfenThis is the authors' version of the following article: "The Scholarly Impact of TRECVid (2003-2009)" published in Journal of the American Society for Information Science and Technology, 62. It is available in its final form at http://dx.doi.org/10.1002/asi.21494.Video retrievalBibliometricsVisualisationResearch evaluationIR test collectionsBibliometricsVisualizationInstitutional repositoriesInformation retrieval--ResearchDigital video--ResearchInformation retrieval--CongressesDigital video--CongressesJournal Article62461362710.1002/asi.21494https://creativecommons.org/licenses/by-nc-sa/1.0/