A multi-criteria evaluation of a user generated content based recommender system

The Social Web provides new and exciting sources of information that may be used by recommender systems as a complementary source of recommendation knowledge. For example, User-Generated Content, such as reviews, tags, comments, tweets etc. can provide a useful source of item information and user preference data, if a clear signal can be extracted from the inevitable noise that exists within these sources. In previous work we explored this idea, mining term-based recommendation knowledge from user reviews, to develop a recommender that compares favourably to conventional collaborative-filtering style techniques across a range of product types. However, this previous work focused solely on recommendation accuracy and it is now well accepted in the literature that accuracy alone tells just part of the recommendation story. For example, for many, the promise of recommender systems lies in their ability to surprise with novel recommendations for less popular items that users might otherwise miss. This makes for a riskier recommendation prospect, of course, but it could greatly enhance the practical value of recommender systems to end-users. In this paper we analyse our User-Generated Content (UGC) approach to recommendation using metrics such as novelty, diversity, and coverage and demonstrate superior performance, when compared to conventional user-based and item- based collaborative filtering techniques, while highlighting a number of interesting performance trade-offs.