Optimizing conflicting objectives in NMF using Pareto simulated annealing

Non-Negative matrix factorization (NMF) has emerged as an important technique for simplifying high-dimension data into interpretable factors. NMF has the attractive characteristic that the factor matrices are naturally sparse, thus allowing them to be readily interpreted. However, there is a tension between the accuracy of the factorization and the sparseness – it is the management of the trade-off between these two criteria that is the subject of this paper. We introduce a multi-criteria Simulated annealing framework that produces a Pareto set of solutions, which are non-dominated on both criteria. We show that solutions at one end of the Pareto front of solutions correspond to NMF factorizations produced with conventional optimization techniques, while solutions at the other end exhibit enhanced sparseness. Clustering is no longer to be observed either in the raw-data form of the matrix, or the generated heat-map form.