Model-Based and Nonparametric Approaches to Clustering for Data Compression in Actuarial Applications

Files in This Item:
File Description SizeFormat 
Clustering Paper NAAJ for research repository (2).pdf3.42 MBAdobe PDFDownload
Title: Model-Based and Nonparametric Approaches to Clustering for Data Compression in Actuarial Applications
Authors: O'Hagan, Adrian
Ferrari, Colm
Permanent link:
Date: 2016
Abstract: Clustering is used by actuaries in a data compression process to make massive or nested stochastic simulations practical to run. A large data set of assets or liabilities is partitioned into a user-defined number of clusters, each of which is compressed to a single representative policy. The representative policies can then simulate the behavior of the entire portfolio over a large range of stochastic scenarios. Such processes are becoming increasingly important in understanding product behavior and assessing reserving requirements in a big-data environment. This article proposes a variety of clustering techniques that can be used for this purpose. Initialization methods for performing clustering compression are also compared, including principal components, factor analysis and segmentation. A variety of methods for choosing a cluster's representative policy is considered. A real data set comprised of variable annuity policies, provided by Milliman, is used to test the proposed methods. It is found that the compressed data sets produced by the new methods, namely model-based clustering, Ward's minimum variance hierarchical clustering and k-medoids clustering, can replicate the behavior of the uncompressed (seriatim) data more accurately than those obtained by the existing Milliman method. This is verified within sample, by examining location variable totals of the representative policies versus the uncompressed data at the five levels of compression of interest. More crucially it is also verified out of sample by comparing the distributions of the present values of several variables after 20 years across 1,000 simulated scenarios based on the compressed and seriatim data, using Kolmogorov-Smirnov goodness-of-fit tests and weighted sums of squared differences.
Type of material: Journal Article
Publisher: Taylor and Francis
Copyright (published version): 2016 Society of Actuaries
Keywords: Actuarial data compressionModel-based clustering
DOI: 10.1080/10920277.2016.1234398
Language: en
Status of Item: Peer reviewed
Appears in Collections:Mathematics and Statistics Research Collection

Show full item record

Google ScholarTM



This item is available under the Attribution-NonCommercial-NoDerivs 3.0 Ireland. No item may be reproduced for commercial purposes. For other possible restrictions on use please refer to the publisher's URL where this is made available, or to notes contained in the item itself. Other terms may apply.