Model-Based and Nonparametric Approaches to Clustering for Data Compression in Actuarial Applications

Files in This Item:
 File SizeFormat
DownloadClustering Paper NAAJ for research repository (2).pdf3.42 MBAdobe PDF
Title: Model-Based and Nonparametric Approaches to Clustering for Data Compression in Actuarial Applications
Authors: O'Hagan, AdrianFerrari, Colm
Permanent link:
Date: 2016
Online since: 2018-05-04T01:00:13Z
Abstract: Clustering is used by actuaries in a data compression process to make massive or nested stochastic simulations practical to run. A large data set of assets or liabilities is partitioned into a user-defined number of clusters, each of which is compressed to a single representative policy. The representative policies can then simulate the behavior of the entire portfolio over a large range of stochastic scenarios. Such processes are becoming increasingly important in understanding product behavior and assessing reserving requirements in a big-data environment. This article proposes a variety of clustering techniques that can be used for this purpose. Initialization methods for performing clustering compression are also compared, including principal components, factor analysis and segmentation. A variety of methods for choosing a cluster's representative policy is considered. A real data set comprised of variable annuity policies, provided by Milliman, is used to test the proposed methods. It is found that the compressed data sets produced by the new methods, namely model-based clustering, Ward's minimum variance hierarchical clustering and k-medoids clustering, can replicate the behavior of the uncompressed (seriatim) data more accurately than those obtained by the existing Milliman method. This is verified within sample, by examining location variable totals of the representative policies versus the uncompressed data at the five levels of compression of interest. More crucially it is also verified out of sample by comparing the distributions of the present values of several variables after 20 years across 1,000 simulated scenarios based on the compressed and seriatim data, using Kolmogorov-Smirnov goodness-of-fit tests and weighted sums of squared differences.
Type of material: Journal Article
Publisher: Taylor and Francis
Journal: North American Actuarial Journal
Volume: 21
Issue: 1
Start page: 107
End page: 146
Copyright (published version): 2016 Society of Actuaries
Keywords: Actuarial data compressionModel-based clustering
DOI: 10.1080/10920277.2016.1234398
Language: en
Status of Item: Peer reviewed
This item is made available under a Creative Commons License:
Appears in Collections:Mathematics and Statistics Research Collection

Show full item record

Citations 50

Last Week
Last month
checked on Sep 12, 2020

Page view(s)

Last Week
Last month
checked on Dec 1, 2022

Download(s) 50

checked on Dec 1, 2022

Google ScholarTM



If you are a publisher or author and have copyright concerns for any item, please email and the item will be withdrawn immediately. The author or person responsible for depositing the article will be contacted within one business day.