PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systems

Files in This Item:
File Description SizeFormat 
portillo_article(PHOEBE_pre)_2017.pdf1.92 MBAdobe PDFDownload
Title: PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systems
Authors: Portillo Dominguez, Andres Omar
Perry, Philip
Magoni, Damien
Murphy, John
Permanent link:
Date: Nov-2017
Abstract: The identification of performance issues and the diagnosis of their root causes are time-consuming and complex tasks, especially in clustered environments. To simplify these tasks, researchers have been developing tools with built-in expertise for practitioners. However, various limitations exist in these tools that prevent their efficient usage in the performance testing of clusters (e.g. the need of manually analysing huge volumes of distributed results). In a previous work, we introduced a policy-based adaptive framework (PHOEBE) that automates the usage of diagnosis tools in the performance testing of clustered systems, in order to improve a tester's productivity, by decreasing the effort and expertise needed to effectively use such tools. This paper extends that work by broadening the set of policies available in PHOEBE, as well as by performing a comprehensive assessment of PHOEBE in terms of its benefits, costs and generality (with respect to the used diagnosis tool). The performed evaluation involved a set of experiments in assessing the different trade-offs commonly experienced by a tester when using a performance diagnosis tool, as well as the time savings that PHOEBE can bring to the performance testing and analysis processes. Our results have shown that PHOEBE can drastically reduce the effort required by a tester to do performance testing and analysis in a cluster. PHOEBE also exhibited consistent behaviour (i.e. similar time-savings and resource utilisations), when applied to a set of commonly used diagnosis tools, demonstrating its generality. Finally, PHOEBE proved to be capable of simplifying the configuration of a diagnosis tool. This was achieved by addressing the identified trade-offs without the need for manual intervention from the tester.
Funding Details: Science Foundation Ireland
Type of material: Journal Article
Publisher: Wiley
Journal: Journal of Software: Practice and Experience, 47, 1837–1874
Volume: 47
Issue: 11
Start page: 1837
End page: 1874
Keywords: Performance testingPerformance analysisCluster computingSystem performance
DOI: 10.1002/spe.2500
Language: en
Status of Item: Not peer reviewed
Appears in Collections:Computer Science Research Collection
PEL Research Collection

Show full item record

Citations 50

Last Week
Last month
checked on Oct 12, 2018

Google ScholarTM



This item is available under the Attribution-NonCommercial-NoDerivs 3.0 Ireland. No item may be reproduced for commercial purposes. For other possible restrictions on use please refer to the publisher's URL where this is made available, or to notes contained in the item itself. Other terms may apply.