Cover Image

Addressing high dimensionality and lack of feature models in testing of software product lines

Software Product Lines (SPLs) allow engineers to systematically build families of software products, defined by a unique combination of features—increments in functionality, improving both the efficiency of the software development process and the quality of the software developed. However, testi...

Full description

Main Author: SOUTO, Sabrina de Figueirêdo
Other Authors: D'AMORIM, Marcelo Bezerra
Format: doctoralThesis
Language: por
Published: Universidade Federal de Pernambuco 2016
Subjects:
Linhas de Produtos de Software
Sistemas Configuráveis
Teste de Software e Depuração
Feature Model
Software Product Lines
Configurable Systems
Software Testing and Debugging
Online Access: https://repositorio.ufpe.br/handle/123456789/15975
Tags: Add Tag
No Tags, Be the first to tag this record!
id ir-123456789-15975
recordtype dspace
spelling ir-123456789-159752019-10-26T01:27:19Z Addressing high dimensionality and lack of feature models in testing of software product lines SOUTO, Sabrina de Figueirêdo D'AMORIM, Marcelo Bezerra http://lattes.cnpq.br/3762670242328435 Linhas de Produtos de Software Sistemas Configuráveis Teste de Software e Depuração Feature Model Software Product Lines Configurable Systems Software Testing and Debugging Feature Model Software Product Lines (SPLs) allow engineers to systematically build families of software products, defined by a unique combination of features—increments in functionality, improving both the efficiency of the software development process and the quality of the software developed. However, testing these kinds of systems is challenging, as it may require running each test against a combinatorial number of products. We call this problem the High Dimensionality Problem. Another obstacle to product line testing is the absence of Feature Models (FMs), making it difficult to discover the real causes for test failures. We call this problem the Lack of Feature Model Problem. The High Dimensionality Problem is associated to the large space of possible configurations that an SPL can reach. If an SPL has n boolean features, for example, there are 2n possible feature combinations. Therefore, systematically testing this kind of system may require running each test against all those combinations, in the worst case. The Lack of Feature Model Problem is related to the absence of feature models. The FM enables accurate categorization of failing tests as failures of programs or the tests themselves, not as failures due to inconsistent combinations of features. For this reason, the lack of FM presents a huge challenge to discover the true causes for test failures. Aiming to solve these problems, we propose two lightweight techniques: SPLat and SPLif. SPLat is a new approach to dynamically prune irrelevant configurations: the configurations to run for a test can be determined during test execution by monitoring accesses to configuration variables. As a result, SPLat reduces the number of configurations. Consequently, SPLat is lightweight compared to prior works that used static analysis and heavyweight dynamic execution. SPLif is a technique for testing SPLs that does not require a priori availability of feature models. Our insight is to use a profile of passing and failing test runs to quickly identify test failures that are indicative of a problem (in test or code) as opposed to a manifestation of execution against an inconsistent combination of features. Experimental results show that SPLat effectively identifies relevant configurations with a low overhead. We also apply SPLat on two large configurable systems (Groupon and GCC), and it scaled without much engineering effort. Experimental results demonstrate that SPLif is useful and effective to quickly find tests that fail on consistent configurations, regardless of how complete the FMs are. Furthermore, we evaluated SPLif on one large extensively tested configurable system, GCC, where it helped to reveal 5 new bugs, 3 of which have been fixed after our bug reports. Software Product Lines (SPLs) allow engineers to systematically build families of software products, defined by a unique combination of features—increments in functionality, improving both the efficiency of the software development process and the quality of the software developed. However, testing these kinds of systems is challenging, as it may require running each test against a combinatorial number of products. We call this problem the High Dimensionality Problem. Another obstacle to product line testing is the absence of Feature Models (FMs), making it difficult to discover the real causes for test failures. We call this problem the Lack of Feature Model Problem. The High Dimensionality Problem is associated to the large space of possible configurations that an SPL can reach. If an SPL has n boolean features, for example, there are 2n possible feature combinations. Therefore, systematically testing this kind of system may require running each test against all those combinations, in the worst case. The Lack of Feature Model Problem is related to the absence of feature models. The FM enables accurate categorization of failing tests as failures of programs or the tests themselves, not as failures due to inconsistent combinations of features. For this reason, the lack of FM presents a huge challenge to discover the true causes for test failures. Aiming to solve these problems, we propose two lightweight techniques: SPLat and SPLif. SPLat is a new approach to dynamically prune irrelevant configurations: the configurations to run for a test can be determined during test execution by monitoring accesses to configuration variables. As a result, SPLat reduces the number of configurations. Consequently, SPLat is lightweight compared to prior works that used static analysis and heavyweight dynamic execution. SPLif is a technique for testing SPLs that does not require a priori availability of feature models. Our insight is to use a profile of passing and failing test runs to quickly identify test failures that are indicative of a problem (in test or code) as opposed to a manifestation of execution against an inconsistent combination of features. Experimental results show that SPLat effectively identifies relevant configurations with a low overhead. We also apply SPLat on two large configurable systems (Groupon and GCC), and it scaled without much engineering effort. Experimental results demonstrate that SPLif is useful and effective to quickly find tests that fail on consistent configurations, regardless of how complete the FMs are. Furthermore, we evaluated SPLif on one large extensively tested configurable system, GCC, where it helped to reveal 5 new bugs, 3 of which have been fixed after our bug reports. 2016-03-15T15:21:11Z 2016-03-15T15:21:11Z 2015-03-31 doctoralThesis https://repositorio.ufpe.br/handle/123456789/15975 por Attribution-NonCommercial-NoDerivs 3.0 Brazil http://creativecommons.org/licenses/by-nc-nd/3.0/br/ application/pdf Universidade Federal de Pernambuco UFPE Brasil Programa de Pos Graduacao em Ciencia da Computacao
institution REPOSITORIO UFPE
collection REPOSITORIO UFPE
language por
topic Linhas de Produtos de Software
Sistemas Configuráveis
Teste de Software e Depuração
Feature Model
Software Product Lines
Configurable Systems
Software Testing and Debugging
Feature Model
spellingShingle Linhas de Produtos de Software
Sistemas Configuráveis
Teste de Software e Depuração
Feature Model
Software Product Lines
Configurable Systems
Software Testing and Debugging
Feature Model
SOUTO, Sabrina de Figueirêdo
Addressing high dimensionality and lack of feature models in testing of software product lines
description Software Product Lines (SPLs) allow engineers to systematically build families of software products, defined by a unique combination of features—increments in functionality, improving both the efficiency of the software development process and the quality of the software developed. However, testing these kinds of systems is challenging, as it may require running each test against a combinatorial number of products. We call this problem the High Dimensionality Problem. Another obstacle to product line testing is the absence of Feature Models (FMs), making it difficult to discover the real causes for test failures. We call this problem the Lack of Feature Model Problem. The High Dimensionality Problem is associated to the large space of possible configurations that an SPL can reach. If an SPL has n boolean features, for example, there are 2n possible feature combinations. Therefore, systematically testing this kind of system may require running each test against all those combinations, in the worst case. The Lack of Feature Model Problem is related to the absence of feature models. The FM enables accurate categorization of failing tests as failures of programs or the tests themselves, not as failures due to inconsistent combinations of features. For this reason, the lack of FM presents a huge challenge to discover the true causes for test failures. Aiming to solve these problems, we propose two lightweight techniques: SPLat and SPLif. SPLat is a new approach to dynamically prune irrelevant configurations: the configurations to run for a test can be determined during test execution by monitoring accesses to configuration variables. As a result, SPLat reduces the number of configurations. Consequently, SPLat is lightweight compared to prior works that used static analysis and heavyweight dynamic execution. SPLif is a technique for testing SPLs that does not require a priori availability of feature models. Our insight is to use a profile of passing and failing test runs to quickly identify test failures that are indicative of a problem (in test or code) as opposed to a manifestation of execution against an inconsistent combination of features. Experimental results show that SPLat effectively identifies relevant configurations with a low overhead. We also apply SPLat on two large configurable systems (Groupon and GCC), and it scaled without much engineering effort. Experimental results demonstrate that SPLif is useful and effective to quickly find tests that fail on consistent configurations, regardless of how complete the FMs are. Furthermore, we evaluated SPLif on one large extensively tested configurable system, GCC, where it helped to reveal 5 new bugs, 3 of which have been fixed after our bug reports.
author2 D'AMORIM, Marcelo Bezerra
format doctoralThesis
author SOUTO, Sabrina de Figueirêdo
author_sort SOUTO, Sabrina de Figueirêdo
title Addressing high dimensionality and lack of feature models in testing of software product lines
title_short Addressing high dimensionality and lack of feature models in testing of software product lines
title_full Addressing high dimensionality and lack of feature models in testing of software product lines
title_fullStr Addressing high dimensionality and lack of feature models in testing of software product lines
title_full_unstemmed Addressing high dimensionality and lack of feature models in testing of software product lines
title_sort addressing high dimensionality and lack of feature models in testing of software product lines
publisher Universidade Federal de Pernambuco
publishDate 2016
url https://repositorio.ufpe.br/handle/123456789/15975
_version_ 1648655390342119424
score 13.657419

Similar Items