In this section, we present how the results from RQ1 and RQ2 have been articulated into guidelines to support organizations building a business case and gathering evidence of RAs. The guidelines have been incrementally constructed based on our action research approach with everis. For its shaping and validation, the construction of the guidelines is respectively divided in two stages: the formative and the summative stages.
5.1 Formative stage: shaping the guidelines
In the formative stage, the goal is to incrementally shape the guidelines from the feedback of the action research. In order to analyze whether other organizations deal with similar problems as everis, we highlight the similarities of RAs designed by everis with other RA contexts that have been reported in the literature and by practitioners. After this analysis, we will only package into the guidelines the material that could be used under other organizations context.
5.1.1 Similar contexts of RAs in practice
The results from our action research in everis are particular to the context described in Section 2.2. IT consulting firms, such as Accenture (Brand [2014]) and Capgemini (Herold and Mair [2013]) also fit into such context (i.e., they use industry-specific RM, and they carry out the three types of projects described there). However, to properly create the guidelines for other RA contexts, it is vital to first characterize RA projects conducted by other companies.
Architecture-centric approaches to develop families of software applications are not new. Deelstra et al. give a classification of these approaches with respect to the level of reuse (Deelstra et al. [2005]): 1) standardized infrastructure; 2) platform: 3) software product line, and 4) configurable product family. In this classification, RAs can be positioned as standardized infrastructures or platforms (Graaf et al. [2005]), whereas software product lines and configurable product families are based on Product Lines Architectures (PLA). Several authors has stated that RAs are more generic than PLAs (Martínez-Fernández et al. [2013b]). Next, we classify RAs in the industry under the two former categories.
On the one hand, standardized infrastructures have been used by public administrations in Germany (Herold and Mair [2013]), in the Netherlands (Galster et al. [2013]), and in Spain (García-Alonso et al. [2010]). These RAs provide software assets as inspiration for the design of applications, but little domain engineering effort is performed (i.e., little domain specific functionality is included in the RA). They are popular in public administration because there is a need to cover multiple organizations from different business domains (i.e., ministries or departments of the government) and little common functionality exist. Also, the high distribution of development teams implies that these RAs play only an informative or instructive role rather than regulative.
On the other hand, platforms additionally “require a certain amount of domain engineering to create and maintain the assets that implement the common functionality” (Deelstra et al. [2005]). There are several business domains that have used this type of RAs.
In the space domain, the NASA has detected that “many Earth science data system components and architectural patterns are reconstructed for each mission” (Mattmann and Downs [2010]). In order to reuse these assets in new systems, they created the NASA’s ESDS RA (NASA [2012]).
In the banking domain, RAs are usually used to integrate legacy systems and migrated service-oriented systems that contain the business logic. The common scenario is that these RAs provide common services that then may be reused in the different front-ends or channels (e.g., desktop applications, web client applications, mobile applications, ATMs and so on). An example is Credit Suisse (Murer and Hagen [2013]).
The most mature domain for RAs may be the embedded systems domain. For instance, Océ, one of the world’s leading copier manufacturers, uses an RA to derive an SA for engines incorporated in a specific series of Océ printers (Graaf et al. [2005]). Besides, in the automotive industry, car manufacturers such as Volvo (Eklund et al. [2005]) and BMW (Reichart and Haneberg [2004]) started to use RAs to develop the software of electronic/engine control unit based on basic software components that were unique to them. As a further step, AUTOSAR has become popular later because it standardizes basic software components for many car manufacturers, suppliers and other related companies (AUTOSAR [2014]). This enables the reuse of software developed by original equipment manufacturers in multiple car manufacturers. This has led to software ecosystems that are characterized by a network of developers rather than a single organization providing the final product to the customer (Eklund and Bosch [2014]).
To sum up, we can conclude that core idea of using an RA for the development and maintenance of a family of software products is common in all these contexts. However, RM are only commonly used in IT consulting firms. As a consequence, the study of RM during the application of the guidelines will be optional so that it can be used in these other contexts.
It is also important to note that all RAs described in this section are based on practical experience in the industry. The guidelines will target this type of RA, also known as classical (Angelov et al. [2012]). Conversely, the guidelines would be hardly applicable in preliminary RAs, i.e., those that are “defined when the technology, software solutions, or algorithms demanded for its application do not yet exist in practice by the time of its design” (Angelov et al. [2012]).
5.1.2 Packaging the guidelines
everis’ results are being suitably packaged with the aim of being applied in prospective RA projects and also in similar organizations. Figure 6 summarizes the guidelines for building a business case and gathering evidence of RAs in industry.
First of all, organizations that may want to use these guidelines need to fit into the context depicted in Section 5.1.1. This means that they need to design an RA based on practical experience, and to use such RA for the development and maintenance of a family of applications in industry. This is because the input for using the guidelines is evidence from real RA projects.
The guidelines support organizations:
-
1.
To decide whether to invest on an RA by providing them with:
-
2.
To improve RA-related practices based on corporate evidence by providing them with:
On the one hand, to analyze whether it is worth to invest on an RA, a checklist of value-driven data that an organization might have is facilitated to check if the provided economic model provided can be executed. On the other hand, to improve RA-related practices with evidence, a set of relevant aspects for RAs is facilitated to check which ones are important for an organization and then use the provided template surveys to gather evidence.
To gather evidence with the help of the aforementioned artifacts, empirical studies can be set up and carried out. Several data collection techniques exist (Lethbridge et al. [2005]). We propose surveys that use interviews and questionnaires, case studies that require documentation analysis and metrics gathering, and focus groups. Below, we explain the dimension, context, objective and method applied in each empirical study of the guidelines. To see the threats to validity in everis, the reader is referred to (Martínez-Fernández [2013]).
I) A survey to check existing value-driven data in RA projects
Dimensions
RQ 1.
Context
Typically, organizations do not have resources to compare the real cost of creating applications with and without an RA and historical data may be scarce. Thus, alternatives should be considered.
Objective
The objective of this survey is to identify the quantitative information that can commonly be retrieved in RA projects in order to quantitatively calculate the costs and benefits of adopting an RA in an organization. This is an initial step to create repeatable techniques for performing a cost-benefit analysis.
Method
Exploratory surveys with personalized questionnaires applied to relevant stakeholders (e.g., manager, architect, developer) to find out the quantitative data that has been collected in RA projects and SA projects. An example of conducting this empirical study and its approach for data collection is described in Section 3.1.
II) A case study to apply REARM to calculate the ROI of adopting an RA
Dimensions
RQ 1.
Context
Before deciding to launch an RA, organizations need to analyze whether to undertake or not the investment. Offering organizations an economic model that is based on former RA projects data can help them to make more informed decisions.
Objective
The objective is to analyze whether it is worth investing on an RA with the help of an economic model, in order to improve the communication among architects and management, and to improve their decisions.
Method
A case study that applies an economic model to calculate the ROI of adopting an RA. Depending on the maturity of the organization, two approaches can be applied. If the organization does not have experience with an RA, the economic model should be fed with estimated data. Nevertheless, when the organization already has experience with RAs (i.e., the case of IT consulting firms), real data can be gathered by means of an exploratory quantitative post-mortem analysis. Then, the economic model quantifies the potential advantages and limitations of using an RA. Some related works explain how to calculate the ROI of a product (Forrester [2014]), software reuse (Frakes and Terry [1996]; Poulin [1997]), and software product lines (Ali et al. [2009]). We suggest the use of REARM, which is an economic model specific for RAs presented in (Martínez-Fernández et al. ([Martínez-Fernández et al. 2013b])). An example of conducting this empirical study and its approach for data collection is described in Section 3.2.
III) A focus group to study the relevant criteria of RAs for an organization
Dimensions
RQ 2.
Context
Typically, organizations drive the design and use of RAs in an unsystematic manner (Eklund et al. [2005]). To drive RA-related practices based on evidence, it becomes fundamental to identify the relevant aspects of RAs as seen by practitioners.
Objective
The objective of this study is to identify the aspects that are important for each organization in order to support RA-related practices.
Method
A focus group with relevant stakeholders (e.g., manager, architect, developer) to find out which aspects of RAs are important to them. A focus group is considered a proven and tested technique to obtain the perception of a group of selected people on a defined area of interest (Lethbridge et al. [2005]). An example of conducting this empirical study and its approach for data collection is described in Section 4.1.
IV) Surveys to gather evidence to improve RA-related practices
Dimensions
RQ 2.
Context
To reuse architectural knowledge and improve RA-related practices in prospective RA projects, organizations need to understand RA’s characteristics, as well as its potential benefits and limitations. Gathering evidence from previous RA projects is a feasible way to start gaining such an understanding.
Objective
The purpose of the survey is to understand the impact of using RAs for designing the SAs of the applications of an information system of a client organization. This is a descriptive survey that measures what occurred while using RAs rather than why. The following questions are important for organizations in order to understand relevant Aspects 1 to 5 of RAs (defined in Table 2):
-
1.
Why is an RA adapted for creating SAs of the organizations’ applications? What type of RA is being designed and used in the organization?
-
2.
What is the state of practice in requirement engineering for RA projects in the organization?
-
3.
What is the state of practice in architectural design for RA projects in the organization?
-
4.
Which tools and technologies are currently being used in RAs projects by the organization?
-
5.
How does the adoption of RAs provide observable benefits to the different actors involved in RA projects in the organization?
Method
Exploratory surveys with personalized questionnaires applied to relevant stakeholders (e.g., architects, developers) to gather their perceptions and needs. An example of conducting this empirical study and its approach for data collection is described in Section 4.2.
Finally, the output for using guidelines for RQ 1 is a business case that evaluates whether it is worth or not to invest on an RA. On the other hand, the output of gathering evidence about Aspects 1 to 5 of Table 2 is a corporate knowledge base about these aspects. It is important to note that, although it is not strictly necessary, it is recommended to gather Aspects 1 to 5 for building the business case. In other words, guidelines are complementary and support each other (e.g., results from a preceding study can be used to corroborate or further develop other results). For instance, in our industry-academia collaboration, the qualitative results about the benefits and drawbacks of RAs supported the unquantifiable benefits, and uncertainties and risk of the business case.
5.2 Summative stage: validating the guidelines
The summative stage will take place once the guidelines will have been adequately shaped and improved. The primary role of this stage will be to validate the version of the guidelines after applying it in everis with more practitioners. This evaluation consists of the use of the guidelines in other organizations to design and conduct empirical studies. Organizations analyzing whether to make the strategic move to RA adoption and organizations that face the design and use of RAs based on evidence will benefit from these guidelines.