A survey of architecture design rationale - Antony Tang, 2006.pdf

of 13
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  A survey of architecture design rationale Antony Tang  a,* , Muhammad Ali Babar  b , Ian Gorton  b , Jun Han  a a Faculty of ICT, Swinburne University of Technology, John Street, Hawthorn, Melbourne, Vic. 3122, Australia b National ICT Australia Ltd. and University of NSW, Australia Received 17 January 2006; received in revised form 25 April 2006; accepted 26 April 2006Available online 15 June 2006 Abstract Many claims have been made about the consequences of not documenting design rationale. The general perception is that designersand architects usually do not fully understand the critical role of systematic use and capture of design rationale. However, there is to datelittle empirical evidence available on what design rationale mean to practitioners, how valuable they consider it, and how they use anddocument it during the design process. This paper reports a survey of practitioners to probe their perception of the value of design ratio-nale and how they use and document the background knowledge related to their design decisions. Based on 81 valid responses, this studyhas discovered that practitioners recognize the importance of documenting design rationale and frequently use them to reason abouttheir design choices. However, they have indicated barriers to the use and documentation of design rationale. Based on the findings,we conclude that further research is needed to develop methodology and tool support for design rationale capture and usage. Further-more, we put forward some specific research questions about design rationale that could be further investigated to benefit industrypractice.   2006 Elsevier Inc. All rights reserved. Keywords:  Design rationale; Software architecture; Survey 1. Introduction Design rationale captures the knowledge and reasoningthatjustifytheresultingdesign.Thisincludeshowthedesignsatisfies functional and quality requirements, why certaindesign choices are selected over alternatives and what typeofsystembehaviourisexpectedunderdifferentenvironmen-tal conditions (Gruber and Russell, 1991; Lee, 1997).Despite the growing recognition of the need for document-ing and using architecture design rationale by researchersand practitioners (Bass et al., 2003; Bosch, 2004; Curtiset al., 1988), there is a lack of appropriate support mecha-nisms and guidelines on what are the essential elements of design rationale, and how to document and reason withdesign rationale in making architecture design decisions.The recently adopted IEEE standards (1471-2000) fordescribing architecture (IEEE, 2000) and the architec-ture documentation methods like Views and Beyond(V&B) (Clements et al., 2002) raise the awareness of andprovide some guidance on documenting design rationale;however, for the reasons discussed in Section 2, each hasits limitations.This article describes our initial investigations into theuse and documentation of design rationale. In the longterm, we aim to develop a conceptual framework and asso-ciated tools to facilitate the capture and use of design ratio-nale. We believe that understanding the current industrypractice of design rationale is one of the most importantsteps towards that goal. However, there is little empiricalresearch that studies what practitioners think about designrationale, how they document and reason with designrationale, and what factors prevent them from document-ing design rationale. 0164-1212/$ - see front matter    2006 Elsevier Inc. All rights reserved.doi:10.1016/j.jss.2006.04.029 * Corresponding author. Tel.: +61 3 92145198; fax: +61 3 98190823. E-mail addresses: (A. Tang), (M.A. Babar), (I. Gorton), jhan@ict. (J. Han). The Journal of Systems and Software 79 (2006) 1792–1804  The need to improve the capture and the use of designrationale in system design and maintenance has beenreported by several researchers (Bosch, 2004; Tyree andAkerman, 2005; Burge, 2005). They allude to a perceptionthat architects generally do not realize the critical role of explicitly documenting the contextual knowledge abouttheir design decisions. Lack of empirical evidence makesit difficult to support or refute these claims. Hence we setout to gather evidence from those who design architectureson a regular basis, in order to examine the attitudes of practitioners who have the most impact on the immediateand future use of design rationale approaches.This article reports the findings of a survey of practitio-ners who have had experience in architecture design in theAsia Pacific region. The findings of this survey shed lighton how design rationale are used, documented, and per-ceived by designers and architects working in this region.The objectives of the study are: ã  To understand the architects’ perceptions about archi-tecture design rationale and the importance of the differ-ent elements of design rationale (such as designconstraints, design strengths and weaknesses). ã  To determine the frequency of documenting and reason-ing with different elements of design rationale, the mainreasons for not documenting design rationale, and thecommon methods, techniques, and tools used to docu-ment design rationale. ã  To identify the potential challenges and opportunitiesfor improving the use and documentation of designrationale in practice.During this study, we have encountered several interest-ing findings which enabled us to identify a set of researchquestions for further investigations. Since a theory explain-ing the attitude and behaviour toward the use of designrationale does not exist, this study employs an inductiveapproach (i.e., using facts to develop general conclusions)as an attempt to move toward such a theory.The article makes three significant contributions to theSoftware Architecture (SA) discipline: ã  It presents the design and results of the first survey-basedempiricalstudy inarchitecturedesign rationalepractices. ã  It provides information about how practitioners thinkabout, reason with, document and use design rationale. ã  It identifies the problems and contradictions of currentdesign rationale practices. As a result, we propose aresearch agenda that aims to explore and enhance cur-rent architecture design rationale practices.We discuss the current approaches to using design ratio-nale in Section 2. We present our research approach inSection 3. Section 4 presents the results of the survey. A discussion of our findings and their limitations are in Sec-tions 5 and 6 respectively. We identify areas for futurework and make some concluding remarks in Section 7. 2. Background and related work  2.1. Design rationale approaches in software engineering  Early work emphasizing the importance of design ratio-naleinsoftwaredesigncanbefoundinParnasandClements(1985) and Potts and Burns (1988). Since then, the softwareengineering community has experimented with severaldesign rationale approaches such as Issue Based Informa-tion Systems (IBIS) (Kunz and Rittel, 1970), Questions,Options, and Criteria (QOC) (Maclean et al., 1996), Proce-dural Hierarchy ofIssues (PHI)(McCall,1987), andDesignRationale Language (DRL) (Lee and Lai, 1996). Most of these methods have been adopted or modified to capturethe rationale for software design decisions (Potts andBurns, 1988) and requirements specifications (Dutoit andPaech, 2002; Lee, 1991; Ramesh and Dhar, 1992). Otherapproaches (e.g. Potts, 1999; Potts et al., 1994) combinerationale and scenarios to elicit and refine requirements.While there are claims of several benefits of using these tocapture design rationale, it is not clear how much or howfar these techniques have been adopted by practitioners.Design rationale havebeen considered animportant partof software architecture since (Perry and Wolf, 1992) laidthe foundation for the evolving community of softwarearchitecture. In the following years, researchers haveemphasized the need for documenting design rationale tomaintainandevolvearchitecturalartifactsandtoavoidvio-lating design rules that underpin the srcinal architecture(Bass et al., 2003; Bosch, 2004). The growing recognitionof the vital role of documenting and maintaining rationalefor architectural decisions has resulted in several efforts toprovide guidance for capturing and using design rationalesuch as the IEEE 1471-2000 standard (IEEE, 2000) andthe Views and Beyond (V&B) approach to document soft-ware architecture (Clements et al., 2002).However, both of these are deficient in several ways. Forexample, the former provides a definition of design ratio-nale without further elaborating on their nature and howthey might be captured. The latter method provides a listof design rationales without justifying why they are impor-tant and how the information captured is beneficial in dif-ferent design context. Moreover, it is unclear if the list of design rationales are complete.Different approaches tend to characterize design ratio-nale with different information. For example, Tyree andAkerman (2005) provides a template that captures certaintypes of information as design rationale; the V&B (Cle-ments et al., 2002) approach considers some other typesof information (e.g. information cross-cutting differentviews) as design rationale; and the Architecture Rationali-zation Method (ARM) uses qualitative and quantitativerationale in design reasoning (Tang and Han, 2005). Thus,there is clearly a need for a common vocabulary or stan-dard guidance so that practitioners understand the issuesin reasoning with and consistently documenting designrationale. A. Tang et al. / The Journal of Systems and Software 79 (2006) 1792–1804  1793   2.2. Generic design rationale According to the Cambridge dictionary, a rationale is areason or intention for a particular set of thoughts oractions. When architects and designers make designdecisions based on their reasoning, what do they con-sider as a reason or an intention? Is a requirement or aconstraint an intention or reason enough for a designchoice? Or is it some generic justification that allowsdesigners to judge that a design choice is better than itsalternatives?The nature of design rationale is different between differ-entdesignactivities.Thearchitecturedecisionprocessisdif-ferenttothedetaileddesign decisionprocess.Bratthalletal.(2000) suggested that architectural level design addressesdecisions with system-wide implications. Lassing et al.(2001) suggested that architecture decisions have a largeinfluence on the quality of the resulting system. Eden andKazman (2003) suggested that what separate architecturedesign activities from other design activities are: (a) archi-tecture is concerned with the selection of architecture ele-ments, their interactions and the constraints where designis concerned with the modularization and detailed inter-faces of design element; (b) architecture is concerned withissues beyond algorithms and data structures of the compu-tations; and (c) architecture focuses on the externally visibleproperties of the software components. Since architecturedesign is at a higher level of abstraction and considers anarray of different inputs, the complexity of the design rea-soning is generally higher than detailed design of a localizedsoftware component. Therefore, design rationale for thetwolevelsofdesignwouldbequitedifferent.Giventhecom-plexity of architecture design, the types of design rationaleand the way they influence architecture decisions are notverywellunderstood.Hencewearemotivatedtoinvestigateinto this area.In this survey, we used nine types of generic design ratio-nales selected from various references to test if and how ourrespondents perceive and use them. This set of genericrationale characterizes different aspects in which reasonscan be portrayed and compared. Their selection is basedon the templates or methods proposed by researchers tocapture design rationale (Clements et al., 2002; Tyree andAkerman, 2005; Bass et al., 2003; Tang and Han, 2005).A generic design rationale is an abstract grouping of rea-sons for justifying decisions that are made in the designprocess. We used common terminologies so that practitio-ners could relate to them. Since this is an exploratorystudy, the list of generic design rationales (below) is com-prehensive but not exhaustive.(1)  Design constraints  are the limitations placed on thedesign. They can be business or technical in nature(Clements et al., 2002).(2)  Design assumptions  are used to describe what areotherwise unknown factors that affect the design(Tyree and Akerman, 2005).(3)  Weakness  of a design describes what the design can-not achieve, which may be functional or technical innature (Bass et al., 2003).(4)  Benefit  of a design describes what benefits the designcan deliver to satisfy the technical or functionalrequirements (Tang and Han, 2005).(5)  Cost ofadesigndescribestheexplicitandimplicitcoststo the system and the business (Tang and Han, 2005).(6)  Complexity  of a design is a relative measure of thecomplexity of the design in terms of implementationand maintenance (Tang and Han, 2005).(7)  Certainty of design , i.e. the design would work, is ameasurement of risk that the design would meet itsrequirements (Tang and Han, 2005).(8)  Certainty of implementation , i.e. the design is imple-mentable, is a measurement of risk that the develop-ment team has the skill and the resources, in terms of schedule and cost, to implement the design (Tang andHan, 2005).(9)  Tradeoffs  between alternative designs is a mechanismto weigh and compare alternatives given each alterna-tive design has its supporting design rationale andpriorities (Bass et al., 2003).Although the above list of design rationales have beensuggested by various researchers and common sense tellsus that they are useful, no empirical studies have been car-ried out to prove that they are actually used in the softwareindustry. In this survey, we asked our respondents to rankthese rationales according to their usefulness and howoften they are being used and documented. The resultsare reported in Section 4. 3. Research approach 3.1. Research method  Considering the objectives of our research and availableresources, we decided to use the survey research method tounderstand architects’ perception of, and current practicesin architecture design rationale. A survey research methodis considered suitable for gathering self-reported quantita-tive and qualitative data from a large number of respon-dents (Kitchenham and Pfleeger, 2001–2002). Our surveydesign was a cross-sectional, case control study. Surveyresearch can use one or a combination of several data gath-ering techniques such as interviews, self-administered ques-tionnaires and others (Lethbridge, 2005). We decided touse a questionnaire as a data collection instrument becausewe wanted to obtain the information from a relatively largenumber of practitioners, many of whom we would not beable to contact personally. 3.2. Survey instrument construction Having reviewed the published literatureon design ratio-nale, we developed a survey instrument consisting of 30 1794  A. Tang et al. / The Journal of Systems and Software 79 (2006) 1792–1804  questionsontheunderstandingandpracticeofdesignratio-nale and 10 questions on demographics (e.g. age, experi-ence, gender, education and others) of the respondents.Some of the demographic questions were designed forscreening the respondents and identifying the datasets tobe excluded from the final analysis. The questions wereplaced in different sections, namely design rationale impor-tance, design rationale documentation, design rationaleusage, and demographic information. Questions on demo-graphics were put in the last section as it is considered goodpractice (Kitchenham and Pfleeger, 2001–2002). In addi-tion, a coversheet explaining the purpose of the study anddefining various terms was attached to the questionnaire. 3.3. Instrument evaluation The questionnaire underwent a rigorous review processfor the format of the questions, suitability of the scales,understandability of the wording, number of questions,and length of time required to complete by experiencedresearchers and practitioners in software architecturedomain. We ran a formal pilot study to further test andrefine the survey instrument. The pilot study was con-ducted with eight people who were considered strongly rep-resentative of the potential participants of our surveyresearch (i.e. practitioners with more than 3 years softwaredesign experience). Data from the pilot study was notincluded in the analysis of the main survey. The feedbackfrom the pilot study helped us refine the questionnaire,which was submitted for ethical committee approval. 3.4. Instrument deployment We decided to use an online web-based survey, as this isusually less expensive and more efficient in data collection(SimsekandVeiga,2001).Inordertoimplementthesurvey,we used an online web-based tool Surveyor (ObjectPlanetInc., 2002). Participants accessed the survey through aURL. 3.5. Target population The inclusion criteria was a software engineer with threeor more years of experience in software development andwho has worked or is working in a design or architect role.We did not have a formal justification for the amount of experience required of valid respondents. We based this onour extensive experience in designing architectures for largesystems that made us believe that people with three or moreyears of experience in software development would be ableto give reliable answers to the type of questions we had. 3.6. Sampling technique The study needed responses from likely time-constrainedsoftware engineers, who we expected were less likely torespond to an invitation from unfamiliar sources. Thismade it hard to apply random sampling. Consequently,we decided to use non-probabilistic sampling techniques,availability sampling and snowball sampling. Availabilitysampling operates by seeking responses from those peoplewhomeet theinclusioncriteriaandareavailable andwillingto participate in the research. Snowballing requires askingthe participants of the study to nominate other peoplewho would be willing to participate. The major drawbackof non-probabilistic sampling techniques is that the resultscannot be considered statistically generalizable to the targetpopulation (Kitchenham and Pfleeger, 2001–2002), in thiscase software designers/architects. However, consider-ing the exploratory nature of our research, we believe thatour sampling techniques were reasonable. 3.7. Invitation mechanics We used two means of contacting potential respondents:personalized contact and professional referrals. The invita-tion letters were sent to a pool of software designers/archi-tects drawn from the industry contacts of the fourinvestigators and past and current students of the post-graduate information technology courses offered by theSwinburne University of Technology and the Universityof New South Wales. We requested the invitees to forwardthe invitation to others who were eligible for participationand provide us the contact for the forwarded invitation. 3.8. Data validation For access control and data validation purposes, thesurvey URL was sent via email. Moreover, the responsesgathered in the survey provided another mechanism of checking the validity of the respondents as genuine soft-ware engineering practitioners. For example, only one of the 81 respondents did not provide a job title and all otherrespondents had relevant job titles. A large number of respondents (55%) provided quite insightful and detailedcomments to several open-ended optional questions. 4. Survey findings The survey questionnaire was divided into seven mainparts. The perception of the importance of design ratio-nale, the use of design rationale and the documentationof design rationale are discussed and analyzed in this articletogether with the profile of the respondents. Architectureevaluation in organizations, architecture enhancementsand risk undertakings in architecture design are the otherthree parts which will be reported separately. Readerswho are interested in the statistics and the questionnaireare referred to Tang et al. (2005). 4.1. Demographic data We directly sent survey invitations to 171 practitioners.Our invitation was forwarded to 376 more people by the A. Tang et al. / The Journal of Systems and Software 79 (2006) 1792–1804  1795
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!