Taxes & Accounting

Testing Database-Driven Applications: Challenges and Solutions

Description
Testing Database-Driven Applications: Challenges and Solutions Gregory M. Kapfhammer Department of Computer Science University of Pittsburgh Department of Computer Science Allegheny College Mary Lou Soffa
Published
of 37
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
Testing Database-Driven Applications: Challenges and Solutions Gregory M. Kapfhammer Department of Computer Science University of Pittsburgh Department of Computer Science Allegheny College Mary Lou Soffa Department of Computer Science University of Pittsburgh Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 1/32 Outline Introduction and Motivation Testing Challenges Database-Driven Applications A Unified Representation Test Adequacy Criteria Test Suite Execution Test Coverage Monitoring Conclusions and Resources Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 2/32 Motivation The Risks Digest, Volume 22, Issue 64, 23 Jeppesen reports airspace boundary problems About 35 airspace boundaries contained in Jeppesen NavData are incorrect, the FAA has warned. The error occurred at Jeppesen after a software upgrade when information was pulled from a database containing 2, airspace boundaries worldwide for the March NavData update, which takes effect March 2. Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 3/32 Testing Challenges Should consider the environment in which software applications execute Must test a program and its interaction with a database Database-driven application s state space is well-structured, but infinite (Chays et al.) Need to show program does not violate database integrity, where integrity = consistency + validity (Motro) Must locate program and database coupling points that vary in granularity Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 4/32 Testing Challenges The structured query language s (SQL) data manipulation language (DML) and data definition language (DDL) have different interaction characteristics Database state changes cause modifications to the program representation Different kinds of test suites require different techniques for managing database state during testing Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 5/32 Testing Challenges The many testing challenges include, but are not limited to, the following: Unified program representation Family of test adequacy criteria Efficient test coverage monitoring techinques Intelligent approaches to test suite execution Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 6/32 Database-Driven Applications P m i D k R 1 A B C D R 2 E F G H m j I R 3 J K L D l Program P interacts with two relational databases Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 7/32 Database Interaction Levels P D 1 D n Database Level A program can interact with a database at different levels of granularity Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 8/32 Database Interaction Levels D 1 P D n UserInfo card_number pin_number user_name Brian Zorman acct_lock Robert Roos Marcus Bittman Geoffrey Arnold Relation Level Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 8/32 Database Interaction Levels D 1 P D n UserInfo card_number pin_number user_name Brian Zorman acct_lock Robert Roos Marcus Bittman Geoffrey Arnold Record Level Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 8/32 Database Interaction Levels D 1 P UserInfo card_number pin_number user_name acct_lock D n Brian Zorman Robert Roos Marcus Bittman Geoffrey Arnold Attribute Level Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 8/32 Database Interaction Levels D 1 P D n UserInfo card_number pin_number user_name Brian Zorman acct_lock Robert Roos Marcus Bittman Geoffrey Arnold Attribute Value Level Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 8/32 Database Interaction Points Database interaction point I r I corresponds to the execution of a SQL DML statement Consider a simplified version of SQL and ignore SQL DDL statements (for the moment) Interaction points are normally encoded within Java programs as dynamically constructed Strings select uses D k, delete defines D k, insert defines D k, update defines and/or uses D k Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 9/32 Database Interaction Points (DML) select A 1, A 2,..., A q from r 1, r 2,..., r m where Q delete from r where Q insert into r(a 1, A 2,..., A q ) values(v 1, v 2,..., v q ) update r set A l = F (A l ) where Q Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 1/32 Refined Database-Driven Application P select * from R1 where A ( select avg(g) from ) R 2 R 1 A B C D m i D k R 2 E F G H m j I R 3 J K L D l update R 3 set J = 5 where L 1 Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 11/32 Test Adequacy Criteria P violates a database D k s validity when it: (1-v) inserts entities into D k that do not reflect real world P violates a database D k s completeness when it: (1-c) deletes entities from D k that still reflect real world In order to verify (1-v) and (1-c), T must cause P to define and then use entities within D 1,..., D n! Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 12/32 Data Flow Information Interaction point: UPDATE UserInfo SET acct lock=1 WHERE card number= + card number + ; ; Database Level: define(bankdb) Attribute Level: define(acct_lock) and use(card_number) Data flow information varies with respect to the granularity of the database interaction Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 13/32 Database Entities UserInfo card_number pin_number user_name acct_lock Brian Zorman Robert Roos Marcus Bittman Geoffrey Arnold A v(i r) = { 1, 32142, Geoffrey Arnold...,, } Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 14/32 The DICFG: A Unified Representation entry lockaccount temp1 = parameter:c_n temp2 = LocalDatabaseEntity:Bank temp3 = LocalDatabaseEntity1:acct_lock temp4 = LocalDatabaseEntity2:card_number Database-enhanced CFG for lockaccount Define temporaries to represent the program s interaction at the levels of database and attribute Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 15/32 The DICFG: A Unified Representation I r qu_lck = UPDATE UserInfo... + temp1 + ; update_lock = m_connect.createstatement() A entry G exit G r2 define(temp3) use(temp4) r 2 entry exit result_lock = update_lock.executeupdate(qu_lck) if( result_lock == 1) completed = true exit lockaccount G G r 1 define(temp2) r1 D Database interaction graphs (DIGs) are placed before interaction point I r Multiple DIGs can be integrated into a single CFG String at I r is determined in a control-flow sensitive fashion Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 15/32 Test Adequacy Criteria all attribute value DUs Database interaction association (DIA) involves the def and use of a database entity all record DUs all relation DUs all attribute DUs all database DUs DIAs can be located in the DICFG with data flow analysis all-database-dus requires tests to exercise all DIAs for all of the accessed databases Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 16/32 Generating Test Requirements Database Seeder System Under Test (P) Database Test Adequacy Criterion (C) Test Case Specification Relational Schema Test Requirements Measured time and space overhead when computing family of test adequacy criteria Modified ATM and mp3cd to contain appropriate database interaction points Soot to calculate intraprocedural associations GNU/Linux workstation with kernel smp and dual 1 GHz Pentium III Xeon processors Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 17/32 Counting Associations and Definitions Assoc & Def Count D R c R l A A v ATM DB mp3cd DB ATM HD mp3cd HD D R c R l A A v Database Granularity DIAs at attribute value level represent 16.8% of mp3cd s and 9.6% of ATM s total number of intraprocedural associations p. 18/32 Measuring Time Overhead System Time sec Time Overhead None D R c R l A A v None D R c R l A A v Database Granularity ATM mp3cd Computing DIAs at the attribute value level incurs no more than a 5 second time overhead p. 19/32 Measuring Average Space Overhead Node & Edge Count None 3 D R c R l A A v None D R c R l A A v Database Granularity N atm N mp3 E atm E mp3 mp3cd shows a more marked increase in the average number of nodes and edges than ATM p. 2/32 Measuring Maximum Space Overhead Node & Edge Count None D R c R l A A v None D R c R l A A v Database Granularity N atm N mp3 E atm E mp3 mp3cd shows a significantly greater maximum space overhead than ATM p. 21/32 Automatic Representation Construction Manual construction of DICFGs is not practical Use extension of BRICS Java String Analyzer (JSA) to determine content of String at I r Per-class analysis is inter-procedural and control flow sensitive Conservative analysis might determine that all database entities are accessed Include coverage monitoring instrumentation to track DIGs that are covered during test suite execution Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 22/32 Tracking Covered DIGs and DIAs PDB m i m j DIG r DIG # 1 DIG s DIG Coverage Table DEF USE {... } {... } q {... } {... } TEST {... } {... } COV? DIA coverage can be tracked by recording which DIGs within a DICFG were executed during testing Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 23/32 Types of Test Suites m T 1 1 T 1 m 1 1 T 1 m 1 ε 1 1 T ε mε 1 e 1 ε e 1 T e m e e T e m e e T e m e e Independent e Non-restricted Partially Independent p. 24/32 Test Suite Execution Independent test suites can be executed by using provided setup code to ensure that all γ = Non-restricted test suites simply allow state to accrue Partially independent test suites must return to ε after T ε is executed by : 1. Re-executing all SQL statements that resulted in the creation of ε 2. Creating a compensating transaction to undo the SQL statements executed by each test after T ε Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 25/32 Representation Extension The execution of a SQL insert during testing requires the re-creation of DICFG(s) The SQL delete does not require re-creation because we must still determine if deleted entity is ever used DICFG re-creation only needed when database interactions are viewed at the record or attribute-value level Representation extension ripples to other methods DICFGs can be re-constructed after test suite has executed, thus incurring smaller time overhead Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 26/32 Test Coverage Monitoring For each tested method m i that interacts with a database and each interaction point I r that involves an insert we must: 1. Update the DICFG 2. Re-compute the test requirements We can compute the set of covered DIAs by consulting the DIG coverage table Test adequacy is : # covered DIAs / # total DIAs Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 27/32 Calculating Adequacy T f m i m j Test Requirements m i DIA COV? def(e1), use(e1) def(e2), use(e2) def(e3), use(e3) def(e4), use(e4) Test Requirementsm j DIA COV? def(e5), use(e5) def(e6), use(e6) def(e7), use(e7) def(e8), use(e8) def(e9), use(e9) def(e1), use(e1) cov(m i ) = 2 4 cov(m j ) = 4 6 cov(t f ) = 6 1 Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 28/32 Related Work Jin and Offutt and Whittaker and Voas have suggested that the environment of a software system is important Chan and Cheung transform SQL statements into C code segments Chays et al. and Chays and Deng have created the category-partition inspired AGENDA tool suite Neufeld et al. and Zhang et al. have proposed techniques for database state generation Dauo et al. focused on the regression testing of database-driven applications Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 29/32 Conclusions Must test the program s interaction with the database Many challenges associated with (1) unified program representation, (2) test adequacy criteria, (3) test coverage monitoring, (4) test suite execution The DICFG shows database interactions at varying levels of granularity Unique family of test adequacy criteria to detect type (1) violations of database validity and completeness Intraprocedural database interactions can be computed from a DICFG with minimal time and space overhead Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 3/32 Conclusions Test coverage monitoring instrumentation supports the tracking of DIAs executed during testing Three types of test suites require different techniques to manage the state of the database SQL insert statement causes the re-creation of the representation and re-computation of test requirements Data flow-based test adequacy criteria can serve as the foundation for automatically generating test cases and supporting regression testing Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 31/32 Resources Gregory M. Kapfhammer and Mary Lou Soffa. A Family of Test Adequacy Criteria for Database-Driven Applications. In FSE 23. Gregory M. Kapfhammer. Software Testing. CRC Press Computer Science Handbook. June, 24. gkapfham/research/diatoms/ Database driven Application T esting tool ModuleS, IBM T.J. Watson Research Center, May 14, 24 p. 32/32
Search
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks