Code coverage: Difference between revisions
Jump to navigation
Jump to search
| [unchecked revision] | [unchecked revision] |
imported>Citation bot Alter: title, pages, template type. Add: isbn, chapter, date. Removed URL that duplicated identifier. Removed access-date with no URL. Removed parameters. Formatted dashes. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | Suggested by GoingBatty | Category:CS1 errors: dates | #UCB_Category | Whitelisted category 321/806 |
|||
| Line 119: | Line 119: | ||
=== Parameter value coverage === | === Parameter value coverage === | ||
'''Parameter value coverage''' (PVC) requires that in a method taking parameters, all the common values for such parameters be considered. The idea is that all common possible values for a parameter are tested.<ref>{{cite web| url = http://www.rhyous.com/2012/05/08/unit-testing-with-parameter-value-coverage-pvc/| title = Unit Testing with Parameter Value Coverage (PVC)}}</ref> For example, common values for a string are: 1) [[Null object|null]], 2) empty, 3) whitespace (space, tabs, newline), 4) valid string, 5) invalid string, 6) single-byte string, 7) double-byte string. It may also be appropriate to use very long strings. Failure to test each possible parameter value may result in a bug. Testing only one of these could result in 100% code coverage as each line is covered, but as only one of seven options are tested, there is only 14.2% PVC. | '''Parameter value coverage''' (PVC) requires that in a method taking parameters, all the common values for such parameters be considered. The idea is that all common possible values for a parameter are tested.<ref>{{cite web| url = http://www.rhyous.com/2012/05/08/unit-testing-with-parameter-value-coverage-pvc/| title = Unit Testing with Parameter Value Coverage (PVC)| date = 8 May 2012}}</ref> For example, common values for a string are: 1) [[Null object|null]], 2) empty, 3) whitespace (space, tabs, newline), 4) valid string, 5) invalid string, 6) single-byte string, 7) double-byte string. It may also be appropriate to use very long strings. Failure to test each possible parameter value may result in a bug. Testing only one of these could result in 100% code coverage as each line is covered, but as only one of seven options are tested, there is only 14.2% PVC. | ||
=== Other coverage criteria === | === Other coverage criteria === | ||
| Line 139: | Line 139: | ||
Full path coverage, of the type described above, is usually impractical or impossible. Any module with a succession of <math>n</math> decisions in it can have up to <math>2^n</math> paths within it; loop constructs can result in an infinite number of paths. Many paths may also be infeasible, in that there is no input to the program under test that can cause that particular path to be executed. However, a general-purpose algorithm for identifying infeasible paths has been proven to be impossible (such an algorithm could be used to solve the [[halting problem]]).<ref>Dorf, Richard C.: ''Computers, Software Engineering, and Digital Devices'', Chapter 12, pg. 15. CRC Press, 2006. {{ISBN|0-8493-7340-9}}, {{ISBN|978-0-8493-7340-4}}; via [https://books.google.com/books?id=jykvlTCoksMC&dq=%22infeasible+path%22+%22halting+problem%22&pg=PT386 Google Book Search]</ref> [[Basis path testing]] is for instance a method of achieving complete branch coverage without achieving complete path coverage.<ref name="SrikantShankar2002">{{cite book|author1=Y.N. Srikant|author2=Priti Shankar|title=The Compiler Design Handbook: Optimizations and Machine Code Generation|year=2002|publisher=CRC Press|isbn=978-1-4200-4057-9|page=249}}</ref> | Full path coverage, of the type described above, is usually impractical or impossible. Any module with a succession of <math>n</math> decisions in it can have up to <math>2^n</math> paths within it; loop constructs can result in an infinite number of paths. Many paths may also be infeasible, in that there is no input to the program under test that can cause that particular path to be executed. However, a general-purpose algorithm for identifying infeasible paths has been proven to be impossible (such an algorithm could be used to solve the [[halting problem]]).<ref>Dorf, Richard C.: ''Computers, Software Engineering, and Digital Devices'', Chapter 12, pg. 15. CRC Press, 2006. {{ISBN|0-8493-7340-9}}, {{ISBN|978-0-8493-7340-4}}; via [https://books.google.com/books?id=jykvlTCoksMC&dq=%22infeasible+path%22+%22halting+problem%22&pg=PT386 Google Book Search]</ref> [[Basis path testing]] is for instance a method of achieving complete branch coverage without achieving complete path coverage.<ref name="SrikantShankar2002">{{cite book|author1=Y.N. Srikant|author2=Priti Shankar|title=The Compiler Design Handbook: Optimizations and Machine Code Generation|year=2002|publisher=CRC Press|isbn=978-1-4200-4057-9|page=249}}</ref> | ||
Methods for practical path coverage testing instead attempt to identify classes of code paths that differ only in the number of loop executions, and to achieve "basis path" coverage the tester must cover all the path classes.{{ | Methods for practical path coverage testing instead attempt to identify classes of code paths that differ only in the number of loop executions, and to achieve "basis path" coverage the tester must cover all the path classes.<ref>{{cite book|last1=Marcozzi|first1=M.|last2=Bardin|first2=Sébastien|last3=Kosmatov|first3=N.|last4=Papadakis|first4=Mike|last5=Prevosto|first5=Virgile|last6=Correnson|first6=Loïc|chapter=Time to clean your test objectives |date=2018 |title=Proceedings of the 40th International Conference on Software Engineering|pages=456–467|doi=10.1145/3180155.3180191 |isbn=978-1-4503-5638-1 }}</ref>{{clarify|date=July 2014}} | ||
==In practice== | ==In practice== | ||
The target software is built with special options or libraries and run under a controlled environment, to map every executed function to the function points in the source code.<ref>{{cite book |last1= Garousi |first1= Vahid |last2= Keleş |first2= Alper Buğra |last3= Balaman |first3= Yunus |last4= Mermer |first4= Alper |last5= Güler |first5= Zeynep Özdemir |chapter= Coverage measurement in model-based testing of web applications: Tool support and an industrial experience report |title= 2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) |date= May 2024 |pages= 37–43 |doi= 10.1109/icstw60967.2024.00019 |isbn= 979-8-3503-4479-0 }}</ref> This allows testing parts of the target software that are rarely or never accessed under normal conditions, and helps reassure that the most important conditions (function points) have been tested. The resulting output is then analyzed to see what areas of code have not been exercised and the tests are updated to include these areas as necessary. Combined with other test coverage methods, the aim is to develop a rigorous, yet manageable, set of regression tests. | |||
In implementing test coverage policies within a software development environment, one must consider the following: | In implementing test coverage policies within a software development environment, one must consider the following: | ||