Software Testing: Levels, Black-box vs White-box, and Quality · সফটওয়্যার টেস্টিং: লেভেল, ব্ল্যাক-বক্স বনাম হোয়াইট-বক্স ও মান

1. Error, fault, failure — and what testing is

Three ideas (error, fault/defect/bug, failure) are constantly confused, and exams test them directly:

• Error (mistake): a human action that produces something wrong — a programmer misunderstands a requirement.
• Fault / defect / bug: the flaw left in the code or document by that error — the wrong line.
• Failure: the observable wrong behaviour at run time when the faulty code executes.

A human makes an error, which puts a fault into the software, which — when run with the right input — causes a failure. A fault that is never executed never causes a failure, but is still a defect.

Software testing is executing a program with the intent of finding failures (and, more broadly, any activity that evaluates quality). The ISTQB foundation syllabus lists seven testing principles; the first two shape everything:

1. Testing shows the presence of defects, not their absence (Dijkstra). Passing tests never prove the software is correct.
2. Exhaustive testing is impossible for any real program, so we select a small, high-value set of tests — which is exactly what the techniques below do.
3. Early testing saves time and money: a defect caught in a requirements review costs far less than one found after release.
4. Defects cluster together: a few modules usually hold most of the defects, so test effort goes where defects have been found.
5. Tests wear out (pesticide paradox): running the same tests again and again stops finding new defects, so tests must be revised.
6. Testing is context dependent: a banking system and a mobile game are tested differently.
7. Absence-of-errors fallacy: software with no known defects can still fail if it does not meet the users' needs.

2. Verification vs validation

This is the single most-asked pair (Barry Boehm's phrasing):

Aspect	Verification	Validation
Question	"Are we building the product right?"	"Are we building the right product?"
Checks against	Specification and design documents	The user's real needs
Typical activities	Reviews, inspections, walkthroughs, static analysis	Actually running the software (unit → acceptance)
Code executed?	Usually not (static)	Yes (dynamic)
Finds	Deviations from the spec, early	Whether it truly solves the user's problem

Textbooks differ: some treat all testing as validation, others (and IEEE 1012) count unit and integration tests as verification and acceptance tests as validation. The safe exam key is: reviews, inspections and walkthroughs → verification; acceptance or beta testing with real users → validation.

So a review is verification; running the finished system with real data is validation. Static testing examines artefacts without running them (reviews, static analysis); dynamic testing runs the code.

3. Levels of testing and the V-model

Testing is organised into levels that mirror the stages of development; the V-model pairs each build stage with the test level that validates it.

Level	What is tested	Who, typically	Note
Unit	One module/function in isolation	Developer	Uses stubs and drivers for the parts it calls or is called by
Integration	The interfaces between modules	Developer / tester	The objective is to find interface errors
System	The whole integrated system against its specification	Independent testers	Includes non-functional testing
Acceptance	Whether the system meets the user's needs	Customer / users	Alpha and beta testing

Integration strategies and the stub/driver trap:

• A driver is throwaway code that calls the module under test (it stands in for a higher, not-yet-ready module).
• A stub is throwaway code that replaces a lower module the unit calls (it returns canned results).
• Top-down integration tests high-level modules first and needs stubs for the lower modules not yet built.
• Bottom-up integration tests the fundamental low-level modules first and needs drivers to call them. ("Which strategy starts with the fundamental components?" → bottom-up.)
• Sandwich / hybrid combines both; big-bang integrates everything at once (hard to localise faults).

Acceptance testing — alpha vs beta:

• Alpha testing: by potential users or customers (or an independent test team) at the developer's site, in a controlled environment, before release.
• Beta testing: by selected real users, at their own sites, in the real environment — the final stage before general release. ("Testing with actual data in the real environment" → beta.)

4. Black-box vs white-box

Point	Black-box (functional)	White-box (structural)
Based on	The specification (behaviour only)	The internal code and its structure
Also called	Functional, behavioural	Structural, glass-box, clear-box
Tester needs code?	No	Yes
Looks for	Wrong outputs for given inputs	Untested paths, logic errors
Techniques	Equivalence partitioning, boundary value analysis, decision tables, state transition	Statement, branch, path coverage; cyclomatic complexity
Typical level	System, acceptance	Unit

Two exam facts: "testing without knowledge of the internal implementation" is black-box, and "structural testing" is another name for white-box. A focus that is not white-box is the user's perspective (that is black-box). Boundary value analysis is a black-box technique. Grey-box testing sits between the two: the tester knows part of the internals (for example the database schema or the API design) but still tests through the external interface.

5. Black-box technique: equivalence partitioning and boundary value analysis (worked)

Equivalence partitioning (EP) splits the input domain into classes that should behave the same, then tests one value per class (valid and invalid). Boundary value analysis (BVA) then tests the edges of each class, where off-by-one defects cluster. Textbooks use different sets: standard BVA (Jorgensen) tests min, min+1, nominal, max−1, max (5 values; 4n + 1 for n inputs); robust BVA adds min−1 and max+1 (7 values; 6n + 1); ISTQB's 2-value BVA tests each boundary and its nearest neighbour across it (−1, 0, 5000, 5001 for the range below), and 3-value BVA adds the neighbour on the inside (−1, 0, 1, 4999, 5000, 5001). Unless a question says otherwise, give the robust set below.

Worked example (a real exam question). A program computes the square root of an input integer between 0 and 5000. Give the equivalence classes and the boundary test cases.

Equivalence classes (three, each tested once):

Class	Rule	Representative	Expected
Invalid (too low)	input < 0	−20	rejected / error
Valid	0 ≤ input ≤ 5000	2500	√2500 = 50
Invalid (too high)	input > 5000	7000	rejected / error

Boundary values (robust set) around min = 0 and max = 5000: {−1, 0, 1, 2500, 4999, 5000, 5001}. Running the program on these confirmed the behaviour: −1 and 5001 are rejected; 0 gives 0; 1 gives 1; the integer square roots of 4999 and 5000 are both 70 (since 70² = 4900 ≤ 4999, 5000 < 71² = 5041), and 2500 gives 50. EP alone needs 3 cases; BVA adds the edges where bugs hide. Exhaustive testing would need all 5001 valid inputs — which is why we partition.

Two more black-box techniques exams name:

• Decision-table testing: list every combination of conditions (for example "member?" × "order over 1000 Tk?") as columns with the expected action for each, then write one test per column — good for business rules.
• State-transition testing: model the system as states and events (for example an ATM card: active → blocked after three wrong PINs) and test each valid transition plus some invalid ones.

6. White-box coverage and cyclomatic complexity (worked)

Coverage metrics, from weakest to strongest:

• Statement coverage: every statement executed at least once.
• Branch (decision) coverage: every decision takes both the true and the false outcome. Branch coverage implies statement coverage, not the reverse.
• Path coverage: every possible path from entry to exit — the strongest, and usually infeasible in full because loops create an unbounded number of paths. (Testing a basis set of V(G) independent paths is the practical substitute, called basis-path testing.)

Cyclomatic complexity V(G) (Thomas McCabe, 1976) counts the linearly independent (basis) paths through a program's control-flow graph: it is the number of test cases basis-path testing needs, and an upper bound on the tests needed for full branch (and statement) coverage. For a single connected graph the three equivalent formulas are:

• V(G) = E − N + 2 (E edges, N nodes) — the common exam form;
• V(G) = D + 1 (D = number of binary decision nodes; a k-way switch counts as k − 1);
• V(G) = number of enclosed regions + 1 in the planar graph.

The general form is V(G) = E − N + 2P, where P is the number of connected components (P = 1 for one program graph, giving E − N + 2).

Worked example. Count the positive numbers in a list:

1  i = 0; count = 0
2  while i < n            // decision
3      if a[i] > 0        // decision
4          count = count + 1
5      i = i + 1
6  return count

Draw its control-flow graph with nodes 1–6 and edges 1→2, 2→3, 2→6 (loop exit), 3→4, 3→5, 4→5, 5→2 (loop back). That is N = 6 nodes and E = 7 edges, with 2 predicate nodes (the while and the if).

• By E − N + 2: 7 − 6 + 2 = 3.
• By D + 1: 2 + 1 = 3.

Both agree, and a rank check of the three basis paths' edge vectors also gives 3. So the basis set has 3 paths, and basis-path testing uses 3 test cases: (a) the list is empty so the loop body never runs; (b) a positive element (the if is true); (c) a non-positive element (the if is false). Branch coverage alone can need fewer — the single input [5, −3] already takes every branch (while true, if true, if false, while false) — which is why V(G) is an upper bound for branch coverage, not a minimum. McCabe recommends keeping V(G) ≤ 10 per module.

Branch-coverage MCQ (worked). For if (A OR B) X = X+1; else X = X−1;, which set of (A, B) achieves decision (branch) coverage? Decision coverage needs the condition A OR B to be both true and false at least once. A OR B is false only when both are false. The options were (a) {(F,T)}; (b) {(F,T),(T,F),(T,T)}; (c) {(F,T),(T,F)}; (d) {(F,F),(T,T)}. Options a, b and c make A OR B true every time, so only d = {(False, False), (True, True)} produces one false outcome and one true outcome and covers both branches — the keyed answer. (It even achieves condition coverage, since each of A and B takes both values.)

7. Functional and non-functional test types

Type	Purpose
Smoke	A quick, broad check that the build's critical functions work — "does it even start?" Decides whether deeper testing is worth it
Sanity	A narrow, focused check that a specific fix or feature now works after a change
Regression	Re-run existing tests after a change to ensure nothing that worked is now broken; prime candidate for automation
Re-testing (confirmation)	Run the specific failed test again after a fix to confirm the defect is gone
Performance	Measure speed, responsiveness and throughput; an umbrella term
Load	Behaviour under expected/peak load
Stress	Push beyond limits to find the breaking point and recovery
Usability	How easy and intuitive it is for real users (a non-functional test)

Note for MCQs: performance and usability testing are non-functional; the test that ensures new changes have not broken existing features is regression.

Common testing tools (asked as "name some testing tools"): JUnit (unit testing in Java), Selenium (web UI automation), JMeter (load and performance), Postman (API testing).

8. Software quality and SQA

• Quality Control (QC) is product-oriented — it finds defects in the product (testing, reviews). Software Quality Assurance (SQA) is process-oriented — planned, systematic activities (standards, audits, reviews, process definition) that build confidence the product meets its requirements. QC detects defects; SQA tries to prevent them.
• Quality models list the characteristics of quality. ISO/IEC 9126 (older, still keyed) names six: functionality, reliability, usability, efficiency, maintainability, portability. ISO/IEC 25010 replaced it: the 2011 edition has eight product-quality characteristics (adding security and compatibility, renaming functionality to functional suitability and efficiency to performance efficiency); the 2023 revision adds safety as a ninth and renames usability to interaction capability and portability to flexibility. When an exam asks for ISO 9126, answer six; for ISO/IEC 25010, eight.

Model	Year	Top characteristics
ISO/IEC 9126	1991/2001	6: functionality, reliability, usability, efficiency, maintainability, portability
ISO/IEC 25010	2011	8: functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, portability
ISO/IEC 25010	2023	9: as 2011 but usability → interaction capability, portability → flexibility, plus safety

Test documentation follows IEEE 829 (the historical Standard for Software Test Documentation — test plan, test case, test log, defect report) and the current ISO/IEC/IEEE 29119 series. A test case states inputs, preconditions, steps and the expected result; it is written by the tester (test engineer), not the developer.