Untested Test (Ad Hoc Testing)

Ensuring quality assurance in software development processes is a fundamental objective that directly impacts product reliability and user satisfaction. In this context, software testing is one of the primary validation methods used to evaluate whether a developed system conforms to its functional and non-functional requirements. Software testing activities can be carried out in a planned, documented, and automation-compatible manner, or they can be performed entirely intuitively, without documentation and spontaneously. This second approach is referred to in the literature as Ad Hoc Testing.

Ad Hoc testing refers to testing activities that fall outside systematic test strategies, are not based on predefined rules, and rely heavily on experience and observation. In this method, test scenarios are not predetermined; instead, the tester interacts directly with the system to discover unexpected aspects of the software. Therefore, ad hoc tests often share characteristics with exploratory testing and are regarded as a complement to classical test types.

In the literature, the software testing process is often viewed as a spectrum: one end consists of fully formal, documented, and automation-integrated testing methods; the other end includes unplanned, undocumented, and intuitive ad hoc testing. Although ad hoc testing occupies the free end of this spectrum, it is frequently employed in dynamic systems or rapidly evolving software projects. In particular, in agile software development approaches such as Agile and Scrum, it is known that ad hoc methods are applied in short-term tests conducted after specific user stories have been completed.

Conceptual Definition and Characteristics

Ad Hoc Testing denotes testing activities conducted without any predefined test plan, test case scenario, or scope documentation. The core of this method involves freely navigating the software system’s user interface or specific functionality to locate errors. The practitioner determines test steps on the spot, and documenting the executed procedures is often an optional choice.

Ad hoc tests are often used as a complement to planned tests. Since predefined scenarios cannot cover all possible cases, potential error patterns arising from user experience or application intuition can be uncovered through this method. Thus, this test type plays a crucial role in identifying marginal conditions that systematic tests may overlook.

Some distinguishing features of this method are summarized below:

Non-Structured Test Process

The most prominent feature of ad hoc testing is its non-structured nature. Since test scenarios and steps are not predetermined, the executed actions are random. Each test session may differ based on the tester’s current knowledge, experience, and intuition. This structure provides flexibility but also introduces challenges in reproducibility.

Lack of Documentation

Ad hoc tests are often not documented. The purpose of the test is not to verify a specific requirement but to intuitively uncover potential errors. Therefore, test outputs and steps are not regularly recorded, making it difficult to reproduce errors after they are detected.

Intuitive and Experience-Based Approach

The success of ad hoc testing largely depends on the tester’s technical experience, system knowledge, and error intuition. A skilled tester can more easily identify system weaknesses and follow targeted test steps. In this regard, ad hoc testing is an approach based on individual competence rather than a formalized process.

Low Reproducibility

Because ad hoc testing does not involve a predefined procedure, its repeatability is low. It is difficult to reapply the same test steps at different times or by different test engineers and achieve the same results. This limits the traceability and retrospective analysis of test outcomes.

Weak Compatibility with Formal Test Processes

Ad hoc testing struggles to integrate with formal processes such as test automation, quality standardization, and certification. Formalized test documentation, requirements traceability matrices, and defect reporting systems are required to comply with standards such as ISO/IEC 29119, but ad hoc testing does not directly conform to this systematic framework.

Similarity to Exploratory Testing

There are significant similarities between ad hoc testing and exploratory testing. In both methods, the testing process has a spontaneous and reactive structure in which learning, designing, and executing occur simultaneously. However, exploratory testing typically involves more systematic feedback and learning cycles, whereas ad hoc testing is distinguished by its more uncontrolled and freeform nature.

Application Areas and Usage Context

Ad Hoc Testing is generally regarded as a complementary technique in software testing processes and can play a critical role in projects conducted under limited resources, time, and documentation. This testing approach has unique application contexts in both traditional software development life cycles and agile software development models.

Testing Requirements Under Time and Resource Constraints

One of the most common application areas of ad hoc testing is environments where project schedules are tight or testing resources are insufficient. Especially as product delivery deadlines approach, when it becomes impossible to systematically execute all test scenarios, developers or testers resort to ad hoc tests to perform rapid error scans on the existing system. This approach enables the detection of unexpected error conditions that were not included in the original test plans.

In Undocumented or Minimally Documented Systems

In prototyping processes, legacy systems, or poorly documented software projects, applying standard test procedures is difficult. In such cases, ad hoc testing provides an opportunity for developers or test engineers to evaluate critical system functions based on their intuition and experience. The absence or incompleteness of test cases is one of the primary conditions that enhance the applicability of ad hoc testing.

Helping Newcomers Gain System Familiarity

Ad hoc testing is also used to help test engineers or developers quickly become familiar with a software system. A new team member working with a module or product for the first time can learn its functionality and develop initial error intuitions by performing ad hoc tests on the system. This method is also used in training environments to develop exploratory testing skills.

Use in Agile Software Development Processes

In agile approaches such as Scrum or Extreme Programming (XP), software is developed and tested in small iterations (sprints). During these processes, it is observed that test teams use ad hoc testing to provide rapid feedback during the sprint review or demo phases at the end of each sprint. This is an effective method for evaluating the overall correctness of functionality before systematic testing.

In GUI (Graphical User Interface) Testing

Graphical user interfaces involve extensive combinations and state transitions based on user interaction. Because GUI tests cannot anticipate every possible user behavior, testers may need to experiment with different paths to observe how the system behaves in practice. In this context, ad hoc testing can provide valuable feedback for understanding user experience, especially in complex interfaces.

Capturing Systemic Surprises (Unanticipated Behaviors)

Some types of errors emerge only under unusual usage scenarios or unexpected input combinations. For example, if a user performs a process out of its normal sequence, the system may crash or produce an unforeseen outcome. It is difficult to include such possibilities in systematic test scenarios, making ad hoc testing effective for uncovering these “corner cases.”

In Ad Hoc Network Systems and Real-Time Applications

Ad hoc testing approaches are not limited to software systems; they are also used in dynamic topology network systems such as wireless ad hoc networks. For instance, network behaviors affected by electromagnetic interference, directional changes, and irregular node movements cannot be tested using predefined scenarios. Therefore, application-specific ad hoc test platforms have been developed to analyze parameters such as mobile node movement and latency through these tests.

Strengths and Weaknesses of Ad Hoc Testing

Ad hoc testing (ad hoc testing) introduces both flexibility and uncertainty into the software testing process due to its intuitive and unstructured nature. As such, it is regarded as a complement to systematic testing methods, but it also carries limiting weaknesses under certain conditions.

Strengths

Flexibility and Rapid Implementation Capability

Ad hoc testing is a highly rapid testing method because it does not require predefined test steps. The test engineer can examine any part of the system on the spot and freely shape the test scope according to current conditions. This enables rapid error scanning in projects under time pressure.

Discovery of New and Unanticipated Errors

Planned tests typically rely on requirements and target predictable situations. In contrast, ad hoc testing can uncover errors that systematic scenarios fail to detect by simulating unexpected user behaviors. The literature emphasizes the importance of this method in illuminating “blind spots.”

Effectiveness for Experienced Testers

If the test engineer possesses sufficient knowledge and intuition about the system, ad hoc testing enables them to uncover more targeted and efficient errors. Particularly in graphical interfaces and user experience testing, intuitive testing by experienced individuals can yield meaningful results.

Contribution with Low Preparation Cost

Since ad hoc testing does not require a specific test environment or data preparation, it can be applied with minimal preparation cost. This is a significant advantage for small-scale projects or limited test teams.

Weaknesses

Low Reproducibility

The most fundamental limitation of ad hoc testing is the difficulty of recreating the same test scenario. Unless test steps are documented, it cannot be guaranteed that a detected error will be reproduced in the same way at a later time. This complicates error tracking and resolution processes.

Lack of Documentation and Traceability

Since most ad hoc tests are not directly recorded, their outputs cannot be integrated into corporate testing processes. The test scope, what was tested, and which data were used are often unclear. This deficiency poses a significant problem in systems requiring quality assurance and regulatory compliance.

Incompatibility with Automation

Test automation requires repeatable, planned, and measurable test scenarios. In this context, ad hoc tests cannot be incorporated into automation frameworks. They perform poorly in processes such as testability, performance monitoring, or integration with continuous integration systems.

Difficulty in Controlling Scope and Depth

In ad hoc testing, quantitative metrics such as which modules were tested and to what extent, or which user scenarios were covered, cannot be tracked. This may lead to incomplete test coverage or the omission of critical conditions.

Excessive Dependence on Individual Ability

The success of testing largely depends on the personal experience and system knowledge of the individual performing it. This can lead to inconsistent results when the same test is applied by different people. Therefore, for corporate quality assurance frameworks, ad hoc testing is considered unsustainable.

Its Place in the Software Testing Process

Ad hoc testing (Ad Hoc Testing) is a technique that operates independently of the formal, planned, and documented stages of the software testing process. Nevertheless, as software projects develop testing strategies, the supportive and complementary role of ad hoc testing is increasingly accepted and more effectively integrated, especially alongside agile methodologies.

Positioning Within the Testing Process

Software testing generally consists of the following fundamental stages:

1. Test Planning
2. Test Design (Test Case Preparation)
3. Test Environment Setup
4. Test Execution
5. Test Result Evaluation and Reporting

This structure represents a formalized testing process emphasized in standards such as ISO/IEC 29119 to ensure measurability and traceability. Ad hoc testing is not directly integrated into this systematic framework; however, it can be applied during the test execution phase as a method of free observation and intuitive inspection. In particular, the contribution of ad hoc testing in verifying the overall stability of the system or user experience after functional testing is complete cannot be overlooked.

Its Position Between Exploratory and Experience-Based Testing Approaches

Ad hoc testing is an approach positioned between planned testing and exploratory testing. Planned testing aims to verify predefined requirements, while exploratory testing seeks to simultaneously learn the system and develop test scenarios. Ad hoc testing, however, primarily involves unplanned but targeted test actions guided by specific system knowledge or intuition.

In agile development environments, testers, when evaluating a working version of the software at the end of each “sprint,” apply not only predefined scenarios but also intuitive ad hoc tests. In this context, it is evident that ad hoc testing is also used in continuous integration processes.

Relationship with Other Test Types

Ad hoc testing does not directly correspond to any of the classical test levels such as unit testing, integration testing, system testing, or acceptance testing. Instead, it should be viewed as a complementary and exploratory testing activity applied at the end of or between these levels. In particular, after system testing, ad hoc testing can be used to emulate user behavior or observe the system’s “unexpected” responses.

Challenges in Integration with Corporate Processes

One of the main challenges of ad hoc testing is its difficulty in being integrated into structured processes. Quality models such as ISO/IEC 25010 require tests to comply with principles of traceability, accuracy, and repeatability. In this context, it is stated that ad hoc tests do not directly align with these standards but can be defined as an auxiliary technique within a suitable testing strategy.