TMMi-P_Syll2.1 TMMi Test Maturity Model Integration Professional (TMMi-P) Questions and Answers

In the process area Advanced Reviews at TMMi Level 4, one of the key specific goals is to measure product quality early in the lifecycle through peer reviews. These reviews are coordinated with dynamic testing to optimise the testing process by incorporating early defect detection and product quality data. This practice emphasises improving product quality from the beginning, aligning peer reviews with the overall test strategy​​.

ISO/IEC 15504, which is known for its process assessment model, was not one of the sources used during the development of the TMMi model. The TMMi model was developed using inputs from sources such as the Capability Maturity Model Integration (CMMI), international testing standards (like those from ISO and IEEE), and industry testing best practices surveys. While the ISO/IEC 15504 model has relevance to general process assessments, it was not specifically referenced in the creation of the TMMi.

For safety-critical systems, reliability testing is a key concern. The identified shortcoming relates to the lack of formal techniques and approaches for reliability testing. The Non-Functional Testing process area, at TMMi Level 3, addresses testing non-functional aspects like reliability, performance, and usability. This process area would need specific attention as it provides a structured approach to assess non-functional risks, define testing techniques, and execute tests to verify the reliability of the product, which is critical for medical systems.

For the Test Policy and Strategy process area, with a 40% achievement, the appropriate compliance rating is "Partly achieved". According to TMMi, a process area is rated as "partly achieved" when a portion of the practices has been implemented, but significant improvements are still needed​​​.

In the Acting phase of the IDEAL improvement model, the focus is on implementing the changes that were identified and planned in the earlier phases. The two main activities in this phase are Create Solution and Implement Solution. This phase generally consumes the most effort, as creating and deploying solutions to improve the processes are resource-intensive activities. These activities align with option D, making it the correct answer .

The TMMi framework is applicable to all levels of testing, including static testing (such as reviews) and dynamic testing, regardless of the phase of the software development lifecycle. TMMi encompasses structured testing activities across various test levels such as component testing, integration testing, system testing, and user acceptance testing, along with both static and dynamic techniques. Therefore, all levels of testing can be improved using the TMMi model.

The specific practice "Establish the Organization’s Test Process Database" is found within the Test Lifecycle and Integration process area, which is part of TMMi Level 3. This process area aims to establish standardized test processes across the organization, including maintaining a test process database that holds process-related information and data for reuse and continuous improvement.

TMMi References:

The Test Lifecycle and Integration process area is responsible for establishing and maintaining organizational test process assets, including the test process database​​.

In the scenario described, testing is performed ad hoc by business analysts who do not have defined test roles or a structured career path within the organisation. The process area that needs attention here is Test Organization (TMMi Level 3). This process area is crucial for establishing a structured test group with dedicated roles and responsibilities for testing, rather than relying on business analysts for testing as a secondary task.

The Test Organization process area focuses on establishing a formalised test organisation within the company, defining test functions and responsibilities, and promoting testing as a recognised profession with defined career paths. This would address the shortcomings of the current setup by creating a dedicated, skilled testing team, thus improving the consistency and quality of testing activities within the bank​​.

A direct benefit of a test improvement program is higher productivity. When test processes are streamlined, better test planning, design, and execution can lead to reduced testing times, fewer defects, and quicker turnaround of quality products. This has a direct impact on organisational efficiency, allowing for more effective use of resources and time.

The specific goals of the "Test Monitoring and Control" process area at TMMi Level 2 focus on monitoring both test progress and product quality. One of the key practices is managing and monitoring test incidents to closure (Specific Practice 3.3). The goal is to ensure that any test-related issues are tracked, addressed, and resolved. This process area emphasises identifying corrective actions when deviations occur and ensuring that they are brought to closure.

Monitoring test incidents helps provide insight into the test status and product quality while ensuring that all critical incidents are resolved effectively, thus aligning with the overall objective of controlling the testing process​​.

The debugging-oriented phase of the evolutionary testing model aligns closely with TMMi Level 1 (Initial). At TMMi Level 1, testing is often chaotic and is not formally separated from debugging. The primary focus is on finding and fixing bugs, which reflects the characteristics of the debugging-oriented phase described in the Gelperin and Hetzel evolutionary testing model. Testing at this level lacks structure and is mainly an ad hoc activity aimed at removing defects from the software.

In the Learning phase of the IDEAL model, one of the critical activities is "Analyse and Validate". This involves reviewing the entire improvement process to determine whether the intended goals were achieved and identifying what worked well and what could be improved. The objective is to learn from the experience to enhance future improvement initiatives and continually refine the ability to implement change​​.

In the scenario described, where an organisation suffers from poor test estimations despite many practices being in place, the process area that requires attention is Test Planning. Specifically, SG 3 (Establish Test Estimates) focuses on developing well-founded estimates for testing effort and cost. This includes creating a top-level work breakdown structure, defining the test lifecycle, and determining estimates for the test effort and costs associated with different test activities. By addressing these practices, the organisation can improve its ability to generate accurate test estimations, leading to better project planning and resource allocation.

Management commitment is highlighted as one of the critical success factors in the context of implementing and sustaining test process improvements. Within the TMMi framework, senior management support is crucial for ensuring that necessary resources, funding, and priority are allocated to the improvement initiatives. This commitment must be sustained throughout the process to overcome potential resistance and ensure that the improvements are institutionalised. Without strong management commitment, the efforts may lose momentum, and the improvement goals may not be achieved.

This factor is critical across several levels of TMMi, as seen in processes related to institutionalising the test process (such as defining organisational policies, resource allocation, and performance evaluation).

The description provided—where testing is multi-level, with distinct test levels (component, integration, system, and acceptance), and where the processes of testing and debugging are differentiated—applies to TMMi Level 2 (Managed). At this maturity level, testing is separated from debugging, and there are formalized processes for different testing levels. A company-wide or program-wide test strategy is established, and specific testing objectives are defined for each test level.

TMMi References:

The TMMi Level 2 framework defines structured testing with multiple levels, including component, integration, system, and acceptance testing, as well as the separation of testing and debugging​​.

In a staged model like TMMi, an organisation follows a predefined improvement path that includes a set of process areas organised by maturity levels. Each level builds upon the previous one, and the organisation progresses in a step-by-step manner through these maturity levels to improve its processes. The staged model is well-structured and prescriptive, offering a clear improvement roadmap for the organisation.

At TMMi Level 3, Test Lifecycle and Integration builds on the practices of Test Planning introduced at TMMi Level 2. This process area focuses on establishing a standardised test lifecycle, integrating the test process with the development lifecycle, and coordinating the testing effort across multiple levels. It also includes creating a master test plan that builds upon the risk assessments, test estimates, and approaches defined at Level 2 during Test Planning.

By further refining these practices and introducing lifecycle integration, Test Lifecycle and Integration ensures that the testing processes are consistent, well-defined, and fully embedded into the overall project lifecycle​​.

Both Test Organization and Test Training Program are process areas associated with TMMi Level 3 (Defined). At this level, the organisation establishes a formal test organisation and ensures that there is a dedicated test training program. This helps to institutionalise testing as a profession and ensures the systematic development of skills needed for testing activities across the organisation.

Advanced Reviews is a TMMi Level 4 process area. At TMMi Level 4 (Measured), organizations focus on making testing a quantifiable process, using metrics and measurements to evaluate and improve product quality and the testing process. Advanced Reviews involve more structured and formalized review processes that are integrated with the dynamic testing approach to measure and improve product quality earlier in the lifecycle.

The Test Policy and Strategy is part of Level 2.

Defect Prevention is part of Level 5.

Test Lifecycle and Integration is part of Level 3.

TMMi References:

TMMi defines Advanced Reviews as a Level 4 process area aimed at improving product quality through more rigorous and measurable review processes​​.

The specific practice "Define product risk categories and parameters" is part of the Test Policy and Strategy process area in TMMi. This practice involves identifying and categorizing the product risks to guide testing efforts and establish a clear understanding of potential product issues. Product risk categories help in determining the focus areas for testing, ensuring that testing is prioritized based on business and technical risks.

TMMi References:

The practice of defining product risk categories is directly linked to the Test Policy and Strategy process area at TMMi Level 2, where risk-based testing approaches are established​​.

The TMMi model is designed to support test process improvement in both software engineering and systems engineering. This is critical because it addresses the needs of total system development, which may include hardware and software. The model covers various testing activities across different disciplines, allowing organisations to improve their test processes regardless of whether they focus solely on software or integrated systems.

In the context of TMMi and test improvement programs, an indirect benefit refers to outcomes that are not directly tied to the technical improvement of the testing process but affect the overall success of the organization in less measurable ways.

Increased personnel motivation is considered an indirect benefit of a test improvement program because it boosts team morale and engagement, leading to better productivity in the long run. This is different from direct benefits such as improvements in defect detection or test execution lead-time, which are quantifiable metrics directly related to the testing process.

TMMi References:

Direct benefits such as defect detection rates and test execution speed are frequently mentioned in TMMi as measurable outcomes from process improvement efforts.

Indirect benefits, like improved motivation, are acknowledged as part of the cultural and organizational improvements that can come from a well-executed test improvement strategy​​.

The generic practice "Establish a defined process" addresses the need to tailor a managed process from the organization’s set of standard processes. This practice is part of Generic Goal 3 (GG 3), which aims to ensure that processes are standardized and defined across the organization. By tailoring processes based on established guidelines, organizations can create specific implementations that fit their projects while maintaining consistency and control across the organization.

TMMi References:

In TMMi, establishing a defined process involves tailoring standard processes to suit specific project needs, ensuring the institutionalization of best practices across projects​​.

For a formal TMMi assessment, both staff interviews and document study are required to collect sufficient evidence. The assessment must adhere to the TMMi Assessment Method Accreditation Requirements (TAMAR), which state that formal assessments require the corroboration of interview data with documentary evidence (artifacts). These can include documents, templates, or screenshots, ensuring that the evaluation is robust, repeatable, and auditable​​.

The specific practice "Define a coordinated approach that coordinates both static and dynamic testing" is part of the Advanced Reviews process area at TMMi Level 4. The goal of this process area is to integrate static and dynamic testing, particularly through peer reviews, which helps optimise both defect detection and product quality management throughout the lifecycle​​​.

This statement is false because the CMMI does not have defined levels specifically for testing maturity. The CMMI model is more general and focuses on overall process improvement across various disciplines, including but not limited to testing. CMMI includes practices like verification and validation but does not specifically address a structured path to testing maturity, unlike TMMi, which defines specific levels for testing process maturity. Therefore, statement D is incorrect.

The Advanced Reviews process area, which is part of TMMi Level 4, is concerned with measuring product quality early in the lifecycle. It aims to enhance the test strategy and test approach by aligning static testing (reviews and inspections) with dynamic testing (execution of test cases). The results and data from static testing are used to optimize the dynamic testing approach, making the overall testing process more efficient and effective.

TMMi References:

Advanced Reviews at TMMi Level 4 focuses on integrating static and dynamic testing approaches to optimize test strategy and product quality evaluation​​.

At TMMi Level 3, the organisation's test processes become more structured, managed, and standardised. It requires defining and implementing a consistent set of test processes across projects and organisational units. One key factor for achieving TMMi Level 3 is the integration of process asset development.

"Process Asset Development" is directly related to establishing organisational test process assets, which is a critical element at TMMi Level 3. This practice area includes developing and maintaining organisational process assets, which encompass the standard processes that will be used across various projects. These assets are required to ensure that testing is a well-defined, repeatable, and efficient process throughout the organisation.

According to the TMMi Framework 1.3, process asset development supports the implementation of defined and repeatable processes and helps with creating guidelines for tailoring processes across the organisation. It provides the necessary framework for consistency in test planning, execution, and reporting, which are key at this maturity level​.

This practice aligns with the goals of TMMi Level 3, where the focus is on defining, standardising, and integrating the test lifecycle within the overall development lifecycle, thus ensuring that test processes are well-established across the organisation. Hence, Process Asset Development (Option B) is critical for the organisation's progression to TMMi Level 3.

The Debugging-oriented phase from Gelperin and Hetzel's evolutionary testing model is associated with Maturity Level 1 "Initial" of TMMi. At this maturity level, testing is not yet a distinct and well-defined process but is often interleaved with debugging. Organizations at this level have chaotic and ad-hoc testing activities, with no separation between testing and debugging.

Maturity Level 1 organizations lack formal processes, and testing is often reactive, performed alongside fixing bugs in the code. This is in line with the debugging-oriented phase, where testing and debugging are seen as part of the same activity.

TMMi References:

The TMMi framework draws directly from the evolutionary model of Gelperin and Hetzel, associating the Debugging-oriented phase with Level 1, which describes a chaotic, unmanaged testing environment​​.

The specific goals of the Test Process Optimization process area at TMMi Level 5 include:

Select test process improvements.

Evaluate new testing technologies to determine their impact on the testing process.

Deploy test process improvements.

However, "Determine, plan and implement test process improvements" (2) and "Monitor product quality against plan and expectations" (5) are not part of the Test Process Optimization goals. Instead, these activities belong to other process areas, such as Test Measurement or earlier maturity levels.

TMMi References:

The goals for Test Process Optimization at TMMi Level 5 focus on selecting and deploying process improvements and evaluating new technologies​​.