TMMi-P_Syll2020 TMMi - Test Maturity Model Integration Professional Questions and Answers

In the scenario described, testing is performed ad hoc by business analysts who do not have defined test roles or a structured career path within the organisation. The process area that needs attention here isTest Organization(TMMi Level 3). This process area is crucial for establishing a structured test group with dedicated roles and responsibilities for testing, rather than relying on business analysts for testing as a secondary task.

TheTest Organizationprocess area focuses on establishing a formalised test organisation within the company, defining test functions and responsibilities, and promoting testing as a recognised profession with defined career paths. This would address the shortcomings of the current setup by creating a dedicated, skilled testing team, thus improving the consistency and quality of testing activities within the bank​​.

Peer ReviewsisNOTaTMMi Level 2process area. Peer Reviews is a process area atTMMi Level 3. At Level 3, organizations formalize their testing processes, including peer reviews, which involve static testing activities like inspections and walkthroughs to identify defects early.

Test Policy and Strategy,Test Monitoring and Control, andTest Environmentare all process areas atTMMi Level 2.

TMMi References:

Peer Reviews are introduced atTMMi Level 3, whereasTest Policy and Strategy,Test Monitoring and Control, andTest Environmentare process areas at Level 2​​.

This statement isFALSEbecause TMMi doesnotfocus only on higher test levels like system and acceptance testing. In fact, TMMi coversall test levels, including lower levels like unit and integration testing, as well as static testing (e.g., reviews and inspections).

TMMi is designed to address testing at all levels of the software lifecycle and goes beyond dynamic testing (e.g., system and acceptance) to include static testing techniques as well. The model is comprehensive and is intended to support testing in bothsystems and software engineering. Furthermore, TMMi addresses the fourcornerstonesof structured testing:lifecycle,techniques,infrastructure, andorganization. These are essential to ensure a thorough and structured approach to improving test processes and aligning them with business goals.

TMMi References:

TMMi documentation clearly indicates that it coversall test levelsincluding static testing (such as peer reviews) and dynamic testing across different stages of development​​.

It is lifecycle-independent and can support different models, such as Agile, DevOps, and traditional V-models​.

The statement thatSpecific Goal 1of theTest Measurement (4.1)process area covers the communication of test measurement results after analysis isFALSE. Specific Goal 1 focuses on aligningtest measurement and analysis activities, which involves setting measurement objectives, specifying the measures to be taken, and establishing procedures for data collection, storage, and analysis. However, the communication of results comes underSpecific Goal 2, which deals with the analysis and reporting of test measurement results.

TMMi References:

Test MeasurementatTMMi Level 4includes defining measures, data collection, and storage as part of aligning test measurement and analysis activities, while communicating results is part of a different goal​​.

TheDiagnosing phasein the IDEAL test improvement cycle is focused on understanding the current state of the organization's testing processes and identifying the areas for improvement. One of the key activities during this phase isDevelop Recommendations, which involves analysing the current process against the goals and proposing areas where improvements can be made to achieve the desired state. Other activities in this phase include characterizing the current and desired states .

TheDebugging-oriented phasefrom Gelperin and Hetzel's evolutionary testing model is associated withMaturity Level 1 "Initial"of TMMi. At this maturity level, testing is not yet a distinct and well-defined process but is often interleaved with debugging. Organizations at this level have chaotic and ad-hoc testing activities, with no separation between testing and debugging.

Maturity Level 1organizations lack formal processes, and testing is often reactive, performed alongside fixing bugs in the code. This is in line with the debugging-oriented phase, where testing and debugging are seen as part of the same activity.

TMMi References:

TheTMMi frameworkdraws directly from the evolutionary model of Gelperin and Hetzel, associating theDebugging-oriented phasewith Level 1, which describes a chaotic, unmanaged testing environment​​.

In theInitiating phaseof a test improvement cycle, one of the key activities is toCharter Infrastructure. This activity involves setting up the structure needed to support the improvement effort, such as establishing a project board and defining responsibilities. This phase focuses on creating the foundational support for the test improvement initiative.

An indirect benefit of a test improvement programme is improved staff motivation. While direct benefits, such as shortened lead time or higher defect detection rates, are focused on measurable improvements, indirect benefits include enhanced morale and engagement of the staff involved. A well-structured test improvement programme can lead to better recognition of the testing function within the organisation, leading to increased satisfaction and motivation among testing professionals.

At TMMi Level 5, theQuality Controlprocess area becomes relevant as it focuses on statistically managing and controlling the test process. Since the organisation is already at TMMi Level 4, moving to Level 5 involves ensuring that the test process is fully predictable, measured, and statistically controlled.Quality Control is a key process area in achieving TMMi Level 5 compliance, alongside other areas such as Defect Prevention and Test Process Optimization.

Defect Preventionis aTMMi Level 5process area. This process area focuses on identifying the root causes of defects and implementing preventive measures to avoid their recurrence. At TMMi Level 5, the goal is to optimize processes and move from defect detection to defect prevention.

Advanced ReviewsandTest Measurementare Level 4 process areas, whileOrganizational Innovation and Deploymentis part of CMMI, not TMMi.

TMMi References:

Defect Preventionis a key process area inTMMi Level 5, focusing on continuous process improvement and defect avoidance​​.

In the TMMi framework, aSpecific Goalis unique to a particular process area and represents the essential outcomes that must be achieved for that process area. In contrast, aGeneric Goalapplies across multiple process areas and ensures that the processes are institutionalised, managed, and aligned with broader organisational objectives. Generic Goals help maintain consistency and effectiveness of processes across the organisation, while Specific Goals focus on the particular needs of an individual process area.

In theLearning phaseof the IDEAL model, one of the critical activities is"Analyse and Validate". This involves reviewing the entire improvement process to determine whether the intended goals were achieved and identifying what worked well and what could be improved. The objective is to learn from the experience to enhance future improvement initiatives and continually refine the ability to implement change​​.

At TMMi Level 3, the organisation's test processes become more structured, managed, and standardised. It requires defining and implementing a consistent set of test processes across projects and organisational units. One key factor for achieving TMMi Level 3 is the integration of process asset development.

"Process Asset Development" is directly related to establishing organisational test process assets, which is a critical element at TMMi Level 3. This practice area includes developing and maintaining organisational process assets, which encompass the standard processes that will be used across various projects. These assets are required to ensure that testing is a well-defined, repeatable, and efficient process throughout the organisation.

According to the TMMi Framework 1.3, process asset development supports the implementation of defined and repeatable processes and helps with creating guidelines for tailoring processes across the organisation. It provides the necessary framework for consistency in test planning, execution, and reporting, which are key at this maturity level​.

This practice aligns with the goals of TMMi Level 3, where the focus is on defining, standardising, and integrating the test lifecycle within the overall development lifecycle, thus ensuring that test processes are well-established across the organisation. Hence, Process Asset Development (Option B) is critical for the organisation's progression to TMMi Level 3.

In the scenario described, where an organisation suffers from poor test estimations despite many practices being in place, the process area that requires attention is Test Planning. Specifically,SG 3 (Establish Test Estimates)focuses on developing well-founded estimates for testing effort and cost. This includes creating a top-level work breakdown structure, defining the test lifecycle, and determining estimates for the test effort and costs associated with different test activities. By addressing these practices, the organisation can improve its ability to generate accurate test estimations, leading to better project planning and resource allocation.

In an Agile context, the emphasis is typically placed on lightweight, flexible documentation. Test documentation should be "just enough" to support the test process, and excessive documentation may be seen as unnecessary overhead. Therefore, calling formore rigorous and thorough documentationisnotin line with Agile principles. Instead, Agile favours working software and collaboration over comprehensive documentation, supported by mentoring and just-in-time guidance rather than formalised documentation​.

Planning pokeris an Agile technique commonly used to estimate effort, and it can be adapted for use in theTest Planningprocess area to help estimate test efforts (SG3: Establish Test Estimates). This collaborative technique brings stakeholders together to estimate the effort required for tasks by assigning numerical values in a consensus-driven manner. It is particularly effective in Agile environments where estimation is iterative and frequently updated based on evolving project needs​.

The generic assessment process in TMMi follows a structured sequence. The correct order for the assessment phases is:

Planning phase: This involves setting the scope, goals, and strategy for the assessment. It includes assembling the assessment team, setting timelines, and defining objectives.

Preparation phase: In this phase, the necessary preparation for the assessment is done, including gathering initial information, documents, and materials. The assessment team familiarizes itself with the organization’s processes.

Interview phase: This phase includes conducting interviews with key stakeholders and collecting direct evidence about the current processes.

Reporting phase: This is the final phase where the results of the assessment are compiled into a report, including findings, recommendations, and a roadmap for improvements.

Hence, the correct sequence isPlanning, Preparation, Interview, Reportingwhich matchesoption B (2, 3, 1, 4).

Each statement about TMMi level 2 in an Agile context requires careful consideration within the framework:

(a): False. While Agile projects indeed use metrics like Velocity, TMMi requires tracking end-results such as escaped defects. Both team performance and outcomes, including defect-related metrics, aresignificant. Test performance indicators at TMMi level 2 in an Agile context should not solely focus on team performance metrics .

(b): True. The product risk assessment process in Agile projects typically adopts a more lightweight approach compared to traditional sequential lifecycle models. Agile teams often conduct risk assessments in a more collaborative and iterative manner .

(c): False. In Agile, monitoring and controlling processes are still critical to ensure goals and quality targets are met, even if the methodology allows for flexibility and adaptation .

(d): False. Traceability is still relevant in Agile, although it may be implemented differently. Agile methodologies such as Scrum do not eliminate the need for traceability between requirements, test conditions, and tests, which is an essential part of TMMi process areas .

(e): True. Agile allows for flexibility in defining the test environment closer to the execution phase, enabling changes that reflect evolving requirements and design decisions .

Thus, the correct combination isA: Statements (a) and (e) are true, while statements (b), (c), and (d) are false​ .

CMMI version 2 recognises the importance of testing in software development and includes two dedicated practice areas:VerificationandValidation. These areas specifically addressthe processes required to ensure that a system or component meets its specified requirements and that the final product is fit for its intended use. While CMMI does not provide a testing-specific maturity model, these practice areas contribute to the overall quality management framework, and organisations adopting CMMI can leverage these processes to improve their testing practices. The TMMi framework complements CMMI by providing more detailed guidelines for testing.

TheActing phaseof a test improvement cycle focuses on executing the planned improvements. Key activities during this phase includecreating solutionsto address identified problems,planning actionsto implement those solutions, andpiloting solutionsbefore full-scale deployment. Thesesteps ensure that the improvements are well-defined, tested in a controlled environment, and refined before being fully implemented across the organisation.

The TMMi framework is applicable to all levels of testing, including static testing (such as reviews) and dynamic testing, regardless of the phase of the software development lifecycle. TMMi encompasses structured testing activities across various test levels such as component testing, integration testing, system testing, and user acceptance testing, along with both staticand dynamic techniques. Therefore, all levels of testing can be improved using the TMMi model.

The specific goal during the implementation of the "Test Planning" process area at TMMi Level 2 is to establish well-founded plans for managing and performing the testing activities. One of the crucial objectives is to"Establish a Master Test Plan"at TMMi Level 3, particularly under the "Test Lifecycle and Integration" process area. This is designed to ensure that a coherent, high-level plan integrates the activities across all levels of testing.

The process includes defining a test approach based on identified risks and involves the creation of the test estimates, schedules, and plans. Test plans should align with organisational strategies and cover test activities for all relevant phases of the development cycle​​.

When implementing theTest Policy and Strategyprocess area in TMMi, one of the specific goals you would be targeting is toEstablish Test Performance Indicators. These indicators are essential for measuring the effectiveness of testing processes and for assessing whether the organization's testing goals, as defined by the test policy, are being met.

Other goals, such as performing product risk assessments or establishing a test organization, are related to different process areas in the TMMi model.

TMMi References:

TheTest Policy and Strategyprocess area in TMMi includes the specific goal of establishingTest Performance Indicatorsto measure and track testing performance​​.

TheTMMi process area Test Planningis indeed supported by severalCMMI process areas, includingProject Planning,Requirements Management, andRisk Management. These CMMI areas provide foundational support for ensuring that the test planning process incorporates adequate project management, requirement traceability, and risk identification, all of which are critical for effective testing.

Specific Goal 2 of the CMMI Validation process arearelates to validation activities, not peer reviews, and therefore does not support the TMMi Peer Reviews process area.

Causal Analysis and Resolutionis linked toDefect Preventionin TMMi, not Quality Control.

Performance Managementdoes not directly support the Non-Functional Testing process area in TMMi.

TMMi References:

TheTMMi Test Planningprocess area is supported by relevant CMMI process areas, such asProject Planning, which helps in aligning project goals with test planning activities​​.

In the context of TMMi, the focus of test process improvement is usually driven by business needs that aim to improve the efficiency, effectiveness, and predictability of the testing process. The typical business reasons for test improvement include:

Increase predictability of testing(Option B): This refers to making the testing process more reliable and consistent, allowing better planning and forecasting of testing activities.

Reduce the costs of failure by improving effectiveness of testing(Option C): This is a direct goal of test process improvement, as it helps catch defects earlier, reducing costs associated with defects in later stages of the lifecycle.

Reduce time to market by increasing efficiency of testing activities(Option D): By making testing more efficient, organizations can release products faster while maintaining or improving quality.

Option A,Introduce a new process improvement framework, isNOTa typical business reason for test improvement. This refers to the introduction of a framework, which could be part of a process improvement strategy but is not a direct business reason for improving testing. Test improvement efforts are generally focused on achieving tangible business benefits, such as cost reduction, risk mitigation, or faster delivery, rather than the implementation of a specific framework for its own sake.

TMMi References:

TMMi emphasizes business-driven goals for process improvement that align with reducing risks, enhancing quality, and improving testing efficiency and effectiveness​.

Required componentsdescribe what an organizationmustcomply with to satisfy a process area in the TMMi model. These components includespecific goalsandgeneric goals, and they form the basis for determining whether a process area has been successfully implemented during assessments.Expected componentsprovide guidance on how to achieve the required goals, but they are not mandatory.

TMMi References:

TMMi definesrequired componentsas those that organizations must achieve to satisfy a process area​​.

Test Process Optimizationis the TMMi Level 5 process area specifically aimed at identifying new testing technologies and transitioning them into the organisation. This process area focuses on continuously improving the test process by evaluating and adopting new tools, methods, and technologies that can enhance testing efficiency and effectiveness. Test Process Optimization encourages organisations to stay ahead of industry developments and integrate innovative testing practices.

The specific practice"Define product risk categories and parameters"is part of theTest Policy and Strategyprocess area in TMMi. This practice involves identifying and categorizing the product risks to guide testing efforts and establish a clear understanding of potential product issues. Product risk categories help in determining the focus areas for testing, ensuring that testing is prioritized based on business and technical risks.

TMMi References:

The practice of defining product risk categories is directly linked to theTest Policy and Strategyprocess area at TMMiLevel 2, where risk-based testing approaches are established​​.

Both Test Organization and Test Training Program are process areas associated with TMMi Level 3 (Defined). At this level, the organisation establishes a formal test organisation and ensures that there is a dedicated test training program. This helps to institutionalise testing as a profession and ensures the systematic development of skills needed for testing activities across the organisation.

A direct benefit of a test improvement program ishigher productivity. When test processes are streamlined, better test planning, design, and execution can lead to reduced testing times, fewer defects, and quicker turnaround of quality products. This has a direct impact on organisational efficiency, allowing for more effective use of resources and time.