2.5.14 Quality Assurance 2.5.14.1 Program Scope and Objectives 2.5.14.1.1 Background 2.5.14.1.2 Authority 2.5.14.1.3 Roles and Responsibilities 2.5.14.1.4 Program Management and Review 2.5.14.1.5 Program Controls 2.5.14.1.6 Terms/Definitions/Acronyms 2.5.14.1.7 Related Resources 2.5.14.2 Introduction 2.5.14.3 Quality Assurance Auditing Process 2.5.14.3.1 Auditing Process Roles and Skills 2.5.14.3.2 Audit Process Control 2.5.14.3.3 Auditing Process Tasks and Flow 2.5.14.3.3.1 Plan to Audit 2.5.14.3.3.2 Perform the Audit 2.5.14.3.3.3 Manage Documents 2.5.14.3.3.4 Monitor and Control Audit Findings 2.5.14.3.4 Audit Process Management 2.5.14.3.5 Audit Process Review Part 2. Information Technology Chapter 5. Systems Development Section 14. Quality Assurance 2.5.14 Quality Assurance Manual Transmittal February 22, 2023 Purpose (1) This transmits revised IRM 2.5.14, Systems Development, Quality Assurance (QA). Material Changes (1) The following revisions were made to align with current Quality Assurance practices and incorporates the revisions made in the Interim Guidance Memo IT-02-04220004. They are as follows: IRM 2.5.14.3.1 - Auditing Process Roles and Skills - AD Quality Assurance Program Office (Quality Specialist) was changed to QA Program Manager, Domain Director was changed to QA Program Staff, Domain POC was changed to Project/Program Manager, Project Manager/Branch Chief was changed to Project/Program Team, Project Team (Quality Analyst) was Project/Program Team IRM 2.5.14.3.2 - Audit Process Control - Added 6 new Process Areas - Risk Management, Privacy, Section 508, Security, Quality Assurance, Process Management, Updated Process Audit to Process Area Audit and Changed Release Review Audit to Process Owner Audit. IRM 2.5.14.3.3.1 - Plan to Audit - Updated QA steps to align with current QA Practices. IRM 2.5.14.3.3.2 - Perform the Audit -Added The auditors coordinate their efforts to ensure consistency in auditing and The Audit Closing Meeting is conducted to communicate high level audit results, global findings, and next steps to the projects and programs. IRM 2.5.14.3.3.3 - Manage Documents -Updated tasks to align with current QA Practices. IRM 2.5.14.3.3.4 - Monitor and Control Audit Findings -Updated tasks to align with current QA Practices. IRM 2.5.14.3.4 - Audit Process Review - Removed Number 2 - “At least quarterly, Program Management will report to senior management on the overall status of task covered by this process.” Effect on Other Documents IRM 2.5.14, dated June 30, 2010, is superseded. This IRM incorporates the revisions made in Interim Guidance Memo IT-02-0422-0004, which were made to align with current QA practices. Audience The audience for this IRM is all Applications Development (AD) personnel responsible for the development and maintenance of Agency software systems identified in the Enterprise Architecture. This IRM applies to all QA activities conducted on projects, including contractor QA functions. It establishes the policy for conducting QA activities and the responsibilities and authority for performing QA across the Applications Development organization. Effective Date (02-22-2023) Nancy Seiger Chief Information Officer 2.5.14.1 (02-22-2023) Program Scope and Objectives Purpose The purpose of this IRM is to provide the framework for conducting Quality Assurance Audit activities within Applications Development (AD). It establishes a standard context for Project Teams, including contractors working for the Applications Development organization, to participate in the QA audit process. Audience The audience for this IRM is all Applications Development personnel responsible for the development and maintenance of Agency software systems identified in the Enterprise Architecture. This IRM applies to all QA activities conducted on projects, including contractor QA functions. It establishes the policy for conducting QA activities and the responsibilities and authority for performing QA across the Applications Development organization. Policy Owner The Associate, Chief Information Officer (ACIO), Application Development establishes all Information Technology (IT) internal controls for this IRM. Program Owner The Application Development Director, Delivery Management and Quality Assurance is the program owner. Primary Stakeholders This policy applies to all IT projects and programs including contractors. Program Goals The objective of this IRM is to establish the overall approach to Quality Assurance, apply Quality Assurance standards, report and control requirements for the QA program as outlined by AD QA Directive and related processes and procedures. 2.5.14.1.1 (02-22-2023) Background GAO/AIMD/GGD-98-54, February 1998, and Clinger-Cohen Act of 1996 2.5.14.1.2 (02-22-2023) Authority GAO/AIMD/GGD-98-54, February 1998, and Clinger-Cohen Act of 1996. 2.5.14.1.3 (02-22-2023) Roles and Responsibilities Below is a list of QA Roles and Responsibilities: QA Program Manager - Manages QA Program QA Program Staff - Serves as liaison from QA Program to project/program personnel and Process Owners Project/Program Manager - Manages Project quality Project/Program Team - Produces quality software products, systems, and documentation Process Owner - Owner of Specific Process Area (e.g., Configuration Management) Enterprise Life Cycle (ELC) Coach - Provides assistance with ELC matters to Project/Program Teams 2.5.14.1.4 (02-22-2023) Program Management and Review Program Reports Audit Report Non-Compliance Summary Report Corrective Action Plan (CAP) Heat Chart DMQA Weekly Status Report Audit Team Daily Status Report Program Effectiveness Measurements show how well products and processes conform to organizational and industry standards, they also indicate how well projects follow documented processes. The Quality Assurance Program reports on process metrics, derived from the number of non-compliances identified during process audits and the number of projects audited to produce process measures. To keep senior management abreast of project status and process performance metrics, the Quality Assurance Program Office will generate, retain, and report process metrics in order to demonstrate Program effectiveness. 2.5.14.1.5 (02-22-2023) Program Controls The following list represents the IRS controls, and mandates applicable to AD projects: IRM 2.5.1 Systems Development IRM 2.5.14 Quality Assurance (QA) IRM 2.16.1 Enterprise Life Cycle (ELC) IRM 2.21 Introduction to Shopping Cart Processing for IT IRM 2.22 Business Planning and Risk Management IRM 2.25 Managed Service for IRS IRM 2.100 Integrated Process Management IRM 2.110 Requirements Engineering IRM 2.120 Engineering IRM 2.125 Change Management IRM 2.126 Enterprise Organizational Readiness IRM 2.127 Testing Standards and Procedures IRM 2.144 Capacity Management IRM 2.149 IT Asset Management IRM 2.150 Configuration Management IRM 2.152 Data Engineering IRM 10.8.1 Information Technology Security Policy and Guidance Federal Information Security Management Act (FISMA) Privacy Act of 1974 Section 508 of the Rehabilitation Act of 1973 Risk, Issue, and Action Item Management Directive Other internal standards are located on the Process owners' SharePoint sites 2.5.14.1.6 (02-22-2023) Terms/Definitions/Acronyms The following tables are a list of terms and acronyms used throughout this IRM Section. Defined Terms Term Definition Applications Development An organization that supports IT components of a system that utilizes IT resources to store, process, retrieve or transmit data or information using IT hardware and software. Clinger-Cohen Act The Clinger-Cohen Act of 1996 (40 U.S.C. 1401(3)), also known as the Information Technology Management Reform Act, was intended, to "reform acquisition laws and information technology management of the Federal Government. Configuration Management Establish and maintain the integrity of work products using configuration identification, configuration control, configuration status accounting, and configuration audits Corrective Action Plan Changes made to bring expected future performance of the project in line with the project plan Daily Metrics Daily status of Quality Assurance Program Staff Enterprise Life Cycle A framework that provides a workflow for projects to follow to move an IT solution from concept to production while making sure that they are in compliance with IRS guidelines and are compatible with the overall goals of the IRS Heat Chart Visual representation of compliance data that uses colors to illustrate project and Domain compliance Internal Revenue Manual Official communications that designate authorities and/or disseminate instructions to staff for IRS officials and employees Lessons Learned Practices for evaluating past performance of activities Non-Compliance Finding and/or weaknesses usually found in a quality audit Peer Review Defect and Resolution Report Used to document Peer Reviews of project artifacts. Acronyms Acronym Definition AD Applications Development ACIO Associate, Chief Information Officer CAP Corrective Action Plan CM Configuration Management DID Data Item Descriptions ELC Enterprise Life Cycle ESC Executive Steering Committee FISMA Federal Information Security Management Act IRM Internal Revenue Manual PRIV Privacy PRM Process Management PP Project Planning PMC Project Monitoring and Control QA Quality Assurance RSKM Risk Management RQEN Requirements Engineering SEC Security SWDEV Software Development SME Subject Matter Expert SM Supplier Management TEST Testing U.S.C. United States Code 2.5.14.1.7 (02-22-2023) Related Resources QA evaluation of the project processes throughout the life cycle is based on the processes defined by the following supporting documents: AD Quality Assurance Directive AD Quality Management Plan AD Quality Assurance Plan AD Quality Assurance Program (processes, procedures, etc.) Internal Revenue Manual (IRM) 2.16 IRS Enterprise Life Cycle IRM 2.5 Systems Development 2.5.14.2 (02-22-2023) Introduction This IRM provides the framework for conducting Quality Assurance Audit activities within Applications Development. It establishes a standard context for Project Teams, including contractors working for the Applications Development organization, to participate in the QA audit process. This IRM establishes: the overall approach to Quality Assurance the applicable Quality Assurance standards the reporting and control requirements for the QA program as outlined by the AD QA Directive and related processes and procedures. The Quality Assurance (QA) Program Office supports the delivery of high-quality products and services by ensuring that projects implement a coordinated set of activities that conform to organizational policies, processes and procedures. Quality Assurance is a systematic, planned set of activities necessary to provide adequate confidence that the product conforms to stated customer requirements. The activities are designed to evaluate the processes (i.e. Project Planning, Project Monitoring and Control, Requirements Management, etc.) by which products are developed. 2.5.14.3 (02-22-2023) Quality Assurance Auditing Process QA Audit Process and Procedures are used to objectively and independently evaluate adherence of the process and work products to applicable directives, processes, standards, procedures, and guidelines. The objectives of the audit process are to: identify and track noncompliance instances communicate and facilitate the resolution of noncompliance issues identify and communicate, to senior management, best practices and opportunities for improvement document Quality Assurance activities; and report quality issues to relevant stakeholders Benefits of the audit process are realized through: consistency in assessing use of organizational processes facilitation of improvements enhanced planning and resource allocation capability 2.5.14.3.1 (02-22-2023) Auditing Process Roles and Skills To meet the objectives and realize the benefits of QA auditing, the following roles and skill sets are needed to perform QA auditing activities. Roles Responsibilities QA Program Manager Manages QA Program Develops, updates, and maintains quality standards and procedures (e.g., plans, processes, activities, templates, checklists, and guidelines) Approves or rejects changes to quality documents Assigns tasks to QA Program Staff Manages Audit Schedule Evaluates auditor feedback and workload Provides QA Program Staff with Audit Checklist Reviews, resolves, or escalates issues to senior program management, if necessary Responsible for the generation of Daily Metrics, Weekly Metrics, and Heat Charts Coordinates quality training based on feedback QA Program Staff Serves as liaison from QA Program to project/program personnel and Process Owners Functions as the Subject Matter Expert (SME) on QA program objectives and procedures Updates quality documents as assigned by QA Program Manager Attends necessary internal training Tailors Audit checklist based on current standards and audit/project specifics Ensures compliance with applicable IRS directives, processes, and organizational standards by conducting audits Reports on quality assessments and the audit findings to QA Program Manager Coordinates with other auditors to establish audit baseline. Coordinates with Project/Program Team on non-compliance items and corrective actions, if necessary Escalates to QA Program Manager, if necessary Receives and acts upon Daily Metrics Schedules a QA Services Team meeting Provides coaching, training, and mentoring to projects/programs Participates in Lessons Learned and continuous improvement activities Project/Program Manager Manages Project quality Commits to compliance with applicable IRS directives, processes, procedures, and organizational standards and their deliverables Has working knowledge of the QA Program objectives and standards; and required deliverables Reviews current QA Program standards and objectives to incorporate into Project Plan Performs project level self assessments Supports and participates in QA audits Receives audit results from the QA Program and responsible for follow-up action, as appropriate Develops strategy for resolution and ensures timely responses to deficiencies with a Corrective Action Plan (CAP) Ensures that quality management activities are scheduled, documented, and performed in accordance with Project Plan. Supports Peer Review activities by allocating personnel, facilities, and time for Peer Reviews Identifies trends in peer reviews and responsible for follow up action, as appropriate Tracks project progress Attends QA Services Team Meeting to discuss non-compliance and resolution Project/Program Team Produces quality software products, systems, and documentation Has working knowledge of QA objectives and standards Submits request to change existing Quality documents Executes quality related activities and produces deliverables Utilizes external Quality documents for support Participates and performs peer reviews, technical reviews, and quality assurance activities Performs independent/self-assessments of project/program artifacts and work products Conducts ELC and quality reviews within various stages of the project lifecycle Supports and participates in audits Reconciles Non-Compliance issues resulting from independent/self-assessments Participates in Lessons Learned and continuous improvement activities Attends quality related training and provides feedback Process Owner Owner of Specific Process Area (e.g., Configuration Management) Participates in Lessons Learned and continuous improvement activities Supports and participates in audits ELC Coach Provides assistance with ELC matters to Project/Program Teams Provides assistance and feedback to projects/programs on organizational standards and actions Collaborates with QA Program Staff to confirm non-compliance items Attends QA Services Team Meeting to discuss non-compliance and resolution 2.5.14.3.2 (02-22-2023) Audit Process Control The audit process is controlled and driven by the Audit Checklists. The checklists are questionnaires used to gather data for the processes and products being audited and to evaluate the project’s level of compliance. The checklists are tailored based on the project or type of audit conducted. The Quality Assurance Program conducts audits on organizational process areas as outlined in the AD Process Framework contained in the IT Process list below. The process areas are: Project Planning (PP) Project Monitoring and Control (PMC) Risk Management (RSKM) Requirements Engineering (RQEN) Software Development (SWDEV) Testing (TEST) Privacy (PRIV) Section 508 (508) Security (SEC) Configuration Management (CM) Supplier Management (SM) Quality Assurance (QA) Process Management (PRM) The Quality Assurance Program conducts three types of audits. The audit types are: Process Area Audit - Evaluates adherence to the standards and procedures of a process area (e.g., Configuration Management) or a group of process areas (e.g., Engineering) within a program or project. Work Product Audit - Reviews work products for conformance to the Enterprise Life Cycle (ELC), Data Item Descriptions (DID) and templates. Work Products can include any work products and deliverables, as well as the standards and/or procedures used to produce them. Process Owner Audit- Evaluates Process Owner policies and procedures based on standards and requirements. This type of audit is performed upon request from the Process Owner or their management. 2.5.14.3.3 (02-22-2023) Auditing Process Tasks and Flow The following tasks constitute the flow of the QA auditing process: Plan to Audit Perform the Audit Manage Documents Manage and Control Audit Findings 2.5.14.3.3.1 (02-22-2023) Plan to Audit The QA Program Manager and QA Program Staff perform Audit Planning activities in conjunction with the overall planning activities. Project/Program Managers and Project/Program Teams support audit planning activities. The following steps describe the audit planning activities: Establish QA Program goals using inputs from previous audits Plan and develop annual QA Audit Plan including audit calendar Plan and develop QA Audit Schedule for each audit Assign projects/programs to auditors Tailor Audit Checklist based on type of audit Coordinate among auditors to ensure consistency in auditing 2.5.14.3.3.2 (02-22-2023) Perform the Audit This task is performed in accordance with the Audit Schedule. The following steps occur during this task: Projects and programs are notified of the audit and provide access to the project/program repository (if required). The auditor reviews the Health Assessment and ReadMe File and makes a preliminary assessment of the quality status of the project/program. The Audit Opening Meeting is conducted to communicate the scope and the objectives of the audit to the projects and programs. The auditor, guided by the appropriate Audit Checklist, will evaluate the level of compliance with standards and processes as well as evaluate associated work products by reviewing instructions and procedures, checking records and through observation. If necessary, project/program staff will be consulted to address questions that arise during the checks and observations. The auditors coordinate their efforts to ensure consistency in auditing. The auditor documents the findings and confirms accuracy with other auditors and (if necessary) ELC Coaches. The Audit Closing Meeting is conducted to communicate high level audit results, global findings, and next steps to the projects and programs. The auditor issues to the auditee a final report of findings and stores it in the appropriate repository based on type of audit. The auditor conducts a QA Services Team meeting with the auditee to discuss audit findings and recommend potential corrective actions. A QA Services Team is not conducted if no deficiencies are found by the auditor. When audit findings require corrective action, a CAP must be submitted by the project/program. 2.5.14.3.3.3 (02-22-2023) Manage Documents This task occurs after an audit is conducted or may occur anytime a document is established, or content/information is gathered relating to the audit process. The following activities occur during this task: All documents generated and/or received as a result of an audit are collected and stored in soft and/or hard copy in the appropriate repository. Changes to QA process assets (i.e., processes, procedures, templates) shall be controlled in accordance with the Document Management Procedure located in the QA Program’s shared repository. As part of this task, when new QA documents are developed and/or requests have been received to modify existing documents, the following activities are executed: The QA Program Office reviews requested changes internally. The QA Program Office implements requested changes, updates repositories, and notifies users of changes to document (if approved). If the change is rejected, the QA Program Office informs the requestor that there will be no change. 2.5.14.3.3.4 (02-22-2023) Monitor and Control Audit Findings The auditor will use the information from the final report and any subsequent actions to update the QA Program database. Using this data, the auditor tracks audit findings to closure. If corrective actions are not completed by the resolution date as outlined in the approved CAP, the auditor escalates the unresolved findings to the QA Program Manager for resolution. This is not required for Process Owner audits. The auditor’s responsibilities are to: Verify resolution of the corrective action. Review updates to the QA Program Database to indicate the status of the audit’s corrective actions. Identify and escalate corrective actions that remain unresolved five days after planned resolution date. Perform trend analysis activities. Prepare audit data and status reports for review activities. Place reports in the appropriate repository. When all findings are resolved and verified, the auditor updates the data repositories, (i.e., shared drive, QA Program database). 2.5.14.3.4 (02-22-2023) Audit Process Management AD Quality Assurance Program Office will regularly maintain measurements on the status and progress of Quality Assurance tasks for the AD portfolio. Process trends shall be analyzed for efficiency and effectiveness. Data will be compiled to develop and report trends in performance and compliance. The reporting will occur through performance trends metrics and compliance metrics. Performance and compliance trends metrics will be reported: At the AD portfolio, domain, and project levels; and For all process areas (by AD portfolio, domain, project levels). 2.5.14.3.5 (02-22-2023) Audit Process Review The QA Program Management will report to Senior Management on the overall status of tasks covered by this process on a regular basis, but no less than annually. More Internal Revenue Manual
