Skip to content

Data/AI · Work role code 672

AI Test & Evaluation Specialist

Performs testing, evaluation, verification, and validation on AI solutions to ensure they are developed to be and remain robust, resilient, responsible, secure, and trustworthy; and communicates results and concerns to leadership.

From DoD DCWF Tool v5.3 (2026-09-25)

Tasks

From DoD DCWF Tool v5.3 (2026-09-25), in the tool's order, with the DCWF KSAT id

Free · DoD DCWF Tool v5.3 (2026-09-25)
  • 508Determine level of assurance of developed capabilities based on test results.Core
  • 550Develop test plans to address specifications and requirements.Core
  • 694Make recommendations based on test results.Core
  • 765BPerform AI architecture security reviews, identify gaps, and develop a risk management plan to address issues.Additional
  • 858ATest, evaluate, and verify hardware and/or software to determine compliance with defined specifications and requirements.Core
  • 858BRecord and manage test data.Core
  • 5120Conduct hypothesis testing using statistical processes.Core
  • 5848Assess technical risks and limitations of planned tests on AI systems.Core
  • 5850Assist integrated project teams to identify, curate, and manage data.Additional
  • 5851Build assurance cases for AI systems that support the needs of different stakeholders (e.g., acquisition community, commanders, and operators).Core
  • 5858Conduct AI risk assessments to ensure models and/or other solutions are performing as designed.Core
  • 5866Create or customize existing Test and Evaluation Master Plans (TEMPs) for AI systems.Core
  • 5873Determine methods and metrics for quantitative and qualitative measurement of AI risks so that sensitivity, specificity, likelihood, confidence levels, and other metrics are identified, documented, and applied.Core
  • 5876Develop machine learning code testing and validation procedures.Core
  • 5877Develop possible solutions for technical risks and limitations of planned tests on AI solutions.Core
  • 5889Identify and submit exemplary AI use cases, best practices, failure modes, and risk mitigation strategies, including after-action reports.Additional
  • 5896Maintain current knowledge of advancements in DoD AI Ethical Principles and Responsible AI.Core
  • 5901Measure the effectiveness, security, robustness, and trustworthiness of AI tools.Core
  • 5910Provide quality assurance of AI products throughout their lifecycle.Core
  • 5914Report test and evaluation deficiencies and possible solutions to appropriate personnel.Core
  • 5916Select and use the appropriate models and prediction methods for evaluating AI performance.Core
  • 5919Test AI tools against adversarial attacks in operationally realistic environments.Core
  • 5920Test components to ensure they work as intended in a variety of scenarios for all aspects of the AI application.Core
  • 5921Test how users interact with AI solutions.Core
  • 5922Test the reliability, functionality, security, and compatibility of AI tools within systems.Core
  • 5923Test the trustworthiness of AI solutions.Core
  • 5926Use models and other methods for evaluating AI performance.Core

Knowledge

From DoD DCWF Tool v5.3 (2026-09-25), in the tool's order, with the DCWF KSAT id

Free · DoD DCWF Tool v5.3 (2026-09-25)
  • 22* Knowledge of computer networking concepts and protocols, and network security methodologies.Core
  • 40Knowledge of organization's evaluation and validation requirements.Additional
  • 108* Knowledge of risk management processes (e.g., methods for assessing and mitigating risk).Core
  • 942Knowledge of the organization's core business/mission processes.Additional
  • 1133Knowledge of service management concepts for networks and related standards (e.g., Information Technology Infrastructure Library, current version [ITIL]).Additional
  • 1157* Knowledge of national and international laws, regulations, policies, and ethics as they relate to cybersecurity.Core
  • 1158* Knowledge of cybersecurity principles.Core
  • 1159* Knowledge of cyber threats and vulnerabilities.Core
  • 6311Knowledge of machine learning theory and principles.Core
  • 6900* Knowledge of specific operational impacts of cybersecurity lapses.Core
  • 6935* Knowledge of cloud computing service models Software as Service (SaaS), Infrastructure as a Service (IaaS), and Platform as a Service (PaaS).Core
  • 6938* Knowledge of cloud computing deployment models in private, public, and hybrid environment and the difference between on-premises and off-premises environments.Core
  • 7003Knowledge of AI security risks, threats, and vulnerabilities and potential risk mitigation solutions.Core
  • 7004Knowledge of AI Test & Evaluation frameworks.Core
  • 7006Knowledge of best practices from industry and academia in test design activities for verification and validation of AI and machine learning systems.Core
  • 7009Knowledge of coding and scripting in languages that support AI development and use.Core
  • 7012Knowledge of current test standards and safety standards that are applicable to AI (e.g., MIL-STD 882E, DO-178C, ISO26262).Additional
  • 7020Knowledge of DoD AI Ethical Principles (e.g., responsible, equitable, traceable, reliable, and governable).Core
  • 7024Knowledge of how AI is developed and operated.Core
  • 7025Knowledge of how AI solutions integrate with cloud or other IT infrastructure.Core
  • 7028Knowledge of how to automate development, testing, security, and deployment of AI/machine learning-enabled software.Core
  • 7029Knowledge of how to collect, store, and monitor data.Core
  • 7030Knowledge of how to deploy test infrastructures with AI systems.Core
  • 7034Knowledge of interactions and integration of DataOps, MLOps, and DevSecOps in AI.Core
  • 7036Knowledge of laws, regulations, and policies related to AI, data security/privacy, and use of publicly procured data for government.Core
  • 7037Knowledge of machine learning operations (MLOps) processes and best practices.Core
  • 7038Knowledge of metrics to evaluate the effectiveness of machine learning models.Core
  • 7040Knowledge of Personal Health Information (PHI), Personally Identifiable Information (PII), and other data privacy and data reusability considerations for AI solutions.Additional
  • 7041Knowledge of remedies against unintended bias in AI solutions.Core
  • 7044Knowledge of testing, evaluation, validation, and verification (T&E, V&V) tools and procedures to ensure systems are working as intended.Core
  • 7045Knowledge of the AI lifecycle.Core
  • 7048Knowledge of the benefits and limitations of AI capabilities.Core
  • 7051Knowledge of the possible impacts of machine learning blind spots and edge cases.Core
  • 7053Knowledge of the user experience (e.g., decision-making, user design, and human-computer interaction) as it relates to AI systems.Core
  • 7054Knowledge of tools for testing the robustness and resilience of AI products and solutions.Core
  • 7111* Knowledge of Zero Trust Security Model Principles.Core

Skills

From DoD DCWF Tool v5.3 (2026-09-25), in the tool's order, with the DCWF KSAT id

Free · DoD DCWF Tool v5.3 (2026-09-25)
  • 182Skill in determining an appropriate level of test rigor for a given system.Core
  • 6490Skill in assessing the predictive power and subsequent generalizability of a model.Core
  • 6630Skill in preparing Test & Evaluation reports.Core
  • 6641Skill in providing Test & Evaluation resource estimate.Core
  • 7065Skill in explaining AI concepts and terminology.Core
  • 7067Skill in identifying low-probability, high-impact risks in machine learning training data sets.Core
  • 7069Skill in identifying risk over the lifespan of an AI solution.Core
  • 7070Skill in integrating AI Test & Evaluation frameworks into test strategies for specific projects.Core
  • 7075Skill in testing and evaluating machine learning algorithms and/or AI solutions.Core
  • 7076Skill in testing for bias in data sets and AI system outputs as well as determining historically or often underrepresented and marginalized groups are properly represented in the training, testing, and validation data sets and AI system outputs.Core
  • 7077Skill in translating operation requirements for AI systems into testing requirements.Core

Abilities

From DoD DCWF Tool v5.3 (2026-09-25), in the tool's order, with the DCWF KSAT id

Free · DoD DCWF Tool v5.3 (2026-09-25)
  • 6060Ability to collect, verify, and validate test data.Core
  • 6170Ability to translate data and test results into evaluative conclusions.Core

Workforce Automation analysis, not part of the source

O*NET jobs

WFA · Part of the paid plan when accounts open

Jobs the ETO-CSET NICE/O*NET crosswalk (2024) maps to this role’s NICE equivalents, with the share of each job’s detailed work activities (DWAs) that has a mapped AI skill.

No O*NET jobs reach this role.

    Equivalents

    NICE work roles

    From DoD DCWF Tool v5.3 (2026-09-25), linked from published ids only, never from titles

    Free · DoD DCWF Tool v5.3 (2026-09-25)

    No published NICE equivalent.

    O*NET jobs it reaches

    Through its NICE equivalents, by the ETO-CSET NICE/O*NET crosswalk (2024); jobs from O*NET 30.3

    Free · O*NET 30.3

    No O*NET job reaches this role.