Webinar Presentation by Cyber Security and Information Systems Information Analysis Center (CSIAC)
Welcome and Introduction
Presenter: Philip Payne, Technical Lead for CSIAC
CSIAC: Defense research partner in Cybersecurity and Information Systems within DoD
Operates under Defense Technical Information Center (DTIC)
Highlights emerging research for DoD
Aims to unlock information and foster collaboration to stimulate innovation
Administrative Notes:
Webinar slides available online
All participants are muted, can use chat for communication
Questions for Q&A to be submitted via Q&A window
Full presentation available online post-webinar
Presenters
Sarah Standard: Retired Navy Captain, Executive Director for Developmental Test Evaluation and Assessments at the Office of the Under Secretary of Defense for Research and Engineering
Nilo Thomas: BS Aerospace Engineering, software and cyber advisor for DOT&E
Primary Topics
Overview of DoD Test and Evaluation Policy and Guidance Updates
Cyber Test and Evaluation (T&E) Policy and Guidance Deep Dive
DoD Test and Evaluation Updates
DODI 5000.89 - DoD Test and Evaluation Documentation
New Publications:
Enterprise T&E Guidebook (2022)
Manually Consolidating memos into DoD Manuals (DoDM)
E.g., DoDM 5000.xC for Cyber T&E, software T&E, electronics management (EMSO), and modeling/simulation testing
Companion Guides - Introduction to T&E testing procedures
Integrated Decision Support Key (IDSK): Document for scheduling and resource allocation for testing to inform critical decisions
Cyber T&E Strategy
Building Cyber T&E Strategy within overall T&E Strategy using IDSK
Cyber Working Group responsible for updating the T&E strategy
Initial Component level testing to full System of Systems level
Types of Testing Incorporated:
Cyber OT&E, Cyber Live Fire, Cyber DT, Contractor T&E, Integrated Contractor Government T&E
Tracking data for decision making
Attack Surface Characterization - Detecting potential vulnerabilities and threat modeling
Ensuring results and observables are integrated into design and operational frameworks continuously (Iterative Process)
Cyber T&E Details
Key Points:
Critical to identify and mitigate engineering technical issues
Focus on measurable requirements for system capabilities (prevent, mitigate, recover, adapt)
Compliance verification and reporting needs integrated into contract requirements
Government Cyber DT - Evaluating systems against compliance standards and ensuring robust capabilities
Operational Cyber Testing and Evaluation (OT&E)
Conducting thorough evaluations under realistic conditions to assess sufficiency
Incorporating representative users, environments, and cyber defenders
Assess operational effects and inform stakeholders about system vulnerabilities and mission impacts
Cyber Live Fire Testing
Concept: Non-kinetic, close cooperation with live fire community to merge testing processes
Pilot projects and developing processes for rigorous system assessments under potential cyber threats
Part of ensuring full system checks including cyber survivability
Cyber T&E Reporting
Focusing on practical outputs for vulnerability exposure, operational impacts, and remediation actions
Enabling adaptive processes and generating decision-support data throughout system lifecycle
Conclusion
Reiterating iterative and data-driven approach for comprehensive cyber evaluations
Leveraging automation and integration for efficient testing processes
Continuous feedback loop in engineering, testing, and decision making
Promoting proactive measures such as integrated testing, realistic scenarios, and detailed reporting
Q&A
Addressing questions on policy, live fire cyber T&E, data-sharing challenges, and impact on operational exercises
Encouraging continued communication and clarity in guidance