Test & Evaluation Overview
(1:30) – An overview of test and evaluation (T&E) principles
and methods for simple products and complex systems, including T&E
tasks from beginning to end of a project. Basic definitions and
concepts for the course, including the differences in test, evaluation,
verification, validation, developmental testing (DT&E or “alpha”),
and operational testing (OT&E or “beta”). Test and
evaluation relationship to other engineering and management disciplines.
Cost-effective T&E, and the cost of quality. Nine basic principles
Test and Evaluation In the Life Cycle
(1:30) – Roles of test and evaluation throughout product
development and support. A useful model of typical test activity
phases (requirements, planning, conduct, analysis & reporting)
and their correlation with program phases. How test concepts fit
into special life cycles: commercial product development, US Department
of Defense acquisitions under DoD-5000, evolutionary development,
iterative development, and agile development.
Developing Test Requirements (2:30)
– Requirements as the primary method for measurement and control
of product development. Where requirements come from in the systems
engineering paradigm. Types of requirements, including traditional,
model-based and agile requirements. Stakeholder values in the form
of Measures of Effectiveness (MOE) and Measures of Performance (MOP).
How to develop a requirements verification matrix (RVM); verification
methods (Inspection, Analysis, Demonstration, Test). Test requirements
differences for prototypes, first articles, production and support.
Designing a Test and Evaluation Program
(2:30) – Understanding stakeholder needs for T&E information
Creating a T&E strategy by evaluating the product/system development
concept in context of the stakeholder needs. An effective outline
of T&E strategy topics, including planning for developmental
and operational testing phases. An example: the US DoD “Test
and Evaluation Master Plan (TEMP)” Trade-offs involved in
test decisions; maturity at different phases; level of detail. What
is a “verification event”? Converting the strategy to
a T&E plan defining specific verification events in terms of
requirements tested, time frame, equipment/skills needed, duration,
and goals. Identification of test enabling products early enough
to affect the development program. Modeling and simulation for test
planning. Risk management as a test planning tool.
Designing Tests and Evaluations
(3:30) – The test procedure as a control for each verification
event. Effective topics to include in a test procedure, as an outline
for developing the test. Identifying the issues and goals in each
verification event. Determining the requirements to include, and
what not to include. “Black box” input/output analysis,
choosing what to measure, and identification of observability issues.
Affecting the product/system design to improve testability. Logical
sequencing of the test procedure based on product/system states,
input controls, and observable measurements. Review of probability
and statistics principles including probability distributions. Analyzing
expected variation in test data, statistical design of tests, sampling
principles, selecting useful statistical methods, design of experiments,
common statistical errors.
Integration Testing (1:00) –
How to successfully manage the intricate aspects of system integration
testing; level of integration planning; managing complex system
integration; work-arounds. Development test concepts; five types
of integration test planning; preferred order of events; component
testing; conducting integration tests for complex systems; work-arounds
for anomalies and failures.
Test Conduct (2:00) – How
to perform testing; differences in testing for prototypes, first
article qualification, recurring production acceptance, support;
rules for test conduct. Testing for different purposes, verification
vs. validation; test records; prerequisites management; test readiness
certification, test constraints, test article configuration; troubleshooting
and anomaly handling; measures of success and indicators of difficulty;
test tools. Test failure analysis.
Robotic Test Challenge (6:00) –
A hands-on class exercise in small groups, handled in four major
segments. Part A analyzes a system concept and requirements, developing
an RVM and specific test requirements,. Part B creates an effective
test program and test procedures for the product system, including
identifying any necessary test equipment to be developed. Part C
builds the robotic systems per assembly instructions, as well as
building any identified test equipment. Part D implements the test
program to evaluate the final robots.
Continuing Education: This course qualifies for 2.1 CEUs or 21