All material on this web site is copyrighted by Honourcode, Inc. 1999-2018.

Applied Test & Evaluation

Test and evaluation (T&E) provide the means to ensure that a product performs in the intended way and with the intended results. Testing starts very early in a product system development, however, because test planning is dependent on good requirements.

This course teaches the details of test and evaluation from product concept through operations, with a view toward application in large system development. The course is highly practical, helping students to understand not only what must be done, but why it is necessary. Students learn about a stakeholder view of test and evaluation, the use of requirements as the primary staypoint for verification, planning a test program in terms of strategy, plans, and procedures, the statistical analysis of a test prior to performing the test, the differences between integration testing and final testing, and how to conduct tests at different phases of product/system development, production and support.

Register here to receive more information on our courses.

Product testing must cover every detail to ensure product success.

Practice the skills on a realistic product system. Design a test program for a typical product system. Build the robotic system. Implement the test program. Analyze the results.

The Robotic Test Challenge Exercise gives students the opportunity to practice the skills taught in the course from end to end during development and support of actual operating robots. The exercise happens in four segments throughout the course, encompassing a total of six hours in highly practical learning.

What you will learn

  • The roles of T&E in development and support
  • Nine basic principles of testing
  • A useful life-cycle model for T&E
  • How to analyze product/system requirements into test requirements that satisfy stakeholders
  • T&E development: strategy, plan, procedures
  • Test design and analysis; statistical methods
  • System integration testing and how it differs from product/system final testing
  • Test methods, test control, test results

The course is aimed at

  • Test Engineers
  • Design Engineers (any engineering discipline)
  • Systems Engineers, Project Engineers; Technical Team Leaders
  • System Support Leaders
  • Technical and Management Staff
  • Project Managers

Columbia, MD - 5-7 Nov 18

Discount Policies

Click here to
Register Now

Course Topics

Test & Evaluation Overview (1:30) – An overview of test and evaluation (T&E) principles and methods for simple products and complex systems, including T&E tasks from beginning to end of a project. Basic definitions and concepts for the course, including the differences in test, evaluation, verification, validation, developmental testing (DT&E or “alpha”), and operational testing (OT&E or “beta”). Test and evaluation relationship to other engineering and management disciplines. Cost-effective T&E, and the cost of quality. Nine basic principles of testing.

Test and Evaluation In the Life Cycle (1:30) – Roles of test and evaluation throughout product development and support. A useful model of typical test activity phases (requirements, planning, conduct, analysis & reporting) and their correlation with program phases. How test concepts fit into special life cycles: commercial product development, US Department of Defense acquisitions under DoD-5000, evolutionary development, iterative development, and agile development.

Developing Test Requirements (2:30) – Requirements as the primary method for measurement and control of product development. Where requirements come from in the systems engineering paradigm. Types of requirements, including traditional, model-based and agile requirements. Stakeholder values in the form of Measures of Effectiveness (MOE) and Measures of Performance (MOP). How to develop a requirements verification matrix (RVM); verification methods (Inspection, Analysis, Demonstration, Test). Test requirements differences for prototypes, first articles, production and support.

Designing a Test and Evaluation Program (2:30) – Understanding stakeholder needs for T&E information Creating a T&E strategy by evaluating the product/system development concept in context of the stakeholder needs. An effective outline of T&E strategy topics, including planning for developmental and operational testing phases. An example: the US DoD “Test and Evaluation Master Plan (TEMP)” Trade-offs involved in test decisions; maturity at different phases; level of detail. What is a “verification event”? Converting the strategy to a T&E plan defining specific verification events in terms of requirements tested, time frame, equipment/skills needed, duration, and goals. Identification of test enabling products early enough to affect the development program. Modeling and simulation for test planning. Risk management as a test planning tool.

Designing Tests and Evaluations (3:30) – The test procedure as a control for each verification event. Effective topics to include in a test procedure, as an outline for developing the test. Identifying the issues and goals in each verification event. Determining the requirements to include, and what not to include. “Black box” input/output analysis, choosing what to measure, and identification of observability issues. Affecting the product/system design to improve testability. Logical sequencing of the test procedure based on product/system states, input controls, and observable measurements. Review of probability and statistics principles including probability distributions. Analyzing expected variation in test data, statistical design of tests, sampling principles, selecting useful statistical methods, design of experiments, common statistical errors.

Integration Testing (1:00) – How to successfully manage the intricate aspects of system integration testing; level of integration planning; managing complex system integration; work-arounds. Development test concepts; five types of integration test planning; preferred order of events; component testing; conducting integration tests for complex systems; work-arounds for anomalies and failures.

Test Conduct (2:00) – How to perform testing; differences in testing for prototypes, first article qualification, recurring production acceptance, support; rules for test conduct. Testing for different purposes, verification vs. validation; test records; prerequisites management; test readiness certification, test constraints, test article configuration; troubleshooting and anomaly handling; measures of success and indicators of difficulty; test tools. Test failure analysis.

Robotic Test Challenge (6:00) – A hands-on class exercise in small groups, handled in four major segments. Part A analyzes a system concept and requirements, developing an RVM and specific test requirements,. Part B creates an effective test program and test procedures for the product system, including identifying any necessary test equipment to be developed. Part C builds the robotic systems per assembly instructions, as well as building any identified test equipment. Part D implements the test program to evaluate the final robots.

Continuing Education: This course qualifies for 2.1 CEUs or 21 PDUs

Qualified Instructors for this course

Dr. Eric Honour, CSEP, INCOSE Fellow, and former INCOSE President, has been in international leadership of the engineering of systems for over 20 years, part of a 40+ year career of complex systems development and operation. His energetic and informative presentation style actively involves class participants. He was the founding Chair of the INCOSE Technical Board in 1994, and served as Director of the Systems Engineering Center of Excellence (SECOE). He was selected in 2000 for Who’s Who in Science and Technology and in 2004 as an INCOSE Founder. He is on the editorial board for Systems Engineering. He has been a successful entrepreneur, systems engineer, engineering manager, and program manager at Harris Information Systems, E-Systems Melpar, and Singer Link, preceded by nine years as a US Naval Officer flying P-3 aircraft. He has led or contributed to the development of 17 major systems.. Dr. Honour has a BSSE (Systems Engineering) from the US Naval Academy, MSEE from the Naval Postgraduate School, and PhD from the University of South Australia based on his ground-breaking work to quantify the value of systems engineering. Dr. Scott Workinger has led innovative technology development efforts in complex, risk-laden environments for 30 years in the fields of manufacturing (automotive, glass, optical fiber), engineering and construction (nuclear, pulp & paper), and information technology (expert systems, operations analysis, CAD, collaboration technology). He currently teaches courses on program management and engineering and consults on strategic management and technology issues. Scott has a B.S in Engineering Physics from Lehigh University, an M.S. in Systems Engineering from the University of Arizona, and a Ph.D. in Civil and Environment Engineering from Stanford University.

Page last modified 17 Jul 18