AMSAA Scorecards
AMSAA Reliability Scorecard
The AMSAA Reliability Scorecard examines a supplier's use of reliability best practices, as well as the supplier's planned and completed reliability tasks. The Scorecard is important for tracking the achievement of reliability requirements and rating the adequacy of the overall Reliability Program. An early Scorecard assessment may be based solely on a Reliability Program Plan, but as time progresses, the Scorecard assessment will become more accurate if information from technical interchange meetings, a Reliability Case, and results from early reliability tests, are included. The Reliability Case documents the supplier's understanding of the reliability requirements, the plan to achieve the requirements, and a regularly-updated analysis of progress towards meeting the requirements.
The Reliability Scorecard uses eight critical areas to evaluate a given program's reliability progress:
- Reliability requirements and planning
- Training and development
- Reliability analysis
- Reliability testing
- Supply chain management
- Failure tracking and reporting
- Verification and validation
- Reliability improvements
There are 40 separate elements among the eight categories in the AMSAA Reliability Scorecard. Each element within a category can be given a risk rating of high, medium, or low (red, yellow, or green) or not evaluated (gray). The Scorecard weights the elements, normalizes the scores to a 100-point scale, and calculates an overall program risk score and eight category risk scores.
Benefits include:
- Poor performers can be identified early
- Engineering and reliability program reviews are made consistent
- An improved mechanism to capture useful metrics is provided
AMSAA Software Reliability Scorecard
The AMSAA Software Reliability Scorecard extends and complements the general reliability scorecard by examining an individual software development effort and assessing the level of risk associated with the software reliability practices being applied. It also complements existing software approaches, such a CMMI, by examining reliability-specific practices within an individual software project.
The Software Scorecard provides a structured and transparent approach to software reliability and maintains a consistent design with the general reliability scorecard. A total of 57 specific elements are examined across seven key areas of software development and sustainment:
- Program Management
- Requirements Management
- Design Capabilities
- System Design
- Design for Reliability (DfR)
- Test and Acceptance
- Fielding and Sustainment
The scorecard provides a structured and transparent instrument for assessing the health of a software development effort in regards to software reliability and is useful in isolating areas for further analysis and work. The discussion and reflection that occurs as the instrument is applied enables multiple disciplines to see the value of good reliability issues and practices in a structured way. The scorecard is invaluable to uncovering areas of weakness so that technical resources can be best prioritized and, subsequently, more reliable software can be developed.
