Consideration of Reliability and Validity Concerns within Assessments of Small and Medium Size Manufacturers

Walden, C., & Greenwood, A. (2010). Consideration of Reliability and Validity Concerns within Assessments of Small and Medium Size Manufacturers. Proceedings of the 2010 Industrial Engineering Research Conference. Cancun, Mexico: Institute of Industrial Engineers.

Abstract

Practitioners have relied for years on the use of enterprise-wide manufacturing assessments (e.g., Shingo Prize and Malcolm Baldridge National Quality Award) to drive enterprise improvement. While these and other assessment approaches have been enormously helpful to practitioners, the efficacy of these types of assessments is highly dependent on the experience, expertise, and skill of the assessor. Therefore, it is important from a research perspective, to define concerns about reliability and validity and to incorporate these concerns into the assessment process. This paper defines reliability and validity within the context of the Taxonomy Based Assessment Methodology (TBAM); an emerging assessment methodology that focuses on the needs of small and medium size manufacturing enterprises (SMEs). A methodology is developed which yields a preliminary set of reliability and validity measures, based on an examination process that utilizes a Case Study-Review Panel approach. Results from this approach are presented and initial inferences are made regarding the reliability and validity measures based on TBAM case studies.