In the last post 'Measure eLearning Course Quality with Six Sigma', we
reviewed how Six Sigma DMAIC Measure phase concepts of process mapping, data gathering, and data summarization applies during eLearning course development process improvement initiative. In this post, we will learn the concepts of measurement system analysis, pattern analysis, pareto chart and process capability.

**Measurement System Analysis**

Issues
such as inaccuracy, bias, imprecision, and lack of reproducibility and
resolution lead to flawed measurement system. Accuracy, Repeatability,
Reproducibility, and Stability are desired measurement characteristics.

**Gauge R&R**

Gauge
R&R measurement system analysis technique uses an analysis of variance
random effects model to assess a measurement system’s reliability and makes
sure that the measurements are not the product of flawed measurement system. It
clarifies if variation in the measurements is due to flawed measurement tool or
due to inconsistent operation of measurement tool. Several factors that affect
the measurement system are measuring systems, operators, test methods,
specification, and parts of samples.

Two key aspects of Gauge R&R are
Repeatability and Reproducibility.

__Repeatability__refers to the variation in measurements taken by a single person or instrument on the same or replicate item and under the same conditions__Reproducibility__refers to the variation occurred when different operators, or instruments measure the same or repeat specimen.

Amongst accuracy and precision, Gauge
R&R deals with the precision of a measurement system.

**Gauge R&R Example**

**Pattern Analysis**

**Control Charts**

Control
chart, also known as process behavior chart is a statistical tool that gives a
graphical representation of process stability or instability over time. Process
stability, one of the most important concepts of any quality improvement
methodology is defined as a state in which a process has shown a certain amount
of consistency in the past and is expected to do so in the future. Control
charts are time-ordered plots of results that are used to statistically
determine if a manufacturing or business process is in a state of statistical
control. The process is in control when only expected variation (variation
resulted from common cause) is present. A process is out-of-control when
special cause variation exists. Control chart differentiates between process
variation resulting from common causes and process variation resulting from
special causes.

**Use a control chart:**

- To track performance over time
- To evaluate progress after process changes/improvements
- To focus attention on detecting and monitoring process variation over time
- To differentiate between special cause and common cause variation
- To achieve and maintain process stability

**How to construct and apply control chart:**

- Select the process to be charted
- Determine sampling method and plan
- Initiate the data collection
- Calculate the appropriate statistics

- UCL=CL+3*S
- LCL=CL-3*S
- These formulas
represent that the Upper Control Limit is 3 standard deviations above the mean and the Lower Control Limit is 3
standard deviations below themean.

- Plot the data values on the first chart (mean, median or individual)
- Evaluate the control chart and determine if the process is “in control”
- Determine whether the system is in control or out-of-control
- Identify special causes (the points above or below the control limit) Determine root causes

- 5 Whys
- Ishikawa or Fishbone Diagram

- Eliminate special cause variation
- Determine root causes

A Run
Chart, also known as a run-sequence plot is an easy way to graphically
summarize and track data trends or patterns. It allows you to analyze data
trends and patterns over a specified period of time. It highlights truly vital
changes in the process. We can observe peaks and drops on Run Chart indicating
variation in the process. Run Chart will help the team monitor
the performance of course development process over a period of time to
determine trends.

**How to construct a Run Chart:**

- Decide on process performance measure
- Gather data (minimum 20 to 25 data points to detect meaningful pattern)
- Create a graph
- Draw a vertical line (Y axis) – Scale related to your variable
- Draw a horizontal line (X axis) – Time or sequence scale

- Calculate median
- Draw a horizontal line at the median value
- Import the data into tool (use excel or any software)
- Ignore points on median
- Identify runs – A run is a series of points on the same side of the median.
- Identify important signals of special causes
- Too few or too many runs
- 6 or more points in a row continuously increasing or decreasing (a “trend”)
- 8 or more points in a row on the same side of the median (“shift”)
- 14 or more points in a row alternating up and down

The
Pareto chart is one of the most prevalent tools that are used to prioritize
quality improvement projects to obtain maximum returns for the resources
invested. A Pareto chart, is a type of chart that contains both bars and line
graph. The individual values are represented by bars and cumulative total is
represented by the line. The purpose of Pareto chart in quality control is to
highlight the most common sources of defects, significant aspects of the
problems, highest occurring types of defects, or most frequent reasons for
customer complaints etc. The Pareto principle, the law of vital few and trivial
many states that very few reasons actually cause most of the defects. This is a
graphical representation of 80-20 rule –showing how 80% of the problems are
caused by which 20% of the issues. This tool helps us analyze just that.

**How to construct a Pareto diagram:**

- Decide which item is to be studied
- Stratify the problem according to sources (by defects, by suppliers etc.) and tabulate the corresponding data
- Preferably the data should be expressed in monitory terms rather than quality or percentage
- Arrange the stratified items in descending order of value and draw a bar diagram
- Draw a curve showing the cumulative % above the bar chart starting from the greatest value

Pareto chart reflects the most important
aspect/defect, ratio of each defect to whole, degree of improvement after
corrective action in some limited area, and progress in each aspect/defect
compared before and after improvement. In the example below, pedagogical gaps,
poor instructional design and technical gaps variables have the greatest
cumulative effect on the eLearning course quality. The improvement of these
variables will yield the greatest benefit.

**Pareto Chart Example****Process Sigma**

Process
Sigma is a statistical representation of the level of quality for the process
or product that is being measured. It is a critical metric that helps
identification of improvement needs and ongoing evaluation of progress. To
measure the process sigma, eLearning course development team needs to define
and measure the opportunities and defects. Opportunity is the defect observable
by customers. An online course could have the opportunities such as,
pedagogical gaps, poor instruction design, technical gaps, complex
interactions, poor media quality, poor connectivity, poor engagement etc. A defect is anything that results in customer dissatisfaction. In order
to calculate yield, we need to subtract the total number of defects from the
total number of opportunities, dividing by the total number of opportunities
and then multiply the result by 100. The following formula is applied to
calculate Defects per Million Opportunities (DPMO).

Sigma conversion table converts Defects Per Million Opportunities to
sigma level. To achieve a
Process Sigma of 6, the process must not produce more than 3.4 defects per
million opportunities. Operating at a process Sigma level of 6 means that the
process is operating at 99.9997% perfection. The
higher the sigma capability, the better the process is performing. As Sigma capability increases, cycle time
reduces, cost reduces and customer satisfaction increases.
However, most companies operate at a 3 to 4 Process Sigma level. This
means these companies bear anywhere between 6210 to 66807 DPMO.

**Process Capability**

Process
capability is a measurable property of a process that indicates how “capable”
the process is to meet customer requirements. It compares process limits to
tolerance limits. Process capability can be articulated as shown below in terms
of 3 Sigma, 4 Sigma, 5 Sigma and 6 Sigma level.

One
more way to express process capability is in terms of the measure known as
Process Capability Ratio

*C*_{p}
In
case the process mean is not centered, C

_{p}overestimates process capability. In such scenario, Process Capability Index*C*_{pk}_{ }is applied.*C*_{pk}_{ }highlights what the process is capable of producing bearing in mind that the mean may not be centered between the specification limits.
USL (Upper Specification Limit) and
LSL (Lower Specification Limit) are set by clients, organization or business
requirements and describe what the team wants a process to achieve. Whereas,
UCL (Upper Control Limit) and LCL (Lower Control Limit) are calculated from the
data and describe what the process is capable of achieving.

**Available information on Measure Phase (Hyperlinks)**

There is a handful lot of other tools and concepts linked with
Measurement phase. I have elaborated few of them in this and previous posts. Share
your views and experiences about how a robust Measurement System leads to a
successful Six Sigma initiative.