|
Background
The purpose of the case study using Harrison InnerView was to determine the ability of the Harrison InnerView software to predict success for the “Pickers and Packers” position. The company provided 91 profiles of current employees. Nineteen profiles were eliminated due to very low consistency scores (less than –110). The low consistency score indicates that the individual either attempted to fool the questionnaire or tried to answer the questions too quickly. Therefore, the total sample population of valid profiles was 72.
All the employees in the sample were rated according to their job performance by supervisors. There were 4 equally weighted criteria used for evaluation: Productivity, Quality, Safety Regulation Compliance, and Attendance.
Performance scoring
Each employee in the sample was rated on each criterion according to the following scale:
Productivity
| 90-100 |
25% or more above standard |
| 80-89 |
20% above standard |
| 70-79 |
15% above standard |
| 60-69 |
10% above standard |
| 50-59 |
Standard |
| 40-49 |
10% below standard |
| 30-39 |
15% below standard |
| 20-29 |
20% below standard |
| 10-20 |
25% below standard |
Quality
| 90-100 |
25% or more above standard |
| 80-89 |
20% above standard |
| 70-79 |
15% above standard |
| 60-69 |
10% above standard |
| 50-59 |
Standard |
| 40-49 |
10% below standard |
| 30-39 |
15% below standard |
| 20-29 |
20% below standard |
| 10-20 |
25% below standard |
Safety
| 90-100 |
No Accidents/Participate in safety committee |
| 80-89 |
No Accidents/No safety violations |
| 70-79 |
No Accidents/One minor safety violation |
| 60-69 |
No Accidents/Two minor safety violations |
| 50-59 |
One Accident/One significant safety violation |
| 40-49 |
One Accident/Two significant safety violations |
| 30-39 |
Two Accidents |
| 20-29 |
Two Accidents plus safety violations |
| 10-20 |
More than two Accidents |
Attendance
| 90-100 |
Perfect |
| 80-89 |
One absence |
| 70-79 |
Two absences |
| 60-69 |
Three absences |
| 50-59 |
Four absences |
| 40-49 |
Six absences |
| 30-39 |
Eight absences |
| 20-29 |
Nine absences |
| 10-20 |
Ten absences |
The total performance score was derived by averaging these four scores.
Since this scale (0-100 is different than the scale related to the suitability predictions in the Harrison InnerView software, it was necessary to adjust these performance scores to match the Harrison InnerView scale. The Harrison InnerView scale is as follows:
| 90-100 |
Excellent to Best |
| 80-89 |
Very Good |
| 70-79 |
Average to Good |
| 60-69 |
Below average |
| 50-59 |
Poor |
To convert the original performance rating into the final performance rating, the following formula was used:
OPR(Original Performance Rating) /2 + 50 = Final Performance Rating
This Study
The aim of the research was to determine the exact traits that make Pickers and Packers successful at the company and to determine the relative accuracy of the InnerView system to predict success based upon the “job template” that incorporates those traits. The template formulates those traits into “traits to have” as well as “traits to avoid”. The template is then used to measure future applicants and serve as a developmental guide for current employees. The template is shown by a graph that indicates how a person scores against each of the required traits and then offers a final “bottom line” score between zero and one-hundred that represents the individual’s level of “total suitability” for a particular position. A score of one hundred represents a person who is completely suitable for that position. Assuming the person is eligible for a position (has the education, experience and technical skills), a suitability score of 75 or greater represents a person who has a good probability of performing effectively in that position. A score of 74 or less represents a person who is considered to be unlikely to perform well in that position.
Accuracy level
The results showed a very high predictive accuracy and a strong correlation between the Harrison InnerView suitability score and the actual job performance. This indicates that the template includes a comprehensive set of traits related to suitability for this position.
The results were analyzed in two ways: the predictive accuracy and the variation between the suitability scores and the performance ratings:
1) Predictive accuracy - A prediction is considered accurate if one or more of the following conditions are met:
a) The suitability score is 75 or greater and the performance score is 75 or greater
b) The suitability score is less than 75 and the performance score is less than 75.
c) The suitability score is within 6 points of the performance score.
The logic behind this definition - If the suitability score were 75 or above and the person were eligible for the position, it would indicate a prediction that the person would probably succeed in the position. If the suitability score were less than 75 and the person was moderately eligible (not highly eligible) for the position then this would indicate a prediction of below average performance. Also, if the suitability score were within 6 points of the performance score it would indicate a very close prediction and thus should also be considered accurate. The suitability score for each person appears on the template in the far right-hand column of the table.
From the research sample, 66 of the 72 employees (92%) showed a correlation between the InnerView suitability score and performance to be accurate, according to the definition above.
2) Variation between the suitability ratings and the performance ratings
The degree of variation between the suitability score and the performance ratings also provides an effective means of understanding the accuracy level of the methodology. The variation was examined in two ways: the average variation between the predictive suitability rating and the performance rating and the percentage of predictions that fall within different ranges of closeness.
The average variation between the suitability score and the performance rating was 7.4 points. This indicates a close relationship between the suitability score and the performance ratings.
Next we examined the percentage of employees whose two scores (performance rating and predicted suitability score) were within 5 points, 8 points, 10 points and 15 points. The table below shows these percentages for each group.
Variation between performance rating and predictive suitability score
| Within 5 points |
Within 8 points |
Within 10 points |
Within 15 points |
| 37% |
51% |
61% |
89% |
If the two scores were within five points, the prediction would be considered to be extremely accurate. If within 8 points, the prediction would be considered to be quite accurate. If within 10 points, the prediction should still be considered to be reasonably accurate. If the two scores are greater than 10 points apart, but within 15 points, it could not be considered to be accurate, but at the same time could not be considered to be very inaccurate. If the two scores are greater than 15 points apart, they should be considered to be inaccurate.
Traits required for success
The results show that the template developed is able to predict job success with a high level of accuracy. The sample size was sufficient to assume reliable results for future applicants.
The research indicates that the success factors for this position include:
A tolerance of less sleep than most people
The willingness to take initiative (this relates to performance, quality and attendance)
Being energetic
Enjoys working with one’s hands
The tolerance of standing (also related to physical/manual work)
The tolerance of pressure of deadlines
The willingness to do physical work
The tolerance of structure (including safety regulations)
The willingness to do work that requires a fast pace
The willingness to be helpful
The tendency to be optimistic
The research indicates that there are also factors that will hinder success in this position. These include:
A severe lack of enthusiasm (or apathy)
The tendency to be harsh or punitive
The desire to avoid responsibility
Having strong desires to earn high pay without having the necessary self-motivation
The tendency to be defensive or have difficulty receiving corrective feedback
A severe lack of diplomacy
A severe lack of precision (poor attention to quality)
A lack of interest in self-improvement (little desire to get better at what one does)
The tendency to be extremely reserved or withdraw from social interaction
There is one additional trait related to success:
The willingness to communicate (good communication)
These traits as formulated in the Harrison InnerView software will enable a future prediction level of success for the Pickers and Packers position of at least 80%.
| 
