Case Study for a Cable Company

Background

The purpose of the case study using Harrison InnerView was to determine the ability of the Harrison InnerView software to predict success at the cable company and to develop a template that could be used effectively for selecting and developing employees. The cable company provided eighty profiles of current employees. Five of the profiles were eliminated from the sample due to having a consistency score of less than zero. This indicates that the individuals either attempted to fool the questionnaire or tried to answer the questions too quickly. Thus the actual sample was 75.

All the employees in the sample were rated according to their job performance by supervisors.

Each employee in the sample was rated or each criteria according to the following scale:

90-100	Excellent to Best
80-89	Very Good
70-79	Average to Good
60-69	Below average
40-59	Poor
Below 40	Failed or will soon fail in the position

The aim of the research was to determine the exact traits that make Salespersons successful at the cable company. Another aim was to determine the relative accuracy of the Harrison InnerView system to predict success based upon the “job template” that incorporates those traits. The template formulates those traits into “traits to have” as well as “traits to avoid”. The template is then used to measure future applicants and serve as a developmental guide for current employees. The template is shown by a graph that indicates how a person scores against each of the required traits and then offers a final “bottom line” score between zero and one-hundred that represents the individual’s level of “total suitability” for a particular position. A score of one hundred represents a person who is completely suitable for that position. Assuming the person is eligible for a position (has the education, experience and technical skills), a suitability score of 75 or greater represents a person who has a good probability of performing effectively in that position. A score of 74 or less represents a person who is considered to be unlikely to perform well in that position.

Results

The results showed a high predictive accuracy and a strong correlation between the Harrison InnerView suitability score and the actual job performance. This indicates that the template includes a fairly comprehensive set of traits related to suitability for this position.

The performance data provided by the cable company showed that the new employees had a considerably lower performance rating than the employees who had been employed for longer. Although it is expected that there will be some difference due to the better performing employees staying longer, the difference was so great that I suspect that the performance ratings were not sufficiently adjusted according to the length of employment. Or, it may be too difficult to evaluate performance until the employee has been working in the position for at least 6 months.

The average performance ratings were as follows:

For an employee of 3 years or greater was 89%.

For an employee of 2-3 years it was 88%.

For an employee of 1-2 years it was 82%.

For an employee of 6-12 month it was 81%.

For an employee of 3-6 month it was 72%.

For an employee of < 3 month it was 74%.

The results were analyzed in two ways: the predictive accuracy and the variation between the suitability scores and the performance ratings:

1) Predictive accuracy - A prediction is considered accurate if one or more of the following conditions are met:

a) The suitability score is 75 or greater and the performance score is 75 or greater

b) The suitability score is less than 75 and the performance score is less than 75.

c) The suitability score is within 6 points of the performance score.

The logic behind this definition - If the suitability score were 75 or above and the person were eligible for the position, it would indicate a prediction that the person would probably succeed in the position. If the suitability score were less than 75 and the person was moderately eligible (not highly eligible) for the position then this would indicate a prediction of below average performance. Also, if the suitability score were within 6 points of the performance score it would indicate a very close prediction and thus should be considered accurate.

Predictive accuracy levels for the two templates

Template number one – all employees – 87% predictive accuracy

Template number two – only employees > 6 months – 91% accuracy

2) Variation between the suitability ratings and the performance ratings

The degree of variation between the InnerView suitability score and the performance ratings given by the supervisors also provides an effective means of understanding the accuracy level of the methodology. The variation was examined in two ways: the average variation between the predictive suitability rating and the performance rating for each group and the percentage of predictions that fall within different ranges of closeness. The average variation between the suitability score and the performance rating for the template using all employees was 8.5 points. The average variation between the suitability score and the performance rating for the template using the employees of greater than 6 months was 6.8 points. This indicates a very close relationship between the suitability score and the performance ratings.

Average variation between suitability and performance rating

Template number one – all employees – 8.5 points

Template number two – only employees > 6 months – 6.8 points

Next we examined the percentage of employees whose two scores (performance rating and predicted suitability score) were within 5 points, 8 points, 10 points and 15 points. The table below shows these percentages for each template.

Variation between performance rating and predictive suitability score

If the two scores were within five points, the prediction would be considered to be extremely accurate. If within 8 points, the prediction would be considered to be quite accurate. If within 10 points, the prediction should still be considered to be reasonably accurate. If the two scores are greater than 10 points apart, but within 15 points, it could not be considered to be accurate, but at the same time could not be considered to be very inaccurate. If the two scores are greater than 15 points apart, they should be considered to be inaccurate. Therefore, in the first template, 15% could be considered to be inaccurate but in the second template only 9%.

Conclusion

The results show the template developed is able to predict job success with a reasonable high level of accuracy. The sample size was sufficient to assume reliable results for future applicants.

Recommendations

If after inquiry, it is determined that the performance ratings were adequately adjusted according to the length of time employed and that performance can adequately be assessed in the first six months, then the first template should be used. If not, then the second template (Cable > 6 months) should be used.

It is possible that there is not a tracking mechanism in place that allows an employee or supervisor to adequately evaluate performance in the first six months of employment. If such a tracking mechanism were put into place, it could in itself reduce unwanted employee turnover. It could be the case that new employees are comparing themselves to the longer employed employee performance and thus unnecessarily becoming discouraged.

Although it is more the case with the first template, both templates are showing more false positive predictions than false negative predictions. Although I think this is partly due to the skewed performance ratings, it is also typical of positions that require a great deal of self-motivation. A detailed examination of the applicants’ goals would be very useful at an interview in order to eliminate some of the false positives. This position requires a great deal of personal drive and self-motivation (indicated by wants challenge, takes initiative and enthusiastic about goals). When that is the case, it is important to try to determine if the employees goals are in alignment with the goals of the company, or if the employee has some other goal to which this job can only be a temporary stepping stone.