Harrison InnerView Case Study

Background

The purpose of the case study using Harrison InnerView was to determine the ability of the InnerView software to predict success for the Lumber Sales Representative. The building supply company provided 35 profiles of current employees. two profiles were eliminated due to very low consistency scores (less than –40). The low consistency score indicates that the individual either attempted to fool the questionnaire or tried to answer the questions too quickly. Therefore, the total sample population of valid profiles was 33.

This case study compares the performance of the employees with the results of the Harrison InnerView profile. All the employees in the sample were rated according to their job performance by supervisors using the performance rating model below. The profiles of the employees were then analyzed in relationship to performance in order to determine the factors that relate to success in the Lumber Sales Representative position. In addition, the job was analyzed in order to assess the traits that were required for this position. These traits were given special attention when analyzing the traits that relate to performance. However, all 130 traits in the Harrison InnerView system were analyzed in relationship to performance.

Performance scoring

The InnerView consultant and the building supply company management team identified five key performance factors for this Lumber Sales Representative position and developed a model that weighted the importance of each factor and adjusted the sales volume factor for the length of time the individual had been in the position. As a side benefit, this model can now be implemented into quarterly performance evaluations.

The five categories tracked are as follows:

1) Volume: average monthly sales volume for previous 12 months

2) Margin: average monthly profit margin on sales volume for previous 12 months

3) Credits: credits to the builder’s account for jobs delivered with extra materials. This costs the company money to pickup and restock.

4) D/R’s(deductions/returns): returned items from builder that were delivered in error. This is important because it costs the company substantially to have to go to the job site and pick-up the order, re-stock and make the correct delivery. In addition, fewer returns relates to greater customer satisfaction. This indicates how accurately the orders are placed by the salesperson.

5) Average Ticket Size: average monthly ticket size for the last twelve months.

Each factor was given a rating based upon a scale between 0 and 100 according to the following:

90-100	Excellent to Best
80-89	Very Good
70-79	Average to Good
60-69	Below average
40-59	Poor
0-39	Extremely poor

Next, each performance factor was given specific criteria for making the ratings. In addition, the sales volume factor was adjusted according to the length of time in the position. The values given below were arrived at through discussion with the senior management team and were based upon the results expected from someone in this position.

In the sales volume model below, the ratings take into consideration the degree of experience that the person has in the position or previous similar positions. For example, if Person A has been in the position for 6 years and is selling $400,000 per month of product, he/she should receive a score of 100. Person B is selling only $160,000 per month of product (which is about average) and thus would receive a performance rating of 75. However, if person B has only been in the position for one year then a more lenient requirement is applied. If, for example, $160,000 is considered to be top performance for having been in the position for only one year, then person B would receive a performance rating of 100 rather than only 70-75. This is to compensate for the cumulative nature of sales.

The table below provides performance expectations for various levels of time in the position starting from six months up to three years. The 5 year category is for seasoned salespersons with five or more years. Those individuals between the three and five year marks should be placed into the category to which they are closest.

Performance rating for average monthly sales volume related to different lengths of employment.

RATING	SIX MO.	1 YR	2 YRS	3 YRS	5 YRS
90-100%	$55k	$100k	$200k	$300k	$400k
80- 89	$50k	$90k	$150k	$200k	$300k
70-79	$45k	$80k	$125k	$175k	$175k
60-69	$40k	$70k	$100k	$150k	$140k
40-59	$35k	$60k	$80k	$125k	$100k
0-39	$30k	$50k	$75k	$100k	$<99k

Performance rating for margins, credits, and D/R’s – used for all experience levels

RATING	MARGIN	CREDITS	D/R’S
90-100%	27%	1-2%	.2%
80- 89	25-26.99%	2%	.35%
70-79	24-25%	2.5%	.5%
60-69	22-23.99%	3.5%	.6%
40-59	20-21.99%	4-5%	.7%
0-39	<20%	>5%	>.7%

Performance levels for Average Ticket Size

RATING	AVERAGE TICKET SIZE
90-100	>$1000 per month
80-89	$801 – 1000 per month
70-79	$701 – 800 per month
60-69	$601 – 700 per month
40-59	$500 – 600 per month
0-39	<$500 per month

In addition, each performance factor above was given its own importance rating based upon its level of importance to overall performance in relationship to the other factors. The various factors were given the following weightings:

Sales Volume	40%
Margins	10%
Credits	15%
D/R’s	15%
Avg Ticket Size	20%

Example: John Doe

Step 1: Rate each category

0	years tenure with average monthly volume at $150,000 – rating of very good @ 89
1	average margin of 23% - rating of below average @ 65
2	average monthly credits of 2.5% -rating of good @ 79
3	average D/R’s of .35% - rating of very good @ 89
4	average invoice size of $875 – rating of very good @ 85

Step 2: Multiply scores by weighted percentage and derive a final performance rating

Monthly volume -	89 X 40% = 35.6
Margins-	65 X 10% = 6.5
Avg. Invoice size-	85 X 20% = 17
Credits-	79 X 15% = 11.85
D/R’s-	89 X 15% = 13.35

Total Performance Rating:

84.3 or rounded off to 84

This Study

The aim of the research was to determine the exact traits that make salespersons successful at the building supply company and to determine the relative accuracy of the system to predict success based upon the “job template” that incorporates those traits. The template formulates those traits into “traits to have” as well as “traits to avoid”. The template is then used to measure future applicants and serve as a developmental guide for current employees. The template is shown by a graph that indicates how a person scores against each of the required traits and then offers a final “bottom line” score between zero and one hundred that represents the individual’s level of “total suitability” for a particular position. A score of one hundred represents a person who is completely suitable for that position. Assuming the person is eligible for a position (has the education, experience and technical skills), a suitability score of 75 or greater represents a person who has a good probability of performing effectively in that position. A score of 74 or less represents a person who is considered to be unlikely to perform well in that position.

Accuracy level of results

The results showed a very high predictive accuracy and a strong correlation between the InnerView suitability score and the actual job performance. This indicates that the template includes a fairly comprehensive set of traits related to suitability for this position.

The results were analyzed in two ways: the predictive accuracy and the variation between the suitability scores and the performance ratings:

1) Predictive accuracy - A prediction is considered accurate if one or more of the following conditions are met:

a) The suitability score is 75 or greater and the performance score is 75 or greater

b) The suitability score is less than 75 and the performance score is less than 75.

c) The suitability score is within 6 points of the performance score.

The logic behind this definition - If the suitability score were 75 or above and the person were eligible for the position, it would indicate a prediction that the person would probably succeed in the position. If the suitability score were less than 75 and the person was moderately eligible (not highly eligible) for the position then this would indicate a prediction of below average performance. Also, if the suitability score were within 6 points of the performance score it would indicate a very close prediction and thus should also be considered accurate. The suitability score for each person appears on the template in the far right-hand column of the table.

From the research sample, 31 of the 33 employees (94%) showed a correlation between the InnerView suitability score and performance to be accurate, according to the definition above.

2) Variation between the suitability ratings and the performance ratings

The degree of variation between the suitability score and the performance ratings also provides an effective means of understanding the accuracy level of the methodology. The variation was examined in two ways: the average variation between the predictive suitability rating and the performance rating and the percentage of predictions that fall within different ranges of closeness.

The average variation between the suitability score and the performance rating was 5.97 points. This indicates a close relationship between the suitability score and the performance ratings.

Next we examined the percentage of employees whose two scores (performance rating and predicted suitability score) were within 5 points, 8 points, 10 points and 15 points. The table below shows these percentages for each group.

Variation between performance rating and predictive suitability score

If the two scores were within five points, the prediction would be considered to be extremely accurate. If within 8 points, the prediction would be considered to be quite accurate. If within 10 points, the prediction should still be considered to be reasonably accurate. If the two scores are greater than 10 points apart, but within 15 points, it could not be considered to be accurate, but at the same time could not be considered to be very inaccurate. If the two scores are greater than 15 points apart, they should be considered to be inaccurate.

The results show that the template developed is able to predict job success with a high level of accuracy. Although the sample size was sufficient to gain accurate results, it is recommended that for incoming employees, records are kept for performance and profiles. When the sample size reaches 200, it is recommended that the template is adjusted using the larger sample size.

Traits required for success

The research indicates that the success factors for this position include:

The research indicates that there are also factors that will hinder success in this position. These include:

These traits as formulated in the Harrison InnerView software will enable a future prediction level of success for the sales position of at least 80%.