Evaluation of Knowledge Based Training Programs Using Repertory Grid

Determining if a knowledge based training program has changed an individual and how much change has been achieved.

The Repertory Grid Interview is an ideal process for determining developmental change because the process maps a person’s knowledge, experience and attitudes. Since the objective of training and development is to improve that person’s knowledge and experience and sometimes alter their attitudes, a before and after grid interview can show how much change has come about. That aids a managerial assessment on value for money.

If an interview is carried out prior to a developmental intervention, trainees can be provided with a head start. The interview process itself, the development of the cognitive map prior to training or development, helps a trainee focus on the subject matter and provides more awareness of the gaps they have in knowledge or development and sets the trainee up to get more out of the training.

Training Evaluation

Knowledge based training enhancement is very simple to determine. Knowledge enhancement is a matter of:

• Knowing more facts
• Knowing more about the facts
• Knowing what is right or wrong about the facts.

For instance you might expect a dairy farm economics student, on completion of a course on cattle breeds, to know more dairy cattle breeds, to know more about the breeds and to know more about how they compare in terms of dairy farm economics.

I assume here that you already know about repertory grid and the meaning of the terms construct, qualifier and element. If you need to review these terms you might like to visit the Enquire Within web site at http://www.enquirewithin.co.nz/site_map.htm

So, to test training effectiveness, we would conduct before and after course evaluation interviews using dairy cattle breeds as elements. The student would name as many breeds as they could for use as elements. Then, using the repertory grid process, a series of two against one comparisons of the elements (breeds) would be made using qualifiers to focus responses such as `… in terms of suitability in different climates or different terrain, feed conversion, profitability, etc’. For example `How are Jersey and Hereford cows similar but different from Holstein in terms of efficiency of feed conversion?’ The compare and contrast process using different combinations of breeds would be continued using laddering and differentiation until the student could go no further producing constructs.

Analyzing the Results

Obviously we would expect the student to be able to name many more breeds after the course than before it.

We would consider what the student regarded as a dairy cow breed. The analyst might be unhappy about the inclusion of beef only breeds when the economics of dairy breeds was being considered. To be proficient, the student might need to clearly discriminate between dairy and beef cattle breeds.

Additionally we would want to know about the number of dimensions used to distinguish between breeds. The number of constructs produced would be expected to be much larger after the course. (In the pre–course test the student would become very aware of his/her inability to distinguish between breeds — the gaps in knowledge — and therefore be much more focussed during the course).

We would also consider on what dimensions does the student evaluate cattle breeds? The before course interview might show that the constructs are largely contextual (where they can be purchased, what they cost, etc.) and personal (what they look like). After the course one might expect more constructs on their suitability for different climates and terrain, the degree of skill needed to manage them and flexibility for use to produce different dairy products (cheese, town milk supply, butter), etc. That is, the number and quality of the constructs produced has increased.

We would look for factual errors in the Grid and it would be expected that these would be reduced.

The process can be used for many types of job training, product knowledge, career training, management training, sales training, technical training, etc., but not procedural training or training where there are relatively few possible constructs.