How to Identify Quality Personality Profiling Tools

Many tests and questionnaires are now available which purport to measure personality, values, motivation, behaviour or other human characteristics.  There is great variation in the quality of these tools.

The category includes only a small number of credible instruments with genuine expert knowledge and research behind them, and a great many ‘cheap and cheerful’ creations which are founded upon little or no research or specialist understanding.

There are also many psychiatric and educational tools.  These are in different categories altogether and are not appropriate to use in the great majority of occupational contexts.

Unsuitable or ineffective tools will at best add no value — you will merely have wasted some time and money.  Far worse, when key personnel decisions are misled by them, they can be damaging to the individuals concerned and to an employer's reputation.

Unfortunately, the differences are not obvious to non-specialists, so here are four clues to pay attention to which may indicate quality personality questionnaire design:

1.  Long questionnaire + moderate report length

There is no magic — it is only possible to get as much out of a questionnaire as you put in.  Therefore, long or detailed text-based reports derived from short or simply worded questionnaires should be regarded with great suspicion.  Such tools are very unlikely to be genuinely measuring what they purport.

Two rules of thumb when choosing a questionnaire to profile the personality of senior personnel are:

2.  Data on scale reliability

The reliability of a measurement is a statistical concept which indicates the integrity, reproducibility or stability of a test result.  For example, if a person completes a questionnaire twice, their results should be very similar.

This is an essential prerequisite for predictive validity — using test results to predict actual on-the-job behaviour — because no predictions can be made if the fundamental measures are fluctuating wildly.

3.  Many scales are more useful than a few

Whilst the theoretical Five-factor model of personality is mathematically and intellectually neat, in most selection and development contexts such tools contribute little.

Most human resources specialists, psychologists and executive coaches find that at least 12 detailed scales are needed to begin adding real value to selection decisions and staff development.  The ideal is about 24 to 33 scales.

Multiple scales avoid pigeon-holing or categorising people into overly generalised boxes, and enable a much better understanding of each person as an individual.  It makes it possible to match people to specific jobs or tasks, and to understand what's driving their performance challenges.

More than 33 scales results in too much overlap between the scales, such that several become redundant.

4.  Scales must be properly normed

Personality questionnaire results have little or no meaning unless compared with what is ‘normal’ or typical for a large and relevant sample of the wider population.  Therefore, results should always be plotted against a comparison group — a norm group — broadly related to the demographic in question.

Un-normed test results presented as stand alone scores, such as 7 out of 10, are largely meaningless and highly vulnerable to misinterpretation.

Unlike ability or aptitude tests which require narrow comparison groups such as Graduates applying for technical roles, personality traits are best compared with a broader, more generic working population.  Otherwise results can skew and become hard to interpret correctly.

For example, results should be able to tell you “Meredith appears to be far more critical-minded than most managers and professionals.“

Important:  Statistically and practically, a norm group of 100,000 people is no better than a norm group of 10,000 or 1000 people.  Why?  Because any differences in a person's results will be only a fraction of a percent and therefore entirely invisible on their profile chart.

Nevertheless, some test publishers strongly promote the notion that a huge norm group gives their tool a special advantage.  To over-emphasise this is misleading and might be intended to distract you from more fundamental flaws in their product, so watch out.  It could indicate nothing more than they are excellent marketers!

Much more critical for you to examine is the statistical construction of the tool.  What matters is the technical skill and diligence of the psychometricians when formulating the questionnaire, running population trials and presenting individual test results.

— Stephanie Thompson