How to identify...
Well
Designed Psychometric Profiling Tools for Business
Use
There
are a lot of tests and questionnaires around - what are the
differences?
Many
tests and questionnaires are now
available which purport to measure personality,
values, motivation, behaviour or other human
characteristics. There is great variation in the quality
of these tools, and the category includes only a small
number of credible instruments. These few have years of
specialist research behind them, compared with the majority ‘cheap and cheerful’
creations which are founded upon little or no research or
specialist knowledge.
There are also
several psychiatric tools. These are in a different
category and are not appropriate to use in the great
majority of occupational contexts.
Unsuitable or ineffective
tools will at best add no value - you will merely have
wasted a little time and money. At worst, when
key personnel decisions are misled by them, they can be
damaging to the point of incurring law suits.
Unfortunately, the differences are often not obvious to the
non-specialist.
So...
What
are the clues which can indicate quality design?
1. Long
questionnaires & moderate report lengths
There is no magic – it is only possible to get as much
out of a questionnaire as you put in. Therefore,
long or
detailed text-based reports derived from short or
simple questionnaires should be regarded with scepticism
and great caution. Such tools are
very unlikely to be genuinely measuring
everything (or anything) that they imply.
A good rule of thumb when
assessing the personality or behavioural style of
professional or executive-level staff, is that questionnaires
should take at least 20 minutes for the fastest
person to complete - and 25 to 40 minutes for most
people. Outputs should be in the form of profile charts
or short text descriptions of
quite narrow traits.
2. Data on
scale 'reliability'
The reliability of a scale is a statistical
concept which indicates the 'integrity' or
‘reproducibility’ (stability) of a test result. For example,
if a person completes a questionnaire twice, their
results should be very similar.
This is an essential
prerequisite for 'predictive validity' –
correlations between test results and actual on-the-job
performance – because no predictions can be made if
the fundamental measures are fluctuating.
(For more about statistics and examples of
reliability and validity figures, look in the Manual &
Interpretation Guide for the
Expert
Psychological Profiler for Business.)
3. Many scales
are usually much more useful than a few
Whilst the theoretical ‘Five-factor
model of personality' is
mathematically and intellectually tidy, in most selection and development
contexts such tools contribute very
limited practical value. Most human resources
specialists, psychologists and executive coaches find
that at least 12 detailed scales are needed
to begin adding value to selection decisions and staff
development work.
The ideal is
about 20 to 32 scales, because this avoids 'pigeon-holing' or
'categorising' people in overly generalised boxes. It enables you to much
better understand each person's individuality, and
thereby to
fit them to specific jobs.
If there are
more than 35 scales there will be too much
overlap between them, and several will be redundant.
4.
Scales must be properly ‘normed’
Personality questionnaire results
have little or no meaning unless compared against what
is ‘normal’ or 'typical' for a large and relevant
sample of the wider population. Therefore, results
should always be plotted against a 'comparison
group' (= 'norm group')
related to the role in question.
For example, a tool should be able to tell you "Meredith describes herself as much less ambitious than
is typical for most Managers and
Professionals".
By contrast, a test result presented as a stand alone
'score' is largely meaningless, and highly vulnerable
to misinterpretation.
Note: Statistically, a norm group of 10,000 people is
no better than a norm group of 1000 people. Why?
Because any differences will be only fractions of a percent,
and therefore completely invisible to the interpreter
when plotted on a profile chart. Nevertheless, some test publishers
'sell' large norm
groups as if they confer an advantage. They do
not, and to imply so is misrepresentative. Very
much more
important are the statistical design of the tool
itself and the care taken when trialling.
Want to know more?
Ask a psychometric expert at
Insight Matters.
Read more about
Expert
Psychological Profiler for Business
TM
