If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!

In the murky past I wrote two article about interviewing programmers. The first covered the basics of interviewing itself and gave (hopefully) a good grounding for those who have never had to interview before. The second was all about the kind of testing you can do to filter out candidates before you commit to doing a full interview. At the time I then promised to write a third section on what types of tests to use during the interview. I kind of got very sidetracked (oops) and have had the draft of the post knocking around for a while. Well I finally got round to finishing it off.

Specific Language Tests

If you are looking for experienced programmers then it can pay to get them to perform specific tests within the language that you require. As I have stated before this can be a dangerous game for anyone but the most experienced. The reason being that every language is a massive subject and it is unreasonable to expect a candidate to know everything. You then get into a game of either making the test too general (and therefore easy to pass) or too specific which then means it only applies to the most experienced.

You have two options when dealing with language testing. 1) Write your own test 2) Use a service that supplies tests. I have tried both and you have to take into account the cost impact of writing your own tests versus cheaper supplied tests which may not be as tailored as you like. My advice is if you have the time and resource (and more importantly someone willing to do it!) then go for write your own as the upsides of tailored tests are worth it.

Use of Pseudo Code

I have recently written an article regarding the use of Pseudo Code (this article in fact prompted me to write it) within development. Although it is a very powerful tool for prototyping it can also be very useful within a testing environment. With the constraints of a specific language removed you can ask candidates to solve logic problems and have them write the results in whatever form of Pseudo code they like. The advantage of this is that you are removing the specifics and hopefully seeing the overall problem solving abilities. The downside is really the opposite in that by removing the specifics you are not gaining any insight into any particular knowledge.

IQ Tests

The only IQ test I have had was at Codemasters who set quite a high threshold for every staff member (around 110 IQ). I have a heavy slant towards math’s / logic puzzles and breezed those sections but struggled on the word association style tests. The reason for me to recount this is that I then went on to recruit a new team (I was heading up a brand new ‘New Media’ department) and first hand came across good and bad examples of using IQ tests.

The good side is that when someone passes and preferably passes well you know your getting someone in the top (n)% of the population. The bad side is that there is a chance that some people are not well suited to passing these kinds of tests. Codemasters had a blanked rule that everyone had to pass. I remember interviewing one guy who came across as a really creative and very talented designer. When he went off to do the IQ test I compared notes with one of the HR girls who also interviewed him and she also gave him a glowing report. The IQ results come back and it turns out he has done disastrously and even if I did want to hire him I would not be allowed too. Later that day I spoke to HR again and they said he came out of the test room shaking, he was definitely not someone who could cope with test situations.

Psychometric Testing - Ability/Aptitude

I have had to perform various psychometric tests in the past and some have been so transparent that I have been able to twist the final results to how I believed the company wanted to view my abilities. You also run the risk that the test is standardized and a candidate may have seen it before and again be able to twist the results which is certainly undesirable. Psychometric is split (in simple form) into Ability and aptitude which I have in the past read up extensively and I have tried to sum them up below. My overall opinion is that this form of testing is for companies with large HR departments with people who have the experience to interpret the results. As with everything results are certainly open to interpretation and thus should only be used as general guidelines.

Ability testing
Ability tests measure a person’s potential. To learn the skills needed for a new job or to cope with the demands of a training course. They focus on what the person is capable of achieving in the future or their potential to learn. Bear in mind that some attainment is required before certain abilities can be measured, for instance, we need a certain knowledge of mathematics before our numerical ability can be measured.

General ability is usually divided up into specific abilities, reflecting the hierarchical structure of intelligence that is generally accepted by most workers in the field. So a general ability test might be composed of specific numerical, verbal and spatial ability scales brought together as a test battery. They can then be scored and interpreted individually as a specific ability or aptitude measure, or together as part of a general ability measure.

Aptitude testing
There is no broadly accepted definition of the difference between ability and aptitude. A lot of people would agree that to some extent the two terms refer to the same thing: aptitude referring to specific ability, and ability referring to general aptitude. Underlying aptitude could be viewed as ability and aptitude is more job related.

Aptitude tests tend to be job related and have names that include job titles such as the Programmers Aptitude Series. Ability tests on the other hand are designed to measure the abilities that underlie aptitude.

Aptitude and ability testing can be used separately to assess specific abilities or together to assess general ability. There are tests which can measure only general ability such as the Standard Progressive Matrices and there are tests which only measure specific abilities. You will find with experience that some tests fall into more than one category and that the distinction between the various categories is not always an easy one to define.

When to use testing

I have mixed opinions on when to use any of the above. In some circumstances having candidates perform tests before you meet them can give you insights which you can then use during the interview itself. If you do put in place the minimum IQ requirement then it can mean less interviews (as some may fail) which is always a bonus as time is always precious and can be given over to more thorough questioning of other candidates.

Splitting the interview into parts and having them perform a test half way through can allow you to go away and think about what to ask in the second part backed up by the results of the test. The downside for me is having your day broken up more than is really necessary.

The last option is to have them perform the test(s) after the interview which for me is the most preferential. You have the most objective opinion because you have finished the interview and can look at the test results side by side.