Issues with Cognitive Reasoning/Critical Thinking Assessments

Charlene Castillo's picture

Overview on Using the

Quantitative/Critical Thinking Assessment

For Selection Decisions

 

 

 

Rationale:

Why use a cognitive test? Cognitive tests have proven to have correlations with job success of around .5 which is among the highest of any type of test. They can especially add significant value to the recruitment process. Using a cognitive test can make a real difference to hire the best talent and avoid the high costs resulting from hiring the wrong candidate. 

What are the weaknesses of using cognitive tests?

1) They take about 90 minutes to complete which can dissuade top talent from applying. 

2) They can be difficult to administer and add costs due to having to supervise the test.

 3) They may also be difficult to interpret because the content is not related to what the person needs to do on the specific job and results are typically divided into multiple sections. 

4) They are often challenged as being biased against specific groups of people because the content is not related to job requirements.

 5) They don’t measure behavior and thus may not discover important tendencies that will either support job success or hinder it.

How does QCT overcome these weaknesses?

1) On average, it is completed in only 18 minutes (average) while providing the same high correlations.

2) It is easy to administer via the internet option and the results are easily verified at the interview.

3) Reports are job specific making it easy to interpret related to the specific jobs.  

4) It is fair to all applicants because the content is directly related to job requirements

5) It is fully integrated with the option to combine with the proven Harrison Suitability Assessment. The combination provides the complete assessment necessary to discover top talent. 

What does QCT cost? 

Standard price is $49

What does QCT measure?

QCT measures verbal reasoning, numerical reasoning, and logic; all within the same set of questions which also helps to shorten the test. Most cognitive tests have separate sections for verbal reasoning, numerical reasoning, logic, and/or spatial reasoning which takes more time and makes the results more difficult to interpret. In addition, some segments such as spatial reasoning only apply to ability needed for a narrow range of jobs. 

How can the QCT achieve the same high correlations in only 8 - 18  minutes?

 QCT focuses on ability and uses sophisticated technology to determine the person’s level. Questions are organized into 10 levels or difficulty. The test begins with the individual selecting the level they believe they can achieve. If they achieve it, they go on to the next more difficult level. If they don’t achieve the selected level, they go to the next lower level. This continues until the level is determined.   Of course if someone is way over-confident, they may end taking 40 minutes or more.

What makes QCT more relevant?

QCT focuses on the ability to solve problems like the ones employees are likely to encounter in their job. Different types of jobs require different levels of ability and thus by connecting the job to the expected level, the test is job specific. QCT shows norms related to the specific jobs and even has an optional automated research engine that can determine the levels that best relate to job success for your specific company. 

Are there other mechanisms QCT uses to shorten the test?

 Yes. QCT avoids multiple choice questions or questions that enable guessing. Instead, specific numerical answers to the problems are entered. This shortens the test because it eliminates the need for additional questions to compensate for guessing.

How does QCT verify that the candidate is the one who completed the test?

When the candidate comes to the interview, the interviewer can give a brief test (QCT Verification Test) that is different from the questions completed on the test but at the same level as that achieved by the candidate. This verifies that the candidate is the one that completed the test.   

What other features and benefits does QCT have?

  • QCT is fully integrated with an optional easy to use applicant tracking system which comes with the QCT for no extra cost. Applicants can be ranked in order of their QCT results, or the combined scores of the behavioral assessment and/or interview. The administrator can require a cover letter and resume and view them along with all assessment results.
  • There is also an option to use and score an extensive library of eligibility questions or create custom eligibility questions that enable you to administer the QCT only to the most qualified individuals. 

Is the QCT timed?  The overall test is not timed but each individual question is timed. If the person knows how to solve the problem, there is sufficient time given to answer the questions as well as check the answers. More difficult questions are given more time. 

Can the person use a calculator or computer?  Yes, in fact it is encouraged because they will also have this option in a real working situation. The aim is to determine if the person knows how to solve the problem, not to test their calculation ability which in these times is nearly always done with a calculator or computer. They can even use a program such as Excel if they know how to use it.

What happens if someone quits before completing the QCT or their internet drops out before they complete it?   They can use the link to log back in and it will resume where they left off. However, any timer that was running at the time will continue until it runs out for that problem. (If that were not the case, there would be no timing because they could just log out, work on the problem and then log back in to enter the answers)

How has the QCT been validated?   The QCT has been validated related to performance. Validation documentation is available by request from Agiledge.

Should the QCT be used for all jobs?   No, it is not designed for jobs that require extremely high levels of mathematics such a mathematician, physicist, or astronomer. For such jobs, it would be best to rely on their academic record, previous experience, or a job specific cognitive assessment.