The International Association of Certified Home Inspector (InterNACHI) has administered more inspection exams than all other sources combined.  It is also the leader in gathering home inspector competence evaluation data, right down to the pass/fail rates of every question ever asked.

 

    Often we will hear someone in the industry complain that "the questions should have been more difficult to answer" on this or that exam.  On the surface it may appear that an exam is improved to the benefit of the consumer if the questions are made to be more difficult.  Let us dispel this myth now. 

 

    For purpose and ease of discussion, let's assume that we have a 100 point True/False exam.  Because a multiple choice question often has at least one clearly wrong answer choice, a multiple choice question is not much different (only a bit more difficult) mathematically than a True/False question.  Let's also assume that we have two exam takers.  One (we'll call him Mr. Veteran) knows 10 times what the other (we'll call him Mr. Newbie) knows about home inspections.

 

    Now we all know that if we make the exam too easy, Mr. Veteran and Mr. Newbie will score similarly.  Just as if we asked a Pulitzer Prize winner and a six year old how to spell the word "CAT" (proof by extremes).  In the inspection industry, an exam that is too easy is harmful to consumers because it makes a weak distinction between Mr. Veteran and Mr. Newbie... they both pass.

 

    But what happens if the questions in the exam are too difficult to answer?  Let's find out using another proof by extremes.   Let's say we create a 100 point True/False home inspection exam that has so many difficult-to-answer questions that Mr. Veteran only knows the answer to 20 of the 100 questions.  That's a pretty hard exam!  Mr. Veteran will have to guess the answer to 80 of the questions.  On average he will score 60 (20 for the ones he knows the answer to plus half of the 80 that he guesses at).  Mr. Newbie knows a tenth of Mr. Veteran.  So he knows the answer to only 2 questions.  On average he will score 51 (2 for the ones he knows the answer to plus half of the 98 he guesses at). 

 

    Now if you flip a coin 10 times, you should get heads 5 times on average.  But often if you flip a coin 10 times you will get more than 5 heads or less than 5 heads.  The same is true for the questions our exam takers are guessing at.  Sometimes Mr. Veteran will score worse than 60 and sometimes Mr. Newbie will score better than 51.  Using an online binomial calculator one finds that 1 in 10 Mr. Newbies will score as well as the average Mr. Veteran and that 1 in 10 Mr. Veterans will score as low as the average Mr. Newbie all based solely on chance!  Often Mr. Newbie will score better than Mr. Veteran simply because the score on an exam that contains a large percentage of difficult-to-answer questions that neither exam taker knows the answers to, is determined solely by luck.  That's not too good for consumers. 

 

    But wait, it gets worse for consumers!  As we make the questions more difficult to answer, the percentage of questions that our exam takers have to guess at goes up, which increases the reliance of the score on chance, which increases the odds that Mr. Newbie will score as well or even better than Mr. Veteran, which weakens the exam's ability to determine who is competent and who isn't, which is worse for consumers.  In fact, on an exam full of so many difficult-to-answer questions that Mr. Veteran only knows the answer to 10 of the 100, Mr. Veteran will (on average) score only 4.5 points better than Mr. Newbie.

 

    But wait, it gets worse for consumers!  By making the questions more difficult-to-answer and increasing the chances that Mr. Newbie will be able to pass by being lucky, we also increase the chance of Mr. Veteran will fail due to bad luck.  If the exam is used for certification (which gives the exam passer a market advantage and denies the exam failer the market advantage) or worse... if the exam is used for licensing (which puts the exam passer into the market and prohibits the exam failer from entering the market), an exam with more difficult-to-answer questions increases the percentage of Mr. Newbies to Mr. Veterans in the market place, by increasing the number of Mr. Newbies and decreasing the number of Mr. Veterans, which is horrible for consumers.

 

    InterNACHI's exams don't rely much on chance.  InterNACHI's exams contain what some would describe as easy-to-answer questions that every Mr. Veteran should know the answer to.  InterNACHI's exams are sometimes criticized for containing such questions.  What these critics don't understand is our superior scoring system.  If the exam taker answers these questions correctly, he gets no credit for them because we can't tell if he answered the questions correctly because he is a Mr. Veteran or if he is a just a lucky Mr. Newbie.  However, if the exam taker answers them incorrectly, we can assume that he is very likely a Mr. Newbie, and the exam taker is severley punished (in terms of score) for failing to answer them correctly.  For you see, it is much easier to determine incompetence than competence.  This system, combined with InterNACHI's high passing cut-off scores, cause InterNACHI's exams to be much better than other exams at distinguishing between the competent and incompetent and far less reliant on chance and luck. 

 

 

 

 

 

 

 

 

Find an Inspector: