A Case of Adverse Impact
Posted on October 20, 2011 by Gail Danneman
In 2009, the United States Supreme Court made a decision about the presence of adverse impact on a promotional test. The New Haven, Connecticut fire department used a test to determine which employees would receive promotions. Nineteen white and one Hispanic firefighter passed the test and should have been eligible for promotion. However, the New Haven fire department decided the test was not valid. The twenty firefighters that were denied promotion argued that they were discriminated against, while the fire department claimed to fear a lawsuit over adverse impact.
The Society for Industrial Organizational Psychology made several arguments in an amicus brief for Ricci v. DeStefano (SIOP amicus brief). The first argument involved the job analysis approach that the fire department conducted to create the test used for promoting firefighters. Another argument was that the test was not as job-related as it should have been. The final argument was that assessment centers rather than a test would have produced more accurate results to base promotions on.
The job analysis conducted by IO Solutions was rather weak according to the amicus brief. Although a job analysis was performed, there are questions to the procedures that were used. The job analysis was not very thorough and failed to use firefighters from New Haven to voice opinions about the test. Instead, a single subject matter expert from Georgia was utilized to examine the test and speak with IO Solutions about the position. Because of this, the Society for Industrial Organizational Psychology did not feel that the job analysis was performed accurately and as in-depth as it should have been.
Industrial organizational psychologists are taught that the practice of job analysis is of utmost importance in creating a test used in selection or promotional decisions. IO Solutions should have used more than one subject matter expert to ensure that the test was valid and reliable. The subject matter experts should have also been from the New Haven fire department. The subject matter expert used may not have been aware of the operations of the New Haven fire department and was a weak source of information for the job analysis. There were several areas of the job that were most likely missed because of failure to use appropriate subject matter experts.
The second argument had to do with the degree to which the test predicted on the job performance. One of the most important skills that firefighters must have to be successful on the job is command presence. The test used by New Haven failed to investigate firefighter’s level of command presence. That being said, the test did not look at the most important skill. Command presence, the ability to make quick decisions in difficult situations, is essential to a firefighter’s success. Because the test did not use command presence as a construct, the test did not truly look at the KSAs of a firefighter.
The final argument was that assessment centers would have been a better option for the New Haven fire department. The Society for Industrial Organizational Psychologists argued that command presence could have been evaluated in an assessment center. Instead of using a test to evaluate firefighters, the assessment center would have identified other important KSAs that a firefighter possessed such as leadership skills, oral communication skills, as well as command presence. Because assessment centers use a number of raters, the level of bias and adverse impact could be greatly reduced.
The case centered on the fact that adverse impact was found to exist in the test used to decide promotional decisions. The Society for Industrial Organizational Psychologists believes that with the use of assessment centers, adverse impact could have been eliminated as a legal issue with this particular case. The New Haven fire department failed to investigate the test that IO Solutions created and may not have considered the possibility of adverse impact. Had assessment centers been used in the promotional decisions, the risk of adverse impact would have been much less.
What are your thoughts on this case?