Week 4, Ch 5 and 6

            When considering validity, I was particularly drawn to one of the threats to validity: construct-irrelevant variance.  As I understand the concept involves using an instrument for a particular purpose, but having other aspects inadvertently confuse the results and therefore impede the proper interpretation and usage of an assessment.  I witnessed this first hand when I helped with the settlement of Burmese refugees in Lancaster.  The students were being assessed for their academic achievement in order to be placed in the correct grade and sections in elementary, middle, and high school.  I accompanied them when an employee from the school district attempted to give them an academic assessment.  Although these children had been in school in refugee camps in Thailand, their very limited English created a situation where they appeared to know less than they did.  In this way the assessment experience had very little validity.  The intent was to use the tests as a tool to discern their academic level so that they could place them in the grade and section appropriate to their academic level.  The tests in English were not able to assess anything but the refugees’ lack of English language skills.  As I continued to work with these children and teens, I continually wondered what their academic skill level and capabilities were.  But neither the school nor I had a tool or set of tools that could even begin to make an accurate assessment.  If they had been Spanish speakers, or some other language that was more common, perhaps an adapted assessment tool would have had some validity. However their native Karen language made it impossible to find any tool to make an adequate assessment.  Even the way they wrote numbers was different than the Arabic numerals used in this country.  I remember coming home from the school district assessment, feeling as though it had been a somewhat useless exercise.  It was district policy to conduct this assessment in either English or Spanish for all new students, but this may be a case where the question of whether a particular assessment scenario is apt to have any validity at all would have been useful before subjecting the refugees to this testing experience.  They were very nervous going to the test, knowing they would not be able to answer anything.  And they left the experience feeling that they had represented themselves poorly.  Additionally they were fearful as to what their failure would mean for them.

            The refugees were placed in grades and sections for school, because the district had to place them somewhere.   But, as this example points out, without assessments and interpretations that fit the intended usage and needs, schools are in a sense flying blind with students. Hopefully, one day, these students will develop English skills enough to get a more accurate and useful assessment of their academic needs and abilities.  In the meantime they were exposed needlessly to yet another intimidating and fearful experience.  

References

Drummond, R. J. and Jones, K. (2010). Assessment Procedures for Counselors and Helping Professionals. Upper Saddle River, New Jersey: Pearson Education, Inc.