Some More Testing Thoughts

1. Is testing bad? Some kinds of testing, such as end-of-unit testing in
high school, seems appropriate and definitely necessary, although there
are schools that seem to get on well without it. This is subject-related
and is directly attached to what's being taught; that seems "good." IQ
and SAT/ACT testing is a different order of testing. They are testing
abstractions, the underlying assumption (esp. with IQ and SAT) being
that there is a "thing" called (or no longer called) "intelligence" or
"aptitude" that is inherent and independent of our formal educations
that can be measured using the proper tools. Stephen Jay Gould took this issue on
very nicely years ago in The Mismeasure of Man. It seems to have been
shown fairly conclusively that these tests (I don't know about the IQ
test so much) can be coached and are influenced by culture, background,
etc. If that's true, they are NOT measuring something inherent but
something acquired. Reason enough to question claims of their power and
inerrancy. That doesn't make them "bad" but it does make them suspect.

2. Is intelligence testing science- or faith-based? I don't mean
religious faith, but the result is the same: It requires a certain (and
I would say VAST) leap to use testing as a TRUE and CONSTANT measure of
whatever it is we expect it to measure. Most of us not involved in the
actual development of tests have no expertise and so we take what
testing agencies say to be true as true, even if we are bothered by it.
I challenged the CB several years ago to tell me who outside the CB
itself had evaluated their new test for its validity; the answer was "No
one." So we accept testing based on our faith in the company that
produces it, not on our own experience (which often contradicts what the
tests "tell" us) or on independent confirmation. We have all grown up
with testing so we think that what was good enough for us must be good
enough for today's students. ("Gimme that old time religion...")

3. Is testing necessary? At the actual classroom level, I'd say yes. In
one way or another we need to evaluate what our students have learned
and testing is a convenient way to do it. There are different kinds of
tests, however, and different contexts in which they may be more or less
useful. A multiple choice test in English seems much less appropriate
than one in math or physics, for example. Is it necessary for college
admission? I argue that it's more necessary for convenience than for
true "measurement" of anything. That is, it's an adaptation of assembly
line techniques to educational evaluation: it's easy to administer,
provides a simple result, can be done with thousands of people at the same
time, and is considered "scientific" which is a kind of gold standard for most people (except in certain areas of Pennsylvania and Kansas these days). Scores can be fed into a machine and a result
pops out. This saves time and effort. For schools with massive applicant
pools, it's a life and budget saver. Imagine having to read
applications! (Although the University of Michigan managed to do it when
they realized they had to.)

4. What about heart surgeons and basketball players? I don't think this
is a fair comparison. There's a big difference between measuring truly
objective characteristics (Can she hold a knife without slitting her own
wrists? Can he toss a ball from the 3-point line and consistently make
the basket?) and measuring a concept that's mystically connected to the
actual behavior you want. (Can she "surge"? Can he "basket"?)
Vocational tests that actually refer to objects in the real world make
sense; testing concepts is a lot trickier.

5. But good testing correlates with good studenthood (great readers,
great math and science kids). OK, but correlation isn't causation, so
what is the test telling us that we don't already know or can't learn
better elsewhere? In other words, so what? The test doesn't cause the good student, and we all know from experience that many good students don't do well on the tests anyway. So while VERY OFTEN the two things correlate, that doesn't necessarily make
them dependent on each other.

6. Can good results come from bad beginnings? If you read The Chosen,
you may have to ask that question. While Conant of Harvard did indeed
use the SAT and a Harvard scholarship program to help find promising
young men from outside New England and New York, he (and the admission
people, Harvard's Board of Governors, etc.) did it in part to help stem
the tide of "undesireable" students (read: Jews): Harvard was fearful of
being "overrun" by smart immigrants and of having them drive out the
wealthy and often less-studious WASPs, so they needed to go farther
afield. (Believe it or not, Harvard once had to scramble for students just
like everyone else.) The SAT helped them do that. The story is more
complex, of course, but we also have to remember that in the 20s and through the 40s
testing mania and eugenics were big. And wasn't it interesting that
those who MADE the tests managed to have takers of their own ethnicity
come out on top?

7. Is it the test or our relationship to the test? Personally, I have
fewer problems with the test itself than I do with the ways it has
become an idol, a crutch, and a crude shorthand replacing our own
judgements about unformed, developing, crazy human beings. I have a
problem with its being called a "standard" when it is used when it's
useful and ignored when it's not or when it provides inconvenient
results. If testing really is a "standard" then it should be used
consistently and brutally no matter what the result (and we know, for
example, that would mean far fewer non-white students entering college,
as we've seen); if not, then people whose scores are less than others'
but who don't get admitted to schools have a right to complain. I have a
problem with being expected to take on faith the word of the company that
controls the test regarding its wonderfulness and usefulness. If we want
testing simply to confirm what we already know or want to know, that's not
science, that's intelligent design.

8. Can we get along without standardized testing in college admission?
Probably not, in one form or another, at least in larger colleges and
universities. There are too many students, too many schools, too many
limitations on time and budgets. Smaller institutions have every reason
NOT to use it; in fact, emphasis on NOT using testing can be a remarkable
distinguishing characteristic for colleges that pride themselves on their
"holistic review" of applicants. Several small college presidents have
recently written excellent essays about why they have made testing
optional or abandoned it. The more I see of that the more I like it.

9. Is there no end to this list? Yes.