Tests of educational achievement

Two stories, both told to me by the same person. The first happened to a family member. The second is from a book by Don Mitchell. Both are about a school system's testing of what a child had learned at home. In both cases, the child was perhaps around 5 or 6 years old.

1. The examiner marked the child down for not knowing something about Pinocchio. The examiner did not ask, and so never found out, that the child could identify on a map all the states in the country.
2. The examiner showed the child a picture and asked the child to say what it was. The child studied it for a long time, and then said, "It could be a unicorn, but I don't see a horn." The examiner wrote, "Is not able to identify a horse." Then the child said, "It looks like a bit like a horse, but the ears are wrong," and the examiner crossed off what she had written.

In both these cases, the child was smarter than the test was prepared to measure. In these examples, the measure of educational achievement was a one-on-one interview, but these stories make me wonder about standardized tests. These days, the makers of standardized tests attempt to be more sophisticated and measure things like critical thinking and writing, rather than memorization. But can a standardized test ever measure anything other than how well people do on standardized tests? The tests do give us a general idea of what people know, and are useful when we want some imperfect information on a large number of people. But we should not think that they tell the whole story of what an individual knows.