From Halcolm’s Evaluation Fairy Tales

    Not so long ago, in a faraway kingdom, there lived a Queen whose beauty was matched only by her vanity. She desired to be assured that her beauty was unparalleled. While she was constantly so assured by her loyal court subjects, she came to doubt their objectivity. She knew they feared the powerful spells she could cast when provoked by something that displeased her, like any suggestion that she might not be the fairest of the fair.

    Having been well-schooled in the lore of queens, she became obsessed with the desire to obtain one of those magic talking mirrors mentioned in ancient tales. Lacking a magic mirror, the Queen stood daily in front of her quite ordinary mirror, repeating hopefully:

                Mirror, mirror, on the wall,

                Who’s the fairest of them all?

    Her distress at receiving no answer grew until one day she shouted in rage:

                Stupid mirror on the wall,

                Why in hell don’t you answer my call?

    The silence that followed her outburst was short-lived, for that quite ordinary mirror suddenly became quite extraordinary:

                Sorry, my Queen, for making you vexed,

                But the question you pose is terribly complex.

    The Queen was startled, but not speechless. In a twinkling, she had overcome all sense of shock, and with queenly cool she proceeded to pose yet again her burning question.

                Mirror, mirror, tell me at last,

                Am I the fairest that this world hast?

    The Queen waited expectantly, but, alas, the magical mirror seemed once again to have lost its magic, or at least its voice. The Queen was becoming convinced that her obsession had made her hallucinatory, when finally the mirror spoke again, this time at length.

                Sorry, my Queen, for this slight delay,

                But I’ve been in search of a practical way

                To approach your question with scientific care.

                There are a few things of which to beware.

                We’d have to observe all the women there are,

                The ones who are near, the ones who are far.

                Even in fairy tales, the task is so great,

                Our evaluation data would soon be out of date.

                Two years in the making such research would be,

                And then the results would be old, you see.

                A judgment of beauty that’s two years old

                Might be thought invalid by those who behold

                A Queen whose appearance must change over years,

                Giving statisticians “maturation effects” fears.

                Now instead of the universe, we might try a sample,

                But the size of the sample would have to be ample,

                And then there’s the chance we’d miss the most pretty.

                That design would leave doubts; a validity pity.

                Nor is measurement easy in a case like this.

                Judgments of beauty are like judgments of bliss.

                What’s beauty to the gander, may not be to the goose;

                The definitions, I fear, are terribly loose.

                Then, too, it would seem, the question needs rephrasing,

                So that, scientifically speaking, it isn’t so fazing.

                It would help if you asked this magic mirror on the wall,

                “Within how many standard deviations does my beauty fall?”

                And then there’s the problem of political propriety,

                When studying beauty in today’s society.

                Some might raise objections reflecting the schism

                Over whether beauty judgments constitute sexism.

                So, you see, my Queen, the study can hardly be done.

                But wait! Here’s data from a new computer run:

                Observations just made at your stepdaughter’s college,

                By fraternity boys with very personal knowledge,

                Definitely show that of all girls in the dorm,

                Your stepdaughter’s beauty is way above the norm.

                Compared to all girls we have on our file,

                Your stepdaughter’s in the 99th percentile.

    The Queen was incensed that her stepdaughter, of whom she had long been jealous, should receive such a judgment, while she, the Queen, received no satisfaction from the mirror. Her rage renewed, she took hold of her jeweled hairbrush to smash the mirror. As she cocked her arm to deliver the fatal blow to scientific reflection, the mirror hastened to add a mitigating observation.

    Your stepdaughter’s ranking is from a small study,

    The results of which are clearly quite muddy.

    There’s really no call for you to feel exasperation

    Over some psych student’s first draft dissertation.

    As for yourself, Queen, I’ve searched far and wide,

    There just is no research that yet can provide

    The statistics on beauty, your question to answer.

    The norms just aren’t there. All the funds went to cancer.

    But this current state of knowledge does permit one conclusion,

    (For the absence of numbers need not result in confusion).

    Based on available data as to who is now most fair,

    Your Queenship’s royal visage is simply beyond compare.

    The Queen relented from her threatened blow. The mirror’s snow white job proved to be quite effective. She immediately arranged for her stepdaughter to have a fatal accident while serving as a control subject in research on dwarfism. But seven dwarfs in the experiment rescued her from the machinations of the evil experimenter, an unscrupulous researcher living off fat royal grants, who, nevertheless, managed to always avoid having his proposals reviewed by the local committee for the protection of human subjects.

    The Queen became a lavish supporter of social science research on new ways of studying beauty. While awaiting a methodological breakthrough, she purchased her own personal Apple IX computer and commissioned a software package that allowed her to use dummy data to run simulations showing that she was, indeed, the fairest of the fair.

    When the breakthrough failed to come, and she had grown tired of alternately running simulated beauty studies and playing Kingdom Invaders on the castle computer, she cast a spell on the whole kingdom. This spell put everyone to sleep for a hundred years in the hope that, by then, somewhere in the rest of the world an evaluator would have developed new methods to deal with the practical problems so arduously detailed by the magical mirror. The spell is timed to wear off any year now. This book was written to warm evaluators worldwide that the Queen’s rage will be turned on them if she awakens to find no progress in practical evaluation.

    Patton, Michael, Practical Evaluation, Sage, 1982, pp. 11-13