Most tests never get past memorization
You can pass a test about a subject and still be unable to use it. Most people have lived some version of this: the exam went fine, and then the first real problem, patient, contract, or bug went badly. The gap is not mysterious. A test measures the level of knowing it demands, and most tests demand the lowest ones.
Knowing has levels
Educational research has long distinguished levels of knowing: remembering a fact, explaining it in your own words, applying it to a new case, analyzing why something behaves the way it does. These are widely known as Bloom’s levels. A question only ever measures the level it demands, whatever the teaching hoped for, and most assessment demands the bottom two.
The levels are easy to see once you look for them. “In what year was the regulation adopted?” is remembering. “Explain what it changed, in your own words” is understanding. “Does it apply to this new situation?” is applying. “Why does it fail in this edge case?” is analysis. Same subject, four different achievements, and being able to do one of them says surprisingly little about the next.
The 93 percent
Momsen and colleagues collected 9,713 exam and quiz questions from 50 university instructors teaching introductory biology and rated each one against Bloom’s levels. 93 percent sat at the bottom two levels, remembering and restating. 6.7 percent asked students to apply anything. Less than 1 percent asked for more than that. The pattern held regardless of class size and institution type, and the stated goals of a course did not predict what its exams asked. Courses promised analysis and tested recall.
There is no reason to think biology is unusual here. The incentives that produce the pattern operate everywhere.
Why tests sink to the bottom
Recall questions are cheap to write, fast to grade, and objective. Recognition is cheaper still: multiple choice puts the right answer on the screen, and with four options, guessing alone yields 25 percent. Production and application are expensive, because someone has to judge an answer against a standard and handle the partial credit. Under time pressure, assessment drifts toward what is cheap to measure, and teaching then drifts toward the assessment. Nobody decides any of this. It accumulates.
What checking above memorization looks like
A check above memorization asks you to produce rather than select: explain in your own words, solve a case you have not seen, write the code and run it, argue a judgment. The standard has to exist before your answer does, as an answer key or a rubric, because a grader that decides the bar after reading your answer is grading its own mood. And delay helps: a check right after teaching measures short-term recall, while the same check later in the session measures what actually stayed.
None of this requires an institution. A study partner with a rubric, or you writing the explanation before checking the source, clears a higher bar than most exams do.
Name the level you are aiming for
A practical habit that costs nothing: attach a level to each thing you learn. “Know what a confidence interval is” and “decide whether a confidence interval applies here” share every keyword and are different targets, reached by different amounts of practice. If you only ever practice the first, the real-world version of the second will find you out. Deciding the level up front also tells you when to stop, which is the part ambitious learners get wrong most.
Where Ulern stands
We build Ulern, a learning system, so read this section knowing that.
Ulern tracks everything you learn as a learning target: the idea plus the level you are aiming for. A target aimed at application is not satisfied by recognition, no matter how many multiple-choice answers you get right along the way. The final check on every target asks you to produce at the target’s own level, and until you have, the target stays open. Progress in Ulern is a list of targets proven and targets still open, not a percentage of pages viewed. This per-target bar is the main thing that separates a session from ordinary quizzing. We have not yet run an external trial; the study above is a large part of why the design refuses to count recognition as mastery.