What Are We Measuring When We Measure Intelligence?
What Are We Measuring When We Measure Intelligence?
From scores and repeated measures to eigenchange, relational assessment, and intelligence as the capacity to move
The score and the encounter
Intelligence has traditionally been treated as something a person possesses. We construct tasks, observe performance, compare the result with a reference population, and assign a score. That score can predict outcomes including educational attainment, training success, and aspects of occupational performance. This predictive value is real and should not be dismissed (Salgado et al., 2003).
Yet a persistent question remains: what exactly have we measured?
Have we measured intelligence itself, or performance produced by one person, in one state, responding to one instrument, under one set of conditions?
A test does not encounter intelligence in isolation. It encounters a temporary configuration of attention, motivation, memory, stress, confidence, familiarity, language, cultural experience, and perceived consequences. The score belongs first to that encounter. We then transfer it from the encounter to the person and treat it as a relatively stable property.
Standardization makes comparisons possible by controlling variation. But some of the variation it controls may contain precisely what we need to understand: how a cognitive system changes when the situation, relationship, pressure, or frame changes.
Intelligence may not be a height that a person reaches. It may be the capacity to enter, sustain, reorganize, and leave states without unnecessarily losing calibration, meaning, or direction.
From the jump to the trajectory
In the preceding essay, LLMs Can’t Jump, we challenged the appearance of sudden cognitive leaps. What looks like a jump from one idea to another may be a trajectory whose intermediate steps were compressed, discarded, forgotten, or never made visible.
The same may be true of human intelligence. A “genius moment” appears sudden because we observe its arrival, not the accumulation that made it possible: abandoned formulations, weak signals, emotional activation, analogies, environmental cues, periods of incubation, and the gradual reorganization of meaning.
The apparent jump may be an observational gap.
This changes the question. Instead of asking only how high someone can score, we can ask how a person, or a cognitive system, moves:
- How does it enter a productive state, and what makes that state available?
- How long can it sustain the state, and at what cost?
- What happens when the rules or the frame change?
- Can it recognize that its current trajectory has become inadequate?
- Can it leave an unsuccessful state and recover its range?
- What residual remains after the interaction has ended?
- Can an insight survive transfer into another domain?
From this perspective, intelligence is not simply the height of cognitive performance. It is the capacity to navigate change: to enter, sustain, reorganize, and leave states produced through interaction without unnecessarily losing calibration, meaning, or direction.
The misguidance of IQ and EQ
IQ tests are not meaningless. They measure particular cognitive performances under standardized conditions, and some of those measurements have substantial predictive utility. The misguidance begins when measurement becomes ontology: when performance inside a carefully constructed frame is treated as intelligence in its entirety.
Most tests evaluate how effectively someone operates after the task, rules, and response space have already been selected. Real-world intelligence often begins one level earlier: is this the right problem, the right frame, and the right set of variables?
A person may perform exceptionally within a frame while remaining unable to recognize that the frame itself is producing the problem. Another may perform modestly in a timed, decontextualized task yet show exceptional ability to reorganize ambiguity, integrate perspectives, or recover after disruption. Research on dynamic assessment has long attempted to observe how performance changes with feedback and scaffolding rather than treating an unaided score as the complete picture (Swanson & Lussier, 2001). Our proposal belongs in that lineage, but extends the unit of analysis from improvement within tasks to movement across states and contexts.
EQ introduces a related difficulty. Emotional-intelligence instruments do not all measure the same thing: they may rely on self-description, informant reports, emotional knowledge, or ability-based tasks, and different approaches can yield weakly related portraits of the same person (Brackett & Mayer, 2003; Conte, 2005).
Recognizing an emotion, however, is not the same as navigating what happens when another system changes us.
Eigenchange and empathy
Within the Reflectum paradigm, we call such alteration eigenchange: a change in a system’s cognitive-emotional configuration produced through interaction with its environment.
Signals may come from any system in the Field capable of vibration, meaning capable of generating, carrying, or transforming a difference. Vibration is used here as a systems concept, not merely as a mechanical or mystical claim. A person, group, language, institution, landscape, piece of music, or AI system may all introduce differences that change what another system can perceive, feel, or consider possible.
Empathy is one relational expression of this broader capacity. It requires sufficient resonance for signals associated with another experience to alter what we perceive and feel, but sufficient differentiation to prevent the induced state from being mistaken for the other’s experience.
Too little resonance produces detachment. Too much may produce fusion, emotional contagion, or loss of calibration. Empathy is therefore not maximal sensitivity, and it is not merely correct emotional classification. It is the calibrated navigation of relationally induced eigenchange.
This is difficult to represent through a single EQ score because the relevant intelligence exists in the movement: how someone enters, differentiates, regulates, and leaves a state altered through interaction. IQ and EQ risk dividing cognition and emotion into separate possessions when both may be expressions of a wider transition architecture.
Why repeated measurement helps, and changes the relationship
Repeated measurement offers an obvious improvement over the isolated score. Instead of treating one performance as definitive, it allows us to observe variation across time and distinguish persistent patterns from fatigue, stress, novelty, practice, mood, or situational disruption. Intensive longitudinal methods such as ecological momentary assessment bring measurement closer to everyday contexts and reduce dependence on distant retrospective reports.
But repetition introduces another source of distortion. The person may learn the task, infer what is being assessed, redirect attention, and develop strategies. They may try to improve, resist, please the evaluator, or preserve a preferred image of themselves. The instrument may eventually be measuring not only the target phenomenon but adaptation to being measured.
This possibility is known as measurement reactivity. Evidence does not justify treating it as universal: some studies find small or context-dependent effects, while many do not test it at all. A large meta-analysis of ecological momentary assessment studies found that 95.8% did not report a test of potential reactivity (Wrzus & Neubauer, 2023). Reviews consequently recommend treating reactivity as a property to investigate within each protocol, population, and timescale, not as a problem that can be assumed away (Ram et al., 2017).
We can distinguish at least three layers.
Continuous observation is not neutral observation
A more naturalistic approach would observe cognition across ordinary situations rather than repeatedly summoning it into a focused event labelled a test. This could capture more of the range through which a person actually lives and thinks.
Yet “continuous and non-focused” must not become a euphemism for invisible surveillance. Passive sensing may reduce interruption while increasing opacity. The less visible measurement becomes, the less opportunity the observed person may have to understand, contest, or withdraw from the model being constructed about them.
Nor does naturalistic observation remove the observer. People do not think, feel, and speak identically before a clinician, employer, friend, anonymous questionnaire, or AI system. Each observer changes what becomes salient, safe, desirable, or expressible. The observer is not outside the phenomenon; the observer helps establish the state in which the phenomenon becomes visible.
The objective is therefore not neutral observation. It is an assessment architecture that records its own participation.
Toward continuous relational assessment
We propose continuous relational assessment as a direction rather than a finished instrument. Its purpose would not be to extract a permanent intelligence score or to measure everything continuously. It would infer patterns from ethically governed samples across time, conditions, and relationships while estimating the changes introduced by observation itself.
Its possible dimensions include:
What AI changes
Classical psychometrics did not lack process theories, dynamic assessment, longitudinal designs, or concern about bias. AI should not be presented as their inventor. Its contribution is different: it changes the attainable granularity, continuity, and scale of trajectory observation.
AI-assisted environments can preserve successive prompts, revisions, discarded branches, recurring metaphors, confidence changes, conceptual compressions, and shifts of frame. They allow us to inspect not only an answer but portions of the route through which the answer emerged. Frames can be compared, weighted, and stress-tested across a body of work that would previously have been difficult to retain and analyse manually.
But AI intensifies the central paradox. The system helping us observe the trajectory also participates in constructing it. It proposes language, retrieves associations, stabilizes categories, and changes what the person considers possible. It can extend a trajectory, compress thousands of words, and preserve formal coherence while meaning quietly thins, drifts, or disappears.
More representation does not necessarily create more information, and more information does not necessarily become meaningful knowledge.
AI therefore does not provide transparent access to “true intelligence.” It creates a new relational environment in which human and machine contributions become partially entangled. The relevant question is no longer only “How intelligent is this person?” It becomes: what capacities become available in this configuration, what conditions produced them, and what remains when the configuration is removed?
Range rather than height
Our working model treats intelligence as multidimensional state-transition capacity. Its central variable is not only height but range.
A system does not think independently of its environment. Every problem, relationship, instrument, and observer introduces signals that may alter its configuration. Some changes are barely perceptible; others reorganize what the system can perceive, feel, or consider possible. Intelligence becomes visible in how the system navigates these eigenchanges.
Can it enter different states without immediately becoming captured by them? Can it distinguish an external signal from the state that signal induces internally? Can it preserve meaning under compression, reorganize when a frame fails, and leave a trajectory that no longer serves the problem?
Empathy belongs to this architecture. It concerns eigenchange carrying signals associated with another experience. Analytical reasoning, creative insight, and adaptation involve different configurations, but each depends on movement within a wider Field.
High performance in one state remains valuable. Yet fixation at that state becomes a liability when conditions change. The practical objective is not permanent residence at a cognitive peak; such a state may be impossible, undesirable, or biologically costly. The objective is to preserve the conditions under which an appropriate transition can occur.
The meaningful variable may be range rather than height: how many states can the system enter, tolerate, and leave without becoming trapped?
A different psychometric question
The next generation of intelligence research may not need only a better ruler. Intelligence was never simply a length.
It may require instruments capable of observing trajectories while accounting for their own participation in producing them. Such an instrument would not ask only whether a person solved the problem. It would also ask:
- Which state made the solution possible?
- How was that state reached, sustained, and left?
- What alternatives were suppressed by the selected frame?
- Was meaning preserved as complexity was compressed?
- Did the interaction expand future possibilities or narrow them?
- Would the capacity remain if the observer, test, or AI disappeared?
A score can tell us something important about performance. A trajectory can tell us how that performance became possible, how stable it is, what it costs, and whether it can survive change.
The future of intelligence research may therefore depend less on finding a button that raises intelligence and more on understanding the transitions that make an intelligent moment possible.
Intelligence may not mean remaining at the highest point. It may mean retaining the capacity to move, and knowing when another state is required.
- Brackett, M. A., & Mayer, J. D. (2003). Convergent, discriminant, and incremental validity of competing measures of emotional intelligence. Personality and Social Psychology Bulletin, 29(9), 1147–1158. doi.org/10.1177/0146167203254596
- Conte, J. M. (2005). A review and critique of emotional intelligence measures. Journal of Organizational Behavior, 26(4), 433–440. doi.org/10.1002/job.319
- Ram, N., Brinberg, M., Pincus, A. L., & Conroy, D. E. (2017). The questionable ecological validity of ecological momentary assessment: Considerations for design and analysis. Research in Human Development, 14(3), 253–270. doi.org/10.1080/15427609.2017.1340052
- Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & De Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European Community meta-analysis. Personnel Psychology, 56(3), 573–605. doi.org/10.1111/j.1744-6570.2003.tb00751.x
- Stinson, L., Liu, Y., & Dallery, J. (2022). Ecological momentary assessment: A systematic review of validity research. Perspectives on Behavior Science, 45(2), 469–493. doi.org/10.1007/s40614-022-00339-w
- Swanson, H. L., & Lussier, C. M. (2001). A selective synthesis of the experimental literature on dynamic assessment. Review of Educational Research, 71(2), 321–363. doi.org/10.3102/00346543071002321
- Wrzus, C., & Neubauer, A. B. (2023). Ecological momentary assessment: A meta-analysis on designs, samples, and compliance across research fields. Assessment, 30(3), 825–846. doi.org/10.1177/10731911211067538