Every conversational AI system now on the market has quietly made a decision on behalf of its users, often without announcing it as a decision at all. That decision concerns personalization: should the system adapt to a person because that person told it to, or because the system inferred it from a growing record of past exchanges? The question sounds procedural, almost technical, the kind of detail that belongs in a settings menu rather than in a serious discussion of how these systems should be built. It is not. It is a question about authorship, about who gets to write the description of a person that a machine then acts upon, and about how easily that authorship can be reclaimed once it has been handed over.
This distinction has a long history that predates large language models by decades. Recommender systems and adaptive hypermedia platforms have always drawn a line between explicit feedback, information a user deliberately provides, and implicit feedback, information a system extracts from behavior the user never framed as a statement of preference. What has changed with conversational AI is the stakes attached to getting the balance wrong. A recommender system that misjudges a person's taste in films produces a bad suggestion. A conversational system that misjudges what kind of answer a person needs right now, based on what kind of answer that person has needed on other occasions, produces something closer to a misunderstanding of the person themselves.
Two roads to personalization
Explicit personalization asks the user to do the work. It takes the form of configuration settings, onboarding questionnaires, direct instructions, and stated preferences. Its virtue is honesty. Whatever the system believes about a person is exactly, and only, what that person has said. Nothing is assumed, nothing is filled in on the system's own initiative. The cost of this honesty is friction. It requires users to have already formed a clear picture of their own needs, to translate that picture into whatever vocabulary the system offers, and to keep revisiting that translation as circumstances change. For many people, in many moments, this is real work, and work unevenly distributed across users with different amounts of time, patience, and technical fluency.
Implicit personalization takes the opposite wager. It infers a model of the user from accumulated behavior: the questions asked, the corrections made, the topics returned to again and again. Its virtue is convenience. It asks nothing of the user directly and can draw on patterns a person would never think, or want, to state out loud. Its cost is subtler and more consequential. Any inference is a claim about a person made without that person's ongoing consent, and unlike an explicit statement, an inference does not expire on its own. It persists until something forces it to be revised, and in most systems built today, very little forces that revision to happen quickly.
Inferential ossification can turn from a mere technical burden into something closer to a curse upon your relationship with the system, freezing its image of you at some past moment, until every change in you collides with a memory that refuses to believe you.
Chief Scientist Setaleur Aplamda
The hidden cost of letting a system decide who you are
The central danger here is not simply that implicit models can be wrong. Wrongness is a manageable problem; any system with feedback loops will occasionally misjudge someone and can, in principle, be corrected. The deeper danger is that these models tend to become sticky. A profile built from historical interaction is, by its nature, a summary of the past, and once a system settles on a version of what a person wants, it tends to serve more of the same, narrowing what that person is offered or how that person is addressed. Content recommendation platforms have already made this dynamic familiar under the name filter bubble, the progressive narrowing of what a person is shown as a system reinforces its own earlier read on their preferences.
Conversational AI faces a structurally identical risk, but at the level of register and depth rather than content alone. A system that has learned to answer a person casually will keep answering casually, not because casualness remains correct, but because casualness is what the evidence supports, precisely at the moment a person most needs something else. This is a problem researchers on non stationary environments have a name for. Most personalization built on historical data assumes that a person's underlying preferences are reasonably stable. When they are not, the resulting mismatch is known as concept drift, and any system that fails to detect it will systematically misread the very people it is trying to serve. Long horizon personalization in conversational AI is a concept drift problem that most current designs do not treat as one. They update slowly, they weight the past evenly, and they rarely distinguish a genuine shift in a person's baseline from a single situational departure from it.
Traits are not states
A second failure sits underneath the first, and it is conceptual rather than statistical. Long run behavioral inference tends to conflate two categories of information that should never be treated as one. There are stable dispositional traits, a general preference for brevity, a home domain of expertise, and there are transient situational states, the fact that on this particular occasion the same person wants unusual depth, or unusual simplicity, regardless of what their historical average would predict. A system trained to detect the former will misapply it to moments that call for the latter, not because its inference was inaccurate on average, but because averages are the wrong instrument for the question being asked. This is the precise experience of a system that decides who a person is and then imports that decision into a request it was never meant to govern. Building implicit personalization responsibly requires more than learning preferences over time. It requires learning which of those inferences are durable and which merely described a moment that has already passed.
Giving implicit personalization its due
A mature position on this question cannot simply declare manual control the safe option and implicit inference the risky one. That framing flatters explicit systems more than they deserve. The cognitive tax of explicit configuration is easy to underestimate. It requires a person to have already formed a clear model of their own preferences, to translate that model into a system's vocabulary of settings, and to revisit the translation as needs evolve, and for many people, in many contexts, this tax exceeds the cost of an occasional misjudgment by an implicit system. The strongest defenders of implicit, behavior driven personalization make a real argument when they say that the most valuable systems are those that anticipate needs rather than requiring people to specify them in advance.
The line between these two paradigms is also less binary in practice than the theoretical picture suggests. People do not simply submit to implicit inference passively. They routinely and deliberately shape their own behavioral signals, engaging repeatedly with certain kinds of responses or conspicuously avoiding others, as an indirect but intentional way of steering a system that offers them no direct controls. This pattern, sometimes called intentional implicit feedback, matters for two reasons. First, it shows that people already treat behavior as a communication channel when no better one is offered, which suggests that some of the apparent success of implicit personalization is a workaround for absent explicit controls rather than proof that explicit controls are unwanted. Second, it points toward the design opportunity that follows.
A framework for judging any system
Any personalization architecture, whichever paradigm it favors, can be evaluated along five dimensions.
Design patterns that work
The choice facing builders is not a binary between explicit and implicit personalization but a design space of hybrid patterns.
An implicit model can be exposed as a concrete, editable artifact rather than a black box, a visible list of inferred facts, each individually removable or correctable. This addresses legibility and correctability at once, converting an opaque, ambient profile into something closer to a jointly maintained record.
Not all inferences carry equal risk. Confidence about a person's preferred tone or level of detail is lower stakes to get wrong, and easier to correct in the moment, than confidence about a person's factual beliefs or substantive goals. Systems that permit continuous, low friction adaptation of style while requiring explicit confirmation before adapting substance concentrate risk where it is cheapest to bear.
Treating every inference as either permanent or entirely temporary is a false choice. A more defensible middle path lets inferences decay in confidence when they are not reinforced by subsequent behavior, so an unreinforced inference loses influence over time rather than being carried forward indefinitely as though it were settled fact.
Above all, an explicit, in context instruction from a person should always take precedence over any prior inference, immediately and without requiring that person to first locate and disable a memory setting buried in a menu. A person who explicitly asks for unusual depth should receive it at once, regardless of what the system has inferred from history, and that correction should never need to outweigh an accumulated statistical prior before it takes effect.
What this means for builders and regulators
For those setting standards in this space, the practical requirements that follow are largely neutral to which paradigm a system favors. People should have a right to inspect, in plain language, any persistent inferred model a system maintains about them. Correction should be available at the level of individual claims, not only through a global reset that discards an entire personalization history as the price of fixing one error. Where an inferred model is used for purposes beyond improving the immediate interaction, for advertising or engagement optimization, that secondary use should require separate, informed consent. And persistent implicit personalization should default to a state a person can easily suspend, rather than demanding active, well informed effort to opt out of a system already collecting and acting on inferences about them.
For those advocating on behalf of users inside product organizations, the demands are more cultural than regulatory, but no less concrete. Insist that explicit instructions always and immediately override accumulated inference. Push for interfaces that make a system's current working model of a person visible by default, not buried where an ordinary user will never look. Draw a clear line between adapting how an assistant communicates and adapting what it assumes about who a person is, since the two carry very different risks when a system gets them wrong. And resist designs in which the commercial beneficiary of a person's inferred profile is anyone other than that person, demanding disclosure wherever this is not the case.
The deeper point beneath all of this is that personalization is not a feature to be maximized. It is a relationship to be governed. A system that learns about a person over time is not inherently a threat to that person's autonomy, but a system that treats what it has learned as a fixed verdict, immune to correction and blind to the difference between who a person generally is and who that person happens to be in this particular moment, has stopped serving the person it claims to know. The task ahead is not to choose between explicit control and implicit inference. It is to build systems honest enough to admit what they believe, humble enough to update it quickly, and disciplined enough to let a person's own words, spoken right now, outrank everything the system thinks it already knows.
For reference: https://doi.org/10.5281/zenodo.21648713