Chapter 15 analyzed how anxiety enters value judgment. When a system claims it can identify risk and offer personalized help, it usually needs more material: study duration, clicks, location, relationships, bodily states, and consumption choices. Data may make services more accurate, but it may also turn a subject's hesitation, recovery, and future intentions into resources that institutions can hold on to indefinitely. Who contributes material, who receives the returns, and who gets to define the uses are distinct questions. The program launches "Growth Navigation," which records study and job-seeking behavior for free and recommends courses to Ning. Ning receives reminders and an organized portfolio; the enterprise receives candidate risk scores; the training provider improves its product with aggregate data; the municipal department allocates subsidies accordingly. A single record simultaneously yields personal, commercial, and public returns, and it also brings misclassification, leakage, raised thresholds, and adverse inferences about those who did not participate. This chapter does not use thought experiments to assert how real platforms handle data, nor does it provide methods for profiling, targeting, or influencing choices. It confronts, head-on, a harsher possibility: a system can profit from observing vulnerability, strip those who refuse observation of opportunity, and then use the behavioral outcomes to prove the profile accurate. Critique must land on permissions, evidence, interests, and repair, not on declaring technical use or commercial gain itself to be harm.
Data Relations, Returns, and Authorization
Ning completes exercises in the Navigation system, and her behavior can be observed by the system; this does not mean the records may be used for hiring, insurance, or public rankings. Visibility is a technical fact; use is an institutional decision. A single act of consent should specify the object, purpose, duration, and consequences of refusal; "the data already exists" cannot answer on authorization's behalf. Certain operational data are necessary to provide the service and may be handled under contract and public rules; additional profiling requires stronger justification. Calling all data user-owned or platform-owned alike obscures the multiplicity of relations involved.
Ning's recommendation letters, chats, and collaboration records involve peers, teachers, and clients. She cannot unilaterally sell the entire relationship to a platform, and the platform cannot fulfill its ethical responsibility merely by obtaining permission from the account holder. Materials concerning multiple people require minimization, de-identification, and task-relevant permissions. Joint production does not mean that every person holds unlimited veto power. Public records and legal liability may require retention; institutions should state the exceptions and avenues of appeal rather than pursue an impossible unanimous consent on each occasion.
Risk scores, capability types, and future predictions are not directly supplied by Ning, yet they are generated from her behavior and from models. Institutions may call them internal knowledge, but Ning bears the consequences for her opportunities. For derived data with high consequences, the subject should be able to learn the principal basis, correct inputs, and challenge uses, without the full model details that would endanger safety or rights having to be disclosed. Inference is not fact. Systems should state probability, time, and applicable scope; they cannot write predictions into personality attributes and then validate themselves with performance measured after the fact.
After Ning turns off tracking, the system may read the missing data as low engagement or high risk. Refusing observation thus itself becomes adverse data, draining consent of meaning. Institutions should establish an "unknown" status, offer other forms of proof, and not infer capability directly from a privacy choice. Some tasks genuinely require records, such as evidence of exercises for safety certification. The necessary scope should be defined by the task; refusal may affect eligibility for that item, but it cannot be expanded into a verdict of general untrustworthiness.
Returns Must Be Disaggregated by Source and by Time
Ning receives one course recommendation, whose value may be limited; the platform uses years of behavior to improve models, sell services, or enter new markets, and those returns persist far longer. Whether the initial exchange was proportionate cannot be judged only by the free features of the moment; future uses, risks, and exit must also be stated. Long-term returns are hard to attribute precisely to any single individual, but that is not grounds to declare the contribution zero. Institutions can adopt use restrictions, collective distribution, service improvement, or public returns, without having to put a price on every click.
Training data, engineering, maintenance, capital, and user feedback jointly produce model capability. Attributing all value to data providers erases the other labor; attributing all of it to the platform treats the material of life as a free natural resource. Distribution should reference contribution, control, risk, and public rules, not be decided by a single metaphor of ownership. When contribution is hard to measure, procedural rights matter more than pseudo-precise profit-sharing: restrict uses, allow exit, disclose categories of return, and let affected groups participate in the decision.
The municipal department uses aggregate data to discover that training is insufficient in a district and can add resources; if individual locations and job-seeking records are identifiable, they may also produce stigma. Anonymization and aggregation help, but small groups, external combination, and long-term tracking can still re-identify people. A public aim does not automatically cancel individual boundaries. Refusing all public use may likewise leave resources allocated to the most visible groups. The more sound course is minimal material, independent governance, effect review, and exit mechanisms, with the public returns actually flowing back to those observed.
The platform gets the data now; Ning may lose opportunities to an old profile only years later; leaks and changes of use also arrive on delay. A one-time short-term subsidy does not automatically settle future risk. Retention periods, deletion, and use updates determine the lifespan of the relation. Long-term research has real value and may be retained under explicit public purpose and protections. Permanent retention cannot be justified merely by "it might be useful later."
The Right to Observe Is an Institutional Capability That Requires Authorization
The Navigation system can record every pause, withdrawal, and mouse path; the existence of the capability does not amount to necessity. Institutions should reason backward from decisions to the minimal fields: what is needed for the recommendation module, what is needed to verify eligibility, and what serves only the convenience of future development. Technical defaults must not answer on behalf of public judgment. Collecting less may reduce personalization, and services should describe the difference honestly. A subject who chooses the generic version should not be excluded from basic opportunities.
A mistaken entertainment recommendation may waste only time; a hiring risk score changes income and career. The latter demands stronger evidence, human review, explanation, and appeal. A model's accuracy rate is empirical material; it cannot make the value choices about the domain of use and the cost of error. When the same data crosses from low-consequence services to high-consequence decisions, authorization and validation should be redone. Historical performance cannot carry over automatically to every domain.
The platform requires Ning to make her learning process visible while disclosing nothing about model versions, commercial relations, or error distributions; visibility is asymmetric. Public accountability should target rules, access, returns, and decision records, so that the observer's power remains verifiable. Employees and managers still retain a private life that is irrelevant here. Transparency does not mean publishing source code or all security details. Verifiability can be built through independent audits, impact reports, case-specific reasons, and appeal outcomes.
That Ning provides data to the training provider does not mean every department within the group may use it. Internal sharing is equally a change of use, requiring roles, logs, and time limits. Copying data to more places "to improve synergy" increases leakage and interpretive drift. Where a joint service genuinely requires sharing, the subject should know the principal participants and who bears responsibility. Errors must not circulate between departments with no one repairing them.
Personalization, Profiling, and Formal Consent
The Navigation system helps Ning find relevant modules among a large number of courses, saves time, and may reveal paths she did not know. This is a real benefit; it does not become false because the platform profits. Evaluation should compare recommendation relevance, alternative entry points, and whether the subject can change the goal. If a recommendation has only one optimal answer, the model's values are embedded in the choice. Displaying multiple reasons, different orderings, and a non-personalized entrance keeps help from becoming command.
The system recommends the same type of skill continuously; Ning's portfolio and relationships gradually form around it, and switching later becomes costly. The path may reflect genuine interest, or it may trace back to early coincidence. Periodically re-asking about goals, allowing history to be cleared, and exploring different fields can restore variability. Frequent resets in turn lose useful continuity. The subject should choose to retain, pause, or separate profiles, rather than the platform deciding for her by convenience.
After seeing her competency score, Ning drills on the scored items; the score rises and the model appears accurate; unmeasured collaboration and care may decline. Feedback is not an external mirror; it enters behavior. Evaluation should look at real tasks and unplanned losses, not only at the self-improvement of the metric. A subject's actively adapting to a standard is not automatically being controlled. She may be learning rationally. The question is whether the standard is relevant, transparent, and appealable, and who bears the cost of adaptation.
The system tells Ning "keep the pace, you are improving," which can lower pressure; the enterprise still has not explained its renewal rules. Emotional support and institutional responsibility are not the same service. A platform cannot fill the information obligations of those who hold power with individual encouragement. Nor must support itself be abolished; it only needs to be made explicit that it cannot guarantee opportunity. The subject should be able to move from advice back to the actual rules and to human consultation.
Profiling Can Form a Self-Validating Loop of Opportunity
The system labels Ning a high attrition risk; the enterprise reduces training; performance then declines; and the outcome is used to justify the classification. This does not mean the initial prediction carried no information; it means the outcome has already been shaped by the decision. Evaluation must record the interventions that followed classification and compare different support paths. Protective resources can also generate loops: for example, when those at high risk receive more help and outcomes improve. Improvement cannot be used to infer that the original risk was false, and deterioration cannot be used to prove character. Prediction, decision, and outcome should be kept separate.
Ning wants to correct her risk score and is asked to submit health, family, and complete study records. Appeal turns into purchasing basic fairness with deeper exposure. Institutions should first open the existing inputs and the reasons for decisions, collect only the material needed to resolve the dispute, and offer a human alternative. Nor can the subject demand the deletion of genuine duty records merely by saying she "dislikes the outcome." Correcting facts, challenging inferences, and objecting to uses are distinct rights.
People who never used the Navigation system may be assigned group scores based on address, school, or occupation. Statistical inference sometimes helps resource planning, but in individual high-consequence decisions it requires stronger relevance and a channel for correction. A subject cannot be required to prove herself an exception to a group average. Banning all group analysis would likewise conceal systemic differences. Public analysis and individual sanction should be kept separate; publishing distributions is not the same as labeling every person.
Records of Ning's early course exit still influence recommendations years later; the system holds to her old identity more stubbornly than she does. Retaining history helps find continuity, but it can also make one period of difficulty permanent. Time limits, decay, and paths to re-prove capability should be tied to the task. Deleting an old record cannot rewrite what happened, but it can limit the record's continued control over an unrelated future. Which responsibilities require long-term retention should be decided concretely by law and public rules.
Data Transactions Cannot Prove Fairness by Formal Consent Alone
If basic enrollment requires accepting advertising profiling, the subject has no real choice. A platform may request proportionate data for optional personalization, while basic eligibility uses the minimal path. Whether bundling is justified depends on the service relation, not on a single blanket consent form. Splitting choices must not, however, pile up into dozens of incomprehensible switches. Key uses, sensitive material, and high-consequence sharing should be prominent, defaults should lean protective, and adjustments can come later.
Ning receives a small discount; this still does not mean the platform may sell her entire behavior in perpetuity. Price is only one part of the exchange; privacy, discrimination, and third-party rights cannot be settled entirely in private. Some uses require public restriction even when paid. Conversely, forbidding individuals to benefit from voluntary data contribution may also overprotect. Limited authorization, clear durations, and collective bargaining can be allowed, so that returns and risks become more proportionate.
The Navigation system charges nothing, and Ning may be paying with data, attention, and inferred risk. Describing this exchange is not a claim that free services are necessarily deceptive; it makes the commercial relation complete. A service can offer clear benefits, a non-profiled version, and a deletion path. A publicly funded free service also has costs, borne by fiscal budgets and governance. It too should state its data boundaries; being non-profit does not earn it a wider right to observe.
If refusing tracking means losing the entrance to interviews, consent carries survival pressure. Institutions should prove that observation is relevant to the task and offer equivalent forms of proof. Real choice does not require zero consequences, but irrelevant privacy cannot be allowed to become a threshold for basic opportunity. Some refusals will genuinely lower service quality, and the platform may say so. Honest consequences and punitive downgrades must be distinguished function by function.
Collective Governance and Starting Over
Ning can hardly know what returns emerge when her data is combined with millions of records, and she cannot negotiate item by item. Industry rules, public regulation, and worker or user representatives can set minimum boundaries. Collective governance does not handle every preference on the individual's behalf; individual viewing, correction, and exit are still needed. Representative bodies can also be captured by suppliers or drift away from their members. Authorization, disclosure of interests, rotation, and appeal keep them limited.
A collective body can manage authorization and returns and reduce individual burden; if it owns member data permanently and refuses exit, the old concentration has merely changed its name. Governance should center on uses and responsibilities, not turn subjects' material into a common asset one cannot leave. The public interest may require retaining a small number of records; the exceptions should have a statutory object, protections, and review. Nor can a community indefinitely own an individual's future.
Individual compensation, reduced service fees, public training, open models, and improved access for the disadvantaged can all return value. No single form is automatically fair. One should ask who the contributors are, on whom the risks fall, and whether the returns are actually reachable, and allow different groups to take part in the choice. Folding everything into micro-payments may understate long-term power; speaking only of public benefit may overlook individual costs. A combined scheme needs an open ledger.
External researchers verifying model bias aids public scrutiny, yet may increase leakage. De-identification, controlled environments, use commitments, and publication of results can be used. Who is granted access cannot be decided unilaterally by the platform; independent governance should take part. Full closure lets the company certify itself; full exposure re-exposes the subjects. The boundary is to be designed around verifiable power, not around maximizing data flow.
Deletion, Portability, and Starting Over
Ning deletes her account; the platform may still hold backups, statutory records, and models already trained. Institutions should state what is deleted immediately, when it is purged, what is retained for responsibility, and whether the model's influence can be reasonably removed. A promise of "one-click disappearance," if technically unachievable, manufactures new misunderstanding. Decisions already made are not automatically revoked by deletion. Wrongful consequences require separate correction and compensation; cleaning up information is not the whole repair.
Ning should be able to export her portfolio, certificates, and whatever study records she may lawfully hold, and move to another service. Standard formats and source verification make the results usable. Portability must not include others' private information or the platform's security material; the boundary must be clear. Portability sharpens competition, but it can also let data spread across institutions. The subject should choose the scope, and the receiving party should state its uses anew; one export must not authorize a permanent chain.
Ning can clear her personalized goals while keeping the qualifications she has earned; she can also require that past behavior no longer influence new recommendations. Starting over protects the capacity to change, while still retaining records tied to real responsibilities. Which layers require continuity should be decided by the task and its consequences. A fully anonymous new identity could be used to evade debts or liability for harm, so limited verification is needed. Abuse prevention must not turn into everyone being permanently trackable.
The platform has already sent a wrong risk score to the enterprise; deleting it internally alone does not restore opportunity. Correction should notify the principal recipients, stop continued use, and let Ning know the scope of the repair. When not all copies can be recalled, the residual risk should be stated honestly. The subject should not bear all the labor of tracking downstream copies one by one. The party that initially controlled the sharing bears the primary duty to notify.
The Right to Observe within a Limited Task
If the Navigation system helps Ning master skills, discover paths, and correct errors, it increases possibility; if it only raises clicks and course purchases, the service's returns stay mainly with the institution. Evaluation should look at task outcomes, exit, and non-purchasers, not only at participation rates. Added capability may also come with data risk, and one benefit cannot settle the account for the other. The ledger needs separate columns.
Whether people who do not use the Navigation system can enroll, submit work, appeal, and obtain public information is the crux of the voluntariness of observation. If visibility extends only to those fully profiled, data consent becomes qualification. Equivalent paths may differ in cost, and their actual reachability should be checked. An identical experience is not required; basic rights and key opportunities must not disappear because of irrelevant tracking.
The model improves matching efficiency, while the total number of positions and wages may be unchanged; the efficiency is real, but it does not solve all scarcity. Institutions should state which problem is improved and how the new returns are distributed, instead of covering choices about resources with technical progress. If efficiency savings convert only into higher targets and more observation, participants remain occupied into the future. Chapter 18 will further trace where efficiency returns go.
For insufficient data, changed environments, or the subject's new goals, the system should output "unknown" rather than maintain a complete profile. The unknown state permits human judgment, supplementary material, and abstention from high-consequence decisions. Institutions need processes configured for uncertainty; a model always having an answer cannot become a commercial promise. The subject also has the right to disagree with an inference, which does not automatically prove the system wrong. Disagreement triggers review; the final conclusion still requires relevant material.
Returning the Right to Observe to the Limited Task
The observation relation should also let subjects know how they are not being observed. If a permission list only enumerates the fields collected while saying nothing about the high-consequence uses expressly forbidden, Ning must still guess whether every pause, click, and silence will enter eligibility judgments. Forbidding the use of learning-navigation data for unrelated hiring sanctions, and requiring fresh authorization for exceptions, is more verifiable than a blanket promise of "responsible use."
An institution can promise never to sell sensitive data and still must explain what counts as selling, and whether affiliated companies and model access are included. The boundary should track actual capability and consequence, accepting independent audit and remediation for violations. This will not eliminate every inference, but it lets subjects know which worries have an institutional answer. Overly broad prohibitions may also obstruct safety, research, and the portability a subject requests. Exceptions should be constituted by object, necessity, permission, and duration, not expanded ad hoc by internal convenience. The distribution of the right to observe must also handle three situations commonly obscured by narratives of return: those who have no data, those inferred from others' data, and those who have changed yet remain decided by old records. Whether Ning receives a one-time compensation answers only part of the existing transaction; if the platform can still infer her from similar populations and lower her opportunities from missing records, the subject's actual control over the observation relation has not been restored.
Ning declines to sync her nighttime activity, and the system may fold "unknown" into higher risk, because those with complete records are easier to predict. This practice is convenient to manage, yet it turns refusing observation itself into punishment. Missing data may stem from a privacy choice, device failure, poverty, or an irrelevant life; it may enter a judgment only when an empirically validated relevance to the task exists, and other paths of proof should remain. Nor can institutions pretend that missing data has no effect. Certain qualifications genuinely require material, and when it cannot be confirmed, a specific decision may be deferred. The key is to state what is not known, why it is needed, and how else it can be proven — not to have the model substitute a hidden score for a public threshold.
Ning has not submitted household income, yet the platform may still infer it from address, contacts, or similar groups. Item-by-item consent cannot alone solve group inference, because one person's sharing changes the visibility of others. High-consequence uses require public limits, group impact assessment, and prohibitive boundaries beyond individual consent; responsibility cannot be left entirely to the person clicking the button. This does not mean no group statistic should ever be used. Public health, transportation, and education planning may require aggregate regularities. The closer the use comes to resource allocation and individual sanction, the higher the demands on data quality, bias, appeal, and independent review; the relevance that justifies general planning cannot, unproven, be converted directly into individual destiny.
The platform says the outcome was produced by a model; the employer says it only bought a service; the data supplier says it only provided samples; a chain forms in which no party holds final responsibility. Whoever chooses the use, the threshold, the resource consequences, and the continuation of use bears the corresponding duty of explanation and repair. Technology suppliers remain responsible for known limitations, data provenance, and security, but they cannot absorb all responsibility on behalf of the actual decision-makers. When multiple parties decide jointly, Ning should be able to see a minimal map of responsibility: whom to petition for correction, who can suspend adverse consequences, who notifies those downstream. A complex supply chain is no reason the subject must guess her way through it layer by layer.
Data projects usually count new matches, transactions, and completion rates, but they do not record the jobs not displayed because of profiling, the directions Ning never attempted, and the time absorbed by the recommended cadence. The counterfactual cannot be fully observed, yet limited evidence can still be formed through randomized audits, comparisons with generic entrances, long-term tracking of exits, and subject interviews. Opportunity loss cannot be set to zero merely because it did not happen. Nor may such audits casually claim that the system stole some particular life. They can only show what observable changes occurred in the choice set, the costs, and the distribution, leaving the value judgments to open discussion. A harsh critique without conditions of failure also reproduces the profiling system's excess of certainty. The Linchuan program may collect minimal study records for course recommendation, use de-identified aggregates for public planning, and forbid risk scores from entering hiring without renewed validation. Ning can view the principal inputs, correct facts, choose the generic service, export her results, and demand downstream correction. The platform keeps a reasonable commercial return, and it must also disclose its principal uses and interests. This arrangement does not apportion the value of data precisely to each person, nor can it eliminate leakage, inference, and model error. What it limits is a specific unlimited expansion: from one convenience to permanent profiling, from group planning to individual sanction, from visible behavior to computable personality, from a free service to ownership of future opportunity.
Data returns and the distribution of the right to observe are ultimately two adjoining questions that cannot be merged. Returns answer where value flows; the right to observe answers who may see whom, and for what. Even if returns were evenly distributed, an over-wide right to observe could still harm subjects; even if observation served a legitimate purpose, returns concentrating in one direction over the long term would still demand explanation. Only when both ledgers are opened together can technology expand choice, rather than merely expanding the prediction of choice. The next chapter takes up the common loss within the rules of competition: when everyone makes locally rational optimizations based on data, why total investment may keep rising while basic capabilities and life do not increase in proportion.