What did half a million people actually consent to?

Between 2006 and 2010, 503,317 people walked into one of 22 assessment centres around the UK. They gave blood and urine, answered questions about their health, sat for measurements like weight and blood pressure, and signed a consent form. Most of them were somewhere between 40 and 69. They agreed that UK Biobank could keep their samples and follow their NHS records for the rest of their lives. The data would go to researchers in the future on request, when they proposed a new study and it had been reviewed and approved by the biobank team. In other words, the participants were consenting to studies that had not been thought of yet, never mind designed.

The consent form is one page with seven statements on it. It is dated November 2006. With it came a much longer participant information leaflet explaining the purpose of the project, what happens on an assessment visit, and the safeguards in place to protect participants’ privacy. The form and the leaflet were written in plain English and designed to be read together. Many biobanks around the world have since been set up on the same model.

At the time, it was a bold project - collect this data ahead of time so that we can do studies in the future that wouldn’t be possible otherwise, for example where you need a long collection period, or where conditions are relatively rare. It seemed a natural extension of the birth cohort studies that had been running for decades.

The world has changed a great deal since 2006 and the consent form has not changed with it. In this post we want to explore what the consent given in 2006 means in the new world of generative AI, and what the implications are.

So what did they actually agree to?

Very little, in terms of specifics, and that was the whole point of it. It was designed to be broad consent, and it did not try to anticipate what the research would be. The form asks participants to give permission for “long-term storage and use of this and other information about me, for health-related research purposes.” It describes a broad aim, “to improve the prevention, diagnosis and treatment of illness and the promotion of health throughout society.” It does not name any technology or method, so it sets no limits on them either. The focus is on the purpose rather than the methodology.

This was a deliberate design. The Ethics and Governance Framework, published in 2007, explains it clearly:

“Because it will be impossible to anticipate all future research uses, consent will be sought for research in general that is consistent with UK Biobank’s stated purpose (rather than for specific research).”

What were the perceived risks then?

“Are there any risks for me in joining UK Biobank?” was one of the leaflet’s own headings, and the answer it gave is really interesting when we look at it through the lens of 2026 and generative AI. The leaflet does not say there are no risks, but it does say those risks are minimal. It talks about the risks of the actual visit - having your blood taken, feeling uncomfortable with some of the questions about your health and lifestyle, and the fact that you will not be given your own results.

On the risk to your personal information, it says this:

“Participation involves a minimal risk in relation to the use of personal information. Great care will be taken to ensure the confidentiality of all data (see below) and the risk to participants of a breach of confidentiality is considered very low.”

And further down:

“Details that might identify you will be removed from any information and samples provided to researchers in order that they cannot be traced back to you. None of your particular test results will be fed back to you, your doctors or anyone else. So, taking part should not have any adverse effects on you (including your employment status or ability to get insurance).”

So we end up with a bit of disagreement between the two documents. The leaflet says the risk of a breach of confidentiality is very low, and the Framework says it will be impossible to anticipate all future research uses. We don’t think those two statements can both be true, particularly when we look at the kinds of things that are now possible with generative AI. This is not to criticise the people who wrote the leaflet, or UK Biobank itself, as they were writing in good faith with the best knowledge they had in 2006. We are focusing on it because it illustrates the sorts of problems that arise when you ask people to consent to research that has not yet been imagined, and when you use data in ways that were not anticipated at the time it was collected.

The kind of future they were imagining

Let’s think about what the people running UK Biobank in 2006, and the other biobanks set up around the same time, imagined the future might look like. The assumption was that research would carry on looking much as it always had. A person has a question. They apply for access, pull out a subset of the cohort, write their analytic code and run it. What comes out is a paper, with its tables and its figures, and the findings go into the world in that form. They could certainly imagine new research methods, but those would be methods very much in keeping with the way research was being done at the time.

That model held for a long time, and safeguards were added to it along the way. Somebody could look at an output before it left and check it for disclosure risk, meaning the small numbers that let you pick a person out of a crowd. In practice this looks like a human reading a table, seeing that a cell contains three people, and suppressing it. In the last 10 years there has been a clear move towards putting data into Trusted Research Environments (TREs), also called secure data environments, rather than sending it out to researchers, which strengthened the safeguards considerably without changing the underlying picture of what research looks like.

Now consider what happens when the output is not a table or a paper. In the last 4 years generative AI has arrived in every part of life, to the point where ChatGPT is a household name. Training a model on a dataset and then using it to produce new content has become an ordinary thing to do, as has handing a pile of information to an AI and asking it to go through the material and tell you what you have missed. What we want to work out is what it means when that approach is applied to health data research.

A model is not an analysis

It helps to split this into two large-scale uses of generative AI on a health dataset. Both have consequences for the consent given in 2006, and they are very different from each other. There are certainly other uses, and we do not want to repeat the mistake we have just described and assume that today’s uses are the only ones there will ever be. These are simply the two that bear most directly on consent right now.

The first is training a model on the dataset, a machine learning algorithm for example, and then releasing that model.

UK Biobank has since published a policy on the use of AI, which says that the use of AI models on the data is “reasonable (and indeed inevitable).”

It also says that if a researcher trains a proprietary model on participant-level data, then the parameters of that model, which the policy explicitly defines as including “neural network weights in a deep learning model”, must be returned to UK Biobank as derived variables and made available to other researchers in the normal way.

From a participant’s point of view that is a very different output from a table or a paper. What gets released is a working artefact that other people can still put questions to, and they can use it to generate new content or to answer questions about the data that the original researcher never thought to ask. The consent form does not contemplate anything of the sort.

The safeguards do not reach this kind of output. With a contingency table you can count the cells and suppress the small ones. A human being can do that, and it is easy to understand what has been done. A neural network offers nothing equivalent to count. Nobody, as far as we can tell, can look at a set of weights and say with any confidence what a determined person could pull back out of them.

To be fair to UK Biobank, their AI policy sees part of this coming. It rules out a trained model that could retrieve or regenerate participant-level data, treating that as the equivalent of handing someone direct access to the data itself, and it flatly bans anyone feeding participant records into a publicly available generative model. Unfortunately, because slices of the data were sent out to researchers before a TRE was set up, and because the policy is new, there is no real way to tell what has already been done. The policy is a good step, but it is not a fix for the consent problem. We probably have to assume at this point that some models have already been trained on the data and released, and that those models are still out there. With the UK Biobank data in particular, slices have already ended up on publicly available GitHub repositories and even on an Alibaba marketplace, so it’s not a stretch to imagine that some of those slices have been used to train models that are now circulating.

New insights

The second use is to point a model at the data and ask it for new insights. Anyone who has worked in business or analytics will recognise this one. You have some data, say the number of people visiting your website each month, and you ask a model to look at it and tell you what is going on. It tells you traffic is up, ties that to your marketing spend or the kind of content you have been publishing, and suggests what to do next. It sometimes surfaces things no human analyst would have thought to look for, and it sometimes just gets to the same place in a fraction of the time.

The question is what happens when you do the same thing to a health dataset. In traditional research, the researcher has a question and everything is contingent on that question: study design, analysis, interpretation. Aside from looking at the data to check for errors, the shape of the distribution and so on, the data is not supposed to be interrogated for its own sake. In fact, doing so is actively discouraged, because it can lead to bad practices such as p-hacking, where you keep looking for patterns until you find one that is statistically significant. The issue is that a generative model does not have to have a question. It can just look at the data and tell you what it finds interesting, and it can do that in a way that is not transparent to the researcher. There is nothing in the 2006 form that anticipates this.

How risky is all this?

This is not about UK Biobank, and it is not really about any one organisation or dataset. The worst case scenario is that the data is used to re-identify people, and that is a risk for any dataset. Linking records across datasets to re-identify people is an old attack and it has been known about for years. What has changed is the skill required to carry it out. In the past, jigsaw re-identification needed somebody with expertise, time and a motive. A generative model collapses all of that into a natural language interface, and the level of expertise required falls a lot.

We already know the demand is there. NHS staff are disciplined, and sometimes prosecuted, for looking up the records of people they know. The motive in those cases is usually just nosiness about an ex-partner, a neighbour, a colleague or somebody off the television, and we should assume the same thing will happen when the lookup can be done through a generative model trained on health data.

Privacy is not the only problem

Everything above treats the risk as a privacy risk. That is the risk the 2006 leaflet was written to address, and it is the risk most existing safeguards are designed around. It is not the only risk, though, and for generative AI it may not be the most important one.

Generative AI is general purpose, so a model trained on participants’ data is not tied to the research question that justified training it, and once it exists it can be put to work on almost anything, including uses that participants would find socially unacceptable. Insurance is a good example. The leaflet promised that taking part should not affect anyone’s ability to get insurance, but that promise was made about individual test results, and it says nothing about a model trained on the cohort being used to price policies for everyone who shares a participant’s risk profile. That use would not require a single person to be re-identified, so there could be no breach of confidentiality at all, and participants could still reasonably feel that their contribution had been used against the spirit in which they gave it.

Who builds these tools also matters. Most generative AI is developed by commercial companies, and companies will tend to repurpose a model wherever there is money in doing so. Decades of research on public attitudes to health data point to two consistent sticking points. The first is commercial access: people are far more uneasy about companies using their data, particularly when those companies stand to profit. The second is purpose: people are generally happy for their data to be used when it serves the public good, and much less happy when it does not. The collapse of care.data, which was closed in 2016, showed what happens when both concerns are ignored at once.

The consent form did set a purpose. Participants agreed to “health-related research”, with the aim of improving health “throughout society”. That purpose was meant to travel with the data, but it is much harder to make it travel with a model, because a model can be copied, fine-tuned and repurposed long after it has left the hands of the researcher who trained it.

Re-consent is not the fix

Re-consenting half a million people is not realistic, and it is not clear that it would actually help. Most participants would tick the box, and a form that said “we may train neural networks on this” in 2026 would only cover the methods we know about today, and would need rewriting again as soon as something new came along. The problem is structural - how do you consent people for things that don’t exist yet, and should you even attempt to do so?

The 2006 form was written by people who knew they could not see the future and said so in the Framework. We are in the same position now, and whatever we write in 2026 will be read in 2046 by somebody wondering what on earth we thought we were doing. So rather than trying to write a better form, or to predict the future, we need a consent process that assumes it will be overtaken by whatever comes next.

We think the answer lies in changing the level of abstraction at which consent operates. Consent that is tied to a technology goes stale as soon as the technology changes, whereas consent that is tied to a purpose and a process can carry on making sense for much longer. Participants can meaningfully agree to two things that will still make sense in 2046. The first is a set of purpose limitations that say what their data, and anything derived from it, may and may not be used for, without naming any technology. The second is a strict governance process that decides whether each new use falls inside those limits.

Dynamic consent

Dynamic consent, where participants can adjust their permissions as circumstances change, has been discussed for years but has not yet been adopted at scale. We feel that this is probably the right direction, but it comes with its own set of problems.

To let somebody revise a permission in 2040, you have to still be able to reach them in 2040. That means holding a live relationship with participants across decades and keeping contact details current, and it means being straight with people that you are asking for something ongoing rather than an afternoon. You also end up introducing biases into your dataset - for example, people who are more transient or less engaged may be underrepresented, or quite frankly may just not want to be bothered. It is a bigger ask, and it would cost us participants, probably quite a lot of them. For these sorts of NHS studies you could reasonably contact people through their GP, but then you have to ask who has consented for their GP to know they are taking part, and what that does for governance.

Then there is the big problem of the people who have died. A cohort recruited in middle age and followed for the rest of participants’ lives will, by design, come to contain a large number of people who have died, and their data remains in the dataset and in any model trained on it.

Even if we could agree that we do want dynamic consent, there is the question of how to implement it technically. Dynamic consent assumes that changing your mind changes something. In traditional research that is true - a row (or rows) of data gets deleted from the dataset. You can’t do anything about the papers that have already been published with it, but you can version the dataset so it is clear whose data went into which cut. The same is not really true of a model that has already been trained on that data. Taking one person out of a trained model is called machine unlearning, and we cannot currently do it reliably. The TOFU benchmark, presented at COLM in 2024, tested the leading methods and found that “none of the baselines we consider show effective unlearning”, and a large group of machine learning researchers and lawyers made the same point in their title at NeurIPS the following year: machine unlearning doesn’t do what you think. The only reliable way to remove a person from a model is to throw the whole thing away and train again from scratch. Dynamic consent only means something if the derivatives stay in the protected environment along with the data.

Some solutions

We don’t have a full solution - there probably isn’t one - but we do have some ideas about the right direction. These split into two groups.

Consent and governance

• A willingness to say on the form, in plain words, that we cannot describe the risks of technologies that do not exist yet.
• Some sort of realistic dynamic consent, with a live relationship with participants and a way to reach them over decades, designed to be as minimally invasive as possible.
• Tiered consent set once at recruitment, with a small number of coarse permissions that are deliberately abstract rather than named after a technology. “No derivative artefact trained on my data may leave the secure environment” is a sentence that will still mean something in 2046, in a way that a named algorithm or method does not. We don’t want to try to guess what the future will look like. The UK Biobank policy on AI does not go far enough, because it still frames the risk in terms of model parameters such as neural network weights, which belong to a methodology that may well be obsolete in 20 years.
• Consent to a governance process and to purpose limitations, rather than to a use. Participants agree to a strict governance process and to a small set of technology-agnostic purpose limitations. A standing body, with participants sitting on it, then adjudicates each novel use against those limitations as it arises. This is close to how we already handle research involving children or adults lacking capacity.
• Purpose limitations that reflect what decades of public trust research tells us people care about. Two examples are “my data, and anything derived from it, may only be used for purposes that serve the public good” and “no commercial organisation may access or use a model trained on my data”.
• Regular audits of models trained on the data.
• Better training of researchers and ethics committees, so that they understand the risks of generative AI. Using a generative AI model to tidy up an image is not without risks when the underlying data is in that image or prompt.
• More staffing for the management and system administration of secure environments, so that they can be properly maintained and monitored. We don’t want a situation where the person deciding whether a model can be introduced into a secure environment is poorly paid and overworked.
• Thinking more widely about what else is in the data. If a participant answered questions about their family history, then they consented on behalf of parents, siblings and children who were never asked and never told. For genomic data this is even more pronounced, as a genome implicates relatives directly, cannot be de-identified in principle, and does not become less identifying with time.

Technical safeguards

• Data within a secure environment with no route out to the internet, and models trained on it staying in that environment rather than circulating as weights.
• Using synthetic data where possible, but not assuming that it is a silver bullet, because it is often generated by models trained on real data and the same risks apply to it. There is also a risk of model collapse. Read more about synthetic data here.
• Using automated tools - that, ironically, involve generative AI - to check outputs for disclosure risk, rather than relying on a human being to do it. For example, they could look at SVGs of tables, which often embed the underlying data within them. One of the advantages of having a natural language interface to a model is that you can ask it to check for disclosure risk, and it can do that at a scale that is not possible for a human being.
• Testing models for their ability to regenerate participant-level data, and treating that as a disclosure risk. This could take place in the secure environment, and membership inference attacks give us a concrete way to do it - we can ask whether a given record was in the training set and get an answer with a number attached to it.
• Training models with differential privacy, which deliberately adds random noise during training so that no single participant’s record can make much difference to what the model learns. How much protection this gives is set by a number which should be reported alongside any model trained on the data.
• Treating models as data. If a model trained on the cohort is itself participant-derived, then it belongs behind the same wall, subject to the same access process and the same withdrawal rules as the records it was trained on.