Expert Comment: Who gets to define care in the age of AI?
Dr Caroline Emmer De Albuquerque Green, Director of Research and Head of Public Engagement at the University of Oxford’s Institute for Ethics in AI, argues that as technology companies adopt the language of care, this must happen with inclusion of the care community.
This summer, the Casey Commission will ask the public in England what adult social care is for and who should provide it. We cannot answer that, or say what responsible technology in care looks like, without first defining what we value about ‘care’ itself. That is the conclusion of two years of co-production with the care community through the Oxford Project on the responsible use of generative AI in adult social care, and of the care-centric approach my colleagues and I set out in The Lancet Healthy Longevity in February.
Yet the definition of care and its role for AI is already being written, and not by the care community but by tech companies who are increasingly adopting the language of care.
In June 2026, OpenAI's Industrial Policy for the Intelligence Age recommended pathways for displaced workers into human-centred work, particularly the care economy. In January, Anthropic published its constitution, announcing it wants Claude to have good personal values, following rules of genuine care and attending to users' wellbeing, autonomy and dignity. These companies are learning to speak the language of care, one of deep moral and relational value, borrowing words from human relationships necessary for people to survive and thrive, evoking authority, virtue, vulnerability, dependency and trust.
The question is whether this reflects genuine concern, or an appropriation of caregiving's moral authority to serve companies' interests. Here are three reasons to be concerned.
“[Tech] companies are learning to speak the language of care, one of deep moral and relational value, borrowing words from human relationships necessary for people to survive and thrive, evoking authority, virtue, vulnerability, dependency and trust. The question is whether this reflects genuine concern, or an appropriation of caregiving's moral authority to serve companies' interests.”
Care-washing
The first concern is 'care-washing': an image and products that sound concerned with wellbeing, dignity and autonomy, but disguise uncaring actions serving companies rather than users. The terminology makes the new seem ethically assured, but words do not guarantee reality.
We have already witnessed the devastating consequences of anthropomorphism and sycophantic conversational AI on humans, especially vulnerable groups, serving companies that want to keep people engaged. Care language without safeguards to spot unhealthy AI relationships, or detect crises, can lead to over-reliance instead of the right support.
The more people use these systems, the more personal data (movement, speech, behaviour and health, especially in clinics and care homes) is extracted and monetised, a practice described in other fields as data colonialism. In care, the stakes are higher because what is captured is vulnerability itself, softened by language and framed as support, without full transparency or the ability to opt out.
These risks cluster into recognisable categories: dependency engineering, data sensitivity, surveillance, workforce displacement, dignity and autonomy. Naming them is the first step toward governing them.
The reframing of what care means
The second concern is more fundamental. Much of what is described as care in AI is sophisticated tools that 'speak care language' and assist in self-care and the care of others, mostly through predicting patterns. But care is more than words or prediction. AI systems do not feel, relate or care in the human sense. Anyone who has given or received care knows it is a relationship, existing in presence, trust, and small, unseen acts of recognition. No technology can replicate that.
Once we accept that technology can 'care,' we quietly lower our expectations of what care is: human interaction becomes optional, and simulation becomes an acceptable substitute for presence, a particular concern for people drawing on long-term care. Technology has a place here, supporting professionals and extending reach, but it must remain a tool, not the definition.
The hidden costs for caregivers
Caregiving carries great moral value, yet caregivers, disproportionately female, have long been underserved and undervalued in the societies where these companies are based. It is ironic that some of the most powerful, male-dominated companies now adopt the moral authority of a group long excluded from politics.
The consequence may be further entrenchment of power structures: companies and governments adopt a caring stance while allocating caregivers fewer resources, treating AI as a panacea for needs requiring investment on multiple levels. This could raise caregiving's visibility, as care sectors grow with an ageing population. AI can genuinely support this population, but only if products primarily serve people, not tech companies, requiring minimum standards, oversight, and governance that puts caregiving communities at the centre.
What’s next?
These concerns require their own governance framework, grounded in caregiving's lived realities rather than general AI ethics guidance. The appropriation of care language must not happen quietly. Rather it needs open, public discussion, with structured means to participate.
At the Institute for Ethics in AI at the University of Oxford, our activities to support such processes include:
- The Caregiving AI Responsibility Initiative (CARI), with FamTech.org, the US technology hub for the care economy, developing a governance framework for frontier AI in caregiving contexts, translated into minimum standards companies can be held accountable to.
- The AI Alliance in Care, a community of good practice born from the Oxford Project on responsible generative AI in adult social care, producing governance recommendations for tech companies and care providers.
- Civic AI, developed with Ambassador Audrey Tang, Senior Fellow of the Institute’s Accelerator Fellowship Programme, grounded in Joan Tronto's ethics of care, proposing a socio-technical framework to make AI and the organisations that build it accountable to the people they serve.
Finally, governments and regulators overseeing health and social care must play a stronger role overseeing 'caring AI.' Rather than taking care's language, companies should put caregiving communities at the core of innovation meant to support them.
For more information about this story or republishing this content, please contact [email protected]
Read the extended version of this article, 'AI companies are adopting ‘care language’: Three reasons why we should be concerned', on the Institute for Ethics in AI blog.
