Last updated: 2026-11-10
Product and Ethical Risk: Should We Build It?
Most risk management asks whether a project will deliver what it set out to deliver. Product risk asks whether that thing should have been the goal at all. A team can retire every schedule risk, ship on the agreed date, pass every test, and still deliver something nobody uses, something that can't be supported, or something that harms the people it touches.
This page covers the questions product risk adds to the process-focused picture, the ethical dimensions that belong on a risk register, a framework of five questions to ask before starting or continuing, why standard scoring tends to underrate ethical risks, and how to record and respond to them.cf. finding risks: systematic ways to spot them
A Project Can Succeed While Its Product Fails FoundationalKnowledge that endures for decades — core principles
Sommerville separates risks to the project (schedule, resources) from risks to the product (quality, performance) and to the business1. The separation matters because success in one doesn't imply success in another. A technically excellent product that reaches the market after a competitor has taken it is a business risk realised; a product delivered on time that solves the wrong problem is a product risk realised, however well the process ran.
| Process (project) risk asks | Product risk asks |
|---|---|
| Will we finish on time and on budget? | Does anyone need this? |
| Do we have the people and skills? | Will people use it, or buy it, once it exists? |
| Will the requirements stay stable? | Could it harm its users, or people who never chose to use it? |
| Will the components integrate? | Can we support and maintain it for as long as people rely on it? |
The left-hand column is where most registers spend their rows, partly because those risks convert easily into weeks and money (see Measuring Risk). The right-hand column is harder to quantify and easier to leave out. What Counts as a Risk sets out the process/product distinction in full.the register often hides product risks behind schedule metrics
Four Product Questions FoundationalKnowledge that endures for decades — core principles
- Need. Does this serve a need users would recognise, or a need the team assumed? Evidence beats conviction: interviews, observation, a prototype in front of the people who'll use it.
- Fit. Will it be adopted? A product that works but doesn't fit how people already work will be worked around.
- Consequences. What happens beyond the intended use: misuse, side effects, effects on people who aren't users at all?
- Sustainability. Can it be maintained, secured and operated for its expected life, financially and technically? Who supports it after the project team moves on?
Underneath all four sits the question the rest of this page develops: just because we can build it, should we?
Ethical Dimensions on the Register Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice
Ethical risks are product risks whose impact falls on people rather than on the schedule or the budget. Four dimensions come up on almost every software project:
| Dimension | Example risk | Typical responses |
|---|---|---|
| Privacy and data | Because we log full request bodies, personal data may be retained beyond its purpose, exposing users if logs leak | Data minimisation; retention limits; a data protection impact assessment (see Data Protection Law and Ethics) |
| Accessibility and inclusion | Because the interface relies on colour and fine pointer control, some users may be unable to complete core tasks | Accessibility standards from the first design; testing with assistive technology and with disabled users |
| Environmental impact | Because the design retrains a large model nightly, energy use may far exceed what the benefit justifies | Measure energy and compute; choose smaller models or less frequent runs; set a budget |
| Bias and fairness | Because training data under-represents some groups, the system may perform worse for them, producing unfair outcomes | Evaluate performance per group; diversify data; human review of consequential decisions (see AI Ethics) |
Technical excellence doesn't offset harm. A fast, reliable, well-tested system that discriminates is a well-engineered harmful system.
Five Questions Before Starting or Continuing Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice
| Question | What it asks | Warning sign |
|---|---|---|
| Transparency | Can we explain what we're building, and why, to the people it affects? | The explanation changes depending on who's asking |
| Accountability | Who is responsible if harm occurs? | Nobody can name a person or role |
| Fairness | Does our approach disadvantage any group? | Nobody has checked |
| Consent | Do users understand and agree to how their data and attention are used? | Consent is buried, bundled, or assumed |
| Reversibility | If this goes wrong, can we undo or limit the damage? | Once deployed, there is no way back |
Reversibility deserves the most attention because it changes over time. Collingridge described a dilemma in controlling any technology: early on, its effects are hard to predict, but it's easy to change; by the time its effects are clear, it has become entrenched and is hard to change2. His response was to keep technologies controllable, favouring decisions that can be revised as more is learned. In software that has a direct architectural meaning: feature flags, staged roll-outs, data that can be deleted, and interfaces that let a component be replaced all keep a product reversible (see architectural practices and Exit Architecture). Reading Collingridge's dilemma as a case for these specific engineering practices is our application, not his.
Who Bears the Risk? Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice
Boehm defines the loss term in risk exposure as the loss to the parties affected if the unsatisfactory outcome occurs3. In practice, registers usually measure loss to the project: weeks, money, reputation. Ethical risks often fall on someone else: users, people whose data is processed, people affected by decisions the system makes, communities near a data centre. If the impact scale only counts loss to the project, a risk that badly harms a small group outside the project scores low on impact and drops to the watchlist.
The fix is to make the parties explicit. Two extra register columns do most of the work:
| Column | Contents |
|---|---|
| Who bears it | The project, the client, users, a specific group of users, non-users, the public |
| Reversible? | Yes, with effort, or no; an irreversible harm to people moves the risk to immediate action whatever its score |
Treating "who bears it" and "reversible?" as overriding the score is our recommendation; it follows from Cox's finding that matrices can rank quantitatively smaller risks above larger ones (see Measuring Risk), which is most damaging when the larger risk falls on people the matrix doesn't count.matrices can misrank risks by magnitude alone
Example Risks and Responses Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice
| Risk | Who bears it | Responses |
|---|---|---|
| An AI tool built for one purpose could enable surveillance or discrimination | Non-users; specific groups | Ethical review before build; restrict capabilities by design; limit who can deploy it |
| An architecture locks users into our ecosystem, so leaving means losing their data | Users | Data export in open formats; documented interfaces; avoid proprietary dependencies where an open one exists |
| Deployment without testing edge cases that mainly affect minority groups | Specific groups of users | Diverse user testing; per-group evaluation; staged roll-out with monitoring |
| A feature designed to maximise engagement encourages compulsive use | Users | Define success in terms of user benefit, not time on site; review metrics with that in mind |
Three organisational responses recur: an ethical review step (a board, a panel, or simply a named reviewer outside the team), testing with a diverse range of users, and an impact assessment before deployment. Each puts a check in the process where it can change the product before harm occurs, which is the same principle as every other proactive risk response.
Professional Duty Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice
For computing professionals, ethical risk management is also an obligation. Codes of conduct such as the BCS Code of Conduct and the ACM Code of Ethics ask members to have regard for the public interest and for the health, privacy and well-being of the people their work affects; Professional Ethics covers them in detail, and Ethics, Security, Sustainability, and Professional Conduct applies them to individual projects. A risk register is one of the few places in a project where those duties become concrete actions with owners and dates.
Related Topics
- Risk Management & Mitigation: the overview and full cycle.
- What Counts as a Risk: process, product and business risk.
- Software Meets Reality: what happens once a system is deployed into an organisation and society.
- AI Ethics: bias, transparency and accountability for AI systems.
- Data Protection Law and Ethics: the legal side of privacy risk.
References
-
Sommerville, I. (2016). Software Engineering (10th ed.). Pearson. Chapter 22: Project Management, section 22.1 Risk management. ↩
-
Collingridge, D. (1980). The Social Control of Technology. Frances Pinter. ↩
-
Boehm, B. W. (1991). Software risk management: Principles and practices. IEEE Software, 8(1), 32–41. https://doi.org/10.1109/52.62930 ↩
Graduate Attributes
Directly developed
- Critical Thinker: Readers apply a framework of five questions to judge whether a product should be built rather than merely whether it can be delivered.
- Sustainability-Minded: Readers consider the long-term societal implications of technology by evaluating reversibility and the distribution of harm.
Modelled on this page
- Subject Specialist: Disciplinary concepts such as the Collingridge dilemma and Boehm's risk exposure are applied to define ethical risk management.
- Inclusive: The analysis explicitly shifts focus from project stakeholders to the people affected by the system, including users and communities.
- Reflective: Standard risk scoring methods are examined for their limitations in capturing ethical impacts on people.
University of Reading Graduate Attributes framework.