By Furaha Maseke Marwa:
A hospital introduces an artificial intelligence system to help identify disease. A bank uses another to assess loan applications. A public institution deploys a chatbot to explain services. Each promises greater speed, wider access and better decisions. But when a patient is missed, an applicant is unfairly rejected or a citizen receives misleading advice, a more difficult question emerges: who established that the system deserved to be trusted, and what evidence supported that judgement?
Africa’s conversation about artificial intelligence must confront this question alongside the excitement about innovation. The continent needs investment, research, infrastructure and locally relevant applications. It also needs institutions capable of distinguishing a convincing demonstration from a dependable system. The ability to build or purchase AI is valuable. The ability to evaluate it, challenge its claims and control its consequences is equally necessary. Without that capacity, adoption can outpace accountability.
Trust cannot rest on a supplier’s reputation, a polished interface or a single accuracy score. It must be earned through evidence about how a system performs, whom it serves, where it fails and what happens when something goes wrong. That evidence must relate to the actual setting in which the technology will operate. A system that performs well elsewhere has not automatically demonstrated that it is suitable for a Kenyan clinic, a South African municipality or a multilingual financial service.
This is the purpose of AI assurance: a structured examination of whether claims about an AI system are supported by sufficient, relevant evidence. It asks whether the system is appropriate for its intended use and whether its risks are understood and controlled. Assurance should help institutions make defensible decisions about deployment, restrictions, improvement or withdrawal. It cannot promise that failure will never occur. It should make uncertainty visible and responsibility clear.
Evidence before trust
Consider an AI screening tool. Reporting that it is accurate does not answer every question a healthcare institution must ask. How often does it miss the condition it is meant to detect? How does it perform across different patient groups, equipment and facilities? What happens when an image is poor or clinical information is incomplete? Can a clinician recognise an unreliable output? Is there a referral pathway when the system cannot provide a useful answer?
The same discipline applies beyond healthcare. A lending system should be examined for unfair outcomes and meaningful routes of appeal. A public service chatbot should be tested for misleading answers, language limitations and inappropriate disclosure of personal information. An autonomous agent should face clear boundaries on the actions it can take. Institutions need evidence proportionate to the consequences of failure. The greater the potential harm, the stronger the justification required before deployment.
Importantly, assurance concerns the whole arrangement around the technology. A technically capable model can become unsafe when connected to poor data, placed in an unsuitable workflow or used by people who misunderstand its limitations. Procurement decisions, staff training, security, accountability and incident response all affect outcomes. Evaluating the model alone leaves these relationships unexplored. Institutions must assess what people and systems will actually do together, under everyday pressures and imperfect conditions.
Africa’s context must shape the assessment
Africa is not a single operating environment. Languages, infrastructure, institutional resources and patterns of access vary within countries as well as between them. An evaluation that overlooks this diversity can conceal important failures. A service may work for fluent English speakers while excluding others. A cloud-dependent application may become unusable during connectivity failures. A decision process may appear accessible on paper while remaining difficult for people with disabilities or limited digital literacy.
Contextualisation must therefore influence testing, design and oversight from the beginning. Relevant communities and professionals should help identify what acceptable performance means and which failures matter most. This does not justify weaker safeguards in settings with fewer resources. It calls for safeguards that people can actually use. A complaints procedure has little value if affected individuals cannot understand it, reach it or obtain a remedy through it.
African institutions also need the capacity to question suppliers. Procurement teams should ask for evaluation results, known limitations, data governance arrangements and evidence of security testing. Contracts should clarify responsibilities when products change or incidents occur. Where necessary, independent reviewers should be able to examine claims without commercial pressure. Building local evaluation expertise is part of technological capability: it enables institutions to negotiate, adapt and make informed choices rather than accept assurances they cannot verify.
Safeguards beyond scores
One danger is reducing assurance to a reassuring number. An overall score may help summarise evidence, but it can hide a serious weakness. Strong predictive performance cannot compensate for exposed patient records. An explainable decision cannot excuse discriminatory treatment. Efficiency gains cannot substitute for an effective way to stop harmful actions. Certain safeguards must be treated as conditions that a system must satisfy, rather than weaknesses that strengths elsewhere can cancel out.
This principle informs my work on the African AI Assurance Framework. The framework is being developed to connect responsible AI commitments with assessable controls and supporting evidence. Its purpose is to make scrutiny practical: what should be checked, what evidence is needed, who reviews it and what findings should prevent deployment? It remains a developing contribution for expert review, not a claim that Africa’s assurance challenges have already been solved.
Human oversight must be effective
The phrase “human in the loop” offers little protection unless that person has the information, competence, time and authority to intervene. Someone expected to approve hundreds of recommendations under pressure may provide a signature without meaningful scrutiny. A professional who cannot challenge the system without penalty does not exercise independent judgement. Organisations must design oversight as a working capability, supported by training, escalation routes and retained manual procedures.
Oversight must continue after launch. Models are updated, user behaviour changes and new vulnerabilities emerge. Systems may gradually be used for purposes their original assessments never considered. Monitoring should therefore track failures, complaints and changes in performance. Material changes should trigger reassessment. Institutions must also be willing to suspend or withdraw a system. Control is incomplete when an organisation can identify a serious problem but cannot stop relying on the technology.
The responsibility for trustworthy AI cannot sit with one actor. Developers must substantiate product claims. Deploying institutions must justify their use. Regulators must clarify obligations and enforce applicable protections. Researchers and independent assessors must test difficult questions. Affected people must have opportunities to challenge consequential decisions. Assurance works when these responsibilities reinforce one another, with clear ownership and scrutiny throughout the system’s life.
Africa should pursue AI with ambition and judgement. The goal is neither unquestioning adoption nor paralysis in search of perfection. It is the institutional confidence to deploy when evidence supports deployment, impose restrictions when limitations demand them and refuse systems whose risks remain unacceptable. When someone asks, “Who assures the AI?”, the answer should identify accountable people, credible evidence and safeguards that work. That is how trust becomes something institutions can demonstrate and communities can question.
Furaha Maseke Marwa is an AI researcher and consultant and Founder and CTO of CognitiveAI Labs. His work focuses on AI assurance, explainable AI and trustworthy autonomous systems in African contexts.