Resources and learning
Rule 8 says that you need to take reasonable steps to make sure any biometric information you hold is accurate, up to date, complete, relevant and not misleading, before you use or disclose it. Part of this requirement is ensuring that your overall biometric system is sufficiently accurate for your purpose and the overall risk and context.
Read our guidance on ensuring the accuracy of personal information. This also applies to biometric information.
How can you ensure that the biometric information you hold and the operation of your biometric system is accurate?
What is reasonable in the circumstances to avoid accuracy errors will change depending on your overall risk profile (the type of information you hold, what it is being used for, your context, and the potential harm that individuals may experience).
Example steps to ensure accuracy:
Download this table as a printable checklist (opens to PDF, 184KB).
|
1
|
Ensure the biometric system is using sufficiently high-quality samples e.g. photos, audio recordings.
|
|
2
|
Keep biometric samples up to date as required and generate new biometric templates when needed (for example, due to aging, surgery or injury).
|
|
3
|
Where necessary, implement manual (human) review of matches by the biometric system before taking action based on the biometric system. You also need to ensure the staff involved have appropriate training and are effectively equipped to challenge the accuracy of results if needed. This is sometimes called having a “human in the loop”. Having a human in the loop will be essential for some uses of biometric information – for example, if you are operating a biometric watchlist or any other context where people could be negatively impacted by the use of their biometric information.
|
|
4
|
Regularly review and refine the sensitivity and specificity settings of the biometric system to ensure the rate of any false positive or false negative matches is appropriate for the use case and not leading to adverse outcomes for individuals.
|
|
5
|
Select a biometric system with appropriate accuracy for your privacy risk and overall context. Some systems show better performance in certain contexts or for certain uses than others, especially in different conditions (i.e. in the wild versus controlled environments). You should refer to independent evaluations (e.g. by NIST) of the accuracy where possible.
|
|
6
|
Train staff and any other users of the biometric system about what a match means, so that they better understand the results of a biometric system and can respond appropriately. For example, a match resulting from a verification process is not a definitive determination of a person’s identity – it reflects the statistical likelihood that this person is same as the identity they are claiming.
|
|
7
|
Have a process or audits in place to identify and resolve errors and issues related to the biometric system, including understanding and mitigating any bias in the system (such as the system being less accurate for a particular demographic group or skin tone). The risk of inaccuracy from bias needs to be addressed both with human review (e.g. training, two person check before acting on an alert) and system checks. Unaddressed risk of bias may compromise the accuracy of the system.
|
|
8
|
Ensure individuals can raise concerns about the accuracy of the system and you have a process in place to respond (see also our guidance on rule 7 – correction of biometric information).
|
|
9
|
Conduct due diligence when choosing or procuring a biometric system or service provider, and consider the suitability for the setting the system will be used in and the New Zealand demographic.
|
Does the biometric system need to have 100% statistical accuracy?
No. Biometric systems, by nature, are probabilistic which means they assess likelihoods not absolutes. So, there’s no such thing as 100% accuracy; a biometric system will always have some margin of error.
However, to comply with rule 8, your biometric system does need to be sufficiently accurate for the overall context, privacy risk and people whose information you are collecting. In most cases, this means your system needs to be accurate in the vast majority of cases (i.e. highly accurate). If you operate a system that is not very accurate, it will be hard to show that it is necessary and effective (therefore, you may be in breach of rule 1). You also need to have a process in place to effectively mitigate the risk of misidentifications, so that the system as a whole is highly accurate.
You should consider how inaccuracies (misidentifications or mis-categorisations) could impact individuals – for example, by causing embarrassment or impacting on a person’s dignity and feelings. For Māori individuals, inaccuracies may also undermine the tapu, mana and mauri associated with Māori biometric information, so you should have a culturally responsive plan to address inaccuracies.
You also need to have processes in place to mitigate the harm from any incorrect identification, verification, categorisation or inference.
The statistical accuracy of your biometric system is also relevant to other rules. For example, rule 1 (your system must be necessary, effective and proportionate with relevant safeguards in place) and rule 4 (making sure your means of collection is fair).
Read our example scenarios of how an organisation might apply rule 8 in context.
