*/
Bad forensic science could jeopardise the integrity of the criminal justice system, so Gill Tully and Anthony Heaton-Armstrong examine the march towards better quality of services and a standing-bearing role for the Bar
The landscape of forensic science provision is a fragmented one and involves several different and mutually independent organisations (see box, p 22). The need for regulation was identified back in 2005 by the House of Commons Select Committee on Science and Technology (House of Commons Select Committee on Science and Technology; Forensic Science on Trial, 7th Report of Session 2004-2005). The report recommended independent regulation and in July 2007, the government announced the creation of the role of Regulator and a Forensic Science Advisory Council (FSAC).
Members of the Bar should be aware of the crucially important work and output of the Regulator’s office and alert to the standards that apply to forensic science, and in particular scientists’ adherence to the over-arching principles that characterise high quality forensic science. This may have significantvalue in any case which includes scientific evidence.
The Regulator’s role is ‘to advise government and the criminal justice system on quality standards in the provision of forensic science’. The remit covers forensic science and forensic pathology, but excludes other allied disciplines including forensic psychiatry, forensic psychology and forensic accountancy. It applies to criminal justice only. Created under Royal Prerogative, the role has no statutory basis and no statutory powers. In 2016, the Home Office published a forensic science strategy (bit.ly/2r1QXQC) which stated the intention to place regulation on a statutory basis. However, the necessary legislation has not yet been passed. All progress to date, therefore, towards adoption of quality standards has been achieved through a combination of persuasion and contractual requirements.
There are four over-arching principles that characterise high quality forensic science: balance, logic, robustness and transparency. (For more on this, see ‘Standards for the formulation of evaluative expert opinion’ (2009) Science & Justice 49, 161-164.)
As codified in the Criminal Procedure Rules, the duty of the expert is to the court, and forensic scientists must remain independent of the commissioner of their work. Setting aside the potential for inappropriate pressure to be exerted on experts, in recent years, the scientific study of cognitive neuroscience has revealed that even highly trained professional scientists are susceptible to cognitive bias (see Dror, I E & Cole, S A Psychonomic Bulletin & Review (2010) 17: 161). This operates at the unconscious level. Areas where there is a high level of subjectivity, such as fingerprint comparison and CCTV image comparison including facial comparison are at particular risk of cognitive bias. A further risk factor is the use of ad hoc methods, rather than formal, validated scientific methods
A key component of balanced forensic science is evaluation of two competing propositions: one relating to the prosecution view and the other to the defence view. Any evaluation that looks at the evidence from only one or the other side is unbalanced and should be rejected. Similarly, use of phrases such as ‘consistent with’ can indicate bias. Take, for example, a case where cell site analysis has been used to infer the location of an individual at the time a phone call was made. A report that the results were ‘consistent with having been used at the crime scene’ do not make clear that the results are also ‘consistent with’ having been used at any address within the range of that cell, which may be a considerable distance. Phrases such as ‘in the vicinity of’ provide no clue to the range of potential locations where a call may have been made, so should not be used unless they are clearly defined.
The Regulator has issued guidance on the subject of cognitive bias (bit.ly/2szC5uG) and it is entirely reasonable to ask an expert through cross-examination, for example, if they are familiar with that guidance, and what steps they have taken to avoid bias.
Clarity and terminology are critically important to ensure that logic is not overturned during court proceedings through either the prosecutor’s fallacy or the defence lawyer’s fallacy.
For a scientific method to be robust, it must have been validated to enable its performance characteristics, strengths and limitations to be understood. There must be sound knowledge underlying the method, and there must be data on which to base interpretation of the results. The Regulator has published a process for validation of forensic science methods in the Codes of Practice and Conduct (bit.ly/2scBrWM) and associated guidance (bit.ly/2rQa9oq). It is critical that any method has been validated for the specific purpose for which it is being used. The test used for detecting explosive residue on the hands of the Birmingham Six had been validated as a screen and it was known that there were other substances which would cross-react. However, what should have been a screening result was used in evidence, without the limitations having been made clear.
Often, data are required in order to interpret findings. For example, in order to determine the evidential significance of a DNA result, data from unrelated individuals are used to determine the match probability or likelihood ratio. This data set is appropriate for the task and ‘match probabilities’ evidence is widely accepted in courts. However, the data available to the scientist making an evaluation of evidential strength in a footwear comparison were criticised by the Court of Appeal, which ruled that statistical methods should not be used where the data were insufficient (R v T [2010] EWCA Crim 2439). This ruling has caused ongoing debate in the scientific community and the Regulator plans to develop a standard for evaluative opinion, working with the Royal Statistical Society, practitioners and the judiciary.
Despite requirements of the Criminal Procedure Rules, it is still the case that not all scientists are highlighting to the court where there is a range of opinion, and justifying their view within that range. The role of the forensic scientist is to provide information and/or opinion in a balanced way: this includes the unknowns and the risks, eg:
The Forensic Science Regulator’s Codes of Practice and Conduct (the Codes) sets out the standards which apply to forensic science. It is legitimate to ask a forensic scientist if they are compliant with these Codes and if not, why not? For those not compliant, questions such as ‘How can you assure the court that your methods are robust?’ and ‘How can the court have confidence that you have conducted the tests and interpreted the results correctly?’ will enable the court to scrutinise risks and make an informed judgement on reliability.
The Regulator stated in her Annual Report (bit.ly/2r1NJwv) that ‘Funding for forensic science across the board, and particularly, perhaps, for defence provision via legal aid, must be at a level that enables the standards to be met. Otherwise we will face the costs, both in criminal justice terms and financially, of quality failures and loss of confidence in forensic science.’
Although compliance with the standards is increasing, in some organisations there is a fundamental lack of understanding of risk, quality, validation and the requirement for assurance. Whilst policing is formally committed to meeting the standards, the level of support at senior levels varies between forces, and the required resources are not always made available.
Whilst the majority of the large commercial and government-funded forensic science providers have a strong quality culture, lack of senior management support is evident in a small minority of providers. In small and micro commercial companies, the costs of achieving accreditation continue to be a barrier to compliance, and the Regulator is continuing to work with policing and the Legal Aid Agency to bring about a position whereby funding for forensic science is at a level that enables the standards to be met.
Full compliance, however, requires that the role of Regulator be placed on a statutory footing.
No quality system can completely prevent errors or indeed malpractice. Accreditation to the required standard provides assurance that the systems and processes of an organisation (including the competence of its people) are appropriate. However, any process that involves human decision making and/or human practical intervention is subject to error. Therefore, proper scrutiny of the evidence in court is necessary.
Manipulation of quality control data has recently been reported within a forensic toxicology provider (see BBC News, 19 February 2017: bbc.in/2mca9sQ); two individuals who were employees of the company concerned have been arrested, although neither has yet been charged with any offence. The ongoing police investigation into these events prevents further comment at the time of writing.
The vast majority of forensic science is commissioned by policing authorities, often at the investigative stage. The priorities of police in their investigations are, quite properly, to obtain rapid detections at minimum cost. However, for contested cases, it is important to recognise that the end ‘customer’ is the court. Has the forensic science commissioned met the needs of the court? Would additional or different analyses have assisted with distinguishing between prosecution and defence hypotheses? Has the forensic scientist recommended further testing and has this been approved or denied by police ‘customers’?
The CPS website states that: ‘Streamlined Forensic Reporting (SFR) has been designed to enable investigators, scientists and prosecutors to comply with the Criminal Procedure Rules, in the interests of justice.’ In principle, this is a proportionate and helpful way of approaching scientific evidence. However, it is important that every SFR is clear and comprehensible, that it is obvious what admissions are being sought and that there is adequate understanding among defence solicitors and barristers regarding the SFR process. If, for example, there is a ‘match’ between a complex mixed DNA profile and a defendant’s DNA, is it clear to the defendant and his or her legal representative that the evaluation of the strength of this match could result in a likelihood ratio of significantly lower than a billion? Is it understood that many other peoples’ profiles may also ‘match’ this mixed profile? Is it understood that there may be multiple ways in which the DNA could have come to be on the item? So in agreeing to the SFR evidence, is the defence agreeing to the DNA ‘match’ or to the mode of transfer? Work is currently ongoing to ensure greater consistency on the SFR process nationally and to improve clarity regarding what admissions are being sought
There is a great deal of excellent forensic science delivered in the CJS, but risks and issues remain, which the Regulator is committed to addressing. The criminal defence community also has a role to play in challenging whether the forensic science in each case has been conducted to the appropriate standards, whether errors and pitfalls have been avoided and whether the limitations of the evidence are clearly stated, in line with the requirements of the Criminal Procedure Rules.
In England and Wales, provision of forensic science spans policing, the private sector, government agencies and even the National Health Service (NHS). Most police forces conduct crime scene investigation, collision investigation, fingerprint enhancement, comparison and interpretation, footwear screening, drug screening, DNA and body fluid search and recovery, digital forensics and firearms classification.
In addition to specialist advice and recovery at a small proportion of crime scenes, private sector forensic science laboratories carry out search recovery of a wide range of forensic traces and the majority of the complex analytical work, such as DNA profiling, drugs and toxicology analysis, firearms and gunshot residue analysis and analysis of trace evidence such as fibres, paint and glass. In addition, the private sector carries out interpretation of forensic science results across these disciplines, resulting in expert evidence.
Provision of forensic pathology is through group practices of primarily self-employed forensic pathologists, who are registered with the Home Office. A few pathologists are employed by the NHS or in academia. Experts from academia also become involved in the supply of evidence in their area of specialism.
Examination of explosives, together with certain DNA and fingerprint work, is carried out by the Defence Science and Technology Laboratory (DSTL). DSTL is an executive agency, sponsored by the Ministry of Defence.
Gill has worked in forensic science for over 25 years. In 2014, she took up the post of Forensic Science Regulator, and is responsible for setting quality standards for forensic science in England & Wales.
The landscape of forensic science provision is a fragmented one and involves several different and mutually independent organisations (see box, p 22). The need for regulation was identified back in 2005 by the House of Commons Select Committee on Science and Technology (House of Commons Select Committee on Science and Technology; Forensic Science on Trial, 7th Report of Session 2004-2005). The report recommended independent regulation and in July 2007, the government announced the creation of the role of Regulator and a Forensic Science Advisory Council (FSAC).
Members of the Bar should be aware of the crucially important work and output of the Regulator’s office and alert to the standards that apply to forensic science, and in particular scientists’ adherence to the over-arching principles that characterise high quality forensic science. This may have significantvalue in any case which includes scientific evidence.
The Regulator’s role is ‘to advise government and the criminal justice system on quality standards in the provision of forensic science’. The remit covers forensic science and forensic pathology, but excludes other allied disciplines including forensic psychiatry, forensic psychology and forensic accountancy. It applies to criminal justice only. Created under Royal Prerogative, the role has no statutory basis and no statutory powers. In 2016, the Home Office published a forensic science strategy (bit.ly/2r1QXQC) which stated the intention to place regulation on a statutory basis. However, the necessary legislation has not yet been passed. All progress to date, therefore, towards adoption of quality standards has been achieved through a combination of persuasion and contractual requirements.
There are four over-arching principles that characterise high quality forensic science: balance, logic, robustness and transparency. (For more on this, see ‘Standards for the formulation of evaluative expert opinion’ (2009) Science & Justice 49, 161-164.)
As codified in the Criminal Procedure Rules, the duty of the expert is to the court, and forensic scientists must remain independent of the commissioner of their work. Setting aside the potential for inappropriate pressure to be exerted on experts, in recent years, the scientific study of cognitive neuroscience has revealed that even highly trained professional scientists are susceptible to cognitive bias (see Dror, I E & Cole, S A Psychonomic Bulletin & Review (2010) 17: 161). This operates at the unconscious level. Areas where there is a high level of subjectivity, such as fingerprint comparison and CCTV image comparison including facial comparison are at particular risk of cognitive bias. A further risk factor is the use of ad hoc methods, rather than formal, validated scientific methods
A key component of balanced forensic science is evaluation of two competing propositions: one relating to the prosecution view and the other to the defence view. Any evaluation that looks at the evidence from only one or the other side is unbalanced and should be rejected. Similarly, use of phrases such as ‘consistent with’ can indicate bias. Take, for example, a case where cell site analysis has been used to infer the location of an individual at the time a phone call was made. A report that the results were ‘consistent with having been used at the crime scene’ do not make clear that the results are also ‘consistent with’ having been used at any address within the range of that cell, which may be a considerable distance. Phrases such as ‘in the vicinity of’ provide no clue to the range of potential locations where a call may have been made, so should not be used unless they are clearly defined.
The Regulator has issued guidance on the subject of cognitive bias (bit.ly/2szC5uG) and it is entirely reasonable to ask an expert through cross-examination, for example, if they are familiar with that guidance, and what steps they have taken to avoid bias.
Clarity and terminology are critically important to ensure that logic is not overturned during court proceedings through either the prosecutor’s fallacy or the defence lawyer’s fallacy.
For a scientific method to be robust, it must have been validated to enable its performance characteristics, strengths and limitations to be understood. There must be sound knowledge underlying the method, and there must be data on which to base interpretation of the results. The Regulator has published a process for validation of forensic science methods in the Codes of Practice and Conduct (bit.ly/2scBrWM) and associated guidance (bit.ly/2rQa9oq). It is critical that any method has been validated for the specific purpose for which it is being used. The test used for detecting explosive residue on the hands of the Birmingham Six had been validated as a screen and it was known that there were other substances which would cross-react. However, what should have been a screening result was used in evidence, without the limitations having been made clear.
Often, data are required in order to interpret findings. For example, in order to determine the evidential significance of a DNA result, data from unrelated individuals are used to determine the match probability or likelihood ratio. This data set is appropriate for the task and ‘match probabilities’ evidence is widely accepted in courts. However, the data available to the scientist making an evaluation of evidential strength in a footwear comparison were criticised by the Court of Appeal, which ruled that statistical methods should not be used where the data were insufficient (R v T [2010] EWCA Crim 2439). This ruling has caused ongoing debate in the scientific community and the Regulator plans to develop a standard for evaluative opinion, working with the Royal Statistical Society, practitioners and the judiciary.
Despite requirements of the Criminal Procedure Rules, it is still the case that not all scientists are highlighting to the court where there is a range of opinion, and justifying their view within that range. The role of the forensic scientist is to provide information and/or opinion in a balanced way: this includes the unknowns and the risks, eg:
The Forensic Science Regulator’s Codes of Practice and Conduct (the Codes) sets out the standards which apply to forensic science. It is legitimate to ask a forensic scientist if they are compliant with these Codes and if not, why not? For those not compliant, questions such as ‘How can you assure the court that your methods are robust?’ and ‘How can the court have confidence that you have conducted the tests and interpreted the results correctly?’ will enable the court to scrutinise risks and make an informed judgement on reliability.
The Regulator stated in her Annual Report (bit.ly/2r1NJwv) that ‘Funding for forensic science across the board, and particularly, perhaps, for defence provision via legal aid, must be at a level that enables the standards to be met. Otherwise we will face the costs, both in criminal justice terms and financially, of quality failures and loss of confidence in forensic science.’
Although compliance with the standards is increasing, in some organisations there is a fundamental lack of understanding of risk, quality, validation and the requirement for assurance. Whilst policing is formally committed to meeting the standards, the level of support at senior levels varies between forces, and the required resources are not always made available.
Whilst the majority of the large commercial and government-funded forensic science providers have a strong quality culture, lack of senior management support is evident in a small minority of providers. In small and micro commercial companies, the costs of achieving accreditation continue to be a barrier to compliance, and the Regulator is continuing to work with policing and the Legal Aid Agency to bring about a position whereby funding for forensic science is at a level that enables the standards to be met.
Full compliance, however, requires that the role of Regulator be placed on a statutory footing.
No quality system can completely prevent errors or indeed malpractice. Accreditation to the required standard provides assurance that the systems and processes of an organisation (including the competence of its people) are appropriate. However, any process that involves human decision making and/or human practical intervention is subject to error. Therefore, proper scrutiny of the evidence in court is necessary.
Manipulation of quality control data has recently been reported within a forensic toxicology provider (see BBC News, 19 February 2017: bbc.in/2mca9sQ); two individuals who were employees of the company concerned have been arrested, although neither has yet been charged with any offence. The ongoing police investigation into these events prevents further comment at the time of writing.
The vast majority of forensic science is commissioned by policing authorities, often at the investigative stage. The priorities of police in their investigations are, quite properly, to obtain rapid detections at minimum cost. However, for contested cases, it is important to recognise that the end ‘customer’ is the court. Has the forensic science commissioned met the needs of the court? Would additional or different analyses have assisted with distinguishing between prosecution and defence hypotheses? Has the forensic scientist recommended further testing and has this been approved or denied by police ‘customers’?
The CPS website states that: ‘Streamlined Forensic Reporting (SFR) has been designed to enable investigators, scientists and prosecutors to comply with the Criminal Procedure Rules, in the interests of justice.’ In principle, this is a proportionate and helpful way of approaching scientific evidence. However, it is important that every SFR is clear and comprehensible, that it is obvious what admissions are being sought and that there is adequate understanding among defence solicitors and barristers regarding the SFR process. If, for example, there is a ‘match’ between a complex mixed DNA profile and a defendant’s DNA, is it clear to the defendant and his or her legal representative that the evaluation of the strength of this match could result in a likelihood ratio of significantly lower than a billion? Is it understood that many other peoples’ profiles may also ‘match’ this mixed profile? Is it understood that there may be multiple ways in which the DNA could have come to be on the item? So in agreeing to the SFR evidence, is the defence agreeing to the DNA ‘match’ or to the mode of transfer? Work is currently ongoing to ensure greater consistency on the SFR process nationally and to improve clarity regarding what admissions are being sought
There is a great deal of excellent forensic science delivered in the CJS, but risks and issues remain, which the Regulator is committed to addressing. The criminal defence community also has a role to play in challenging whether the forensic science in each case has been conducted to the appropriate standards, whether errors and pitfalls have been avoided and whether the limitations of the evidence are clearly stated, in line with the requirements of the Criminal Procedure Rules.
In England and Wales, provision of forensic science spans policing, the private sector, government agencies and even the National Health Service (NHS). Most police forces conduct crime scene investigation, collision investigation, fingerprint enhancement, comparison and interpretation, footwear screening, drug screening, DNA and body fluid search and recovery, digital forensics and firearms classification.
In addition to specialist advice and recovery at a small proportion of crime scenes, private sector forensic science laboratories carry out search recovery of a wide range of forensic traces and the majority of the complex analytical work, such as DNA profiling, drugs and toxicology analysis, firearms and gunshot residue analysis and analysis of trace evidence such as fibres, paint and glass. In addition, the private sector carries out interpretation of forensic science results across these disciplines, resulting in expert evidence.
Provision of forensic pathology is through group practices of primarily self-employed forensic pathologists, who are registered with the Home Office. A few pathologists are employed by the NHS or in academia. Experts from academia also become involved in the supply of evidence in their area of specialism.
Examination of explosives, together with certain DNA and fingerprint work, is carried out by the Defence Science and Technology Laboratory (DSTL). DSTL is an executive agency, sponsored by the Ministry of Defence.
The beginning of the legal year offers the opportunity for a renewed commitment to justice and the rule of law both at home and abroad
By Louise Crush of Westgate Wealth Management sets out the key steps to your dream property
A centre of excellence for youth justice, the Youth Justice Legal Centre provides specialist training, an advice line and a membership programme
By Kem Kemal of Henry Dannell
By Ashley Friday of AlphaBiolabs
Providing bespoke mortgage and protection solutions for barristers
Joanna Hardy-Susskind speaks to those walking away from the criminal Bar
From a traumatic formative education to exceptional criminal silk – Laurie-Anne Power KC talks about her path to the Bar, pursuit of equality and speaking out against discrimination (not just during Black History Month)
Yasmin Ilhan explains the Law Commission’s proposals for a quicker, easier and more effective contempt of court regime
Irresponsible use of AI can lead to serious and embarrassing consequences. Sam Thomas briefs barristers on the five key risks and how to avoid them
James Onalaja concludes his two-part opinion series
