Challenge 11.5

What is the problem, and how does it affect the Challenge Sponsor organisation, service users and/or People of Scotland?

Impact assessments (IAs) are a key tool for supporting good policy making and ensuring that the Scottish Government meets its legal obligations. For key decisions, government officials are required to complete impact assessments which clearly set out the impacts of decisions (on different groups, the environment and business).

The Scottish Government is required to complete up to eight different IAs when making policy decisions (seven of these are a statutory requirement). The system of IAs has evolved in an ad hoc fashion over the last 25 years since devolution and has become complex, fragmented and uncoordinated.

The system of impact assessments is difficult and time consuming for teams to navigate. Each of the eight impact assessments follows slightly different processes, although there are significant stages in common between most assessments. Each assessment has bespoke and lengthy guidance, training and templates to complete and is supported by different teams in government. 

Often teams struggle to properly resource the production of high quality and timely assessments. Impact assessments typically take the form of written documents that run from 5 to 50 pages depending on the complexity of the decision, and the Scottish Government produces and publishes several hundred impact assessments each year at considerable cost. Templates need to be completed off-line and information in the templates is not pre-populated or transferred from one impact assessment to another. 

Policy teams may start the impact assessment process with a limited understanding of the evidence relating to policy decisions. There may also be particular unintended bias and blind spots in policy teams’ thinking.

There are significant opportunities to improve Scotland’s system of impact assessment. Impact assessment is often conducted as a standalone activity and it may be possible to better draw on evidence from previously completed impact assessments on similar topics when commencing impact assessments – thereby building the evidence base in an iterative way.

As well as automating and digitalising the processes for undertaking impact assessments to make it simpler and quicker to complete there may be opportunities to use AI/ machine learning to identify potential impacts associated with policy decisions early as a basis for broadening the initial consideration of impacts.

There may also be opportunities to use digital approaches to allow the outputs from Impact Assessment to be more transparently available to the general public allowing them to directly search and interrogate outputs.

Any technological solution will need to be sufficiently flexible and adaptable to changes in the overall system of impact assessments in Scotland - for example the introduction of new impact assessments, work to integrate existing impact assessments or changing requirements to existing impact assessments.

How will we know the Challenge has been solved?

We will know the challenge has been solved when:

1. The process for completing impact assessments has been simplified for SG policy teams
   Impact assessments will take less time to complete and be less resource intensive. The process for completing impact assessments will be less repetitive across different assessments and more intuitive and will support policy teams to focus more on evidence and outcomes – rather than navigating complex guidance and templates. Impact assessments will be completed earlier in the process and there will be less demand for central support in completing impact assessments.

2. We have better quality impact assessments
   Impact assessments will include a greater depth and breadth of evidence (perhaps supported by AI) which better articulates the range of potential impacts of decisions and options to address these and successfully builds on the evidence from other completed impact assessments and evaluation evidence. Better and more up to date evidence underpins policy decisions, including evidence of 'what works' We may also receive feedback from interested stakeholders on the quality of impact assessments, when they review the published end products.

3. Decision makers are better placed to consider the range of potential impacts associated with policies and weigh up trade-offs
   Ministers and other senior decision-makers receive coherent advice from all required assessments that is evidence based and influences their decisions. This advice clearly set out potential trade-offs for example tensions between environmental goals and socio-economic goals etc. The quality of advice would be enhanced as result of impact assessment outputs being more consistent and higher quality.

4. Policy makers skills and knowledge are improved
   Policy makers have a stronger understanding of why particular impact assessments are required and used. Policy makers spend less time on form filling / bureaucracy and more time on thinking, gathering and processing evidence, engaging with stakeholders and considering trade-offs and alternative policy approaches.

5. Scotland’s system of impact assessments is recognised as effective by stakeholders and internationally
   Solving the challenge is likely to result in increased level of confidence across the policy profession in being able to meet requirements for impact assessments, in increased understanding of the value of impact assessment and in increased perception of trust among stakeholders. If successful we would receive praise from external stakeholders on the quality of published IAs and other governments and public bodies would look to Scotland and be keen to learn from the solution implemented here.

Who are the end users likely to be?

Policy and delivery teams in Scottish public bodies, with an initial focus on Scottish Government teams involved in conducting impact assessments. The solution will also be of interest to stakeholders with a role in scrutinising Scottish Government policy making processes and decisions. The ultimate beneficiaries, should the solution be successful, will be people, businesses and communities in Scotland, via improvements in the efficiency and effectiveness of impact assessments.

Has the Challenge Sponsor attempted to solve this problem before?

No. We have a range of internal resources to support teams with individual impact assessments mostly comprising web-based guidance and templates in Word, Excel or PDF format.

Are there any interdependencies or blockers?

If the solution is to be used on Scottish Government IT hardware and through the SCOTS network then there will be security and other restrictions on what software can be installed / used and on what can be web-accessed. Any solutions must take account of the separate legal requirements underpinning each impact assessment, which are distinct (and should be flexible enough to change to reflect any future changes to those legal requirements). There are likely to be sensitivities with using AI tools to consider the potential impacts of sensitive policies – SG policy on AI use will need to be followed.

Will a solution need to integrate with any existing systems / equipment?

If the solution is to be used on Scottish Government IT hardware and through the SCOTS network then there will be security and other restrictions on what software can be installed / used and on what can be web-accessed. Any solutions must take account of the separate legal requirements underpinning each impact assessment, which are distinct (and should be flexible enough to change to reflect any future changes to those legal requirements). There are likely to be sensitivities with using AI tools to consider the potential impacts of sensitive policies – SG policy on AI use will need to be followed.

Is this part of an existing service?

Yes – impact assessments underpin successful policy development and delivery of all public services.

Any technologies or features the Challenge Sponsor wishes to explore or avoid?

AI may have particular potential to provide a means of ensuring that, at the outset of the assessment process, policy teams are fully appraised of relevant evidence around both context and policy options. Currently teams are typically asked to ‘scope’ potential impacts themselves before undertaking detailed evidence gathering, and a system that could automatically summarize existing relevant evidence in response to the stated aims of a draft policy may offer a means to start IAs with a better grasp of evidence early in development.

There may be an opportunity to draw on machine learning as an explicit feature of AI. Although this Challenge is more focused on the earlier stages of policy development there may be opportunities, for example, to feed in data once a policy has been implemented and reviewed and the tool learns from that to refine further the approach to IAs.

What is the commercial opportunity beyond a CivTech contract?

Other public bodies beyond SG are subject to the same duties to assess impacts and are likely to be interested in the potential of a solution to this challenge.  There may also be specific benefit to other public sector customers in Scotland of knowing that using this solution means their processes are aligned to SG processes. 

There may be interest in other countries of the UK and overseas in a technical system for streamlining impact assessments, as long as the solution is flexible enough to be applied to differently defined duties or different combinations of duties.

Who are the stakeholders?

Impact assessment teams across the Scotland Government

• Office of the Chief Economic Adviser and other analytical teams

• Policy Profession team

• Scottish Delivery Bodies Group – this includes the CEOs and others of larger public bodies with delivery functions.

• Stakeholders whose interests are represented in impact assessments: Businesses, organisations representing disadvantaged groups, island communities and others

• Users of impact assessments for the purposes of scrutiny, e.g Scottish Parliament, EHRC, Poverty and Inequality Commission

• Policy and delivery teams in Scottish Government

• Policy and delivery teams elsewhere in Scottish public sector

• ITECS

Who’s in the Challenge Sponsor team?

This project would be supported by a small team of policy and analytical officials who are leading work to improve our current system of impact assessments. The project would also be supported by leads for each of the individual impact assessments.

What is the policy background to the Challenge?

This challenge relates to wider work initiated by Scottish Government Executive Team to improve the Scottish Governments system of impact assessments. The eight impact assessments which currently apply to policy decisions are:

Business and Regulatory Impact Assessment (BRIA) – no legal requirement – introduced 2010. The BRIA was introduced to consider the impacts on businesses of policy/ regulations. The current version was developed in response to the specific recommendations of the New Deal for Business, which was introduced to help improve the SG relationship with business and how we worked with them in developing policy and regulations.

Consumer Duty Impact Assessment – required since April 2024. The Consumer Duty Impact Assessment aims to put consumer interests at the heart of strategic decision-making across the public sector to deliver better policy outcomes for Scotland. This approach should result in better quality services and outcomes for consumers as users of public services. The assessment applies to any SG decisions of a strategic nature that could impact on consumers, which includes small businesses.

Child Rights and Wellbeing Impact Assessment (crwia) – legally required since July 2024 (introduced on non-legislative basis in 2015. CRWIA is a process, tool and report through which the anticipated impact of any legislative provision (such as bills, and Scottish Statutory Instruments), and decisions of a strategic nature (including budgeting decisions) relating to the human rights and wellbeing of children can be identified, researched, analysed and recorded. The CRWIA began as a Ministerial expectation in 2015 as a means of supporting Ministers to meet their duties and became a legal requirement in July 2024.

Data protection impact assessment (dpia) – required since May 2018. The Scottish Government is required to consult with the Information Commissioner’s Office (ICO) when policy proposals or secondary legislation could significantly impact the processing of personal data. The ICO may require a legislative DPIA for proposals deemed particularly sensitive. The data controller (e.g., the Scottish Government or other public bodies) is responsible for conducting a DPIA, focusing on the processing aspect. The introduction of this DPIA requirement aims to prevent compliance failures, protect individuals' rights, and avoid regulatory penalties.

Equality Impact Assessment (eqia) - required since May 2012. EQIAs are a tool to assess policies, strategies, legislation, etc against each of the three needs of the Public Sector Equality Duty (PSED).  The EQIA process helps identify how policies can be adjusted or implemented to better meet these needs/aims: eliminating discrimination, harassment, victimisation, and other prohibited conduct; advancing equality of opportunity between people who share a protected characteristic and those who do not; and fostering good relations between people who share a protected characteristic and those who do not. The assessment applies to all new and revised policies and practices.

fairer Scotland Duty Assessment (FSDA) – required since April 2018. The Fairer Scotland Duty requires public bodies in Scotland to actively consider how they can reduce inequalities of outcome caused by socio-economic disadvantage. The FSDA is to support compliance with the legal duty. The assessment applies to decisions of a strategic nature about the exercise of functions, which have the potential to impact on inequalities of outcome for socio-economically disadvantaged persons.

Island Communities Impact Assessment (Icia) – required since December 2020. The ICIA aims to ensure SG considers the unique circumstances of island communities in the development of their policies, strategies or services. The impact assessment is a statutory process stemming from the Islands (Scotland) Act 2018 which introduced the requirement to have due regard for island communities in decision making – including, where necessary completing Island Communities Impact Assessments.

Strategic Environmental Assessment (sea) - required since July 2004. SEA is a means to assess the likely significant impacts of a qualifying public plan on the environment and to seek ways to avoid or minimise negative effects and enhance positive effects. The assessment applies to public bodies and those organisations preparing plans of a 'public character'. The assessment is a legal requirement of Environmental Assessment (Scotland) Act 2005, during the preparation of a qualifying plan or programme. In this Act the environmental assessment is the preparation of an environmental report, the carrying out of consultations and the taking into account of the environmental report and the result of the consultations in decision-making.