Automated decision-making: Balancing efficiency and legality post-Robodebt

By Nicola Johnson and Michelle Harradine

Increased availability of automated technologies presents a unique challenge for administrative decision-makers: can an appropriate balance be struck between embracing the benefits of technology and ensuring government decisions are lawfully made?

The Robodebt Scheme

Administered between 2016 and 2019, the Robodebt Scheme (the Scheme) is an infamous example of not achieving the right balance of automation for efficiency and the fundamental need for lawfulness in administrative decision-making. It involved raising of Centrelink debts using averaged income data obtained by comparing PAYG incomes reported by the ATO against wages reported to Centrelink.

Deemed unlawful in 2019, the Scheme was later the subject of a Royal Commission. The final report was tabled on 7 July 2023 and contained 56 recommendations, two relating specifically to automated decision-making.

Recommendation 17.1: Reform of legislation and implementation of regulation

The Commissioner recommended that the Commonwealth consider legislative reform to introduce a consistent legal framework in which automation in government services could operate.

Where automated decision-making is implemented:

1. there be a clear path for those affected by decisions to seek review.
2. departmental websites contain information advising that automated decision-making is used and explaining in plain language how the process works.
3. business rules and algorithms be made available, to enable independent expert scrutiny.

Recommendation 17.2: Establishment of a body to monitor and audit automated decision-making

The Commissioner recommended that the Commonwealth consider establishing a body, or expanding an existing body, with the power to monitor and audit automated decision-making processes about their technical aspects and their impact in respect of fairness, the avoiding of bias, and client usability.

Government response

The Government provided its response to the Royal Commission in November 2023, accepting all 56 of the Commission’s recommendations in full or in principle.

Regarding the two recommendations relating to automation, the Government specifically agreed to continue to implement the Digital Service Standard, progress legislative reforms to ensure automated decisions are made under a consistent legal framework, consider establishing a body to monitor automated decision-making and ensure Australians had greater control over their own personal information (see our previous article, Privacy Act changes on the horizon: Federal Government response to Privacy Act Review report, on the Government’s response to the Privacy Act review).

The Commissioner’s recommendations have been addressed in part by the Administrative Review Tribunal Bill 2024. The Bill proposes to establish the new Administrative Review Tribunal which will:

1. have a Guidance and Appeals Panel with the power to review decisions referred to it by the President that raise an issue of significance to administrative decision-making.
2. adopt a broad definition of ‘decision maker’ to provide a ‘fall-back’ where there is ambiguity or uncertainty about the person who made a decision, such as in cases of automation.
3. re-establish the Administrative Review Council to monitor the integrity of the Commonwealth administrative law system, inquire into the adequacy of procedures used in relation to the making of administrative decisions, and inquire into systemic issues related to the making of administrative decisions.

Where to from here?

While significant steps have been taken since the Report to create meaningful change, further progress is required to promote administrative law principles of fairness, consistency, lawfulness, impartiality, and accountability where automation is used in government decision-making.

Importance of human involvement

There are obvious examples of scenarios where fully automated decisions are preferable. These simple cases may include where someone is eligible for the grant of a benefit, visa or payment and this can be processed efficiently and expediently through automation. This needs to be balanced against more complex decisions where human involvement is necessary and integral to the validity of the decision. Discretion plainly limits the scope and accessibility of automated decision-making. It is fundamental that, if a decision involves the exercise of discretion, the decision maker must exercise that discretion personally and not be fettered in doing so.

Uniform approach to judicial oversight

Significant legislative reform is needed to introduce a consistent legal framework in which automation in government services can operate in Australia.

1. As a starting point, legislative amendments must ensure that the relevant statute expressly authorises the use of automation, where it is intended for a statutory power to be exercised by using automated decision-making.
2. The legislation might require periodic assessments of the suitability of the proposed automated system to exercise the statutory power and require regular auditing, testing and reporting obligations.
3. It also might require that an automated decision be reduced to a statement of reasons to allow the person affected by the decision to understand the decision-making process.
4. The legislation must provide a clear pathway for administrative review of the decision.

Expert input into design of automated systems

Responsibility for the design and implementation of a new automated decision-making system cannot be left solely to the IT department. Automated systems should be designed with statutory interpretation at the forefront and involve a team of lawyers, policymakers, and operational and technical experts to ensure legislative compliance and safeguard against unintended results.

Quality assurance

It is crucial that assumptions in automation are repeatedly challenged and tested. Rigorous and ongoing legal testing should take place to ensure the relevant law has been correctly interpreted into code. Validation and accuracy testing can ensure the machine is making the same decisions as humans on the same data. Data sets used in the design of the system may contain information with historical biases or inequalities, so it is essential to also test for algorithmic bias. Automated decision-making systems should have integrated monitoring, review and audit to ensure decisions are legal, accurate, fair and consistent.

Transparency and accountability

Transparency and responsible disclosure around the use of automation in decision making is critical to ensuring people understand when they are engaging with it and can challenge outcomes. A recent report of the NSW Ombudsman tabled in Parliament, mapped 275 automated decision-making systems reported to be in use, or in development, in the NSW public sector. To promote transparency, similar reports could be published in each State and Territory and by the Federal Government, with a move towards mandatory rather than voluntary participation.

The lack of clarity and technical complexity of automated systems can make it difficult for individuals to identity and challenge administrative errors. Individuals whose rights are affected by automated decisions need to be equipped with the information to allow them to understand, review and question those decisions.

Conclusion

The Commissioner’s recommendations and the Government’s response to the Report make clear that the future of automated decision-making in Australia must focus on transparency and accountability. Most critically, governments must determine which decisions require human involvement and the degree and timing of that involvement; that question can only be answered by reference to the particular agency’s functions and the legislation being applied.