Technology Participant FAQs

Standards-Based Evaluation Plan 

Our approach is in accordance with Australia’s National Quality Infrastructure established by the Department of Industry, Science and Resources. This layered approach gives businesses and consumers confidence in the goods and services they are developing, using or trading. Accreditation takes place in an international mutual recognition treaty through JAS-ANZ(for organisations, products and people) and NATA (for laboratories)in global partnership with UKAS. Beneath this layer, conformity assessment bodies, like ACCS, provide product testing, laboratory testing and management systems certification of products and services to international standards. 

Our evaluation reporting and modelling, developed in accordance with ISO/IEC 25040:2024 Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Quality evaluation framework will provide robust, repeatable and recognisable evaluation to international standards. 

Our evaluation reporting and modelling, developed in accordance with ISO/IEC25040:2024 Systems and software engineering —Systems and software Quality Requirements and Evaluation (SQuaRE)—Quality evaluation framework will provide robust, repeatable and recognisable evaluation to international standards.  

The product quality model is generic to any information technology system, but covers the key aspects of functional suitability, performance efficiency, compatibility, interaction capability, reliability, security, maintainability, flexibility and safety of the technology. It therefore offers a robust and appropriate generic framework upon which to build evaluation activity. 

We have developed a detailed evaluation plan covering: 

• Age verification technologies
• Age estimation technologies
• Age inference technologies
• Successive validation
• Parental Consent/Control
• Technology stack deployments
• Technology readiness
• Implementation factors for age assurance technologies

Testing will involve a combination of automated functional and non-functional tests, manual usability and acceptance tests, manual functional tests and static reviews. 

We evaluate technologies against the agreed framework in the Institute of Electrical and Electronics Engineers (IEEE) Standard for Online Age Verification (IEEE 2089.1) and consider the draft ISO/IEC 27566-1 which is still under development but has reached a stable draft international standard (DIS) stage. 

It is expected that the assessment process considers the extent to which an age assurance approach is consistent with the  principles and upholds any rights as set out in the United Nations Convention on the Rights of the Child. This includes the rights to privacy, harm protection, and not unduly restricting access to services, including during the trial. 

• Accuracy (how well the technology can detect a user’s age) 

• Interoperability (how well the technology can be used across multiple online platforms) 

• Reliability (how consistently the technology can produce the same result) 

• Ease of use (how simple the technology is to operate) 

• Freedom from bias (how well the technology avoids racial or other bias) 

• Protection of privacy (how well the technology protects users’ personal information, including data minimisation techniques) 

• Data security (how well the technology safeguards users’ personal information from unauthorised access, breaches or theft through, for example, the use of security by design principles and resistance to presentation attacks) 

• Human rights protections (i.e. accessibility for all users, including people with disability, as well as applicable rights under the UN Convention on the Rights of the Child) 

Age assurance technologies can be applied in different contexts and use cases, different verification or estimation methods (and/or a combination of these methods) can be used to establish age with varying levels of certainty proportionate to the risk of harm. Different levels of risk or legislative requirements may require different levels of assurance. 

For example, a facial age estimation technology may provide a level of assurance that is considered sufficient to access services with a lower risk of harm, but would not establish enough indicators of confidence to permit access to a service with a higher risk of harm. Depending on the circumstance, establishing a higher level of assurance may be achieved by using one method with a higher confidence indicator (e.g. government-issued credential) or by combining two or more methods with lower confidence indicators.  

This will allow assurance methods to be benchmarked against levels of assurance using established indicators of confidence, so that different assurance technologies can be evaluated against a common benchmark. 

The trial will consider assurance methods across the technology stack i.e. including at the device, operating system, digital platform, app store and/or application level. 

We identify, test, and assess known assurance methods that may be appropriate to determine a user being between 13-16 years of age, including but not limited to:  

• biometric age estimation 

• account confirmation processes (such as verified parental consent through a credential) 

• email verification processes (and other algorithmic profiling solutions) 

• parental controls 

• age-appropriate tokenised attribute exchange models 

• device or operating system-level interventions 

• double-blind tokenised attribute exchange models (as described in the eSafety Commissioner’s Roadmap for Age Verification). 

• non-government-issued hard identifers e.g. credit cards or open banking. 

The project timeline is as follows:

1. Project Plan - by 31 Oct 2024
2. Trial Development - by 31 Dec 2024
3. Preliminary Report - by 30 Apr 2025
4. Final Report - by 30 Jun 2025

The Age Check Certification Scheme, operated by AVID Certification Services Limited, was appointed to lead the trial. We test that ID and age check systems work. 

We are a global conformity assessment body based in the UK (with an AU subsidiary company), accredited by the UK Accreditation Service under ISO/IEC 17065, which assesses age verification, age estimation and age inference systems all over the world, including existing clients in Australia. 

We are accredited by the United Kingdom Accreditation Service under: 

• ISO/IEC 17065:2012 - Conformity assessment - Requirements for bodies certifying products, processes and services
• ISO/IEC 17021-1:2015 - Conformity assessment - Requirements for bodies providing audit and certification of management systems - Part 1: Requirements
• ISO/IEC 17025:2017 - General requirements for the competence of testing and calibration laboratories (Pending Accreditation – our assessment for this is on 5th & 6th November 2024)

We are the appointed auditor for the UK’s Proof of Age Standards Scheme operated by PASSCO CIC, including the development and management of their standards suite: 

• PASS 0:2022 – Proof of Age Standards Scheme – General Principles and Definitions
• PASS 1:2022 – Proof of Age Standards Scheme – Requirements for Identity and Age Verification
• PASS 2:2020 – Proof of Age Standards Scheme – Requirements for e-ID Validation Technology
• PASS 3:2020 – Proof of Age Standards Scheme – Requirements for Data Protection and Privacy
• PASS 4:2022 – Proof of Age Standards Scheme – Requirements for Proof of Age Card Design and Construction
• PASS 5:2023 – Proof of Age Standards Scheme – Requirements for Digital Presentation of Proof of Age

Our certification criteria are formally approved the Information Commissioner’s Office in accordance with the Commissioner’s tasks and powers under Articles 57(1)(n) and 58(3)(f) pursuant to Article 42(5) of the UK General Data Protection Regulation. 

We are appointed as an authorised certification body by the UK’s Department for Science, Innovation and Technology for the Digital ID and Attributes Trust Framework. 

We provide a comprehensive range of certifications essential for business operations and compliance. Our offerings include ID verification, age assurance, presentation attack detection, information security management, ISO standards across various domains, and GDPR compliance. Each certification is meticulously crafted to meet industry best practices and regulatory requirements, ensuring our clients uphold the highest standards of security, efficiency, and legal compliance. Explore www.accscheme.com for our diverse certification portfolio. 

We have teamed up with Australian company, KJR software, Artificial Intelligence and User Experience testing consultants based in Canberra and Sydney to develop a comprehensive programme of evaluation and testing of age assurance systems in this trial. 

KJR is an Australian software quality engineering consultancy business which believes in harnessing technology for meaningful impact. Specialising in software testing and AI implementation, KJR is the partner of choice for many of Australia’s leading enterprises, SMEs and technology challengers; they pride themselves on being thought leaders on the impact and risk of digital technologies and enabling customer success. With over 27 years in the tech industry, KJR is committed to delivering world-class professional services at a human scale. 

Our 20-person strong core team is led by Project Director Tony Allen, Subject Matter Expert on ID & Age Assurance, Editor of ISO/IEC 27566 Age assurance systems international standards. 

Tony is a Chartered Trading Standards Practitioner and Global Subject Matter Expert on Age Assurance, including: 

• As the Technical Editor of the ISO/IEC 27566 series of international standards on age assurance
• As the Chair of IST 33/5/5 – BSI Committee on Age Assurance
• As the Chair of the UK Government’s Expert Panel on Age Restrictions
• As a contributing specialist to the IEEE 2089.1 standards on online verification
• As the expert witness in Paxton v FSC currently being considered by the Supreme Court of the United States (one of a series of cases in which he has given evidence in the US about age assurance)
• As the author of the Law of Age Restricted Sales in England & Wales

Our Deputy Project Director is Andrew Hammond, Software Testing Specialist, Contributing Expert on ISO/IEC 29119 Software testing international standards and General Manager at KJR.  

We have put together an eminent team of specialist scientists, independent validation, stakeholder advisory, communications, project management and delivery partners. Our objective is not only to deliver scientific testing of the full range of age assurance methods and approaches, but also to gather feedback from a diverse population of Australian users by conducting the trial as much as possible in the context of existing real-world deployments. 

We then have four PhD level eminent and specialist scientists to design and implement the evaluation plan; seven product test engineers; a team of three graphic designers and data visualisation specialists for report production; three stakeholder and engagement specialists and a Legal Counsel to the Trial 

Understanding the desired statistical significance for the evaluation activity is critical, because this has the single biggest impact on the overall cost variables of the project. Statistical validity is principally based around the number of trial participants that you need to have confidence in the evaluation results, like the classification accuracy measures (false accept rate, false reject rate, failure to acquire rate), binding accuracy (age assurance output relates to the correct individual), outcome error parity (freedom from bias). 

The level of statistical validity required is an important variable factor that we would be willing to discuss further with the Department. As a starting point, taking the 26m population of Australia and applying a generally accepted in research methodology confidence interval of 95% (giving a Z score of 1.96), and a proposed margin of error of 0.03; this leads to a sample size of 1067+ for analysis. 

We would apply this to population wide analysis, but apply a wider margin of error of 0.05 to specific population sub-categories (making the sample size 384+). 

We are establishing an Ethics Panel chaired by George Billinge (former Senior Policy Manager for Age Assurance at the UK Regulator, OFCOM) and our usual ACCS Impartiality Panel required as a part of our existing accreditation. 

ACCS are an existing accredited conformity assessment body under ISO/IEC 17065. This requires us to have in place a mechanism for safeguarding impartiality, which we achieve through our independent impartiality panel. This will include the creation of a conflict of interest declaration and register which will be applicable throughout the project. 

Our project will be guided by an Ethics Handbook and overseen by an Ethics Committee, Chaired by George Billinge, a former Policy Manager on Age Assurance for UK Online Safety Regulator, OFCOM, supported by Lyn Nicholson from Holding Redlich acting as Counsel to the Trial Project. This is critically important for a project that involves the collection of biometric data and data about under 18’s and working with human test subjects. This task includes liaison with the Office of the Australian Information Commissioner (OAIC) and the application of safeguarding of children policies in the AU context. These are existing ACCS policies that need to be applied and adapted to the specifics of the technology trial. 

Our ethical approach will include analysis and extent of Aboriginal and Torres Strait Islander peoples and multi-ethnic diverse communities in the demographic spread of human test subjects including through the application of the AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research. Working with KJR’s Cultural Advisor, Johnny Fejo, the project will include initial and ongoing Equality, Diversity and Inclusion Monitoring to identify where any aspects of the project have EDI consequences and put in place appropriate measures and controls to address them. 

We respectfully acknowledge the Traditional Custodians of the lands and water in Australia where we will be conducting our trial and we pay respects to their Ancestors and Elders past, present and emerging and are proud to support their communities through their inclusion and careful consideration throughout the design, implementation, communication and reporting of the trial. 

We have built in independent scrutiny including validation of our proposed approach to evaluation by Prof. Toby Walsh, Laureate Fellow, Scientia Professor of AI and Chief Scientist at the University of New South Wales’s AI Institute.  

It is critically important for the overall quality of the project, to ensure that the results are accepted, robust, recognised and capable of supporting the Department in their ongoing policy development. In order to achieve this we propose that the approach to evaluation is independently validated. We intend to invite eminent Professor Toby Walsh FAA FAAAI FAAAS FACM FEurAI FRSN, who is a Laureate Fellow, Scientia Professor of AI and Chief Scientist at the University of New South Wales’s AI Institute to independently validate our project evaluation plan and project report. 

Toby Walsh is Scientia Professor of Artificial Intelligence at the University of New South Wales in Sydney and CSIRO’s Data61. He is the winner of the prestigious Celestino Eureka Prize for Promoting Understanding of Science and was named on the international “Who’s Who in AI” list of influencers. He has given over 100 talks on the subject of AI to a wide range of audiences, from industry associations, conferences for the general public, corporate events, board meetings, policy meetings and many others. 

As well as speaking at leading companies and stages globally like CeBIT, the World Knowledge Forum, World Summit AI and TEDx, Toby also appears regularly on TV and radio, has been profiled by the New York Times and has authored four books on AI for a general audience, the most recent ones entitled “Machines Behaving Badly” and “Faking It: Artificial Intelligence in a Human World” (Fall 2023). He is a Fellow of the Australia Academy of Science and was named by the newspaper The Australian as one of the “rock stars” of Australia’s digital revolution. He has won both the Humboldt Prize and the NSW Premier’s Prize for Excellence in Engineering and ICT. His Twitter account was voted in the top ten to follow to keep abreast of developments in AI. 

The specific methodology for testing is still under development, but from our experience of undertaking testing elsewhere, we anticipate that the following description of how we would likely deploy the tests for each technology will feature: 

1. Age verification methods - Ensure that users are accurately verified as being above or below a required age threshold using valid identity documents or other verification methods. This will involve document verification testing using our dataset of 4,000+ legitimate, fake, and altered identity documents (e.g., passports, driver's licenses) from various regions, nationally and internationally. We present the documents to the UX gateway for participant age assurance providers to evaluate whether the system correctly accepts valid documents and rejects fraudulent or tampered ones. We also test with documents that are worn, slightly damaged or poorly scanned to assess how tolerant the system is to imperfect inputs. We will perform security testing to identify vulnerabilities that could allow users to bypass verification, including biometric comparisons and deep fake and video injection presentation attack.
2. Age estimation methods - Test the system’s ability to estimate a user's age based on biological or behavioural features that vary with age. We will augment our existing test dataset with additional images of users of varying ages, ethnicities and genders which reflect the full diversity of the Australian population (27.6% of whom were born overseas), including high-quality and lower-quality images (e.g., different lighting conditions, angles and facial expressions). We will upload images and check how accurately the system estimates age, comparing the results with the actual ages and perform tests with both static photos and real-time camera feeds. We’ll also explore voice, hand geometry or typing speed analysis, test different age groups and analyse the estimation results. We will test with outliers, such as individuals with facial features that may not correspond to their chronological age (e.g., younger adults with premature aging, or children with adult-like features). We will undertake outcome error parity analysis.
3. Age inference methods – we will test systems that infer a user’s age based on data inputs such as purchase history, possession of other age-related evidence, browser behaviour or online activity. We will analyse the assumptions and reliability of the inference behind decision making tools. We will simulate various user behaviours online (e.g., browsing child-friendly content vs. adult-targeted services) using our existing and new avatar test accounts that mimic different personas, including children, teenagers, and adults, with different browsing histories and online purchase behaviours. We will ensure that the system does not violate privacy policies and correctly anonymizes data where required.
4. Parental Consent Technology – we will ensure that parental consent systems accurately verify a parent or guardian’s identity and relationship to the child before granting access to services. This includes simulating different parent-child relationships, testing various identity verification methods (e.g., credit card verification, email, document upload). We will also test for cases like foster parents, legal guardians, or step-parents to ensure the system accommodates complex family structures. We will simulate requests for access from children of different ages and assess how the system handles multiple requests, including revoking consent and we will attempt to simulate potential fraudulent behaviour, such as children trying to impersonate parents, or bad-actors seeking to groom children through offers of fraudulent parental consent. Ensure the system is resistant to such efforts.
5. Parental Control Technology – we will test how effectively parental control technologies can limit access to content or services based on a child’s age and how parents can monitor or restrict activity. We will create test user accounts for different age groups and test access to restricted services (e.g. age-restricted content or apps) and examine if content is appropriately filtered or blocked based on user profiles. We will test different levels of parental controls, from time restrictions to content categories and evaluate how easy it is for parents to configure these settings. We will examine how intuitive the parental control interface is for setting restrictions, monitoring activity and receiving notifications, ensuring parents can reasonably easily modify settings without the need for technical expertise. We will simulate attempts by children to bypass parental controls (e.g. changing settings or creating fake accounts) to assess the robustness of the parental control measures.

One aspect of the trial is to test that solutions are secure and manage personal data properly.  This aspect of the trial will be completed before any personal data is used for the trial itself. 

Yes.  We have appointed a legal counsel to the trial from the leading law firm Holding Redlich, to advise us on compliance with all relevant laws and regulations. 

Understanding the desired statistical significance for the evaluation activity is critical, because this has the single biggest impact on the overall cost variables of the project. Statistical validity is principally based around the number of trial participants that you need to have confidence in the evaluation results, like the classification accuracy measures (false accept rate, false reject rate, failure to acquire rate), binding accuracy (age assurance output relates to the correct individual), outcome error parity (freedom from bias). 

The level of statistical validity required is an important variable factor that we are still in the process of determining. As a starting point, taking the 26m population of Australia and applying a generally accepted in research methodology confidence interval of 95% (giving a Z score of 1.96), and a proposed margin of error of 0.03; this leads to a sample size of 1067+ for analysis. 

This is part of the testing process, and we will be assessing not only if the result is incorrect, but also how far out it is.  Some methods, such as estimation, will never determine an exact age, but aim to estimate as close as possible to the real age.