Skip to main content

AI Trustworthiness

What is it?​

AI Trustworthiness solutions encompass a suite of services and tools designed for evaluating the reliability, safety, and ethical considerations of AI technologies. Whether through a human-led process (such as red-teaming) 1 or automated test (e.g., benchmarking tools), companies in this category evaluate the data inputs used in AI, the trained frontier AI models, downstream AI systems, an organisation's safety standards and procedures, as well as even the AI application tech stack—i.e., the combination of machine learning frameworks, data processing tools, development and deployment environments, specialised hardware, network infrastructure, and more involved in implementing an AI use case.

While sometimes referred to as AI Auditing, here we distinguish risk audits from compliance audits, by calling the former "evaluations". By evaluations, we mean testing that is not always for the purposes of documenting conformity with regulatory requirements or industry standards; AI evaluations can be repeatedly performed above and beyond the specifications of legislation for reasons related to competitive advantages, product-market-fit, risk management (e.g., minimising asset losses, litigations, or insurance costs), and more. Likewise, we can think of AI Trustworthiness solutions as divided into two sub-domains: (1) data, model, or system-focused evals, and (2) external compliance audits or organisational conformity assessments.

Data, model, or system -focused evaluations include rigorous tests to identify and mitigate bias, errors, and potential risks in AI datasets, models and systems. By examining AI model inputs, such as training data or fine-tuning 2 data, as well as the behaviour of AI models, both procedural and automated evaluations aim to promote fairness, reliability, and safety in AI technologies (among other desirable characteristics of trustworthy AI). Techniques such as adversarial testing, interpretability and explainability studies 3, or formal verification assessments could be utilised pre-deployment and post-deployment to continuously enhance the reliability and ethical standards of AI applications 4.

External compliance audits and organisational conformity assessments represent another crucial dimension of AI Trustworthiness. These are conducted to ensure that AI technologies and their deploying organisations meet requisite laws and industry standards, covering both pre-deployment readiness and periodic reviews after deployment. Third-party compliance audits may include on-site inspections of an AI application's technology stack, reviewing data certifications, or scrutinising the results of the above-mentioned evaluations. Each of these activities play a pivotal role in organisational accountability.

Why is it imperative?​

AI Trustworthiness solution providers primarily help to mitigate risks from AI Technical Failures, whether driven by Unreliability or Misalignment. However, because they may also evaluate the adversarial robustness of models—that is the ability of the system to withstand or overcome adversarial conditions—they may help prevent AI Misuse in all its forms. Demand for auditing solutions will be driven by industry imperatives to minimise legal and reputational risks, enhance user trust, and ensure operational viability. Moreover, it is possible for some AI systems to become more dangerous after deployment, emphasising the importance of regular post-deployment auditing.

The most pertinent compliance requirements that can be addressed through evaluations or audits might include requirements for companies to: Monitor and attest to AI model robustness against misuse, Implement measures to address errors, faults or inconsistencies, Identify systemic risks and implement standardised risk mitigation measures, and Implement robust data governance framework, to name a few.

How might it work?​

SUB-DOMAINEXAMPLE SOLUTION IDEASSOLUTION TYPE
EvaluationsAlgorithmic bias detection and mitigation toolsSoftware
EvaluationsAdversarial testing for model robustnessSoftware / Service
EvaluationsInterpretability tools and platformsSoftware
Compliance auditsPre-Deployment compliance auditService
Compliance auditsData-focused evaluationsSoftware / Service
Compliance auditsOrganisational governance procedures auditService
Compliance auditsHardware-specific auditsService
Compliance auditsIndustry-specific AI risk evaluationsService
Compliance auditsCompliance auditsService
Compliance auditsCompliance auditsService

Data, Model, & System Evaluations​

Automated and procedural evaluations are critical for the integrity and effectiveness of AI-centric systems. These evaluations can be conducted holistically as end-to-end service offerings, or at strategic points in the AI lifecycle—including during development, for pre-deployment certifications, or as periodic checks in live applications. However, it is important to note that no comprehensive audit has yet been universally adopted. Many evaluations exist, but—contrary to some popular beliefs—there lacks a generally accepted taxonomy of which combinations of evaluations and audits constitute a complete measure of AI Trustworthiness. There are now a number of automated open-source tools as well as commercial software-as-a-service (SaaS) solutions designed to uncover and score AI models for hidden biases or under-representative datasets 5. By employing algorithmic bias detection and mitigation tools, evaluators can assist with the identification and rectification of subtle yet impactful trends that could negatively affect AI performance as well as company reputations 6. Bias in AI-driven hiring decisions, medical diagnoses, or financial advice, may be detected through statistical analysis techniques such as fairness indicators that visualise model performance across defined groups 7, or through tools that test AI outputs under hypothetical scenarios and visualise model behaviour 8. Some methods, such as relabelling 9, may help to improve datasets and ensure representativeness of disadvantaged or protected classes of people 10. It is however necessary to note that fairness in AI systems might be a partially intractable challenge. Because there are many complex sources of bias—some societal and some technical—it is not possible to fully “debias” a system. The goal instead is to mitigate bias or fairness-related harms as much as reasonably possible 11.

Continuous, real-time testing and periodic bespoke evaluations are crucial in the rapidly evolving landscape of AI breakthroughs, where new threats can emerge just as quickly as new technologies. Technological solutions in this domain include services designed to scrutinise behavioural patterns of AI systems through human-in-the-loop procedural assessments, uncovering what the AI can learn or has learned. An example is the scanning of prompts to identify methods for circumventing established boundaries through techniques like role-playing or reverse psychology 12. Further bolstering AI security within the models themselves, adversarial testing for robustness could also be deployed through automated software platforms. One advantage of automated software tools is that they are more discreet, allowing for ongoing improvement without disrupting system operations 13. Beyond that, there are more extensive and targeted red-team assessments that may require greater access provisions in exchange for bespoke robustness evaluations. It is important to note that the more standardised and scalable adversarial testing is, the more likely it is to miss failure modes that a more involved and tailor-made process might uncover 14. In either case, details of these evaluations could be recorded, providing full auditability for later reviews by compliance audits seeking to reasonably assure an AI system is safe, secure, robust, reliable, and so on.

Both the major AI-labs and downstream AI-adopting enterprises will increasingly demand interpretability tools to help determine whether models are understood enough and predictable enough to be deployed to end-users. These tools may include automated interpretability software that can equip AI developers with more efficient capacities to generate model-agnostic analyses and assess how changes in AI inputs affect their outputs; where more rigorous third-party evaluators may make use of ensembles of approaches that analyse the underlying model weights, subnetworks within the overall neural network, or how a model thinks when encountering conditions outside of its training data 15. These evaluators might also incorporate a range of inner interpretability methods such as "activation patching", which identifies the activations in a neural network that are most responsible for producing a particular output 16. The demand versus innovation gap when it comes to interpretability solutions for diagnostics, debugging, robustness, and benchmarking, is among one of the most significant gaps in the field of AI safety. Without these tests, organisations might fail to realise that their AI-driven systems are prone to inaccurate or biassed results—especially when put in unanticipated situations.

External Compliance Audits​

Businesses that audit AI models and applications will also see demand in relation to forthcoming compliance regimes. Soon regulators will enforce the policies discussed in "Risk Management in the Age of AI", requiring companies to demonstrate accurate, fair, adversarially robust, truthful and secure AI-driven systems. Model, and system -focused evaluators may have transferable service offerings in this regard; they can facilitate regulatory conformity through pre-deployment compliance audits of AI application tech stacks. Such services would aim to identify AI system vulnerabilities and assert when client AI systems meet jurisdiction-specific requirements 17.

Data-focused evaluators may also have transferable service offerings that can apply to data-focused compliance auditing. A variety of training data, inference data, and fine-tuning data audits can systemically review data processing pipelines for minimum standards. Such audits would verify the effectiveness of companies in managing data ethically and securely, encompassing an examination of data quality, the presence of bias, and the proper application of data anonymisation techniques 18. Data Protection Impact Assessments (DPIA) help organisations to proactively and systematically identify, assess, and mitigate the privacy risks associated with the lifecycle of sensitive data use 19. Importantly, implementing a DPIA is integral to achieving compliance with EU GDPR.

Moreover, future organisational governance procedures audits may focus more on a company's governance procedures. These are the people, processes, and technologies surrounding AI systems, which greatly influence the safety and security integrated in the design, deployment, and oversight of AI 20 Governance-related assessments are likely to include examinations of the mechanisms for organisational procedures, incentive structures, and management systems. For example, auditors may conduct conformity assessments to ensure that AI companies have reasonably effective reporting channels and whistleblowing protocols in place, in order to help advance goals related to contestability and redress. Similarly, auditors may review an organisation's hardware-related governance procedures through hardware-specific conformity audits, focused on the quality of organisational safeguards around AI chipsets, data centres, and confidential information. This may involve on-site inspections that rigorously test the hardware-enabled security of specialised components, or validating whether a company has adhered to protocols that limit employee access to model weights. Both of which may prevent sensitive information leaks.

Finally, it is also important to recognise that each sector faces distinct challenges concerning AI deployment, and thus, distinct standards and compliance requirements. A multifaceted AI assurance ecosystem will be needed to offer industry-specific AI risk evaluations for AI in healthcare, energy, finance, and other industry contexts that necessitate expert knowledge 21.

Who needs it?​

Prospective CustomerEst. WTP for AI EvalsEst. WTP for Compliance Audit
High-Tech ManufacturersHighRequired
Data Collectors & BrokersLowRequired
Cloud Compute ProvidersHighRequired
Frontier AI LabsHighRequired
AI Systems DevelopersHighRequired
Industry EnterprisesMediumRequired

The demand for AI Trustworthiness solutions is driven by the imperative to ensure AI systems are reliable, safe, and ethically sound. These assessments are crucial not just for ethical AI practices but also as a strategic business move to avoid scandals that could damage reputations and alienate users, thereby affecting marketability and profitability. Moreover, with tightening regulatory frameworks on AI transparency and fairness, investing in AI Trustworthiness solutions can mitigate legal risks and position companies as leaders in responsible AI. Understood as more than merely checkbox exercises, these evaluations serve as strategic investments, reducing potential litigation risks and insurance premiums while enhancing reputation and user trust. Such evaluations also facilitate smoother market entry into high-stakes sectors (e.g., critical infrastructure, healthcare, banking) for AI innovations by preemptively addressing potentially harmful behaviours.

For data and bias -focused auditing, the primary buyers may include Chief Information Officers or data management teams responsible for data quality across the enterprise. As it relates to AI model and system risk evaluations, it is likely that Safety, Risk, or Ethics teams along with their oversight boards will be involved in the appointment of external auditors. As AI labs and downstream developers increasingly recognise the business imperative of AI risk evaluations, in addition to forthcoming regulatory requirements, such teams and oversight boards have become a necessity.

As described in the "Risk Management in the Age of AI" section, compliance audits for both companies developing AI and enterprises adopting AI applications will soon be mandatory and enforced. Depending on the level of computational power, there will soon be stringent requirements to assure an AI system's safety, fairness, security, robustness, and explainability, as well as organisational transparency, accountability, and redress 22. This auditing process entails a rigorous review of AI models, their training data, and the processes and technologies involved in each AI-centric system implementation. Most companies will likely employ Chief Compliance Officers, legal counsels, and risk management teams akin to those of the Financial Services sector, to support them in navigating this compliance landscape.

What else do investors & founders need to know?​

As it relates to data-focused evaluations, despite increased standards and regulation, highlighted by the ISO TR 24027 and NYC Bias Auditing Law 23, AI auditors point to a lack of regulatory oversight and standards 24. For example, the case of O'Neil Risk Consulting & Algorithmic Auditing (ORCAA) and their unrepresentative evaluation of HireVue. This case emphasises the crucial role of methodology in AI auditing: ORCAA conducted a post-model bias assessment of HireVue, however, the models they assessed weren’t representative of the broader range of AI systems HireVue offers to clients 25. In order to establish accountability and ensure quality, there are proposals for data-focused auditors to perform mandatory registrations and disclosures of their methodologies 26. Such market governance would enhance transparency and reliability, allowing regulatory bodies to identify flaws in AI bias assessments performed by the third-party assurance ecosystem 27. Nonetheless, policymakers are recognizing the challenges of executing and scaling effective AI safety evaluations amidst market pressures 28. Without regulatory oversight and industry standards, AI labs or downstream developers will be pressured to choose the most efficient evaluators. Yet for some kinds of AI risks, this efficiency may come at the cost of diluting the quality of AI risk evaluations. Accordingly, market governance may soon involve multi-level accreditation mechanisms, potentially restricting high-compute AI model evaluations to AI Trustworthiness providers that have the pre-specified qualifications 29. Acknowledging the sensitive nature of data handled by AI risk evaluators (e.g., model weights), following strict data privacy protocols and maintaining confidentiality is paramount 30. Finally, besides averting the competitive costs and legal costs of breaching privacy expectations, evaluators with advanced accreditations for dangerous capability risk 31 evaluations must ensure the highest cybersecurity standards to mitigate the potentially irreversible proliferation of models with systemic risks 32.

In the historical record of financial assurance and compliance, auditors in pursuit of financial stability and growth have often prioritised client satisfaction and long-term business relations above all else 33. Such incentives may pose a potential risk for the scrutiny of AI compliance auditing, a situation where the consequentiality of mistakes could be disastrous 34. It is expected that AI policymakers will draw inspiration from the Sarbanes-Oxley Act of 2002 (i.e., Section 203, and Section 201), which mandates that audit teams rotate every five years and restricts non-auditing ancillary services to maintain auditor independence and objectivity 35. This regulatory framework would mean that public authorities define the desired outcomes of AI-driven companies, and outsource the process of procedural validation to licensed, independent private entities 36. Lastly, whistleblower protection policies—already in place in some jurisdictions—could play a crucial role in preventing undesirable practices. Until such policies take effect, voluntary anonymous reporting mechanisms can enhance public and client confidence in AI auditors 37 38.

CASE STUDY

Auditing the Frontier: Lessons from Financial Assurance Reform

  • Year: 2001
  • Companies: Arthur Anderson, Enron, WorldCom
  • Industry: Financial Assurance / Auditing

The Enron scandal, unfolding in 2001 with the bankruptcy of Enron Corporation, highlighted significant flaws in financial assurance and auditing practices. This corporate catastrophe, one of the largest in U.S. history, was marked by dubious accounting practices, including the use of Special Purpose Vehicles (SPVs) to obscure debt and inflate the company's financial health​​​​ 39.

The scandal not only led to the dissolution of Enron but also the downfall of Arthur Andersen, one of the world's leading auditing firms, thereby underlining the acute conflicts of interest and the lack of transparency within the financial sector​ 40​.

In response to the Enron debacle, the Sarbanes-Oxley Act (SOX) was enacted in 2002, ushering in stringent reforms aimed at bolstering corporate governance and financial reporting. SOX mandates critical measures such as internal controls for financial reporting, the establishment of independent audit committees, the creation of the Public Company Accounting Oversight Board, and the requirement for CEOs and CFOs to certify the accuracy of financial statements 41​​.

This legislation was a direct outcome of the Enron scandal, designed to restore investor confidence and ensure that third-party auditors maintain objectivity and independence from the companies they audit.

The implications of the Enron case and subsequent legislative reforms extend into contemporary concerns, such as the governance of third-party AI assurance—both compliance auditing and safety evaluations.

Just as SOX addressed conflicts of interest and mandated rigorous financial controls to prevent future frauds, similar governance frameworks will be necessary to uphold pivotal standards for the AI assurance ecosystem. This scandal is a lesson for founders that want to build future-proof companies, anticipating similar AI assurance market requirements from AI regulations and industry standards.

Sample Companies from the Venture Landscape:​

As at 15 April 2024: Of the 100 startups in our AI Assurance Technology landscape scan, we uncovered 36 offering AI Trustworthiness solutions. These companies included 30 seed/early stage startups, and 6 growth/late stage companies. Here's a brief sampling of those startups:

placeholder

placeholder
Prism Eval

placeholder

We systematically evaluate AI systems to understand their capabilities, limitations, and potential risks through rigorous testing protocols.

We are an AI Safety and Alignment company. We advance & leverage research in LLM Psycho-cognition to build a deep understanding of the cognition of AI systems. We then use it to evaluate SOTA AI systems and help robustify those systems at scale.

Armilla AI is proud to be leading the charge in AI/ML model auditing and risk assessments. Their cutting-edge technology and automated testing techniques ensure that your models are reliable and secure to inform the most accurate underwriting process for your AI warranty.

placeholder

placeholder

placeholder

Advai provides a suite of tools to test, evaluate and help you trust AI systems, based on next-gen research.

BABL AI Audits and Certifies AI Systems. We offer our clients a streamlined and flexible audit process that provides trust and assurance at a fraction of the time and cost.

Establish trust in your AI systems. Conduct audits of your AI systems to showcase the trustworthiness of your technology.

Note

The above companies may also offer products and services that fit in one of the other three solution domains. All relevant domain classifications and the full list of companies surfaced through our landscape scan can be reviewed in the Appendix: "AIAT Landscape Logo Map"

Footnotes​

  1. Red-teaming (def.): Using manual or automated methods to adversarially probe a language model for harmful outputs. "Red-teaming" is used synonymously to "adversarial testing" in this report (as is common in the existing literature). ↩

  2. Fine-tuning (def.): Additional AI model training where the model learns from a smaller, bespoke dataset. This process harnesses the model's fundamental strengths, but also recalibrates it for specific applications. [Bergmann, Dave] ↩

  3. Note: The terms (mechanistic) interpretability and explainability are used inconsistently by AI researchers and commentators: The terms are sometimes used interchangeably, both describing the extent to which humans are able to understand how and why an AI system makes decisions in a certain way. Other authors have proposed definitions that distinguish between the two; a prominent example comes from the US National Institute of Standards and Technology (NIST): "Explainability refers to a representation of the mechanisms underlying AI systems’ operation, whereas interpretability refers to the meaning of AI systems’ output in the context of their designed functional purposes. Together, explainability and interpretability assist those operating or overseeing an AI system, as well as users of an AI system, to gain deeper insights into the functionality and trustworthiness of the system, including its outputs." [NIST AI RMF 2023] ↩

  4. Formal Verification (def.): Testing to assert whether an AI model or system satisfies pre-specified criteria, often through automated methods or systematic mathematical validations. [ScienceDirect] ↩

  5. Credo AI. "Gen AI Ops Landscape." Accessed April 9, 2024. https://www.credo.ai/gen-ai-ops-landscape; and Williams, Ian. "LLM Tools." ianww.com. Accessed April 9, 2024. https://ianww.com/llm-tools#open-source ↩

  6. Roselli, Drew, Jeanna Matthews, and Nisha Talagala. "Managing bias in AI." In Companion proceedings of the 2019 world wide web conference, pp. 539-544. 2019. https://doi.org/10.1145/3308560.3317590 ↩

  7. Tensorflow. “Fairness-indicators.” Last modified 2021. https://www.tensorflow.org/tensorboard/fairness-indicators. Accessed April 10, 2024. ↩

  8. Tensorflow. "what_if_tool." Model Understanding with the What-If Tool Dashboard. Last modified 2023. https://www.tensorflow.org/tensorboard/what_if_tool . Accessed March 12, 2024. ↩

  9. Data labelling (def.): The process of annotating text, images, and other data types with machine-interpretable information (or metadata), so that AI models can be trained or fine-tuned using that data. ↩

  10. Bantilan, Niels. "Themis-ml: A fairness-aware machine learning interface for end-to-end discrimination discovery and mitigation." Journal of Technology in Human Services 36, no. 1 (2018): 15-30. https://doi.org/10.1080/15228835.2017.1416512 ↩

  11. Lewicki, Kornel, Michelle Seng Ah Lee, Jennifer Cobbe, and Jatinder Singh. "Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service.” CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. April 23-28, 2023, Hamburg, Germany. Accessed April 10, 2024. https://doi.org/10.1145/3544548.3581463 ↩

  12. CalypsoAI. "CalypsoAI Launches New Solution to Secure Companies Using ChatGPT and Other Large Language Models." Last modified 2023. https://moderator.calypsoai.com/calypsoai-moderator-launch. Accessed March 12, 2024. ↩

  13. Robust Intelligence. "Automated red-teaming to satisfy AI risk standards and regulatory compliance." Last modified 2023. https://www.robustintelligence.com/platform/continuous-validation. Accessed March 12, 2024. ↩

  14. SaferAI. “Distinguishing Audits, Evals, and Red-Teaming” Last modified 2024. https://www.safer-ai.org/post/distinguishing-audits-evals-and-red-teaming. Accessed April 11, 2024. ↩

  15. Räuker, Tilman, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell. "Toward transparent ai: A survey on interpreting the inner structures of deep neural networks." In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 464-483. IEEE, 2023. https://api.semanticscholar.org/CorpusID:251104722. ↩

  16. Meng, Kevin, David Bau, Alex Andonian, and Yonatan Belinkov. "Locating and editing factual associations in GPT." Advances in Neural Information Processing Systems 35 (2022): 17359-17372. https://proceedings.neurips.cc/paper_files/paper/2022/file/6f1d43d5a82a37e89b0665b33bf3a182-Paper-Conference.pdf ↩

  17. LaBrie, Ryan C., and Gerhard Steinke. "Towards a framework for ethical audits of AI algorithms." (2019). https://web.archive.org/web/20200318085901id_/https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1398&context=amcis2019; and Dekra. “DEKRA and LatticeFlow Launch AI Safety Assessment Services.” Last modified 2023. https://www.dekra.com/en/dekra-and-latticeflow-launch-ai-safety-assessment-services/. Accessed March 15, 2024. ↩

  18. Accenture. “AI ethics & governance.” Last modified 2023. https://www.accenture.com/us-en/services/applied-intelligence/ai-ethics-governance. Accessed March 15, 2024. ↩

  19. Henriksen-Bulmer, Jane, Shamal Faily, and Sheridan Jeary. "Dpia in context: applying dpia to assess privacy risks of cyber physical systems." Future internet 12, no. 5 (2020): 93. https://doi.org/10.3390/fi12050093 ↩

  20. Brundage, Miles, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf et al. "Toward trustworthy AI development: mechanisms for supporting verifiable claims." arXiv preprint arXiv:2004.07213 (2020). https://doi.org/10.48550/arXiv.2004.07213 ↩

  21. Ashmore, Rob, Radu Calinescu, and Colin Paterson. "Assuring the machine learning lifecycle: Desiderata, methods, and challenges." ACM Computing Surveys (CSUR) 54, no. 5 (2021): 1-39. https://doi.org/10.1145/3453444 ↩

  22. Werner, Jeremy. “What to Consider when Navigating Global AI Compliance” Last modified 2024. https://babl.ai/what-to-consider-when-navigating-global-ai-compliance/. Accessed March 20, 2024.; and NIST. “AI Risk Management Framework.” Voluntary Guidance. National Institute of Standards and Technology, U.S. Department of Commerce, January 26, 2023. https://www.nist.gov/itl/ai-risk-management-framework. ↩

  23. New York City Bias Auditing Law (def.): The New York City Bias Auditing Law, passed in January 2022, mandates algorithmic bias audits of municipal agencies' automated decision-making systems to ensure fairness and accountability. [NYC local law 144-21 ↩

  24. Costanza-Chock, Sasha, Inioluwa Deborah Raji, and Joy Buolamwini, "Who Audits the Auditors?," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea: ACM, 2022), https://doi.org/10.1145/3531146.3533213. ↩

  25. Edwin Farley, "AI Auditing: First Steps Towards the Effective Regulation of Artificial Intelligence Systems" (SSRN Scholarly Paper, December 26, 2023), https://ssrn.com/abstract=4676184. ↩

  26. Inioluwa Raji et al., "Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea: ACM, 2022), 557-571, https://doi.org/10.1145/3514094.3534181. ↩

  27. Costanza-Chock, Sasha, Inioluwa Deborah Raji, and Joy Buolamwini, "Who Audits the Auditors?," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea: ACM, 2022), https://doi.org/10.1145/3531146.3533213. ↩

  28. Benjamin Faveri and Graeme Auld, "Informing Possible Futures for the Use of Third-Party Audits in AI Regulations" (Carleton University, School of Public Policy and Administration, 2023), https://doi.org/10.22215/sppa-rgi-nov2023. ↩

  29. Ross Gruetzemacher et al., "An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI" (arXiv, January 15, 2023), https://doi.org/10.48550/arXiv.2310.14455. ↩

  30. Brundage, Miles, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf et al. "Toward trustworthy AI development: mechanisms for supporting verifiable claims." arXiv preprint arXiv:2004.07213 (2020). https://doi.org/10.48550/arXiv.2004.07213 ↩

  31. Dangerous capability risks (def.): The potential for AI technologies to directly or indirectly cause extreme harms by way of its capacities for deception, persuasion & manipulation, weapons design, cyber attacks, long-horizon planning, self-proliferation, and more. [Shevlane, Toby et al] ↩

  32. Sella Nevo et al., "Securing Artificial Intelligence Model Weights: Interim Report" (RAND Corporation, 2023), https://www.rand.org/pubs/working_papers/WRA2849-1.html. ↩

  33. Elisa S. Moncarz et al., "The Rise and Collapse of Enron: Financial Innovation, Errors and Lessons," Contaduría y Administración, no. 218 (January-April 2006): 17-37. ↩

  34. Edwin Farley, "AI Auditing: First Steps Towards the Effective Regulation of Artificial Intelligence Systems" (SSRN Scholarly Paper, December 26, 2023), https://ssrn.com/abstract=4676184. ↩

  35. Sarbanes-Oxley Act of 2002, Pub. L. No. 107-204, 116 Stat. 745 (codified as amended in scattered sections of 11, 15, 18, 28, and 29 U.S.C.), https://www.congress.gov/107/plaws/publ204/PLAW-107publ204.pdf.; and Inioluwa Raji et al., "Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance," in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea: ACM, 2022), 557-571, https://doi.org/10.1145/3514094.3534181. ↩

  36. Gillian K. Hadfield and Jack Clark, "Regulatory Markets: The Future of AI Governance" (arXiv, April 10, 2023), http://arxiv.org/abs/2304.04914. ↩

  37. Gregor Thüsing and Gerrit Forst, eds., Whistleblowing - A Comparative Study, Ius Comparatum - Global Studies in Comparative Law (Springer Cham, 2016) ↩

  38. Alexis Ronickher and Matthew LaGarde, "Despite Regulation Lag, AI Whistleblowers Have Protections," 2023. ↩

  39. Bondarenko, Peter. "Enron scandal." Encyclopedia Britannica. Accessed April 2, 2024. https://www.britannica.com/event/Enron-scandal; and SOX Law. "Unveiling the Enron Scandal: Deception, Fallout, and Lessons Learned." Accessed April 1, 2024. https://www.soxlaw.com/unveiling-the-enron-scandal-deception-fallout-and-lessons-learned/ ↩

  40. Bondarenko, Peter. "Enron scandal." Encyclopedia Britannica. Accessed April 2, 2024. https://www.britannica.com/event/Enron-scandal ↩

  41. International Accounting Standards Board Plus. "PCAOB." Accessed April 1, 2024. https://www.iasplus.com/en/resources/regional/pcaob; and SOX Law. "Unveiling the Enron Scandal: Deception, Fallout, and Lessons Learned." Accessed April 1, 2024. https://www.soxlaw.com/unveiling-the-enron-scandal-deception-fallout-and-lessons-learned/ ↩