Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing
Risk Insights: Yusuf Moolla
27 episodes
7 months ago
Spoken by a human version of this article. TL;DR (TL;DL?) Testing is a core basic step for algorithmic integrity.Testing involves various stages, from developer self-checks to UAT. Where these happen will depend on whether the system is built in-house or bought.Testing needs to cover several integrity aspects, including accuracy, fairness, security, privacy, and performance.Continuous testing is needed for AI systems, differing from traditional testing due to the way these newer systems chang...
All content for Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing is the property of Risk Insights: Yusuf Moolla and is served directly from their servers
with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.
Spoken by a human version of this article. TL;DR (TL;DL?) Testing is a core basic step for algorithmic integrity.Testing involves various stages, from developer self-checks to UAT. Where these happen will depend on whether the system is built in-house or bought.Testing needs to cover several integrity aspects, including accuracy, fairness, security, privacy, and performance.Continuous testing is needed for AI systems, differing from traditional testing due to the way these newer systems chang...
Spoken by a human version of this article. TL;DR (TL;DL?) Testing is a core basic step for algorithmic integrity.Testing involves various stages, from developer self-checks to UAT. Where these happen will depend on whether the system is built in-house or bought.Testing needs to cover several integrity aspects, including accuracy, fairness, security, privacy, and performance.Continuous testing is needed for AI systems, differing from traditional testing due to the way these newer systems chang...
Spoken by a human version of this article. One question that comes up often is “How do we obtain assurance about third party products or services?” Depending on the nature of the relationship, and what you need assurance for, this can vary widely. This article attempts to lay out the options, considerations, and key steps to take. TL;DR (TL;DL?) Third-party assurance for algorithm integrity varies based on the nature of the relationship and specific needs, with several options.Key factors to ...
Navigating AI Audits with Dr. Shea Brown Dr. Shea Brown is Founder and CEO of BABL AI BABL specializes in auditing and certifying AI systems, consulting on responsible AI practices, and offering online education. Shea shares his journey from astrophysics to AI auditing, the core services provided by BABL AI including compliance audits, technical testing, and risk assessments, and the importance of governance in AI. He also addresses the challenges posed by generative AI, the need for con...
Spoken by a human version of this article. AI literacy is growing in importance (e.g., EU AI Act, IAIS). AI literacy needs vary across roles. Even "AI professionals" need AI Risk training. Links EU AI Act: The European Union Artificial Intelligence Act - specific expectation about “AI literacy”.IAIS: The International Association of Insurance Supervisors is developing a guidance paper on the supervision of AI.About this podcast A podcast for Financial Services leaders, where we discuss fai...
Navigating AI Governance and Compliance Patrick Sullivan is Vice President of Strategy and Innovation at A-LIGN and an expert in cybersecurity and AI compliance with over 25 years of experience. Patrick shares his career journey, discusses his passion for educating executives and directors on effective governance, and explains the critical role of management systems like ISO 42001 in AI compliance. We discuss the complexities of AI governance, risk assessment, and the importance of clear ...
Mitigating AI Risks Ryan Carrier is founder and executive director of ForHumanity, a non-profit focused on mitigating the risks associated with AI, autonomous, and algorithmic systems. With 25 years of experience in financial services, Ryan discusses ForHumanity's mission to analyze and mitigate the downside risks of AI to benefit society. The conversation includes insights on the foundation of ForHumanity, the role of independent AI audits, educational programs offered by the ForH...
Spoken (by a human) version of this article. Public AI audit reports aren't universally required; they mainly apply to high-risk applications and/or specific jurisdictions.The push for transparency primarily concerns independent audits, not internal reviews.Prepare by implementing ethical AI practices and conducting regular reviews.Note: High-risk AI systems in banking and insurance are subject to specific requirements Links AI and algorithm audit guidelines vary widely and are not universal...
Spoken by a human version of this article. Knowing the basics of substantive testing vs. controls testing can help you determine if the review will meet your needs.Substantive testing directly identifies errors or unfairness, while controls testing evaluates governance effectiveness. The results/conclusions are different.Understanding these differences can also help you anticipate the extent of your team's involvement during the review process. Links This article details a (largely) s...
Spoken by a human version of this article. Ongoing education helps everyone understand their role in responsibly developing and using algorithmic systems. Regulators and standard-setting bodies emphasise the need for AI literacy across all organisational levels. Links ForHumanity - join the growing community here. ForHumanity - free courses here.IAIS: The International Association of Insurance Supervisors is developing a guidance paper on the supervision of AI.DNB: De Nederlandsche Ba...
Spoken by a human version of this article. The terminology – “audit” vs “review” - is important, but clarity about deliverables is more important when commissioning algorithm integrity assessments. Audits are formal, with an opinion or conclusion that can often be shared externally. Reviews come in various forms and typically produce recommendations, for internal use. Regardless of the terminology you use, when commissioning an assessment, clearly define and document the expected deliverable...
Spoken (by a human) version of this article. Outcome-focused accuracy reviews directly verify results, offering more robust assurance than process-focused methods.This approach can catch translation errors, unintended consequences, and edge cases that process reviews might miss.While more time-consuming and complex, outcome-focused reviews provide deeper insights into system reliability and accuracy.This article explains why verifying outcomes is preferred over tracing through processes, an...
Spoken (by a human) version of this article. Documentation makes it easier to consistently maintain algorithm integrity. This is well known. But there are lots of types of documents to prepare, and often the first hurdle is just thinking about where to start. So this simple guide is meant to help do exactly that – get going. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla. P...
Spoken (by a human) version of this article. Banks and insurers are increasingly using external data; using them beyond their intended purpose can be risky (e.g. discriminatory). Emerging regulations and regulatory guidance emphasise the need for active oversight by boards, senior management to ensure responsible use of external data. Keeping the customer top of mind, asking the right questions, and focusing on the intended purpose of the data, can help reduce the risk. Law and guideline men...
Spoken (by a human) version of this article. Banks and insurers sometimes lose sight of their customer-centric purpose when assessing AI/algorithm risks, focusing instead on regular business risks and regulatory concerns. Regulators are noticing this disconnect. This article aims to outline why the disconnect happens and how we can fix it. Report mentioned in the article: ASIC, REP 798 Beware the gap: Governance arrangements in the face of AI innovation. About this podcast A podcast for Fi...
Spoken (by a human) version of this article. With algorithmic systems, an change can trigger a cascade of unintended consequences, potentially compromising fairness, accountability, and public trust. So, managing changes is important. But if you use the wrong framework, your change control process may tick the boxes, but be both ineffective and inefficient. This article outlines a potential solution: a risk focused, principles-based approach to change control for algorithmic systems. Resourc...
Spoken (by a human) version of this article. The integrity of algorithmic systems goes beyond accuracy and fairness. In Episode 4, we outlined 10 key aspects of algorithm integrity. Number 5 in that list (not in order of importance) is Security: the algorithmic system needs to be protected from unauthorised access, manipulation and exploitation. In this episode, we explore one important sub-component of this: deprovisioning user access. Link from article: U.S. National Coordinator for Criti...
Spoken (by a human) version of this article. When we're checking for fairness in our algorithmic systems (incl. processes, models, rules), we often ask: What are the personal characteristics or attributes that, if used, could lead to discrimination? This article provides a basic framework for identifying and categorising these attributes. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yu...
Spoken (by a human) version of this article. Legislation isn't the silver bullet for algorithmic integrity. Are they useful? Sure. They help provide clarity and can reduce ambiguity. And once a law is passed, we must comply. However: existing legislation may already applynew algorithm-focused laws can be too narrow or quickly outdatedstandards can be confusing, and may not cover what we need"best practice" frameworks help, but they're not always the best (and there are several,...
Spoken (by a human) version of this article. Even in discussions among AI governance professionals, there seems to be a silent “gen” before AI. With rapid progress - or rather prominence – of generative AI capabilities, these have taken centre stage. Amidst this excitement, we mustn't lose sight of the established algorithms and data-enabled workflows driving core business decisions. These range from simple rules-based systems to complex machine learning models, each playing a crucial r...
Spoken (by a human) version of this article. In a previous article, we discussed algorithmic fairness, and how seemingly neutral data points can become proxies for protected attributes. In this article, we'll explore a concrete example of a proxy used in insurance and banking algorithms: postcodes. We've used Australian terminology and data. But the concept will apply to most countries. Using Australian Bureau of Statistics (ABS) Census data, it aims to demonstrate how postcodes can ...
Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing
Spoken by a human version of this article. TL;DR (TL;DL?) Testing is a core basic step for algorithmic integrity.Testing involves various stages, from developer self-checks to UAT. Where these happen will depend on whether the system is built in-house or bought.Testing needs to cover several integrity aspects, including accuracy, fairness, security, privacy, and performance.Continuous testing is needed for AI systems, differing from traditional testing due to the way these newer systems chang...
