So, why are autonomous weapons systems such a big deal? Aren't they just weapons like the rest of them? Well, the black box problem with algorithmically controlled systems raises challenges different from those of "fire and forget" munitions.
Four AI experts explain.
The first part of this episode clarifies what an algorithm is when the black box appears and why it's important.
The second part clarifies how the data the algorithms rely on can be biased and that constant maintenance and updates do not fix the problem.
Shownotes:
Producer and host: Sune With sunewith@cas.au.dk
Cover art: Sebastian Gram
References and literature:
- Algorithmic bias, Wikipedia (Accessed April 8. 2025)
https://en.wikipedia.org/wiki/Algorithmic_bias
- Black Box, Wikipedia (Accessed April 8. 2025)
https://en.wikipedia.org/wiki/Black_box
- Blouin, Lou; Rawashdeh, Samir, March 2023, “AI´s mysterious “black box” problem. Explained”, NEWS University of Michigan-Dearborn (Accessed April 8. 2025)
https://umdearborn.edu/news/ais-mysterious-black-box-problem-explained
- Co-Coders (Accessed April 8. 2025)
- ExekTek (Accessed April 8. 2025)
- Hatherley, J. J., 2020, ”Limits of Trust in Medical AI. ” Journal of Medical Ethics, 46(7), 478-481.
- Hatherley, J., Sparrow, R., & Howard, M. (2024).”The Virtues of Interpretable Medical AI. ” Cambridge Quarterly of Healthcare Ethics, 33(3), 323-332.
- Hatherley, J. (2025).”A Moving Target in AI-Assisted Decision-Making: Dataset Shift, Model Updating, and the Problem of Update Opacity. ” Ethics and Information Technology, 27, 20.
https://link.springer.com/article/10.1007/s10676-025-09829-2 - citeas
- Hyperight, “The Black Box: What We´re Still Getting Wrong about Trusting Machine Learning Models” (Accessed April 8. 2025)
- Lystlund, Lise Bach (Accessed April 8. 2025)
- Nygreen, Jonas (Accessed April 8. 2025)
https://www.linkedin.com/in/jonasnygreen/
Music: Sofus Forsberg
