Research Papers

With increasing integration of Large Language Models (LLMs) into areas of high-stakes human decision-making, it is important to understand the risks they introduce as advisors. To be useful advisors, LLMs must sift through large amounts of content, written with both benevolent and malicious intent, and then use this information to convince a user to take a specific action. This involves two social capacities: vigilance (the ability to determine which information to use, and which to discard) and persuasion (synthesizing the available evidence to make a convincing argument). While existing work has investigated these capacities in isolation, there has been little prior investigation of how these capacities may be linked. Here, we use a simple multi-turn puzzle-solving game, Sokoban, to study LLMs' abilities to persuade and be rationally vigilant towards other LLM agents. We find that puzzle-solving performance, persuasive capability, and vigilance are dissociable capacities in LLMs. Performing well on the game does not automatically mean a model can detect when it is being misled, even if the possibility of deception is explicitly mentioned. However, LLMs do consistently modulate their token use, using fewer tokens to reason when advice is benevolent and more when it is malicious, even if they are still persuaded to take actions leading them to failure. To our knowledge, our work presents the first investigation of the relationship between persuasion, vigilance, and task performance in LLMs, and suggests that monitoring all three independently will be critical for future work in AI safety.

AI systems comprise a range of interactions across the technical and organisational components of a range of actors. These components work together to provide the systems' functionality. This socio-technical assemblage is increasingly described as an algorithmic supply chain. Given their role in supporting a wide range of systems, foundation models (FMs) are increasingly a key part of many algorithmic supply chains. In practice, various technical and non-technical components work to mediate, adapt, and augment the behaviour of models, such as FMs, both in general, and for their use in specific application contexts. However, many AI safety evaluations tend to focus on capabilities of FMs themselves and/or assess these components independently, at certain levels of abstraction, with less consideration on how these components could interact, influence, reinforce or counteract each other. In this paper, we introduce the cascade as a concept for supporting more holistic AI evaluations. The cascade captures how the interactions between socio-technical components along the AI supply chain can compound and produce cumulative effects with downstream consequences. Specifically, we (i) identify gaps in current AI auditing approaches, using LLMs as our case study; (ii) demonstrate how cascade problems manifest in deployed AI systems through a characterisation of cascades through different viewpoints (component and stakeholder), and (iii) propose research directions for assessing the cascade specifically as an object of analysis. As these cascades can significantly impact transparency, accountability, security, and safety, we advocate for a paradigm shift in AI system auditing towards systems-oriented audits that incorporate cascading effects, to complement model centric evaluations.

The emergence of autonomous, high-velocity Agentic AI systems is creating an internal assurance scalability crisis. Point-in-time, document-based audits cannot keep pace with non deterministic behaviour and distributed deployments of agents across rapidly evolving environments. The crisis is dual-scale: vertically, governance and control obligations change faster than frameworks can operationalise them; horizontally, assurance mechanisms fail to scale across complex, heterogeneous systems and evidence sources. Risk-based regulation now requires organisations to demonstrate ongoing control adequacy and effectiveness, yet existing Trustworthy AI Assurance and Audit frameworks remain fragmented and largely manual. Drawing on the evolution of cybersecurity posture management, this paper reframes trustworthiness as a continuously generated signal rather than a static certificate. It contributes 1) A Trustworthy AI Assurance Ontology modelling the end-to-end pathway from regulatory obligation to verifiable evidence 2) An ontology-driven, evidence-gated benchmark of thirteen leading frameworks, revealing a posture readiness gap 3) The Trustworthy AI Posture (TAIP) framework, which operationalises the NIST AI RMF Test,Evaluate,Verify,Validate (TEVV) cycle as reusable AI Assurance Objects. TAIP decouples policy content ('what') from execution semantics ('how'), enabling composable, automatable assurance across jurisdictions and agentic systems. Evidence from heterogeneous tools is normalised and recursively aggregated into posture at claim, system, organisational, and ecosystem levels. A use case mapping Australian AI Guardrails to Microsoft 365 Copilot demonstrates claim decomposition, evidence binding, and posture computation in practice. By standardising execution while allowing policy variation, TAIP enables scalable, machine-speed trust signal generation.

Binary neural networks (BNNs) are increasingly deployed in edge computing applications due to their low hardware complexity and high energy efficiency. However, verifying the robustness of BNNs against input perturbations, including adversarial attacks, remains computationally challenging because the underlying decision problem is inherently combinatorial. In this paper, we propose an Ising- and quantum-inspired framework for BNN robustness verification. We show that, for a broad class of BNN architectures, robustness verification can be formulated as a Quadratic Constrained Boolean Optimization (QCBO) problem and subsequently transformed into a Quadratic Unconstrained Boolean Optimization (QUBO) instance amenable to Ising and quantum-inspired solvers. We demonstrate the feasibility of this formulation on binarized MNIST by solving the resulting QUBOs with a free energy machine (FEM) solver and simulated annealing. We also show the deployment of this framework on quantum annealing and digital annealing platforms. Our results highlight the potential of quantum-inspired computing and Ising computing as a pathway toward trustworthy AI systems.

Anthropomorphic design is routinely used to make conversational agents more approachable and engaging. Yet its influence on users' perceptions remains poorly understood. Drawing on psychological theories, we propose that anthropomorphism influences risk perception via two complementary forms of trust, and that domain knowledge moderates these relationships. To test our model, we conducted a large-scale online experiment (N = 1,256) on a financial decision-support system implementing different anthropomorphic designs. We found that anthropomorphism indirectly reduces risk perception by increasing both cognitive and affective trust. Domain knowledge moderates these paths: participants with low financial knowledge experience a negative indirect effect of perceived anthropomorphism on risk perception via cognitive trust, whereas those with high financial knowledge exhibit a positive direct and indirect effect. We discuss theoretical contributions to human-AI interaction and design implications for calibrating trust in anthropomorphic decision-support systems for responsible AI.

Chain-of-thought (CoT) reasoning is fundamental to modern LLM architectures and represents a critical intervention point for AI safety. However, CoT reasoning may exhibit failure modes that we note as pathologies, which prevent it from being useful for monitoring. Prior work has identified three distinct pathologies: post-hoc rationalization, where models generate plausible explanations backwards from predetermined answers; encoded reasoning, where intermediate steps conceal information within seemingly interpretable text; and internalized reasoning, where models replace explicit reasoning with meaningless filler tokens while computing internally. To better understand and discriminate between these pathologies, we create a set of concrete metrics that are simple to implement, computationally inexpensive, and task-agnostic. To validate our approach, we develop model organisms deliberately trained to exhibit specific CoT pathologies. Our work provides a practical toolkit for assessing CoT pathologies, with direct implications for training-time monitoring.

We make three contributions in this paper. First, we present an approach for compiling a random forest classifier into a set of circuits, where each circuit directly encodes the instances in some class of the classifier. We show empirically that our proposed approach is significantly more efficient than existing similar approaches. Next, we utilize this approach to further obtain circuits that are tractable for computing the complete and general reasons of a decision, which are instance abstractions that play a fundamental role in computing explanations. Finally, we propose algorithms for computing the robustness of a decision and all shortest ways to flip it. We illustrate the utility of our contributions by using them to enumerate all sufficient reasons, necessary reasons and contrastive explanations of decisions; to compute the robustness of decisions; and to identify all shortest ways to flip the decisions made by random forest classifiers learned from a wide range of datasets.

Debate is efficient with your time

Jonah Brown-Cohen, Geoffrey Irving, Simon C. Marshall +3

AI safety via debate uses two competing models to help a human judge verify complex computational tasks. Previous work has established what problems debate can solve in principle, but has not analysed the practical cost of human oversight: how many queries must the judge make to the debate transcript? We introduce Debate Query Complexity}(DQC), the minimum number of bits a verifier must inspect to correctly decide a debate. Surprisingly, we find that PSPACE/poly (the class of problems which debate can efficiently decide) is precisely the class of functions decidable with O(log n) queries. This characterisation shows that debate is remarkably query-efficient: even for highly complex problems, logarithmic oversight suffices. We also establish that functions depending on all their input bits require Omega(log n) queries, and that any function computable by a circuit of size s satisfies DQC(f) <= log(s) + 3. Interestingly, this last result implies that proving DQC lower bounds of log(n) + 6 for languages in P would yield new circuit lower bounds, connecting debate query complexity to central questions in circuit complexity.

This study investigates students' perceptions of Artificial Intelligence (AI) grading systems in an undergraduate computer science course (n = 27), focusing on a block-based programming final project. Guided by the ethical principles framework articulated by Jobin (2019), our study examines fairness, trust, consistency, and transparency in AI grading by comparing AI-generated feedback with original human-graded feedback. Findings reveal concerns about AI's lack of contextual understanding and personalization. We recommend that equitable and trustworthy AI systems reflect human judgment, flexibility, and empathy, serving as supplementary tools under human oversight. This work contributes to ethics-centered assessment practices by amplifying student voices and offering design principles for humanizing AI in designed learning environments.

Trade-offs in Financial AI: Explainability in a Trilemma with Accuracy and Compliance

Patricia Marcella Evite, Ekaterina Svetlova, Doina Bucur

As Artificial Intelligence (AI) becomes increasingly embedded in financial decision-making, the opacity of complex models presents significant challenges for professionals and regulators. While the field of Explainable AI (XAI) attempts to bridge this gap, current research often reduces the implementation challenge to a binary trade-off between model accuracy and explainability. This paper argues that such a view is insufficient for the financial domain, where algorithmic choices must navigate a complex sociotechnical web of strict regulatory bounds, budget constraints, and latency requirements. Through semi-structured interviews with twenty finance professionals, ranging from C-suite executives and developers to regulators across multiple regions, this study empirically investigates how practitioners prioritize explainability relative to four competing factors: accuracy, compliance, cost, and speed. Our findings reveal that these priorities are structured not as a simple trade-off, but as a system of distinct prerequisites and constraints. Accuracy and compliance emerge as non-negotiable "hygiene factors": without them, an AI system is viewed as a liability regardless of its transparency. Operational levers (speed and cost) serve as secondary constraints that determine practical feasibility, while ease of understanding functions as a gateway to adoption, shaping whether AI tools are trusted, used, and defensible in practice.

This paper presents the outputs of the exploratory phase of a global audit of Large Language Models (LLMs) project. In this exploratory phase, I used the Global AI Dataset (GAID) Project as a framework to stress-test the Llama-3 8B model and evaluate geographic and socioeconomic biases in technical AI governance awareness. By stress-testing the model with 1,704 queries across 213 countries and eight technical metrics, I identified a significant digital barrier and gap separating the Global North and South. The results indicate that the model was only able to provide number/fact responses in 11.4% of its query answers, where the empirical validity of such responses was yet to be verified. The findings reveal that AI's technical knowledge is heavily concentrated in higher-income regions, while lower-income countries from the Global South are subject to disproportionate systemic information gaps. This disparity between the Global North and South poses concerning risks for global AI safety and inclusive governance, as policymakers in underserved regions may lack reliable data-driven insights or be misled by hallucinated facts. This paper concludes that current AI alignment and training processes reinforce existing geoeconomic and geopolitical asymmetries, and urges the need for more inclusive data representation to ensure AI serves as a truly global resource.

Accurate currency recognition is essential for assistive technologies, particularly for visually impaired individuals who rely on others to identify banknotes. This dependency puts them at risk of fraud and exploitation. To address these challenges, we first build a new Bangladeshi banknote dataset that includes both controlled and real-world scenarios, ensuring a more comprehensive and diverse representation. Next, to enhance the dataset's robustness, we incorporate four additional datasets, including public benchmarks, to cover various complexities and improve the model's generalization. To overcome the limitations of current recognition models, we propose a novel hybrid CNN architecture that combines MobileNetV3-Large and EfficientNetB0 for efficient feature extraction. This is followed by an effective multilayer perceptron (MLP) classifier to improve performance while keeping computational costs low, making the system suitable for resource-constrained devices. The experimental results show that the proposed model achieves 97.95% accuracy on controlled datasets, 92.84% on complex backgrounds, and 94.98% accuracy when combining all datasets. The model's performance is thoroughly evaluated using five-fold cross-validation and seven metrics: accuracy, precision, recall, F1-score, Cohen's Kappa, MCC, and AUC. Additionally, explainable AI methods like LIME and SHAP are incorporated to enhance transparency and interpretability.