ExplainabilityTrustworthinessSafetycs.LG
Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc
Doug Lenat, Gary Marcus
July 31, 2023
arXiv.org
41 citations
4 influential
Key Takeaways
-
Current LLMs prioritize plausibility over accuracy: The paper notes that large language models are "trained to produce outputs that are plausible, but not necessarily correct," meaning they can sound convincing while being factually wrong—a critical limitation for trustworthy AI systems.
-
Transparency and auditability are essential for trust: The paper argues that trustworthy AI should provide "full step-by-step line of reasoning" with documented "provenance of the knowledge used" that can be audited, contrasting sharply with current LLMs which are described as "unpredictable and uninterpretable."
-
Explicit knowledge and rules can improve reasoning: An alternative approach using "curated pieces of explicit knowledge and rules of thumb" with inference engines could theoretically address LLM limitations by enabling logical deduction rather than statistical pattern matching.
-
Speed versus expressiveness presents a real tradeoff: The paper identifies a fundamental challenge: highly expressive logical systems that can represent complex reasoning run too slowly, while faster systems sacrifice the ability to represent nuanced meaning—suggesting hybrid approaches may be necessary.
-
Hybrid AI systems may be the path forward: The abstract concludes that "any trustworthy general AI will need to hybridize" different approaches, suggesting that combining generative AI's strengths with symbolic AI's interpretability and reasoning capabilities could address current trustworthiness gaps.