Knowledge Prompting: How Knowledge Engineers Use Large Language Models

Prompting skills are critical but undervalued: The study found that the ability to effectively prompt LLMs is an essential skill for knowledge engineers, yet this capability is not widely recognized or trained in the field, suggesting a gap between current practices and emerging needs.

LLMs improve efficiency but complicate quality assurance: While participants believed LLMs could enhance productivity in knowledge graph engineering, the technology simultaneously intensified existing challenges around evaluating and validating the quality of knowledge engineering work—a concern that responsible AI practitioners should address.

Knowledge engineers lack ethical training for AI integration: The research revealed that most knowledge engineers receive limited ethical training, creating a vulnerability when deploying LLMs in their work; the authors propose "KG cards" (modeled after data cards) as a practical framework to guide responsible knowledge graph construction.

Responsible AI requires deliberate attention to risks and harms: The study emphasizes that integrating LLMs into knowledge engineering tasks must be "mindful of potential risks and harms," indicating that responsible AI practices cannot be an afterthought but must be built into workflows from the start.

Natural language processing expertise is becoming more broadly necessary: The findings suggest that NLP skills, traditionally specialized knowledge, may become relevant across more roles in knowledge graph construction as LLMs become standard tools in the field.