Alex Garnett, Senior Curriculum Developer at Temporal Technologies, presented on effective and principled uses of AI in documentation, focusing on Retrieval Augmented Generation and the Kappa AI platform.
Key discussion points included practical AI integration in documentation workflows, the importance of citations and feedback loops, accessibility, privacy considerations, moderation, and ethical challenges in LLMs.
The session included attendee Q&A on workflow adaptation, AI ingestion of community content, the feedback loop process, accessibility, regulated industry constraints, ethical/licensing issues, technical implementation, and moderation strategies.
The importance of maintaining high standards, context, and authoritativeness in documentation when leveraging AI tools was emphasized.
Action Items
None assigned or volunteered in the transcript.
AI in Documentation: Approaches and Principles
AI is increasingly promoted for use in documentation, but common large language models remain prone to errors (“hallucinations”), requiring oversight and domain knowledge.
Effective uses of AI include automating repetitive or reactive documentation tasks, enabling writers to focus on their preferred or value-adding activities.
The most interesting AI applications go beyond writing to user interaction, such as embedding chatbots in documentation sites that draw answers from existing, trusted corpora.
Retrieval Augmented Generation (RAG) allows leveraging a company’s own documentation and community-generated content to inform AI-generated responses while relying on robust citation mechanisms.
Integration of support, documentation, and community content (forums, Slack, GitHub, LMS) enhances the knowledge base available to users via AI tools.
Practical Implementation: Kappa AI Platform at Temporal
Temporal uses Kappa AI to provide a chatbot interface on documentation sites, enabling users to interactively query across multiple content sources.
Kappa’s analytics dashboard provides metrics on usage, certainty of answers, and source distribution (docs, Slack, courses, GitHub, etc.), facilitating ongoing content improvement.
The AI flags uncertain answers, encouraging users to further research and highlighting documentation gaps for writers.
Temporal did not need significant additional preparation for Kappa ingestion; existing markdown-based Docusaurus docs were compatible by default.
The feedback loop from AI analytics is operationalized by reviewing flagged “uncertain” queries and generating Jira tickets to address documentation deficiencies, especially to clarify compatibility and negative cases explicitly.
Community-generated content ingestion (e.g., public Slack, Discourse) is pruned regularly to maintain quality and prevent misinformation from contaminating the chatbot’s knowledge base.
Accessibility, Ethics, and Industry Considerations
Increasing access routes to documentation via AI is framed as an accessibility gain, provided content remains authoritative and well-moderated.
Challenges in regulated industries include the risk of AI-generated misinformation with potentially severe consequences; in these contexts, AI chatbot implementation may be inadvisable.
Ethical and licensing concerns are significant: there is skepticism about major actors’ respect for licensing, and the best recourse is to support good vendors and practices that elevate standards, such as robust citation and transparency.
Active moderation of community channels ensures quality control, but automating pruning with additional AI may introduce further complexity and unintended consequences.
Decisions
No significant decisions or approvals were made or recorded during the session.
Open Questions / Follow-Ups
None explicitly called out for follow-up; several attendee questions were answered live, including technical, ethical, and implementation aspects of documentation AI integration.