pakhom: AI-Assisted Thematic Analysis with Methodology-as-Architecture

Conducts AI-assisted thematic analysis – reflexive, codebook, and framework modes – with methodology codified at the architectural level. Three methodologically-distinct operating modes (Reflexive Scaffold, Codebook Collaborative, Framework Applied) shape the AI's role explicitly so the chosen epistemic stance is visible to reviewers, replicable across runs, and stamped onto every output.

The name

pakhom (Coptic eagle) is the native Coptic Egyptian form of the name of Saint Pachomius the Great (c. 292-348 CE), the desert abbot whose written Rule established the genre of methodology-as-written-document in Christian tradition. The Pachomian Rule was the first codified framework for organized communal practice; it transformed the unruly anchorite tradition into reproducible, inspectable, transmissible discipline. This package is a digital descendant of that tradition: AI behavior in qualitative analysis is constrained at the architectural level by methodologically-coherent rules, not at the configuration level by user discipline. Pakhom codified the Rule; pakhom codifies the methodology-as-permission- structure.

Three methodology modes

Each mode encodes a different posture for AI agency. The mode declaration is mandatory in every config (no default); it is locked at run start, stamped on every output, and any change creates a fork run with parent_run_id linkage.

reflexive_scaffold (Mode 1): AI as Socratic gadfly (Sarkar 2024). The researcher authors codes and themes (typically in NVivo / ATLAS.ti); pakhom contributes the provocateur loop that surfaces counter-narratives, absent voices, alternative interpretations, disconfirming evidence, and assumption-surfacing terms. The AI never names themes or codes. Use run_mode1. Per AC6 (symmetric engagement affordances), Mode 1's re-engagement lever is reflexive memos – typed Markdown notes round-tripped via YAML frontmatter (add_memo, persist_memos).
codebook_collaborative (Mode 2): AI proposes codes; researcher gates each at the codebook + theme review pause-points. The auto-pipeline of run_analysis; this is what users coming from a codebook TA / template TA tradition will recognize. IRR + saturation are quality diagnostics; researcher review points interleave with the AI's progressive coding pass.
framework_applied (Mode 3): Researcher provides a theoretical framework (e.g., Theory of Planned Behavior, COM-B, Theoretical Domains Framework – pre-built specs ship in inst/extdata/frameworks/); AI applies it verbatim and flags entries that resist the framework as anomalies per the framework's anomaly_handling policy. Use run_analysis with config$methodology$framework_spec_path set. The framework spec is archived byte-equivalently into the run dir and its sha256 is stamped into run_metadata.json.

Architectural commitments (AC1-AC10)

These commitments are load-bearing and do not weaken across modes.

AC1: AI is scaffold by architecture, not by configuration.
AC2: Three modes; no fourth.
AC3: No default mode; explicit declaration mandatory.
AC4: Methodology stamped on every output (ClinicalTrials.gov pattern – run_metadata.json, every CSV/JSON header, HTML stamp).
AC5: Soft-lock with audit trail; methodology change creates a new run with parent_run_id linkage.
AC6: Symmetric researcher-engagement affordances across modes: the same re-engagement levers in every mode, opt-in for Modes 2/3 (anti-gaming).
AC7: Universal Tier-0 transparency requirements (T0.1 quote provenance, T0.2 participant spread, T0.3 coverage) in all modes.
AC8: Modes share one architecture and primitive layer; Modes 2 and 3 are config branches of run_analysis() and Mode 1 (run_mode1()) layers its provocateur loop on the same primitives.
AC9: Methodology rules generated from config and injected into the model context every turn.
AC10: Stage-gating via filesystem state.

Algorithm-level commitments (C1-C8)

Distinct from the mode-design ACs above, these govern how the coding, clustering, statistics, and reporting layers behave; see the README ("For methodologists: rewrite-direction commitments") for the full text.

C1: The AI decides when to stop (no hardcoded theme/ saturation thresholds).
C2: Codes are preserved through clustering (themes group codes, never rewrite them).
C3: Live tracking artifacts during processing.
C4: Dataset-agnostic (no hardcoded column-name allowlists).
C5: No catch-all / "Other" buckets in the inductive modes.
C6: Arbitrary research-question length/complexity.
C7: Mode-aware behaviour only where modes genuinely differ.
C8: Publication-quality per-subtheme output shape.

Tier-0 universal transparency requirements

Three commitments mandatory in every mode, addressing the most-cited empirical critiques of LLM-for-TA tools.

T0.1 – Quote provenance + verification ladder: Every AI-attributed verbatim claim runs through strict offline match, normalized match, substring search, and embedding similarity. Fabricated quotes are dropped silently and logged to fabrication_log.csv. Mode 1 + Anthropic + framework_applied constraints handled per provider.
T0.2 – Participant spread per theme: Every theme reports n_distinct_contributors + Gini coefficient + top contributor share, so themes that look prevalent but rest on one heavy poster get surfaced (Jowsey et al. 2025 "Frankenstein" finding).
T0.3 – Whole-corpus coverage assertion: Modes 2/3 assert every preprocessed entry reached the LLM (entry-level coverage; within-entry truncation against the per-entry character cap is measured and disclosed) via compute_corpus_coverage; Mode 1 asserts "no silent skip across themes x provocation categories" via compute_mode1_coverage. Both inherit a virtual Tier0Coverage parent class so the report dispatches uniformly via render_tier0_coverage_card.

Main entry points

run_analysis: Modes 2 + 3 orchestrator (data load -> coding -> sentiment -> themes -> correlations -> report -> finalize_run).
run_mode1: Mode 1 orchestrator (provocateur loop + memos + Mode 1 report). Mirrors run_analysis's scaffolding but routes through run_provocateur_questioning.
create_config / config_wizard_app: Create a config programmatically or via a Shiny wizard.
load_framework_spec: Load a theoretical framework for Mode 3 (built-in: "tpb", "comb", "tdf").
add_memo / persist_memos: Mode 1 reflexive memo CRUD + Markdown round-trip.
compare_runs / compare_models: Cross-run and inter-model reliability comparisons.

Provider support

OpenAI (G family) and Anthropic (Claude family) with a unified ai_complete abstraction. Mode 3 + Anthropic structurally precludes the Citations API (forced tool_use schema and Citations API output are mutually exclusive on the same response); the Mode 3 report renders an explicit footnote disclosing this rather than letting reviewers infer a bug.

Author

Maintainer: Abanoub J. Armanious armaniousabanoub@gmail.com (ORCID)