Skip to contents

Compute corpus coverage from a completed coding run

Source: R/corpus_coverage.R
compute_corpus_coverage.Rd

Asserts the LLM saw every entry that survived preprocessing (entry-level coverage). Returns a CorpusCoverage S3 object summarising the funnel from preprocessed data to LLM-processed entries to coded entries, plus the no_silent_truncation flag that pakhom uses as the headline Tier-0 assertion. Within-entry truncation against the per-entry character cap is measured separately and disclosed via n_entries_truncated / chars_sent_to_llm (or reported as untracked for coding states recorded before these fields existed) – a configurable, surfaced cap is intentional behavior, distinct from the silent corpus truncation the headline flag guards against.

Usage

compute_corpus_coverage(
  coding_state,
  data,
  n_raw_loaded = NA_integer_,
  n_after_preprocessing = NA_integer_,
  test_mode_sample_size = NA_integer_
)

Arguments

coding_state

A finalized ProgressiveCodingState (the one returned by run_progressive_coding).

data

The standardized + preprocessed tibble that was fed to the coding step (must have std_id and std_text). Used to compute byte counts and to verify every entry has a matching entry_results record.

n_raw_loaded

Optional integer: rows loaded from the database before preprocessing. NA_integer_ when unknown (e.g., resumed run where the raw count wasn't preserved across the checkpoint).

n_after_preprocessing

Optional integer: rows after preprocessing but before any test-mode sampling. Defaults to NA_integer_.

test_mode_sample_size

Optional integer: when test mode was on, the sub-sample size used. NA_integer_ when test mode was off.

Value

A CorpusCoverage S3 object (a list with class).

Details

Pre-preprocessing counts (e.g., raw rows from the database before deduplication and length filtering) can be supplied via n_raw_loaded and n_after_preprocessing; when omitted, the coverage object reports them as NA_integer_ and the card degrades gracefully. The headline assertion (entry-level coverage: no silent entry-level truncation) does not depend on pre-preprocessing counts.