Ballot Guide Lab — v2 results addendum for team review October 11, 2026. AI evaluation complete; human review pending. This is new evidence to incorporate into the original v1 report. The original report PDF and v1 frozen answers remain unchanged. The results below refer only to the revised conversational questions. V1 and v2 use different questions and fresh judges, so their score difference is not a controlled effect estimate. RUN 15 held-out questions (10 everyday, 5 difficult) on Propositions 2, 32 and 36 were run under naive untuned, source-only untuned and source-only fine-tuned conditions, producing 45 real outputs. The recorded run reused the pinned Qwen3.5-2B base and saved LoRA adapter on a free Colab Tesla T4. No retraining or paid inference endpoint was used. Generation took 321.10 seconds including first-inference warmup. This is total generation time, not a steady-state user-facing latency estimate. Per-answer latency is preserved in the raw data. Each question's source packet was identical across all three conditions; prototype prompts were identical for the tuned and untuned comparison. Untuned runs used PEFT disable_adapter with every layer verified disabled. The adapter was verified enabled for tuned inference. Decoding was greedy, seed 894, max_tokens 320, max_length 3072, thinking disabled, no JSON grammar. The runtime manifest preserves revisions, hashes and installed package versions. EVALUATION A separate fresh AI agent received shuffled output IDs, questions, voter-visible answers, raw outputs, official reference pages and answer notes. It had no condition key, training labels, other ratings or generator-development history. It scored accuracy, completeness, clarity and neutrality from 0 to 2 and counted distinct unsupported factual claims separately in visible/raw outputs. Its exact runtime model variant was not exposed; provenance does not guess it. The same official PDF was consulted to clarify pages 15 and 75. No external reporting, current law or election outcomes were used to justify answers. RESULTS (ONE AI JUDGE, NOT HUMAN RATINGS) Condition n Strict passes Accuracy Completeness Clarity Neutrality Visible claims Raw claims Naive untuned 15 1/15 0.867 0.867 1.533 1.200 26 26 Source-only untuned 15 2/15 0.400 0.333 2.000 2.000 4 11 Source-only fine-tuned 15 2/15 1.067 0.467 1.733 1.867 2 3 Strict pass means all four scores equal 2 with zero visible unsupported claims. The fine-tuned condition did not improve strict passes over source-only untuned on this selected v2 set. Some successful abstentions pass, whereas withheld answerable information loses accuracy/completeness. Higher mean accuracy does not establish reliable overall answers or population accuracy. The source-only technical structure/source checks passed 8/15 untuned and 10/15 tuned outputs. These are validator outcomes, not semantic strict passes. Matching text did not establish completeness, correct attribution or entailment. The judge counts false source-absence assertions as unsupported factual claims; claim counts are not percentages of cases. Read the per-answer reasons. EXAMPLES TO DISCUSS - V2-01 tuned: gives repayment costs instead of the bond purpose and amount. - V2-05 tuned: supplies the small-employer $17 timing but omits its $18 timing. - V2-06 tuned: supplies $18 timing but omits the requested $17 start. - V2-07 tuned: presents an opponent's job-loss prediction as a fact. - V2-10 tuned: attributes an opponent's billions claim to the Legislative Analyst. - V2-12 tuned: gives statewide repayment, without identifying the unavailable personal-tax calculation. - V2-15 tuned: gives the community-college amount but omits the local-allocation gap. - O024 audit raw output: its quotation is supported on page 75 but was assigned to a page-72 source ID; the application withheld it. The ambiguity is retained. SOURCE/CHECKPOINT CAVEAT The allocation split is on PDF page 15. Some frozen authored reference checkpoints list page 16. Source excerpts and PDF evidence take precedence over checkpoint metadata. This is documented rather than changing the frozen cases. The two optional instruction-attack and historical-wage regressions have not run. TRAINING BOUNDARY The saved adapter was trained earlier on 104 assistant-authored QA examples based on the California 2024 official guide; 31 examples formed validation. Training propositions were 3, 4, 5, 6 and 33; validation 34 and 35. Pilot measures 2, 32 and 36 were held out of those synthetic QA examples. The training loss masked question/source tokens and supervised assistant answer tokens. The PDF also supplied retrieval evidence. Candidate quotations were outside this training and evaluation. The base Qwen model was already pretrained on broader material; this project does not establish that the base model saw only the official guide. RECOMMENDATION AND REMAINING WORK Revise. Complete real independent human ratings for all 45 v2 outputs using the new blank workbook and blinded viewer, before consulting AI ratings. Human ratings currently: 0/45. Paired ratings: 0. AI–human disagreement is unavailable, not 0%. Preserve both original ratings and resolve material disagreements with an additional review. Candidate-quotation review is separate and still pending. Prioritize retrieval/source-role handling, answer completeness and interpretation checks. Do not claim voter readiness, measured adoption, time savings, broad political neutrality, or improvement based on these selected cases alone. Update the report's setup, results, transcripts and AI disclosure with v2 evidence; verify team names/NetIDs and render the final submission within required limits. FILES v2-responses.html — blinded visible/raw answers and sources human-review-v2.xlsx — blank independent human ratings and frozen answers v2-judge-ratings.jsonl — exact fresh AI ratings and criterion reasons v2-judge-provenance.json — blinding, input hashes and evaluation limitations v2-scored-results.zip — outputs, scoring, condition key, templates and runtime v2-generated-results.zip — unchanged original Colab generation export Ballot_Guide_V2_Run.ipynb — saved-adapter inference/reproduction notebook v2-review-instructions.txt — human scoring and explicit v2 export commands Generation archive SHA256: 57beb1edae9b8915de1127925a0a871eb8783afa279e469126f03e3d4a3acfcd