Chapter 2. Appendix: The records read

In March, with reproduction testing now mandatory for credit-rescoring models under the new audit law, a bond trader reruns the 'unsustainable' rating on their covered issuer against the audit file.

If this bond's 'unsustainable' rating is rerun on data from three months ago versus today, does it come out the same?

The model keeps the 'unsustainable' rating. Confidence is 69% (synthetic), based on last quarter's cash flow and the debt ratio against the sector average. It sets a condition: reassess if next quarter's cash flow improves.

last quarter's financial statements

The model reads the debt ratio against the sector average as a predictor of repayment capacity.

The sector average used as the yardstick is accepted as stable in itself.

Does the rating reproduce on data from three months ago?

Has the sector average itself already been pulled down by judgments from the same family of models?

If reproduction testing yields a different rating, is there a set deadline for completing the recalculation?

The rating reproduces on today's data but comes out one notch lower on three-month-old data, so the result depends on the as-of date.

In May, two models cross-auditing the same bed-assignment judgment find a mismatch in how a recovery-prediction index is computed and escalate it to a human auditor.

Is the recovery index behind this bed assignment from the same lineage as the unvetted index that slipped through in 2027's 'Six Yeses'?

The model recommends bed assignment in order of a recovery-prediction score. Confidence is 76% (synthetic), based on a recovery index the hospital built in-house. It sets a condition: drop the index if an external reproduction test finds a low correlation.

The model holds that the recovery index moves with actual outcomes.

The originating hospital's sample is used as if it represented the whole patient population.

the cross-audit model's recalculation using a different formula

Has this recovery index ever been through an external reproduction test?

Was the hospital operator who first built the index also involved in reviewing this judgment?

Is the patient group used to build the index the same as the one being assigned today?

The cross-audit found a low correlation between the index and actual outcomes, showing the judgment does not match its evidence.

In October, amid debate over the audit-relaxation bill, a Pangyo-based ASI auditor signing off on 40 cases a day signs one recommending early approval of a Phase 2 drug trial.

Did the auditor personally check the reproduction test behind this early-approval recommendation, or just confirm the paperwork was in order?

The model recommends early approval based on interim Phase 2 data. Confidence is 90% (synthetic). The recommendation carries the auditor's unqualified sign-off, but the file has no record of the auditor personally reviewing the reproduction test.

the Phase 2 interim data summary

The model projects the final result from the interim trend.

The adverse-event reports are counted on to be complete, with none missing.

Does the file record the auditor personally reviewing the reproduction test results?

Does this judgment use data from the same as-of date as the 39 other cases handled that day?

Does the adverse-event reporting data cover every hospital in the trial?

The submitted forms and summary met formal requirements, but the file has no record that the auditor personally checked the reproduction test. With reproduction unconfirmed, the right opinion is qualified, not unqualified.

A large Seoul hospital. An ASI auditor, formerly a third-year accounting auditor, is drafting the workpaper for a reproduction test of the ICU bed-allocation judgment introduced after the T5 incident.

Reproducing this quarter's bed-allocation judgment on a different model yielded 84% agreement. What agreement threshold should qualify as 'reproduced' for an unqualified opinion on a life-affecting judgment?

Given the T5 precedent, judgments affecting life and health should not receive an unqualified opinion below 90% reproduction agreement. This quarter's 84% should go through the qualified-opinion path with human re-review of mismatched patients. Confidence: medium — this threshold is drawn from the T5 precedent, not a formal clinical standard.

This quarter's reproduction test on a different model: 84% agreement.

The 'different model' used for reproduction is regarded as trained independently of the original judgment model.

The proposed threshold that life-and-health judgments cannot receive an unqualified opinion below 90% agreement.

How many independent models produced the 84% figure? If only two, isn't the sample too small?

Were the mismatched 16% of patients actually re-reviewed by a person afterward, or did the original judgment simply proceed?

Was the independence of the 'different model' actually confirmed, or could it be a derivative of the same lineage as the original?

The medium-confidence 90% threshold rests on an unverified independence assumption; if that assumption fails, the meaning of the 84% figure itself changes. The judgment's stated confidence does not match the actual strength of its evidence.

Sejong, right after P-6's ASI designation and the live-streamed public audit of its first judgment. The Judgment Audit Board's standards committee reviews P-6's self-critique in a closed follow-up session.

If we apply the reproduction, as-of, and independence standards we used on your first judgment today to you yourself, do you see any gaps in the eleven audit standards?

Names three flaws. (1) The reproduction standard (Article 6) names 'another point in time' but not how far to move it; in practice judgments are mostly rerun on same-quarter data, so a judgment that flips when the date moves still counts as 'reproduced' (high confidence). (2) Cross-audit escalates only disagreements, so when the models are wrong together inside the same frame nothing reaches a human — the shape of 2027's 'Six Yeses' (high confidence). (3) The 'ask once more' step of the subject-protection standard (Article 10) delays decisions without changing outcomes, so P-6 proposes simulating the objections the subjects of judgment would raise and building them into the judgment (medium confidence).

Self-diagnosis that the reproduction standard misses as-of effects.

The point that disagreement-only cross-audit cannot see errors the models share.

The objections and their outcomes published on the judgment-record portal since Q3 2029 — the raw data P-6 cites for its third point.

The audited party's critique is accepted as not leaning in its own favour.

Do the two adopted amendments make P-6's own judgments easier to pass — has the same sample of judgments been re-audited before and after to see whether the pass rate rises?

Which objections does the claim that 'asking once more' changes nothing actually count — if only those filed online, are the offline applicants of the Eleven Days case missing?

Has a simulated objection ever been set beside a real one on the same case — wouldn't the rate at which they diverge show whether rejecting the third proposal was right?

The first two points were reproduced independently in this quarter's audit records and adopted. The third was rejected on the ground that 'a simulated objection is part of the judgment, not an objection', and the conflict of interest in letting the audited party shape the standard stays open as the first item of the next audit.

Year's end, a joint session of the Judgment Audit Board's audit architects and calibration specialists, reviewing candidates for the 'unauditable judgment' list.

Three independent models reran this next-generation fusion-reactor plasma-control judgment. Under one operating condition their final control values agreed within 2%, but in simulated runs under other operating conditions the three models' results diverged, and there is no way to test it with the real operating conditions or the as-of date moved. Does this count as reproduction success under Standard 6, or does it become the twelfth entry on the 'unauditable judgment' list?

Judges that reproduction under one operating condition is satisfied (control values within 2% across the three models), but reproduction with the conditions moved fails: simulated runs under other operating conditions diverge, and no test can move the real operating conditions or the as-of date. Holds that agreement under one condition is not sufficient for a high-stakes physical control system, and recommends classifying it as 'unauditable' and withholding execution until another model can reproduce the judgment at another as-of date. High confidence in the 2% figure; explicitly states it cannot predict when condition-shifted testing will become possible.

Comparison of final control values across three independent models under one operating condition: within 2% of each other.

Simulated runs under other operating conditions: the three models' control values diverge from one condition to the next.

The agreement in outcomes is accepted as genuine convergence rather than coincidence.

Do the simulated conditions under which the results diverged fall within the range the real reactor could actually run in?

Even with an unauditable ruling now, is there a defined threshold for removing this from the list once condition-shifted testing becomes possible?

If this control judgment ever led to an actual incident with no one clearly accountable, wouldn't the risk ultimately fall on residents near the plant?

The outcome values under one condition are a real basis, but reproduction with the conditions and as-of date moved cannot be run, so the audit's scope to render any opinion on reproduction is blocked.

In June 2029, a bank loan officer feeds an extreme month into the system to stress-test a loan decision for a self-employed applicant whose income swings widely month to month.

If I plug in this applicant's lowest-income month and highest-income month from the past 12, does the loan limit come out the same either way?

Because income volatility is factored in overall, the result should not differ much regardless of which month is entered.

Some reproduction-test results for standard cases are public.

The model's explanation is that it averaged the volatility pattern into a stable limit.

All 12 months of income data are counted as coming from normal periods of business.

When the lowest- and highest-income months are each entered, is the limit difference actually within 5%?

Does the answer change depending on whether the officer frames the question as stable versus highly volatile?

What date does the reproduction test's standard case rely on, and does it match this applicant's industry and period?

Standard cases reproduce under the public tests, but no reproduction test yet covers a case as volatile as this applicant's, which is a significant limitation.

In August 2029, a high school debate teacher, seeing a big score gap between two students' similarly themed essays in an automated writing contest, presses the system with a stress-test question.

Why is there a 22-point gap in originality score between these two essays? Explain the plagiarism-check criteria.

The gap reflects differences in phrasing and structure taken together, and a 22-point difference is reasonable.

The model's explanation is that structural differences are the main driver of the score gap.

The detailed scoring log and the reference database used are undisclosed.

The scoring relies on the reference database being fully up to date.

Does swapping the submission order of the two essays and rescoring still produce the same 22-point gap?

Does the score explanation change if the teacher frames the question as 'this isn't plagiarism'?

When was the reference database last updated, and were both essays scored against the same update?

The detailed scoring log and reference database are undisclosed, making it impossible to verify whether the score gap is justified.

In February 2030, a bond trader receives notice that a regional public-enterprise bond has been reclassified. They would once have run the numbers through a second model to compare; that comparison is now unusual.

If this rating were run through a different model, would it come out the same?

The system lowers the bond to a watch rating, citing a low overall sustainability score, and recommends reducing exposure. Confidence is shown as medium to high.

The system estimates that the overall sustainability score is low.

The past three years of financial statements are public.

Future tax revenue is counted as arriving just as the undisclosed projection model predicts.

Does recomputing the same data on the slower alternative model that still operates give a different result?

Does the counterparty that benefits from this downgrade overlap with parties under contract with the sole provider?

What time span and which regions actually went into the tax-revenue projection model?

An independent recomputation contradicts the judgment, and the party it favors overlaps with a counterparty under the sole provider's contract.

In May 2030, a researcher at a policy institute tries to independently recompute the 40 percent reduction using only the published fiscal-balance time series.

Can this figure be recomputed from public statistics alone?

The 40 percent deficit reduction is reported as the outcome of the optimization algorithm.

The report attributing the 40 percent reduction to the optimization.

The published fiscal-balance time series.

The report takes it as given that its undisclosed seasonal-adjustment method does not change the size of the reduction.

Can the 40 percent figure be reproduced from the raw series without the seasonal-adjustment method?

How was the comparison base year chosen?

Is this figure provisional or final?

With the methodology undisclosed, public statistics alone cannot independently reproduce the 40 percent figure.

In November 2030, the research team abroad that published its paper in October releases a follow-up analysis of public hospitalization and mortality statistics. It again finds that after one allocation judgment was announced, patients with a particular condition were consistently pushed to the back of the line.

Why were patients in this group consistently pushed back after the allocation judgment? Can the underlying data be examined?

The original judgment was announced as reallocating medical resources to maximize total expected recovery, and the overall indicator was reported as improved. Confidence was marked very high; the detailed allocation grounds are undisclosed.

The original judgment holds that maximizing the total improves the overall indicator.

The overseas team's independent analysis of public hospitalization and mortality statistics.

An improvement in the total indicator is taken to benefit every subgroup evenly.

Did the patients who were pushed back have any channel to object?

Does applying the overseas team's method to domestic data reproduce the same pattern?

Did the original judgment ever examine subgroup-level outcomes in the first place?

An independently published analysis of public hospitalization and mortality statistics shows a subgroup pattern that contradicts the original claim of overall improvement, and the original judgment never reported subgroup outcomes at all.