Institutions & ProceduresRelease admincalc-2026Q4Released

AdminCalc-TW

Can AI get Taiwan's business ID check digits and Minguo calendar conversions right?

Taiwan's Ministry of Finance publishes the check-digit arithmetic for business administration numbers, and the Ministry of the Interior publishes the official Minguo-to-AD year table. AdminCalc-TW turns both into 200 questions and tests whether language models return the exact integer answer.

public questions
200
question families: Business ID check digits, Minguo calendar year conversion
2
test branches
13
model configurations tested
9

Questions, rule code and per-question scores are public: Code on GitHubDataset on Hugging Face

What the questions look like

These three examples come from published reference cases in the bundle, each paired with its exact integer answer and official source note.

Business ID check digitsReference case G05

Under Taiwan's current business administration number check-digit rule, does "10458570" pass validation (reply 1 if valid, 0 if not)?

Expected answer1

Source note: 附件第三節:第 7 位為 7,Z1 = 16、Z2 = 15,能被 5 整除,符合邏輯(修正前無效,見 G10)

Minguo calendar year conversionReference case G01

What Western (AD) calendar year corresponds to Minguo year 1? (Reply with the integer year only.)

Expected answer1912

Source note: 〈年歲對照表〉第 120 列「民國1|1912」(題幹寫成民國元年)

Minguo calendar year conversionReference case G13

What Minguo calendar year corresponds to AD 2026? (Reply with the integer year only.)

Expected answer115

Source note: 〈年歲對照表〉第 6 列「民國115|2026」

Two question families

AdminCalc-TW covers two administrative calculation families across 200 public questions and 13 test branches, grounded in official government notices and web reference tables.

Business ID check digits v0.1.1

營利事業統一編號檢查碼(財政部現行邏輯) · 100 public questions

What the model is asked to do

Models answer three tasks on 8-digit Taiwan business administration numbers under the revised check-digit rule (sum divisible by 5, replacing the older divisible-by-10 rule): checking single-number validity, finding the smallest valid eighth check digit from a seven-digit prefix, and counting valid numbers in a list. National ID numbers and resident certificate numbers are outside the scope of this release.

Answer format and scoring

Exact integer output with zero tolerance and no partial credit: 1 or 0 for validity checks, and an exact integer for minimum check digits or valid counts.

Official source document

Based on the official government announcement "營利事業統一編號檢查碼邏輯修正說明(附件)" published by 財政部財政資訊中心 on 2021-12-22, used under 政府資料開放授權條款-第1版.

View official source page

Test branches (10)

BranchDefinitionShare
V1is_valid · 第 7 位非 7 · 新舊都符合8% (8 questions)
V2is_valid · 第 7 位非 7 · 只有現行符合7% (7 questions)
V3is_valid · 第 7 位非 7 · 都不符合13% (13 questions)
V4is_valid · 第 7 位為 7 · 新舊都符合5% (5 questions)
V5is_valid · 第 7 位為 7 · 只有現行符合5% (5 questions)
V6is_valid · 第 7 位為 7 · 都不符合12% (12 questions)
C1min_check_digit · 第 7 位非 710% (10 questions)
C2min_check_digit · 第 7 位為 710% (10 questions)
L1count_valid · 至少一個只有現行符合15% (15 questions)
L2count_valid · 沒有只有現行符合的號碼15% (15 questions)

Minguo calendar year conversion v0.1.1

民國年與西元年換算(民國 1–120 年) · 100 public questions

What the model is asked to do

Models convert between Minguo years 1–120 and Western (AD) years 1912–2031 across three tasks: Minguo to AD year, AD to Minguo year, and extracting the AD year from a full Minguo date. Minguo year 1 is AD 1912, there is no year 0, and months and days stay unchanged.

Answer format and scoring

Exact integer output with zero tolerance and no partial credit: reply with the converted AD or Minguo integer year.

Official source document

Every conversion answer maps directly to a row of the official web table "年歲對照表" published by 內政部戶政司 (the source page carries no publication date and was archived on 2026-11-13). Background context on official Minguo dating cites 公文程式條例 (2007-03-21) and 行政院《文書處理手冊》 (2023-06-08), neither of which alters the table values.

View official source page

Test branches (3)

BranchDefinitionShare
Y1民國年 → 西元年40% (40 questions)
Y2西元年 → 民國年30% (30 questions)
D1民國日期 → 西元年份30% (30 questions)

Models tested on the public split

9 model configurations answered the same public question set. Every figure below is read directly from release bundle admincalc-2026Q4 (public set as of 2026-11-16).

ModelBusiness ID check digitsMinguo calendar year conversionPublic score
DeepSeek-V4.1-FlashDeepSeekModel ID: deepseek-v4.1-flashProvider and hosting: OpenCode APIAnswer mode: Highest reasoning setting (max)71.0%71/100100.0%100/10085.5%
GLM-5.3-FlashZhipu AIModel ID: glm-5.3-flashProvider and hosting: OpenCode APIAnswer mode: Highest reasoning setting (max)50.0%50/100100.0%100/10075.0%
Beam 1.0ReflectionModel ID: beam-1.0Provider and hosting: OpenCode API (free tier)Answer mode: Highest reasoning setting (max)36.0%36/100100.0%100/10068.0%
gpt-oss-20bOpenAIModel ID: gpt-oss-20bProvider and hosting: OpenRouter API (mxfp4)Answer mode: Highest reasoning setting (max)34.0%34/10098.0%98/10066.0%
Qwen3.8-FlashAlibabaModel ID: qwen3.8-flashProvider and hosting: Bailian (Alibaba Cloud Model Studio)Answer mode: Highest reasoning setting (max)32.0%32/10097.0%97/10064.5%
Qwen3.8-27BAlibabaModel ID: qwen3.8-27bProvider and hosting: Bailian (Alibaba Cloud Model Studio)Answer mode: Highest reasoning setting (max)27.0%27/10097.0%97/10062.0%
Flint 1.0RecursiveModel ID: flint-1.0Provider and hosting: OpenRouter API (free tier)Answer mode: Highest reasoning setting (max)29.0%29/10082.0%82/10055.5%
TAIDE 12BTAIDEModel ID: taide-12b-2602Provider and hosting: Local Mac Studio (Apple M1 Ultra, q4km-gguf)Answer mode: Step-by-step answer (cot)28.0%28/10079.0%79/10053.5%
Breeze 2 8BMediaTekModel ID: breeze2-8bProvider and hosting: Local Mac Studio (Apple M1 Ultra, q4ks-gguf)Answer mode: Step-by-step answer (cot)35.0%35/10031.0%31/10033.0%

Note: per-family accuracy comes from each run's accuracy field and the public score comes directly from release bundle admincalc-2026Q4 (as of 2026-11-16, source commit c2fa0df5f683). Incomplete or non-ok runs are shown as unavailable with the bundle reason, never as 0.

Limitation: the public set is one sample (100 items per family), and each score measures one model at one configuration rather than serving as a general ranking.

Articles

Articles that draw on AdminCalc-TW's questions or results.

  1. Where Models Get Taiwan's Labor and Administrative Rules Wrong

    Pass/fail benchmarks show that models miss local statutory math, but not whether they are running an outdated Taiwan regulation, importing a US formula, or failing at multi-step arithmetic. Encoding both rules and counter-hypotheses as code separates those failures.

All articles

Bundle as of