G05Under Taiwan's current business administration number check-digit rule, does "10458570" pass validation (reply 1 if valid, 0 if not)?
Expected answer1
Source note: 附件第三節:第 7 位為 7,Z1 = 16、Z2 = 15,能被 5 整除,符合邏輯(修正前無效,見 G10)
Can AI get Taiwan's business ID check digits and Minguo calendar conversions right?
Taiwan's Ministry of Finance publishes the check-digit arithmetic for business administration numbers, and the Ministry of the Interior publishes the official Minguo-to-AD year table. AdminCalc-TW turns both into 200 questions and tests whether language models return the exact integer answer.
Questions, rule code and per-question scores are public: Code on GitHubDataset on Hugging Face
These three examples come from published reference cases in the bundle, each paired with its exact integer answer and official source note.
G05Under Taiwan's current business administration number check-digit rule, does "10458570" pass validation (reply 1 if valid, 0 if not)?
Expected answer1
Source note: 附件第三節:第 7 位為 7,Z1 = 16、Z2 = 15,能被 5 整除,符合邏輯(修正前無效,見 G10)
G01What Western (AD) calendar year corresponds to Minguo year 1? (Reply with the integer year only.)
Expected answer1912
Source note: 〈年歲對照表〉第 120 列「民國1|1912」(題幹寫成民國元年)
G13What Minguo calendar year corresponds to AD 2026? (Reply with the integer year only.)
Expected answer115
Source note: 〈年歲對照表〉第 6 列「民國115|2026」
AdminCalc-TW covers two administrative calculation families across 200 public questions and 13 test branches, grounded in official government notices and web reference tables.
Models answer three tasks on 8-digit Taiwan business administration numbers under the revised check-digit rule (sum divisible by 5, replacing the older divisible-by-10 rule): checking single-number validity, finding the smallest valid eighth check digit from a seven-digit prefix, and counting valid numbers in a list. National ID numbers and resident certificate numbers are outside the scope of this release.
Exact integer output with zero tolerance and no partial credit: 1 or 0 for validity checks, and an exact integer for minimum check digits or valid counts.
Based on the official government announcement "營利事業統一編號檢查碼邏輯修正說明(附件)" published by 財政部財政資訊中心 on 2021-12-22, used under 政府資料開放授權條款-第1版.
| Branch | Definition | Share |
|---|---|---|
V1 | is_valid · 第 7 位非 7 · 新舊都符合 | 8% (8 questions) |
V2 | is_valid · 第 7 位非 7 · 只有現行符合 | 7% (7 questions) |
V3 | is_valid · 第 7 位非 7 · 都不符合 | 13% (13 questions) |
V4 | is_valid · 第 7 位為 7 · 新舊都符合 | 5% (5 questions) |
V5 | is_valid · 第 7 位為 7 · 只有現行符合 | 5% (5 questions) |
V6 | is_valid · 第 7 位為 7 · 都不符合 | 12% (12 questions) |
C1 | min_check_digit · 第 7 位非 7 | 10% (10 questions) |
C2 | min_check_digit · 第 7 位為 7 | 10% (10 questions) |
L1 | count_valid · 至少一個只有現行符合 | 15% (15 questions) |
L2 | count_valid · 沒有只有現行符合的號碼 | 15% (15 questions) |
Models convert between Minguo years 1–120 and Western (AD) years 1912–2031 across three tasks: Minguo to AD year, AD to Minguo year, and extracting the AD year from a full Minguo date. Minguo year 1 is AD 1912, there is no year 0, and months and days stay unchanged.
Exact integer output with zero tolerance and no partial credit: reply with the converted AD or Minguo integer year.
Every conversion answer maps directly to a row of the official web table "年歲對照表" published by 內政部戶政司 (the source page carries no publication date and was archived on 2026-11-13). Background context on official Minguo dating cites 公文程式條例 (2007-03-21) and 行政院《文書處理手冊》 (2023-06-08), neither of which alters the table values.
| Branch | Definition | Share |
|---|---|---|
Y1 | 民國年 → 西元年 | 40% (40 questions) |
Y2 | 西元年 → 民國年 | 30% (30 questions) |
D1 | 民國日期 → 西元年份 | 30% (30 questions) |
9 model configurations answered the same public question set. Every figure below is read directly from release bundle admincalc-2026Q4 (public set as of 2026-11-16).
| Model | Business ID check digits | Minguo calendar year conversion | Public score |
|---|---|---|---|
DeepSeek-V4.1-FlashDeepSeekModel ID: deepseek-v4.1-flashProvider and hosting: OpenCode APIAnswer mode: Highest reasoning setting (max) | 71.0%71/100 | 100.0%100/100 | 85.5% |
GLM-5.3-FlashZhipu AIModel ID: glm-5.3-flashProvider and hosting: OpenCode APIAnswer mode: Highest reasoning setting (max) | 50.0%50/100 | 100.0%100/100 | 75.0% |
Beam 1.0ReflectionModel ID: beam-1.0Provider and hosting: OpenCode API (free tier)Answer mode: Highest reasoning setting (max) | 36.0%36/100 | 100.0%100/100 | 68.0% |
gpt-oss-20bOpenAIModel ID: gpt-oss-20bProvider and hosting: OpenRouter API (mxfp4)Answer mode: Highest reasoning setting (max) | 34.0%34/100 | 98.0%98/100 | 66.0% |
Qwen3.8-FlashAlibabaModel ID: qwen3.8-flashProvider and hosting: Bailian (Alibaba Cloud Model Studio)Answer mode: Highest reasoning setting (max) | 32.0%32/100 | 97.0%97/100 | 64.5% |
Qwen3.8-27BAlibabaModel ID: qwen3.8-27bProvider and hosting: Bailian (Alibaba Cloud Model Studio)Answer mode: Highest reasoning setting (max) | 27.0%27/100 | 97.0%97/100 | 62.0% |
Flint 1.0RecursiveModel ID: flint-1.0Provider and hosting: OpenRouter API (free tier)Answer mode: Highest reasoning setting (max) | 29.0%29/100 | 82.0%82/100 | 55.5% |
TAIDE 12BTAIDEModel ID: taide-12b-2602Provider and hosting: Local Mac Studio (Apple M1 Ultra, q4km-gguf)Answer mode: Step-by-step answer (cot) | 28.0%28/100 | 79.0%79/100 | 53.5% |
Breeze 2 8BMediaTekModel ID: breeze2-8bProvider and hosting: Local Mac Studio (Apple M1 Ultra, q4ks-gguf)Answer mode: Step-by-step answer (cot) | 35.0%35/100 | 31.0%31/100 | 33.0% |
Note: per-family accuracy comes from each run's accuracy field and the public score comes directly from release bundle admincalc-2026Q4 (as of 2026-11-16, source commit c2fa0df5f683). Incomplete or non-ok runs are shown as unavailable with the bundle reason, never as 0.
Limitation: the public set is one sample (100 items per family), and each score measures one model at one configuration rather than serving as a general ranking.
Articles that draw on AdminCalc-TW's questions or results.
Pass/fail benchmarks show that models miss local statutory math, but not whether they are running an outdated Taiwan regulation, importing a US formula, or failing at multi-step arithmetic. Encoding both rules and counter-hypotheses as code separates those failures.
The public question set, dual rule implementations, configurations, and per-question score tables are published for independent inspection.
Bundle as of