Code-driven Check: Japan’s Open Data and the Machine-Readable Gap

IT Policy Proposals
Code-driven Check: Japan’s Open Data and the Machine-Readable Gap

どうも〜おかむーです! Today I’ll take a tech-first look at how Japan’s central government publishes data and where engineers actually hit friction. エンジニア的に言うと、政策をコードで検証できるかがすべてなんですよ〜

  • Japan has solid portals (Digital Agency, Japan Dashboard, e-Stat) but many datasets are still PDF-locked.
  • New machine-readability rules are promising, but rollout and enforcement will decide real impact.
  • Practical fixes: publish CSV/JSON, provide OpenAPI specs, and add dataset versioning + provenance.

結論

Government data infrastructures in Japan have matured — dashboards and APIs exist (e-Stat, Japan Dashboard, Digital Agency) — but the day-to-day reality is mixed: many KPIs, grant reports, and guidance live in PDFs or Excel that break automation. 要するに、政策の検証を自動化するにはまだやることが多いということです。

Report

What I checked

  • Portals: digital.go.jp (Digital Agency) and Japan Dashboard surface lots of stats; dashboard.e-stat.go.jp exposes an API for statistical tables.
  • Policy docs: Digital Denen strategy and grant KPI guidance (cas.go.jp / chisou.go.jp) often publish roadmaps and KPI tables, but reports are frequently PDF.
  • Rulemaking: Recent documents from the Digital Agency outline machine-readability rules (levels, Level1 = CSV/Excel/CSV required) and a formal decision on machine-readability — promising signal but implementation timeline matters.

Technical pain points (code folks will sympathize)

  • PDF-locked tables: automated extraction needs brittle tools (tabula, Camelot) and manual QA.
  • Mixed formats: Excel with merged cells, inconsistent schemas across local governments.
  • Lack of APIs or unstable endpoints: scraping dashboards is fragile.

Quick code example (how to fetch e-Stat data)

import requests

import pandas as pd

API_KEY = "YOUR_KEY"

res = requests.get('https://api.e-stat.go.jp/rest/3.0/app/json/getStatsData',

params={'appId':API_KEY,'statsDataId':'000342xxxx'})

data = res.json()

parse into dataframe according to structure

要するに、API一本で取得できれば検証は爆速です。

Concrete improvements

  • Mandate CSV/JSON + OpenAPI for every dataset declared Level1/2.
  • Central dataset catalog with schema, last-updated, DOI-like stable IDs.
  • Validation CI that lints datasets (schema, types, missing-values) before publish.
  • Example: provide a Git-backed catalog (Data Package / Frictionless) so municipalities can push changes via PRs.

Policy measurement gap

  • Grant KPI docs (chisou.go.jp) set targets but performance reporting often lags and is unstructured. Automating KPI pipelines would allow near-real-time monitoring and reduce discretion errors.

まとめ

Public data in Japan is heading the right way: institutions and dashboards exist, and rulemaking on machine-readability is happening. But to turn policy into verifiable code, we need consistent CSV/JSON publishing, stable APIs, dataset schemas, and tooling (linting, CI, catalogs). これが揃えば、政策をコードで語れる世界になります!

おかむーから一言

Tech + politics = unstoppable combo. データがちゃんとAPIで出れば、誰でも政策をテストできる。僕はそれを本気でやりたいんです!