Skip to content
jc-rs
GitHub

Method

How the number is produced

The product is a compatibility number anyone can check, not speed. That premise is worth nothing unless the way it is measured is on the table too.

One rule

A fixture pair enters the denominator only when the reference implementation matches that fixture under the published structural JSON comparison.

tests/differential/validate.py walks every .json fixture in the pinned corpus, runs the reference against the same input, and only then compares jc-rs. If the reference does not match its own expected output under this comparison, neither implementation is being tested and the pair is reported instead of counted. The reference is jc 1.25.7, the original Python tool. The comparison decodes JSON or NDJSON and ignores object-key order. Numerically equal integers and floats count as equal; array order, fields, other types, and values must agree.

What that leaves out, in the open

Tested934The reference matches the fixture under the comparison, so the result means something.
Matching934jc-rs has no structural or value differences under the published comparison. Dividing this count by Tested gives the published percentage.
Oracle reject9The reference does not match its own fixture here. Nothing to measure against.
Unmapped149The fixture filename resolves to no parser. Reducing this set will lower the headline number before it raises it, which is the trade the harness exists to make visible.
No input18The corpus ships the expected output without the input that produced it.

Two things that decide whether it means anything

The fixtures are a verbatim mirror

tests/fixtures/ is copied verbatim from the pinned reference and make check-fixtures fails the build if any shared file differs. Without that, a failing parser could be made to pass by rewriting the expected output, and the compatibility number would measure nothing at all.

The run is pinned to TZ=PST8PDT

The fixtures carry *_epoch fields computed in local time, and the reference's own test runner pins that zone. In any other timezone the oracle rejects every timestamp-bearing fixture and 146 pairs quietly leave the denominator, which understates coverage as surely as a silent skip overstates it.

Per parser

200 parsers have oracle-valid fixtures behind them. The other 37 are implemented but unproven by the corpus, and this page says so rather than rounding them into the total.

ParserTestedMatch
mdadm4040ping4040netstat3737ping_s3636ls3535dig2424git_log1919git_log_s1919nmcli1818iostat1414iostat_s1414sfdisk1414rsync1313rsync_s1313arp1212iptables1212csv1111csv_s1111last1111ls_s1111traceroute1111traceroute_s1111blkid1010df1010ifconfig99lsusb99proc99uname99vmstat99vmstat_s99ps88who88acpi77dir77mount77path_list77route77time77curl_head66free66nsd_control66ntpq66pip_show66stat66ufw66ufw_appinfo66upower66crontab_u55ethtool55ini55pip_list55stat_s55top55top_s55typeset55w55zpool_status55cbt44date44du44env44hciconfig44iwconfig44lsattr44lsblk44lsof44mpstat44mpstat_s44path44pidstat44pidstat_s44plist44ss44sysctl44timedatectl44uptime44xml44certbot33crontab33dmidecode33dpkg_l33file33findmnt33finger33group33lspci33needrestart33pacman33passwd33proc_version33wc33x509_cert33zipinfo33cksum22csv_ih22csv_ih_s22efibootmgr22find22fstab22git_ls_remote22gshadow22history22host22hosts22http_headers22id22ip_route22jobs22lsmod22m3u22os_release22pkg_index_apk22proc_consoles22proc_cpuinfo22proc_net_tcp22proc_stat22proc_zoneinfo22shadow22srt22swapon22systemctl22systemctl_ls22systemctl_luf22toml22tsv_ih22tsv_ih_s22wg_show22x509_csr22yaml22zpool_iostat22airport11airport_s11apt_cache_show11apt_get_sqq11cef11cef_s11chage11debconf_show11gpg11hash11lsb_release11openvpn11pgpass11pkg_index_deb11postconf11proc_buddyinfo11proc_crypto11proc_devices11proc_diskstats11proc_driver_rtc11proc_filesystems11proc_interrupts11proc_iomem11proc_ioports11proc_loadavg11proc_locks11proc_meminfo11proc_modules11proc_mtrr11proc_net_arp11proc_net_dev11proc_net_dev_mcast11proc_net_if_inet611proc_net_igmp11proc_net_igmp611proc_net_ipv6_route11proc_net_netlink11proc_net_netstat11proc_net_packet11proc_net_protocols11proc_net_route11proc_net_unix11proc_pagetypeinfo11proc_partitions11proc_pid_fdinfo11proc_pid_io11proc_pid_maps11proc_pid_mountinfo11proc_pid_numa_maps11proc_pid_smaps11proc_pid_stat11proc_pid_statm11proc_pid_status11proc_slabinfo11proc_softirqs11proc_swaps11proc_uptime11proc_vmallocinfo11proc_vmstat11rpm_qi11syslog11syslog_s11systemctl_lj11tracepath11tsv11tsv_s11tune2fs11udevadm11x509_crl11xrandr11

Check it yourself

None of this is a claim you have to take on trust. Clone the repository, pull the reference submodule and run the same harness CI runs.

git clone --recurse-submodules https://github.com/OlegSotnikov/jc-rs.git
cd jc-rs
make submodule deps-py
make differential

CI runs it with --fail-under 100, so the number cannot drop without the build going red.