How to convert logs to JSON without guessing the schema

Choose the log parser by format

Before converting a log file to JSON, inspect a few complete records. The producer and its configured output format matter; the filename does not. Syslog, CLF, and CEF are separate grammars with different fields and escaping rules.

The streaming variants have their own parser names ending in -s. They emit one JSON value per input record. The -u option tells jc-rs to flush each value as soon as it is ready.

A complete access-log conversion

This Combined Log Format record contains an address, timestamp, HTTP request, status, byte count, referrer, and user agent. Splitting on spaces would break as soon as a quoted value contains a space.

203.0.113.7 - - [11/Aug/2026:14:22:09 +0000] "GET /health HTTP/1.1" 503 19 "-" "curl/8.7.1"

jc-rs parses the full record. jq is used only after that conversion to make the example output shorter:

printf '%s\n' '203.0.113.7 - - [11/Aug/2026:14:22:09 +0000] "GET /health HTTP/1.1" 503 19 "-" "curl/8.7.1"' |
  jc-rs --clf |
  jq '.[0] | {host, request_method, request_url, status, bytes}'

{
  "host": "203.0.113.7",
  "request_method": "GET",
  "request_url": "/health",
  "status": 503,
  "bytes": 19
}

Convert a finished file or follow a live one

Finished file: write one JSON array

Use the regular parser when the input has an end. The result is one valid JSON array, convenient for archiving or passing to a program that expects a complete document.

jc-rs --clf < /var/log/nginx/access.log > access.json
jc-rs --syslog < exported-syslog.log > syslog.json
jc-rs --cef < security-events.cef > security-events.json

Growing file: emit NDJSON immediately

A file followed by tail -F may never reach EOF. Use a streaming parser so each complete line becomes a JSON object while the writer stays open. jq accepts successive JSON values and, with -c, writes one compact result per line.

tail -F /var/log/nginx/access.log |
  jc-rs -u --clf-s |
  jq -c 'select(.status >= 500)'

This output is NDJSON, not a JSON array. Keep it line-oriented for another streaming consumer, or collect it later with jq -s '.'. See the NDJSON, JSONL and JSON guide for the memory and recovery trade-offs.

If the producer already emits JSON, keep it

Parsing is for existing human-readable logs and systems whose output you cannot change. When you control the application, configure structured logging at the source. When a command already has a stable JSON mode, send that JSON straight to jq.

journalctl -o json --since today |
  jq -c 'select(.PRIORITY == "3")'

Do not run already-valid JSON through --syslog, and do not serialize JSON inside a message string if the logger can emit fields natively. Source-side structure preserves types and avoids a parsing step altogether.

Validate a sample before converting the archive

Successful parsing guarantees valid JSON serialization, not a correct interpretation of the source. Check that the input matched the expected grammar and that the fields mean what your downstream job assumes.

1. 01Run ten representative records, including missing fields, unusual user agents, and error lines. Compare the parsed values with the originals.
2. 02Count records carrying an unparsable field. The CLF and syslog parsers preserve lines they cannot classify instead of quietly inventing fields.
3. 03Keep diagnostics on stderr. Never merge stderr into the log stream with 2>&1; an error message is not a log record in the selected grammar.
4. 04Inspect timestamps and nulls explicitly. Not every source carries a UTC offset or every optional CLF field, and a missing value should stay missing.

jq '{
  records: length,
  unparsable: [.[] | select(has("unparsable"))] | length
}' access.json

Log files often contain credentials, session identifiers, and customer data. A local CLI conversion keeps the file on the machine; still apply the same access controls to the JSON output as to the original log.

Queries worth keeping

Count HTTP statuses

jq 'group_by(.status)
  | map({status: .[0].status, requests: length})
  | sort_by(-.requests)' access.json

Find high-severity CEF events

jc-rs --cef < security-events.cef |
  jq '[.[] | select((.agentSeverityNum // 0) >= 7)]'

Watch emergency through error syslog severities

tail -F exported-syslog.log |
  jc-rs -u --syslog-s |
  jq -c 'select(.priority != null and (.priority % 8) <= 3)'

The syslog PRI value combines facility and severity. Taking it modulo 8 recovers the severity number; 0 through 3 mean emergency, alert, critical, and error.