Shell pipeline guide
Use jc-rs and jq safely in Bash
jq reads JSON; it does not parse the columns printed by ps, df, or ss. When the command has no suitable JSON mode, jc-rs establishes the schema before jq runs. A production Bash script also needs to preserve quoting, keep stderr out of the data, and notice failures from every stage.
By Oleg Sotnikov · Published · Commands checked with Bash, jq 1.7, and the jc-rs CLI
jc-rs creates JSON; jq queries it
A reliable Bash and jq pipeline has three contracts. The command writes the format its parser expects. jc-rs converts that format to a documented JSON schema. jq selects, reshapes, or renders values from that schema.
LC_ALL=C ps aux |
jc-rs --ps |
jq 'map(select(.mem_percent >= 5))'ps auxProcess collection
Not: JSON quoting or field names
jc-rs --psColumns, types, nulls, and JSON serialization
Not: Which processes you want
jqFiltering and output shape
Not: The human-readable ps grammar
If the producer has a stable native JSON flag, prefer it and omit jc-rs. The extra parser is useful only when human-readable output is the available interface.
Decide whether the next stage needs an array
Most non-streaming command parsers return an array of records. jq can preserve that array for another JSON consumer, or unwrap it into a stream for text processing.
map(select(.use_percent >= 80))one JSON array
API payload, file, or another JSON stage
.[] | select(.use_percent >= 80)successive JSON objects
streaming into another jq-aware command
-r '.[] | .mounted_on'raw text lines
human output or a line-oriented shell command
-c '.[]'compact JSON, one value per line
NDJSON output
LC_ALL=C df -h |
jc-rs --df |
jq 'map(select(.use_percent >= 80))'LC_ALL=C df -h |
jc-rs --df |
jq -r '.[]
| select(.use_percent >= 80)
| [.mounted_on, ((.use_percent | tostring) + "%")]
| @tsv'Use -r only at the boundary where JSON becomes text. Keeping JSON longer avoids a second round of shell splitting and escaping.
Single-quote the jq program; pass data with arguments
In Bash, a jq program normally belongs in single quotes. That prevents the shell from expanding $variables, backslashes, and wildcard characters before jq sees them. Values from Bash should cross the boundary with --arg or --argjson, never by building jq source code.
String input
wanted_user=alice
LC_ALL=C ps aux |
jc-rs --ps |
jq --arg user "$wanted_user" 'map(select(.user == $user))'Numeric input
limit=${LIMIT:-80}
case $limit in
(''|*[!0-9]*) printf 'LIMIT must be an integer\n' >&2; exit 2 ;;
esac
LC_ALL=C df -h |
jc-rs --df |
jq --argjson limit "$limit" 'map(select(.use_percent >= $limit))'Avoid
Building a double-quoted jq program that interpolates $wanted_user.
Quotes or jq syntax inside the value can break the program or change its meaning.--arg serializes it as data instead.
Make an upstream failure fail the script
Bash normally reports the status of the last command in a pipeline. If the producer or jc-rs fails but jq exits successfully, the script can appear healthy. Enable pipefail and use jq's -e mode when the jq result itself is a condition.
#!/usr/bin/env bash
set -Eeuo pipefail
if ! {
LC_ALL=C df -h |
jc-rs --df |
jq -e '
if type == "array"
then map(select(.use_percent >= 90))
else error("expected an array from the df parser")
end
'
} >full-filesystems.json 2>pipeline.err; then
printf 'disk report failed; see pipeline.err\n' >&2
exit 1
fiset -e alone is not a substitute for pipefail. An empty JSON array is also a valid, truthy jq value; test its length separately if “no records” is an error for your job.
Keep stderr separate from structured data
Pipes carry stdout. That is exactly what you want: diagnostics remain on stderr while parseable data moves right. Redirect the whole pipeline's stderr to a separate file when a scheduled job needs a record of failures.
Good
{ command | jc-rs --parser | jq ...; } 2>pipeline.errThe parser receives only the command's stdout.
Bad
command 2>&1 | jc-rs --parserAn error message is mixed into the data and may become a bogus row.
Locale is part of the input contract too. For commands whose headings or numbers are localized, set LC_ALL=C on the producer, as the examples above do. It does not need to be exported for the whole script.
Define what empty input means for this job
Parsers do not all treat empty input identically: for some formats an empty collection is meaningful; for others it is malformed input. After a successful parse, decide whether zero records is acceptable rather than relying on implicit jq behavior.
if ! LC_ALL=C ps aux |
jc-rs --ps |
jq -e --arg user "$wanted_user" '
map(select(.user == $user))
| if length > 0 then . else error("no matching process") end
' >processes.json
then
printf 'process lookup failed or returned no rows\n' >&2
exit 1
fiFor one scalar destined for a shell variable, use raw output and require a value. Keep the expansion quoted afterward:
pid=$(LC_ALL=C ps aux |
jc-rs --ps |
jq -er --arg user "$wanted_user" '
first(.[] | select(.user == $user)) | .pid
')
printf 'first PID: %s\n' "$pid"The same rules apply to a live stream
Streaming parsers produce NDJSON: each record is a complete JSON value. jq naturally reads that sequence. Use compact output to preserve the one-record-per-line contract.
tail -F /var/log/nginx/access.log |
jc-rs -u --clf-s |
jq -c 'select(.status >= 500)'Because this pipeline is intentionally long-lived, its final status is available only when it exits. Run production monitors under a supervisor that restarts failed processes and captures stderr.
Check the parser schema before writing jq
The parser pages show fixture-derived field names and output shapes. Check one before committing a jq expression to a script, especially when the source command varies by operating system.