Piping Data Into a CLI AI Agent

Here's something most people never realize about terminal agents: they're Unix filters. The same agent you chat with interactively will also read from standard input and write to standard output, which means you can drop it into a pipeline between

grep

jq

This guide shows you how to do it well, because the naive version wastes tokens and the good version is genuinely powerful.

Quick-Start: Copy This Right Now

The core move is

-p

[object Object], error.log | claude -p ,[object Object],

What this does: Pipes a log file into Claude Code in one-shot mode. The agent reads the piped text as its input, produces a summary, and exits — no interactive session, no UI. It behaves like

sort

uniq

That single pattern —

cat something | agent -p "instruction"

⚡ Pro tip: Redirect the output too, and you've got a complete pipeline stage.

cat data.csv | claude -p "flag rows with negative balances" > flagged.txt

Pipe Data the CLI Agent Understands

The counterintuitive part is that more data is often worse. When you pipe data the CLI agent must process, every byte becomes tokens you pay for and context the model has to sift. Piping a 200MB log file doesn't give you a better summary — it gives you a huge bill and a truncated, confused answer. The skill is pre-filtering with the deterministic tools before the agent stage.

grep ERROR app.log | ,[object Object], -500 | claude -p ,[object Object],

What this does: Uses

grep

tail

This is the inversion beginners miss. The agent is the most expensive stage in your pipeline, so it should receive the least data, not the most. Let

grep

awk

jq

head

⚡ Pro tip: Ask for structured output when the next stage is another program. Adding

--output-format json

jq

Step-by-Step: Building a Real Pipeline

Let's build something useful — a pipeline that triages a failing test run.

Step one: capture the raw input deterministically.

pytest 2>&1 | ,[object Object], test-output.txt

What this does: Runs the tests and saves the full output to a file while also showing it. You want the raw material captured before the agent touches it, so you can re-run the agent stage without re-running the tests.

Step two: filter to the relevant slice.

grep -A 5 FAILED test-output.txt | ,[object Object], -100

What this does: Pulls the failure lines plus five lines of context each, capped at 100 lines. This is the pre-filter — the agent will only see failures, not the thousands of passing-test lines.

Step three: pipe the filtered slice through the agent for the judgment call.

grep -A 5 FAILED test-output.txt | ,[object Object], -100 | \
  claude -p ,[object Object], \
  --output-format json > triage.json

What this does: Sends only the failure slice to the agent and asks for structured triage as JSON. The agent does the one thing the deterministic tools can't — reason about shared root causes — and writes a machine-readable result the next stage can consume.

Step four: consume the structured output downstream. Now

triage.json

The Cost Math That Makes or Breaks a Pipeline

The economics of piping deserve their own moment, because they're where a clever pipeline turns into an expensive mistake. The agent stage bills by tokens, and tokens scale with input size. A pipeline that pipes a full day of logs through the agent every hour isn't a smart automation — it's a recurring bill that grows with your log volume. The deterministic pre-filter isn't just about answer quality; it's the difference between a pipeline that costs cents and one that costs dollars every single run, compounding across a schedule.

When the input genuinely is large and you can't filter it down, the move is to summarize in cheap passes and pipe the summaries, not the raw data. Split the input into chunks, run a cheap model over each to extract just the relevant signal, then pipe the combined signal to the expensive model for the final judgment.

[object Object],
,[object Object], -l 1000 huge.log chunk_
,[object Object], c ,[object Object], chunk_*; ,[object Object],
  ,[object Object], ,[object Object], | claude -p ,[object Object], --model haiku >> signal.txt
,[object Object],
,[object Object], signal.txt | claude -p ,[object Object],

What this does: Runs a cheap, fast model over each chunk to distill the signal, then a single expensive pass to reason over the distilled result. You pay the premium model once, over a small input, instead of over the whole raw haystack — the same map-reduce pattern that makes big-data processing affordable, applied to agents.

⚡ Pro tip: Route pipeline stages to the cheapest model that can do each stage's job. Extraction and filtering rarely need a frontier model; the final judgment might. Assigning a cheap model to the mechanical stages and reserving the expensive one for the single reasoning step is the biggest cost lever you have in any agent pipeline.

Pro-Level Variations

A site-reliability engineer pipes the last hour of structured logs through an agent that classifies incidents by severity, writing a JSON summary that a downstream script routes to the right on-call channel. The agent handles the fuzzy classification; the routing stays deterministic.

A data analyst pipes a messy CSV through an agent to normalize inconsistent category labels ("NY", "New York", "new_york" → "New York"), then pipes the cleaned output straight into their normal analysis tools. The agent is a smart

sed

A security engineer pipes

git log

git ,[object Object], --since=,[object Object], --oneline | \
  claude -p ,[object Object], \
  --output-format json

What this does: Pipes recent commit subjects through the agent to surface ones worth a closer look, as structured data. It's judgment applied at scale, wired into a pipeline that acts on the result.

⚡ Pro tip: Cap the work with

--max-turns 1

Troubleshooting Common Issues

⚠️ Common mistake: Piping raw, unfiltered data and blaming the agent for a vague answer. If you pipe a 10,000-line log with no pre-filter, the agent either truncates it (missing the part you cared about) or drowns in noise. The answer quality problem is almost always a data-volume problem. Fix the pipe, not the prompt: filter harder before the agent stage.

The second issue is non-deterministic output breaking downstream parsing. If your next stage expects JSON and the agent occasionally wraps it in prose or markdown fences, your pipeline breaks intermittently. Be explicit — "output only valid JSON, no markdown, no preamble" — and strip fences defensively in the consuming script.

⚡ Pro tip: Test the agent stage in isolation before wiring the whole chain. Save a representative input to a file, pipe just that file through the agent, and confirm the output shape is stable across a few runs. A pipeline is only as reliable as its least predictable stage, and the agent is usually that stage — so pin its behavior down alone before you trust it in the chain.

For long-running agent stages that produce output gradually,

--output-format stream-json

Your Turn

When you pipe data the CLI agent can use, you stop treating it as a chat window and start treating it as a composable transform — the reasoning stage in a pipeline of deterministic tools. Filter hard before it, ask for structured output after it, cap the turns, and it slots between

grep

jq

The pipelines that work — the filter-then-reason chains, the JSON-output prompts, the triage one-liners — are worth saving as reusable snippets. Keep them in PromptABCD so your next automation starts from a proven pipeline stage instead of rediscovering that the secret is feeding the agent less, not more.