STDIN, STDOUT, and STDERR

Have started scripting again, and have forgotten all about stdin, stdout, and stderr - and the various syntaxes. Time for a refresher.

Ok, so if I was to do an ELI5:
Every process is initialized with three open file-descriptors: stdin (standard input), stdout (standard output), and stderr (standard error).

• stdin (file handle 0) is where the process gets input from you (usually from the keyboard or from pipes)
• stdout (file handle 1) is where process output is displayed (usually sent to the console or a file or a pipe)
• stderr (file handle 2) is basically stdout for errors (usually to stdout unless redirected elsewhere)

Let's take this for example:
myscript <infile 2>errfile

• A process is created for myscript.
• infile is opened as standard input for myscript (file handle 0)
• errfile is opened as standard error (file handle 2)
• myscript output is sent to /dev/tty by default (file handle 1)

Separating stderr from stdout is a great idea when you know command execution will result in errors, and you don't want them cluttering up valid entries in your stdout. Great example is when you're grepping for something. Most of the time, you get one or two hits, yet stdout is filled with "Permission denied" entries for protected directories. What to do?

grep needle /home/haystack/* > results.txt
Valid results redirected to results.txt; bad results displayed on the console.

grep needle /home/haystack/* 2> errors.log
This time, stderr (2) is redirected to errors.log, while valid results are displayed on the console.

grep needle /home/haystack/* &> output.txt
With this, everything (stdout and stderr) is redirected to output.txt, with nothing sent to tty.

If you want all results displayed on screen and saved on file, pipe them to tee:
grep needle /home/haystack/* |& tee output.txt

If your version of bash is giving you a syntax error, try this less elegant version:
grep needle /home/haystack/* 2>&1 | tee output.txt
This basically redirects stderr (2) to stdout (1).
1>&2 redirects stdout (1) to stderr (2).

What if you simply do:
grep needle /home/haystack/*  | tee results.txt
You get stdout on the screen and in the file, but the console will also display stderr.

Note that the above examples will overwrite if the file exists. Use >> and tee -a to append.