Output redirection empties the file before the command reads it

Output redirection empties the file before the command reads it

reads as

`sort data.txt > data.txt` exits zero and data.txt is still there. Conclusion drawn: the file was sorted in place.

actually

The shell performs redirections before running the command, and for output redirection 'if the file does not exist it is created; if it does exist it is truncated to zero size'. sort then opens an empty file, reads nothing and writes nothing, correctly and successfully. The original contents are gone.

blind because

The exit code belongs to a command that did exactly what was asked of it with the input it was given. The destruction happened in the shell, before the command started, and produced no status of its own.

the check

Compare the line count before and after in the same command. Observed on bash 5.2.21: a three-line data.txt held zero lines after `sort data.txt > data.txt`, with sort exiting 0; `grep -v DEBUG conf.txt > conf.txt` left conf.txt at zero bytes, with grep exiting 1 because it had nothing to match.

cost of missing

The file that was supposed to be filtered is now empty, and emptiness is valid input to whatever reads it next: configuration becomes all-defaults, a dataset becomes zero records, and neither state raises an error.

mitigation

Write to a new name and rename over the original, or use a tool with an explicit in-place mode such as `sed -i` or `sponge`.

generalises to

Every operation that opens its destination before reading its source: in-place archive rewrites, dumps piped over their own file, copies where source and destination alias.

source

gnu.org