FileIO.md

File Operations

One of the most useful aspects of shells like Bash is how we can manipulate files. Let's see how it's done!

Redirection

Many programs produce a text output. By default, this output is displayed on your terminal. We can redirect this output in a couple ways.

The > operator redirects the output of a program to a file, completely replacing its contents. For instance, ls > list.txt would create a new file with a list of other files in the current folder.

The >> operator works similarly, accept that it appends the output to the end of the file instead of rewriting it. If you ran ls >>list.txt twice, list.txt would contain two complete lists of files.

The < operator instead reads a file into a program's input. wc -l <myfile> will print the number of lines in <myfile>

Lastly, the | (pipe) redirects the program's output, but instead to a second program instead of a file. For example, ls | head -n 1 will print the first file that ls finds. You can chain multiple pipes together.

Useful Programs

cat simply takes every input and sends it to its output unchanged.

echo prints each of its arguments to the output.

diff compares two files.

Filters

A filter is a program that manipulates streams of input text and produces more streams of output text. Most of these programs can operate either on an input (from a pipe or redirection) or on a filepath.

sort is a filter that sorts each line of its input.

uniq removes adjacent matching lines from its input, while uniq -d lists only lines that are duplicated.

wc (for word count) gives the length of its inputs in lines, words, and bytes.

tee copies its input to one or more outputs.

head and tail print the first and last few lines of their inputs respectively. Both take an -n <linecount> flag to control how many lines are displayed.

grep accepts as an argument a pattern to search for, and outputs matching lines from its input.

Exercise

Technically, we can now edit any text file, though rather tediously.

1. Use echo and the >> operator to create a file. Append a few different lines to the file.
2. run sort < myfile
3. run sort < myfile | uniq
4. run wc -l myfile. The -l flag specifies that we want to count lines.

Now, choose a word that appears in your file. Use grep <myword> to search for lines containing that word.

Try the same search again, but using grep -v. What is different about your output?

Use grep to filter out certain lines of your file, making sure to redirect the output to a new file. Compare the two with diff.