sed

Overview

sed is a command that parses and transforms text, line-by-line. It is a form of stream-editing.

sed’s first parameter is an input script which describes how to transform the text. It can either be passed on the command-line ( -e option, and is the default), or through a file ( -f option).

sed’s second parameter is the input stream to operate on. By default, sed prints output to stdout . This can be redirected with the >> operator, to say, a file, as shown in this example:

Bruce Barnett’s Cheat Sheet for SED.

The -exec option is used so that you can execute a shell command on every output.

Comments

sed allows you to put comments inside of script. Comments begin with the # character.

Substitution

One of the most popular operations is the substitution operation, s. It accepts a regular expression for the search pattern and standard text for replacement. To replace some (but probably not all, see the part below on sed’s line by line processing) occurances of the word old with the word new in inputFile.txt and save it to outputFile.txt, you would write:

Now, the part where I mentioned that sed will replace some but probably not all occurances…well sed operates line-by-line on text, and it’s default behavour once a match has been found is to print the result, and start with the next line of text. This means any further occurances of the word old on the same line will not get replace by the word new.

To get around this, you can use the global replacement character ( g).

The Delimiter

The forward slash is conventional delimiter, used to separate variables in a sed search pattern. However, the forward slash can get confusing when trying to manipulate path names, as these have forward slashes also, which means you need to escape them with a prefixed backslash. This can quickly get messy (the picket fence look):

Deleting Lines

You can delete lines from files using the pattern ‘3d’, which will delete the 3rd line from the file, or ‘3,5d’ which delete lines 3, 4 and 5. So the full command for deleting the 3rd line would be:

To remove the last name of a file:

d is the delete operator. D is another delete operator, although it’s behavour is slightly different (it doesn’t print what’s in the current pattern space).

Pattern Found Variable (&)

When using the substitution command, the & character represents the found pattern. It can be used in the replacement string to insert the pattern found into the replacement text. This is useful for say, when you want to modify some text instead of replacing it completely.

A simple example is if you wanted to double a certain string of text, anywhere it was found.

This command finds any 2, or series of 2’s (that is what the regex 22* does), and doubles the found string (this is what the && does).

Complex Examples

These examples may combine sed with other programs such as find and xargs.

Variables in total upper-case are designed to be modified to suit your own needs.

Find All C++ Files, Copy To New Directory, Then Replace Strings

This is useful if you want to copy a directory with C++ file inside, and then globally find/replace strings without touching the original files. This assumes C++ header files have the .hpp file extension (you can change this if needed).

Change The Format Of Dates

This example looks for all occurences of a date in the format 2014/11/03 and replaces it with the format 2014-11-03. It does not check to make sure it is a valid date (e.g. 2013/13/01 will still replaced, even though there is no 13th month).

External Resources

Sed – An Introduction And Tutorial by Bruce Barnett is a very detailed page on sed. It is really easy to follow due to it’s step-by-step approach to increased complexity.

Posted: January 6th, 2014 at 8:50 am
Last Updated on: January 23rd, 2017 at 11:22 pm