Advanced sed: managing text streams in Linux. Using the sed streaming text editor in Linux Other sed editor commands

Last time we talked about functions in bash scripts, in particular how to call them from the command line. Our topic today is a very useful tool for processing string data - a Linux utility called sed. It is often used to work with texts in the form of log files, configuration and other files.

If you're doing any kind of data manipulation in bash scripts, you'll want to be familiar with the tools sed and gawk. Here we will focus on sed and working with texts, as this is a very important step in our journey through the vast expanses of bash script development.

Now we will look at the basics of working with sed, and also look at more than three dozen examples of using this tool.

Basics of working with sed

The sed utility is called a streaming text editor. In interactive text editors like nano, you work with texts using the keyboard, editing files, adding, deleting or changing texts. Sed allows you to edit data streams based on developer-specified rule sets. Here's what this command looks like:

$sed options file
By default, sed applies the rules specified when called, expressed as a set of commands, to STDIN . This allows you to pass data directly to sed.

For example, like this:

$ echo "This is a test" | sed "s/test/another test/"
This is what happens when you run this command.

A simple example of calling sed

In this case, sed replaces the word “test” in the line passed for processing with the words “another test”. To formulate a rule for processing text enclosed in quotation marks, forward slashes are used. In our case, we used a command like s/pattern1/pattern2/ . The letter “s” is an abbreviation of the word “substitute”, that is, we have a replacement command. Sed, executing this command, will look through the transmitted text and replace the fragments found in it (we'll talk about which ones below) corresponding to pattern1 with pattern2.

Above is a primitive example of using sed to get you up to speed. In fact, sed can be used in much more complex text processing scenarios, such as working with files.

Below is a file containing a piece of text and the results of processing it with this command:

$ sed "s/test/another test" ./myfile

Text file and the results of its processing

This uses the same approach we used above, but now sed processes the text stored in the file. However, if the file is large enough, you will notice that sed processes the data in chunks and displays what is processed on the screen, without waiting for the entire file to be processed.

Sed does not change the data in the file it is processing. The editor reads the file, processes what it read, and sends the resulting output to STDOUT. In order to make sure that the source file has not changed, it is enough to open it after it has been transferred to sed. If necessary, sed output can be redirected to a file, possibly overwriting the old file. If you're familiar with one of the previous ones in this series, which deals with redirecting input and output streams, you should be able to do this.

Executing command sets when calling sed

To perform multiple actions on data, use the -e switch when calling sed. For example, here's how to organize the replacement of two text fragments:

$ sed -e "s/This/That/; s/test/another test/" ./myfile

Using the -e switch when calling sed

Both commands are applied to each line of text from the file. They must be separated by a semicolon, and there should be no space between the end of the command and the semicolon.
To enter multiple text processing patterns when calling sed, you can, after entering the first single quote, press Enter, and then enter each rule on a new line, not forgetting the closing quote:

$ sed -e " > s/This/That/ > s/test/another test/" ./myfile
This is what will happen after the command presented in this form is executed.

Another way to work with sed

Reading commands from a file

If you have a lot of sed commands that you need to process text with, it's usually best to pre-write them to a file. To specify a sed file containing commands, use the -f switch:

Here are the contents of the mycommands file:

S/This/That/ s/test/another test/
Let's call sed, passing the editor a file with commands and a file to process:

$ sed -f mycommands myfile
The result when calling such a command is similar to that obtained in the previous examples.

Using a command file when calling sed

Replace command flags

Take a close look at the following example.

$sed "s/test/another test/" myfile
This is what is contained in the file and what will be obtained after sed processes it.

Source file and results of its processing

The replace command normally handles a multi-line file, but only the first occurrences of the searched text on each line are replaced. In order to replace all occurrences of a pattern, you need to use the appropriate flag.

The scheme for writing a replacement command when using flags looks like this:

S/pattern/replacement/flags
The execution of this command can be modified in several ways.

When transmitting the number, the serial number of the occurrence of the pattern in the string is taken into account, and it is this occurrence that will be replaced.
The g flag indicates that all occurrences of the pattern in the string should be processed.
The p flag indicates that the contents of the original string should be printed.
The w file flag tells the command to write the text processing results to a file.

Let's consider using the first version of the replacement command, indicating the position of the replaced occurrence of the required fragment:

$ sed "s/test/another test/2" myfile

Calling the replacement command indicating the position of the fragment to be replaced

Here we specified the number 2 as the replacement flag. This led to the fact that only the second occurrence of the desired pattern in each line was replaced. Now let's try the global replacement flag - g:

$ sed "s/test/another test/g" myfile
As can be seen from the output results, such a command replaced all occurrences of the pattern in the text.

Global replacement

The replace command flag p allows you to print lines where matches are found, while the -n option specified when calling sed suppresses normal output:

$ sed -n "s/test/another test/p" myfile
As a result, when sed is launched in this configuration, only the lines (in our case, one line) in which the specified text fragment is found are displayed.

Using the replace command flag p

Let's use the w flag, which allows you to save the results of text processing to a file:

$ sed "s/test/another test/w output" myfile

Saving text processing results to a file

It can be clearly seen that during the operation of the command, the data is output to , while the processed lines are written to the file whose name is indicated after w .

Separator characters

Imagine replacing /bin/bash with /bin/csh in the /etc/passwd file. The task is not that difficult:

$ sed "s/\/bin\/bash/\/bin\/csh/" /etc/passwd
However, this doesn't look very good. The thing is that since forward slashes are used as delimiter characters, the same characters in the strings passed to sed have to be escaped. As a result, the readability of the command suffers.

Luckily, sed allows us to define our own delimiter characters to use in the replace command. The separator is the first character that appears after s:

$ sed "s!/bin/bash!/bin/csh!" /etc/passwd
In this case, an exclamation mark is used as a separator, as a result the code is easier to read and looks much neater than before.

Selecting text fragments for processing

So far, we have called sed to process the entire stream of data passed to the editor. In some cases, using sed you need to process only some part of the text - a specific line or group of lines. To achieve this goal, you can use two approaches:

Set a limit on the numbers of rows to be processed.
Specify the filter whose corresponding rows need to be processed.

Let's consider the first approach. There are two possible options here. The first, discussed below, involves specifying the number of one line that needs to be processed:

$sed "2s/test/another test/" myfile

Processing only one line, the number specified when calling sed

The second option is a range of strings:

$ sed "2.3s/test/another test/" myfile

Processing a range of rows

In addition, you can call the replace command so that the file is processed from a certain line to the end:

$ sed "2,$s/test/another test/" myfile

Processing the file from the second line to the end

In order to use the replace command to process only rows that match a given filter, the command must be called like this:

$ sed "/likegeeks/s/bash/csh/" /etc/passwd
By analogy with what was discussed above, the template is passed before the command name s .

Processing rows that match a filter

Here we used a very simple filter. In order to fully exploit the capabilities of this approach, you can use regular expressions. We will talk about them in one of the following materials in this series.

Removing rows

The sed utility is useful not only for replacing one sequence of characters in strings with another. With its help, namely, using the d command, you can delete lines from a text stream.

The command call looks like this:

$sed "3d" myfile
We want the third line to be removed from the text. Please note that this is not a file. The file will remain unchanged, the deletion will only affect the output generated by sed.

Removing the third line

If you do not specify the line number to delete when calling the d command, all lines in the stream will be deleted.

Here's how to apply the d command to a range of strings:

$sed "2,3d" myfile

Delete a range of rows

Here's how to delete lines starting from the given one and up to the end of the file:

$sed "3,$d" myfile

Deleting lines to the end of the file

Rows can also be deleted using the following pattern:

$sed "/test/d" myfile

Removing rows using a pattern

When calling d, you can specify a pair of patterns - the lines in which the pattern occurs, and those lines that are between them, will be deleted:

$ sed "/second/,/fourth/d" myfile

Removing a range of rows using wildcards

Inserting text into a stream

With sed you can insert data into a text stream using the i and a commands:

The i command adds a new line before the given one.
The a command adds a new line after the given one.

Let's look at an example of using the i command:

$ echo "Another test" | sed "i\First test"

Team i

Now let's look at the a command:

$ echo "Another test" | sed "a\First test"

Command a

As you can see, these commands add text before or after the data from the stream. What if you need to add a line somewhere in the middle?

Here we will be helped by indicating the number of the reference line in the stream, or template. Please note that addressing strings as a range will not work here. Call the i command, specifying the line number before which you need to insert a new line:

$ sed "2i\This is the inserted line." myfile

Command i with reference line number

Let's do the same with the a command:

$ sed "2a\This is the appended line." myfile

Command a with reference line number

Note the difference in how the i and a commands work. The first one inserts a new line before the specified one, the second - after it.

Replacing strings

The c command allows you to change the contents of an entire line of text in a data stream. When calling it, you need to specify the line number, instead of which new data should be added to the stream:

$ sed "3c\This is a modified line." myfile

Replacing an entire string

If you use a pattern in the form of plain text or a regular expression when calling a command, all strings matching the pattern will be replaced:

$ sed "/This is/c This is a changed line of text." myfile

Replacing strings using a pattern

Replacing characters

The y command works with individual characters, replacing them according to the data passed to it when called:

$sed "y/123/567/" myfile

Replacing characters

When using this command, you need to take into account that it applies to the entire text stream; it cannot be limited to specific occurrences of characters.

Displaying line numbers

If you call sed using the = command, the utility will print the line numbers in the data stream:

$sed "=" myfile

Displaying line numbers

The stream editor displayed line numbers before their contents.

If you pass a pattern to this command and use the sed -n switch, only line numbers matching the pattern will be printed:

$ sed -n "/test/=" myfile

Printing line numbers matching a pattern

Reading insert data from a file

Above, we looked at techniques for inserting data into a stream, indicating what needs to be inserted directly when calling sed. You can also use a file as a data source. To do this, use the r command, which allows you to insert data from a specified file into a stream. When calling it, you can specify the line number after which you want to insert the contents of the file, or a pattern.

Let's look at an example:

$sed "3r newfile" myfile

Inserting file content into a stream

Here the contents of the newfile file have been inserted after the third line of the myfile file.

This is what happens if you use the pattern when calling the r command:

$ sed "/test/r newfile" myfile

Using a wildcard when calling the r command

The contents of the file will be inserted after each line that matches the pattern.

Example

Let's imagine such a task. There is a file in which there is a certain sequence of characters, which in itself is meaningless, which must be replaced with data taken from another file. Namely, let this be a newfile file in which the character sequence DATA plays the role of a placeholder. The data that needs to be substituted for DATA is stored in the data file.

You can solve this problem by using the r and d commands of the sed stream editor:

$ Sed "/DATA>/ ( r newfile d)" myfile

Replacing the placeholder with real data

As you can see, instead of the DATA placeholder, sed added two lines from the data file to the output stream.

Results

Today we looked at the basics of working with the sed stream editor. In fact, sed is a huge topic. Learning it can be compared to learning a new programming language, but once you understand the basics, you can master sed at any level you need. As a result, your ability to process texts with it will be limited only by your imagination.

That's all for today. Next time we'll talk about the awk data processing language.

Dear readers! Do you use sed in your daily work? If yes, please share your experience.

Now we will look at the basics of working with sed, and also look at more than three dozen examples of using this tool.

Basics of working with sed

$sed options file
By default, sed applies the rules specified when called, expressed as a set of commands, to STDIN . This allows you to pass data directly to sed.

For example, like this:

$ echo "This is a test" | sed "s/test/another test/"
This is what happens when you run this command.

A simple example of calling sed

Above is a primitive example of using sed to get you up to speed. In fact, sed can be used in much more complex text processing scenarios, such as working with files.

Below is a file containing a piece of text and the results of processing it with this command:

$ sed "s/test/another test" ./myfile

Text file and the results of its processing

Sed does not change the data in the file it is processing. The editor reads the file, processes what it read, and sends the resulting output to STDOUT. In order to make sure that the source file has not changed, it is enough to open it after it has been transferred to sed. If necessary, sed output can be redirected to a file, possibly overwriting the old file. If you are familiar with one of the previous articles in this series, which talks about redirecting input and output streams, you should be able to do this.

Executing command sets when calling sed

To perform multiple actions on data, use the -e switch when calling sed. For example, here's how to organize the replacement of two text fragments:

$ sed -e "s/This/That/; s/test/another test/" ./myfile

Using the -e switch when calling sed

$ sed -e " > s/This/That/ > s/test/another test/" ./myfile
This is what will happen after the command presented in this form is executed.

Another way to work with sed

Reading commands from a file

If you have a lot of sed commands that you need to process text with, it's usually best to pre-write them to a file. To specify a sed file containing commands, use the -f switch:

Here are the contents of the mycommands file:

S/This/That/ s/test/another test/
Let's call sed, passing the editor a file with commands and a file to process:

$ sed -f mycommands myfile
The result when calling such a command is similar to that obtained in the previous examples.

Using a command file when calling sed

Replace command flags

Take a close look at the following example.

$sed "s/test/another test/" myfile
This is what is contained in the file and what will be obtained after sed processes it.

Source file and results of its processing

The scheme for writing a replacement command when using flags looks like this:

S/pattern/replacement/flags
The execution of this command can be modified in several ways.

When transmitting the number, the serial number of the occurrence of the pattern in the string is taken into account, and it is this occurrence that will be replaced.
The g flag indicates that all occurrences of the pattern in the string should be processed.
The p flag indicates that the contents of the original string should be printed.
The w file flag tells the command to write the text processing results to a file.

Let's consider using the first version of the replacement command, indicating the position of the replaced occurrence of the required fragment:

$ sed "s/test/another test/2" myfile

Calling the replacement command indicating the position of the fragment to be replaced

$ sed "s/test/another test/g" myfile
As can be seen from the output results, such a command replaced all occurrences of the pattern in the text.

Global replacement

The replace command flag p allows you to print lines where matches are found, while the -n option specified when calling sed suppresses normal output:

$ sed -n "s/test/another test/p" myfile
As a result, when sed is launched in this configuration, only the lines (in our case, one line) in which the specified text fragment is found are displayed.

Using the replace command flag p

Let's use the w flag, which allows you to save the results of text processing to a file:

$ sed "s/test/another test/w output" myfile

Saving text processing results to a file

It can be clearly seen that during the operation of the command, data is output to STDOUT, while the processed lines are written to a file whose name is indicated after w.

Separator characters

Imagine replacing /bin/bash with /bin/csh in the /etc/passwd file. The task is not that difficult:

Luckily, sed allows us to define our own delimiter characters to use in the replace command. The separator is the first character that appears after s:

$ sed "s!/bin/bash!/bin/csh!" /etc/passwd
In this case, an exclamation mark is used as a separator, as a result the code is easier to read and looks much neater than before.

Selecting text fragments for processing

Set a limit on the numbers of rows to be processed.
Specify the filter whose corresponding rows need to be processed.

Let's consider the first approach. There are two possible options here. The first, discussed below, involves specifying the number of one line that needs to be processed:

$sed "2s/test/another test/" myfile

Processing only one line, the number specified when calling sed

The second option is a range of strings:

$ sed "2.3s/test/another test/" myfile

Processing a range of rows

In addition, you can call the replace command so that the file is processed from a certain line to the end:

$ sed "2,$s/test/another test/" myfile

Processing the file from the second line to the end

In order to use the replace command to process only rows that match a given filter, the command must be called like this:

$ sed "/likegeeks/s/bash/csh/" /etc/passwd
By analogy with what was discussed above, the template is passed before the command name s .

Processing rows that match a filter

Removing rows

The sed utility is useful not only for replacing one sequence of characters in strings with another. With its help, namely, using the d command, you can delete lines from a text stream.

The command call looks like this:

$sed "3d" myfile
We want the third line to be removed from the text. Please note that this is not a file. The file will remain unchanged, the deletion will only affect the output generated by sed.

Removing the third line

If you do not specify the line number to delete when calling the d command, all lines in the stream will be deleted.

Here's how to apply the d command to a range of strings:

$sed "2,3d" myfile

Delete a range of rows

Here's how to delete lines starting from the given one and up to the end of the file:

$sed "3,$d" myfile

Deleting lines to the end of the file

Rows can also be deleted using the following pattern:

$sed "/test/d" myfile

Removing rows using a pattern

When calling d, you can specify a pair of patterns - the lines in which the pattern occurs, and those lines that are between them, will be deleted:

$ sed "/second/,/fourth/d" myfile

Removing a range of rows using wildcards

Inserting text into a stream

With sed you can insert data into a text stream using the i and a commands:

The i command adds a new line before the given one.
The a command adds a new line after the given one.

Let's look at an example of using the i command:

$ echo "Another test" | sed "i\First test"

Team i

Now let's look at the a command:

$ echo "Another test" | sed "a\First test"

Command a

As you can see, these commands add text before or after the data from the stream. What if you need to add a line somewhere in the middle?

$ sed "2i\This is the inserted line." myfile

Command i with reference line number

Let's do the same with the a command:

$ sed "2a\This is the appended line." myfile

Command a with reference line number

Note the difference in how the i and a commands work. The first one inserts a new line before the specified one, the second - after it.

Replacing strings

$ sed "3c\This is a modified line." myfile

Replacing an entire string

If you use a pattern in the form of plain text or a regular expression when calling a command, all strings matching the pattern will be replaced:

$ sed "/This is/c This is a changed line of text." myfile

Replacing strings using a pattern

Replacing characters

The y command works with individual characters, replacing them according to the data passed to it when called:

$sed "y/123/567/" myfile

Replacing characters

When using this command, you need to take into account that it applies to the entire text stream; it cannot be limited to specific occurrences of characters.

Displaying line numbers

If you call sed using the = command, the utility will print the line numbers in the data stream:

$sed "=" myfile

Displaying line numbers

The stream editor displayed line numbers before their contents.

If you pass a pattern to this command and use the sed -n switch, only line numbers matching the pattern will be printed:

$ sed -n "/test/=" myfile

Printing line numbers matching a pattern

Reading insert data from a file

Let's look at an example:

$sed "3r newfile" myfile

Inserting file content into a stream

Here the contents of the newfile file have been inserted after the third line of the myfile file.

This is what happens if you use the pattern when calling the r command:

$ sed "/test/r newfile" myfile

Using a wildcard when calling the r command

The contents of the file will be inserted after each line that matches the pattern.

Example

You can solve this problem by using the r and d commands of the sed stream editor:

$ Sed "/DATA>/ ( r newfile d)" myfile

Replacing the placeholder with real data

As you can see, instead of the DATA placeholder, sed added two lines from the data file to the output stream.

Results

That's all for today. Next time we'll talk about the awk data processing language.

Dear readers! Do you use sed in your daily work? If yes, please share your experience.

Now we will look at the basics of working with sed, and also look at more than three dozen examples of using this tool.

Basics of working with sed

$sed options file
By default, sed applies the rules specified when called, expressed as a set of commands, to STDIN . This allows you to pass data directly to sed.

For example, like this:

$ echo "This is a test" | sed "s/test/another test/"
This is what happens when you run this command.

A simple example of calling sed

Above is a primitive example of using sed to get you up to speed. In fact, sed can be used in much more complex text processing scenarios, such as working with files.

Below is a file containing a piece of text and the results of processing it with this command:

$ sed "s/test/another test" ./myfile

Text file and the results of its processing

Sed does not change the data in the file it is processing. The editor reads the file, processes what it read, and sends the resulting output to STDOUT. In order to make sure that the source file has not changed, it is enough to open it after it has been transferred to sed. If necessary, sed output can be redirected to a file, possibly overwriting the old file. If you are familiar with one of the previous articles in this series, which talks about redirecting input and output streams, you should be able to do this.

Executing command sets when calling sed

To perform multiple actions on data, use the -e switch when calling sed. For example, here's how to organize the replacement of two text fragments:

$ sed -e "s/This/That/; s/test/another test/" ./myfile

Using the -e switch when calling sed

$ sed -e " > s/This/That/ > s/test/another test/" ./myfile
This is what will happen after the command presented in this form is executed.

Another way to work with sed

Reading commands from a file

If you have a lot of sed commands that you need to process text with, it's usually best to pre-write them to a file. To specify a sed file containing commands, use the -f switch:

Here are the contents of the mycommands file:

S/This/That/ s/test/another test/
Let's call sed, passing the editor a file with commands and a file to process:

$ sed -f mycommands myfile
The result when calling such a command is similar to that obtained in the previous examples.

Using a command file when calling sed

Replace command flags

Take a close look at the following example.

$sed "s/test/another test/" myfile
This is what is contained in the file and what will be obtained after sed processes it.

Source file and results of its processing

The scheme for writing a replacement command when using flags looks like this:

S/pattern/replacement/flags
The execution of this command can be modified in several ways.

When transmitting the number, the serial number of the occurrence of the pattern in the string is taken into account, and it is this occurrence that will be replaced.
The g flag indicates that all occurrences of the pattern in the string should be processed.
The p flag indicates that the contents of the original string should be printed.
The w file flag tells the command to write the text processing results to a file.

Let's consider using the first version of the replacement command, indicating the position of the replaced occurrence of the required fragment:

$ sed "s/test/another test/2" myfile

Calling the replacement command indicating the position of the fragment to be replaced

$ sed "s/test/another test/g" myfile
As can be seen from the output results, such a command replaced all occurrences of the pattern in the text.

Global replacement

The replace command flag p allows you to print lines where matches are found, while the -n option specified when calling sed suppresses normal output:

$ sed -n "s/test/another test/p" myfile
As a result, when sed is launched in this configuration, only the lines (in our case, one line) in which the specified text fragment is found are displayed.

Using the replace command flag p

Let's use the w flag, which allows you to save the results of text processing to a file:

$ sed "s/test/another test/w output" myfile

Saving text processing results to a file

It can be clearly seen that during the operation of the command, data is output to STDOUT, while the processed lines are written to a file whose name is indicated after w.

Separator characters

Imagine replacing /bin/bash with /bin/csh in the /etc/passwd file. The task is not that difficult:

Luckily, sed allows us to define our own delimiter characters to use in the replace command. The separator is the first character that appears after s:

$ sed "s!/bin/bash!/bin/csh!" /etc/passwd
In this case, an exclamation mark is used as a separator, as a result the code is easier to read and looks much neater than before.

Selecting text fragments for processing

Set a limit on the numbers of rows to be processed.
Specify the filter whose corresponding rows need to be processed.

Let's consider the first approach. There are two possible options here. The first, discussed below, involves specifying the number of one line that needs to be processed:

$sed "2s/test/another test/" myfile

Processing only one line, the number specified when calling sed

The second option is a range of strings:

$ sed "2.3s/test/another test/" myfile

Processing a range of rows

In addition, you can call the replace command so that the file is processed from a certain line to the end:

$ sed "2,$s/test/another test/" myfile

Processing the file from the second line to the end

In order to use the replace command to process only rows that match a given filter, the command must be called like this:

$ sed "/likegeeks/s/bash/csh/" /etc/passwd
By analogy with what was discussed above, the template is passed before the command name s .

Processing rows that match a filter

Removing rows

The sed utility is useful not only for replacing one sequence of characters in strings with another. With its help, namely, using the d command, you can delete lines from a text stream.

The command call looks like this:

$sed "3d" myfile
We want the third line to be removed from the text. Please note that this is not a file. The file will remain unchanged, the deletion will only affect the output generated by sed.

Removing the third line

If you do not specify the line number to delete when calling the d command, all lines in the stream will be deleted.

Here's how to apply the d command to a range of strings:

$sed "2,3d" myfile

Delete a range of rows

Here's how to delete lines starting from the given one and up to the end of the file:

$sed "3,$d" myfile

Deleting lines to the end of the file

Rows can also be deleted using the following pattern:

$sed "/test/d" myfile

Removing rows using a pattern

When calling d, you can specify a pair of patterns - the lines in which the pattern occurs, and those lines that are between them, will be deleted:

$ sed "/second/,/fourth/d" myfile

Removing a range of rows using wildcards

Inserting text into a stream

With sed you can insert data into a text stream using the i and a commands:

The i command adds a new line before the given one.
The a command adds a new line after the given one.

Let's look at an example of using the i command:

$ echo "Another test" | sed "i\First test"

Team i

Now let's look at the a command:

$ echo "Another test" | sed "a\First test"

Command a

As you can see, these commands add text before or after the data from the stream. What if you need to add a line somewhere in the middle?

$ sed "2i\This is the inserted line." myfile

Command i with reference line number

Let's do the same with the a command:

$ sed "2a\This is the appended line." myfile

Command a with reference line number

Note the difference in how the i and a commands work. The first one inserts a new line before the specified one, the second - after it.

Replacing strings

$ sed "3c\This is a modified line." myfile

Replacing an entire string

If you use a pattern in the form of plain text or a regular expression when calling a command, all strings matching the pattern will be replaced:

$ sed "/This is/c This is a changed line of text." myfile

Replacing strings using a pattern

Replacing characters

The y command works with individual characters, replacing them according to the data passed to it when called:

$sed "y/123/567/" myfile

Replacing characters

When using this command, you need to take into account that it applies to the entire text stream; it cannot be limited to specific occurrences of characters.

Displaying line numbers

If you call sed using the = command, the utility will print the line numbers in the data stream:

$sed "=" myfile

Displaying line numbers

The stream editor displayed line numbers before their contents.

If you pass a pattern to this command and use the sed -n switch, only line numbers matching the pattern will be printed:

$ sed -n "/test/=" myfile

Printing line numbers matching a pattern

Reading insert data from a file

Let's look at an example:

$sed "3r newfile" myfile

Inserting file content into a stream

Here the contents of the newfile file have been inserted after the third line of the myfile file.

This is what happens if you use the pattern when calling the r command:

$ sed "/test/r newfile" myfile

Using a wildcard when calling the r command

The contents of the file will be inserted after each line that matches the pattern.

Example

You can solve this problem by using the r and d commands of the sed stream editor:

$ Sed "/DATA>/ ( r newfile d)" myfile

Replacing the placeholder with real data

As you can see, instead of the DATA placeholder, sed added two lines from the data file to the output stream.

Results

That's all for today. Next time we'll talk about the awk data processing language.

Dear readers! Do you use sed in your daily work? If yes, please share your experience.

Introduction

The sed command is a Stream EDitor for automatic text editing. "Stream editor" - in the sense that it can edit the incoming data stream continuously, say, as part of a program channel (pipe). Automatically - this means that as soon as you set the editing rules, the rest occurs without your tedious participation. In other words, the sed editor is not interactive.

The sed program is more complex than the commands that we have already discussed in previous articles in the HuMan series. It includes an arsenal of its own commands, therefore, in order to avoid tautology and confusion, in this article the sed command will henceforth be called a “program” or “editor”, and the sed editor commands will simply be called commands.

The sed program can do complex tasks, and it takes time to learn how to formulate these tasks.

But along with complex actions, the sed command has simple but very useful capabilities, which are no more difficult to master than other Unix commands. Don't allow yourself to be overwhelmed by the complexity of mastering the entire program.

We'll start from simple to complex, so you can always figure out where to stop.

Command s - substitution (replacement)

The sed program has many commands of its own. Most users only know the s command, and this is enough to work with the sed editor. The s command replaces PATTERN with REPLACE:

sed s/SAMPLE/REPLACEMENT/

$ echo day | sed s/day/night/ (Enter) night

It couldn't be simpler. And here is an example with input from the zar.txt file:

In the mornings he did exercises.

I didn't put s/SAMPLE/REPLACE/ in quotes because this example doesn't need quotes, but if it had metacharacters, then quotes would be required. In order not to break your head every time, and not to accidentally make a mistake, always put quotation marks, preferably the “stronger” single ones, this is a good habit. You can't spoil porridge with oil. I, too, will not skimp on quotation marks in all subsequent examples.

As we can see, the replacement command s has four components:

S the command itself /.../.../ separator PATTERN pattern for searching and subsequent replacement REPLACE expression that will replace the PATTERN if one is found.

The forward slash (/) is used as a separator by tradition, since sed's ancestor, the ed editor, uses them (as does the vi editor). In some cases, such a separator is very inconvenient, for example, when you need to change paths to directories that also contain a forward slash (/usr/local/bin). In this case, you have to separate the forward slashes with backslashes:

Sed "s/\/usr\/local\/bin/\/common\/bin/"

This is called a “picket fence” and looks very ugly, and most importantly, incomprehensible.

What's unique about sed is that it allows you to use any delimiter, such as the underscore:

$ echo day | sed s_day_night_ night

or colon:

$ echo day | sed s:day:night:night

If, while searching for a delimiter you like, you get the message "incomplete `s command", then this character is not a good separator, or you simply forgot to put a delimiter or two.

In this article, I have to use the traditional separator (/) to avoid confusing the reader, but if necessary, I will use the tilde (~) as a separator.

Regular expressions (RE)

(Regular expressions, regexp, RE)

The topic of regular expressions is so vast that entire books are devoted to it (see links at the end of the article). However, talking seriously about the sed editor without using regular expressions is as counterproductive as talking about trigonometry using adding sticks. Therefore, it is necessary to talk at least about those regular expressions that are often used with the sed program.

With Or any other letter. Most letters, numbers, and other non-special characters are considered regular expressions that represent themselves.

* An asterisk following any symbol or regular expression means any number (including zero) repetitions of this symbol or regular expression.

\+ Indicates one or more repetitions of a character or regular expression.

\? Means none or one repeat.

$i$ Means exactly i repetitions.

$i,j$ The number of repetitions is in the range from i to j inclusive.

$i,$ The number of repetitions is greater than or equal to i.

\(,j\) The number of repetitions is less than or equal to j.

$RE\) Remember the regular expression or part of it for future use as a whole. For example, \(a-z$* will search for any combination of any number (including zero) lowercase letters.

. Matches any character, including newline.

^ Indicates a null expression at the beginning of a line. In other words, whatever is preceded by this sign must appear at the beginning of the line. For example, ^#include will search for lines starting with #include.

$ The same as the previous one, only applies to the end of the line.

[LIST] Means any character from the LIST. For example, it will search for any English vowel letter.

[^LIST] Means any character except those in the list. For example, [^aeiou] will look for any consonant. Note: LIST can be an interval, for example [a-z], which will mean any lowercase letter. If you need to include ] (square bracket) in the LIST, indicate it first in the list; if you need to include - (hyphen) in the LIST, then indicate it first or last in the list.

RE1\|RE2 Means PB1 or PB2.

RE1RE2 Means combining the regular expressions РВ1 and РВ2.

\n Indicates a newline character.

\$; \*; \.; \[; \\; \^ Mean accordingly: $; *; .; [; \; ^

Attention: The remaining backslash (\) conventions of the C language are not supported by sed.

\1 \2 \3 \4 \5 \6 \7 \8 \9 Indicates the corresponding part of the regular expression, stored using the signs $and $.

A few examples:

abcdef Means abcdef

a*b Represents zero or any number of a's and one b's. For example, aaaaaab; ab; or b.

a\?b Means b or ab

a\+b\+ Represents one or more a's and one or more b's. For example: ab; aaaab; abbbbb; or aaaaaabbbbbbb.

.* Means all characters on a line, on all lines, including empty ones.

.\+ Matches all characters on a line, but only on lines that contain at least one character. Empty strings do not match this regular expression.

^main.*(.*) It will search for lines that begin with the word main, and also contain opening and closing brackets, and there may be any number of characters before and after the opening bracket (or there may not be any).

^# Will search for lines starting with a # sign (eg comments).

\\$ Will search for lines ending with a backslash (\).

Any letters or numbers

[^ ]\+ (The square bracket, in addition to the ^ symbol, also contains a space and a tab) -- Means one or any number of any characters, except a space and a tab. Usually this means a word.

^.*A.*$ Indicates a capital A in the exact middle of a line.

A.$9$$ Indicates a capital letter A, exactly the tenth letter from the end of the line.

^.$,15$A Indicates a capital letter A, exactly the sixteenth from the beginning of the line.

Now that we've seen some regular expressions, let's return to the s command in sed.

Using the & symbol when the PATTERN is unknown “How is it unknown?”, you ask, “Don’t you know what you want to replace?” I will answer: I want to put in brackets any numbers found in the text. How to do it? Answer: use the & symbol.

The symbol & (ampersand), when placed as part of a REPLACEMENT, means any PATTERN found in the text. For example:

$ echo 1234 | sed "s/*/(&)/" (1234)

An asterisk (asterisk) after the interval is needed so that all numbers found in the sample are replaced. Without it it would have been:

$ echo 1234 | sed "s//(&)/" (1)234

That is, the first digit found was taken as a sample.

Here is an example with a completely meaningful load: let’s create a formula.txt file:

A+432-10=n

and apply the command to it:

$ sed "s/*-*/(&)/" formula.txt a+(432-10)=n

The mathematical formula acquired an unambiguous meaning.

Another ampersand symbol can be used to double the PATTERN:

$ echo 123 | sed "s/*/& &/" 123 123

There is one subtlety here. If we make the example a little more complicated:

$ echo "123 abc" | sed "s/*/& &/" 123 123 abc

as you would expect, only the numbers are doubled since there are no letters in the PATTERN. But if we swap parts of the text:

$ echo "abc 123" | sed "s/*/& &/" abc 123

then no doubling of numbers will work. This is a feature of the regular expression * - it matches only the first character of the string. If we want to double the digits wherever they are, we need to modify the regular expression in REPLACE:

$ echo "abc defg 123" | sed "s/*/& &/" abc defg 123 123

then the numbers will double, regardless of the number of previous “words”.

Using the escaped parentheses $, $ and \1 to process part of a PATTERN The escaped parentheses $and $ are used to store part of a regular expression.

The symbol \1 means the first remembered part, \2 means the second, and so on, up to nine remembered parts (the program does not support more). Let's look at an example:

$ echo abcd123 | sed "s/$*$.*/\1/" abcd

Here $*$ means that the program must remember all alphabetic characters in any quantity; .* means any number of characters after the first remembered part; and \1 means we only want to see the first part remembered. That’s right: in the program output we see only letters and no numbers.

In order to swap words, you need to remember two sub-PATTERNS, and then swap them:

$ echo stupid penguin |sed "s/$[a-z]*$ $[a-z]*$/\2 \1/" stupid penguin

Here \2 means the second sub-PATTERN, and \1 means the first. Note the spacing between the first expression $[a-z]*$ and the second expression $[a-z]*$. It is necessary for two words to be found.

The \1 sign does not have to be present only in REPLACEMENT; it can also be present in SAMPLE, for example, when we want to remove duplicate words:

$ echo penguin penguin | sed "s/$[a-z]*$ \1/\1/" penguin

Command substitution modifiers s

Replacement modifiers are placed after the last delimiter. These modifiers determine what the program will do if there is more than one match to the PATTERN in the string, and how to perform the replacement.

Modifier /g

Global replacement

The sed program, like most Unix utilities, reads one line at a time when working with files. If we order a word to be replaced, the program will replace only the first word that matches the PATTERN on the given line. If we want to change every word that matches the pattern, then we should enter the /g modifier.

Without the /g modifier:

$ echo this cat was the most ordinary cat | sed "s/cat/kitten/" this kitten was the most ordinary cat

The editor replaced only the first word that matched.

And now with the global replacement modifier:

$ echo this cat was the most ordinary cat | sed "s/cat/kitten/g" this kitten was the most ordinary kitten

All matches in this string have been replaced.

And if you need to change all the words, say, put them in brackets? Then regular expressions will come to the rescue again. To select all alphabetic characters, both upper and lower case, you can use the [A-Ya-Ya] construction, but it will not include words like “something” or “s”ezd.” The [^] construction is much more convenient ]*, which matches all characters except space. So:

$ echo the stupid penguin timidly hides | sed "s/[^ ]*/(&)/g" (stupid) (penguin) (timid) (hides)

How to choose the right match from several

If you do not apply modifiers, the sed program will replace only the first word that matches the PATTERN. If you apply the /g modifier, the program will replace every matched word. How can you select one of the matches if there are several of them on the line? - Using the conventional symbols $and $ we are already familiar with, remember the sub-SAMPLES and select the one you need using the symbols \1 - \9.

$ echo stupid penguin | sed "s/$[a-z]*$ $[a-z]*$/\2 /" penguin

In this example, we remembered both words, and, putting the second (penguin) in first place, removed the first (stupid) by putting a space in its place in the REPLACEMENT section. If we replace the space with a word, it will replace the first one (stupid):

$ echo stupid penguin | sed "s/$[a-z]*$ $[a-z]*$/\2 smart /" smart penguin

Numeric modifier

This is a one/two/three-digit number that is placed after the last separator and indicates which match is to be replaced.

$ echo very stupid penguin | sed "s/[a-z]*/good/2" very good penguin

In this example, each word is a match, and we have told the editor which word we want to replace by placing a modifier 2 after the REPLACE section.

You can combine the numeric modifier with the /g modifier. If you need to leave the first word unchanged, and replace the second and subsequent ones with the word “(deleted)”, then the command will be like this:

$ echo very stupid penguin | sed "s/[a-z]*/(deleted)/2g" very (deleted) (deleted)

If you really want to remove all subsequent matches except the first, then you should put a space in the REPLACE section:

$ echo very stupid penguin | sed "s/[a-z]*/ /2g" very

Or don’t put anything at all:

$ echo very stupid penguin | sed "s/[^ ]*//2g" is very

The numeric modifier can be any integer from 1 to 512. For example, if you need to put a colon after the 80th character of each line, the command will help:

$ sed "s/./&:/80" filename

Modifier /p - output to standard output (print)

The sed program already outputs the result to the standard output (for example, a monitor screen) by default. This modifier is used only with the sed -n option, which just blocks the output of the result to the screen.

Modifier /w

Allows you to write the results of text processing to the specified file:

$ sed "s/SAMPLE/REPLACE/w filename

/e modifier (GNU extension)

Allows you to specify a shell command (not a sed program) as a REPLACEMENT. If a match to the PATTERN is found, it will be replaced with the output of the command specified in the REPLACE section. Example:

$ echo night | sed "s/night/echo day/e" day

/I and /i modifiers (GNU extension)

Makes the replacement process case insensitive.

$ echo Night | sed "s/night/day/i" day

Modifier Combinations

Modifiers can be combined when it makes sense. In this case, the w modifier should be placed last.

Conventions (GNU extension) There are only five of them:

\L converts REPLACE characters to lowercase \l converts the next REPLACE character to lowercase \U converts REPLACE characters to upper case \u converts the next REPLACE character to uppercase \E undoes a translation started by \L or \U For obvious reasons, these conventions are used alone. For example:

$ echo stupid penguin | sed "s/stupid/\u&/" Stupid penguin

$ echo little puppy | sed "s/[a-z]*/\u&/2" little Puppy

We've covered almost every aspect of the sed command. Now it's time to look at the options of this program.

sed program options

The program has surprisingly few options. (Which somewhat compensates for the excess of commands, modifiers and other functions). In addition to the well-known options --help (-h) and --version (-V), which we will not consider, there are only three of them:

Option -e--expression=command_set

One way to execute multiple commands is to use the -e option. For example:

Sed -e "s/a/A/" -e "s/b/B/" filename

All of the previous examples in this article did not require the -e option simply because they contained a single command. We could have used the -e option in the examples, it would not have changed anything.

Option -f If you need to execute a large number of commands, it is more convenient to write them to a file and use the -f option:

Sed -f sedscript filename

Sedscript here is the name of the file containing the commands. This file is called a sed program script (hereinafter simply a script). Each script command should occupy a separate line. For example:

# comment - This script will change all lowercase vowels to uppercase vowels s/a/A/g s/e/E/g s/i/I/g s/o/O/g s/u/U/g

You can name the script whatever you want, it is important not to confuse the script file with the file being processed.

Option -n The sed -n program does not print anything to standard output. To receive a withdrawal you need a special instruction. We have already become familiar with the /p modifier, which can be used to give such an indication. Let's remember the zar.txt file:

$ sed "s/1-9/&/p" zar.txt In the mornings he did exercises.

Since no matches were found with the PATTERN (there are no numbers in the file), the s command with the /p modifier and the & sign as a REPLACEMENT (remember that the ampersand means the PATTERN itself) works like the cat command.

If PATTERN is found in the file, then lines containing PATTERN will be doubled:

$ sed "s/exercise/&/p" zar.txt In the mornings he did exercises.

In the mornings he did exercises.

Lightning is an electric charge.

Now let's add the -n option:

$ sed -n "s/exercise/&/p" zar.txt In the mornings he did exercises.

Now our program works like the grep command - it returns only lines containing PATTERN.

Selecting the desired elements of the edited text
Using just one s command, we have seen the incredible capabilities of the sed editor. But everything he does comes down to search and replace. Moreover, during operation, sed edits each line one by one, without paying attention to the others. It would be convenient to limit the rows that need to be changed, for example:
Select lines by numbers
Select rows in a certain range of numbers
Select only rows containing a certain expression
Select only lines between some expressions

Select only lines from the beginning of the file to some expression

Select only lines from some expression to the end of the file

The sed program does all this and more. Any sed editor command can be used address-wise, in a certain range of addresses, or with the above restrictions on the range of lines. The address or constraint must immediately precede the command:

Sed "address/restrict command"

Selecting rows by numbers

This is the simplest case. Just indicate the number of the required line before the command:

$ sed "4 s/[a-z]*//i" gumilev.txt What a strange bliss In the early twilight of the morning, In the melting of spring snow, to all that is perishing and wise.

$ sed "3 s/В/(В)/" gumilev.txt What a strange bliss In the early twilight of the morning, (In) the melting of spring snow, In everything that perishes and is wise.

Selecting rows in a range of numbers

The range is indicated, not surprisingly, separated by commas:

$ sed "2.3 s/В/(В)/" gumilev.txt What a strange bliss (In) the early twilight of the morning, (In) the melting of spring snow, In everything that perishes and is wise.

If you need to specify a range up to the last line of a file, but you don’t know how many lines there are, then use the $ sign:

$ sed "2,$ s/in/(in)/i" gumilev.txt What a strange bliss (in) the early twilight of the morning, (in) the melting of spring snow, (in) everything that perishes and is wise.

$ sed "/morning/ s/in/(in)/i" gumilev.txt What a strange bliss (in) the early twilight of the morning, In the melting of spring snow, In everything that perishes and is wise.

Selecting rows in the range between two expressions

Just as in the case of line numbers, the range is specified separated by commas:

$ sed "/morning/,/wise/ s/in/(in)/i" gumilev.txt What a strange bliss (in) the early twilight of the morning, (in) the melting of spring snow, (in) everything that perishes and wisely .

Selecting lines from the beginning of the file to a certain expression

$ sed "1,/snow/ s/in/(in)/i" gumilev.txt What a strange bliss (in) the early twilight of the morning, (in) the melting of spring snow, In everything that perishes and is wise.

Selecting lines from a certain expression to the end of the file

$ sed "/snow/,$ s/in/(in)/i" gumilev.txt What a strange bliss In the early twilight of the morning, (in) the melting of spring snow, (in) everything that perishes and is wise.

Other sed editor commands

d (delete) command

Removes the following lines from standard output:

$ sed "2 d" gumilev.txt What a strange bliss In the melting of spring snow, In everything that perishes and is wise.

And more often they write it simpler (without a space):

Sed "2d" gumilev.txt

Everything that was said in the previous section about addressing strings also applies to the d command (as well as to almost all commands in the sed editor).

Using the d command, it is convenient to throw away the unnecessary “header” of some mail message:

$ sed "1,/^$/d" filename

(Delete lines from the first to the first empty line).

Get rid of comments in the configuration file:

$ sed "/^#/d" /boot/grub/menu.lst

And you never know where you need to remove extra lines!

p (print) command

The English word "print" is translated as "print", which in Russian is associated with a printer, or at least with a keyboard. In fact, this word in the English context often simply means output to a monitor screen. So the p command does not print anything, but simply displays the specified lines.

When used by itself, the p command doubles the lines in the output (after all, the sed program prints a line to the screen by default, but the p command prints the same line a second time).

$ echo I have a cat | sed "p" I have a cat I have a cat

There are uses for this property, such as doubling empty lines to improve the appearance of text:

$ sed "/^$/ p filename

But the p command reveals its true colors in combination with the -n option, which, as you remember, prevents lines from being printed on the screen. By combining the -n option with the p command, you can get only the required lines in the output.

For example, look at lines one through ten:

$ sed -n "1.10 p" filename

Or just comments:

$ sed -n "/^#/ p" /boot/grub/menu.lst # GRUB configuration file "/boot/grub/menu.lst".

# generated by "grubconfig". Sun 23 Mar 2008 21:45:41 # # Start GRUB global section # End GRUB global section # Linux bootable partition config begins # Linux bootable partition config ends # Linux bootable partition config begins # Linux bootable partition config ends

This is very reminiscent of the grep program, which we already encountered when we talked about the -n option with the /p modifier. But, unlike the grep command, the sed editor makes it possible not only to find these lines, but also to change them, replacing, for example, everywhere Linux with Unix:

$ sed -n "/^#/ p" /boot/grub/menu.lst | sed "s/Linux/Unix/" # GRUB configuration file "/boot/grub/menu.lst".

# generated by "grubconfig". Sun 23 Mar 2008 21:45:41 # # Start GRUB global section # End GRUB global section # Unix bootable partition config begins # Unix bootable partition config ends # Unix bootable partition config begins # Unix bootable partition config ends

Team!

Sometimes you need to edit all rows except those that match the PATTERN or selection. The exclamation point (!) character inverts the selection. For example, let’s delete all lines except the second from Gumilyov’s quatrain:

$ sed "2 !d" gumilev.txt In the early twilight of the morning,

Or select all lines, except comments, from the /boot/grub/menu.lst file:

$ sed -n "/^#/ !p" /boot/grub/menu.lst default 1 timeout 20 gfxmenu (hd0,3)/boot/message title SuSe on (/dev/hda3) root (hd0,2) kernel /boot/vmlinuz root=/dev/hda3 ro vga=773 acpi=off title Linux on (/dev/hda4) root (hd0,3) kernel /boot/vmlinuz root=/dev/hda4 ro vga=0x317

Command q (quit)

The q command terminates the sed program after the specified line. This is convenient if you need to stop editing after reaching a certain point in the text:

$sed "11 q" filename

This command will finish when it reaches the 11th line.

The q command is one of the few sed commands that does not accept string ranges. The command cannot stop working 10 times in a row if we enter:

Sed "1.10 q" Absurd!

w (write) command

We will receive a gum.txt file containing the last two lines of Gumilyov's quatrain from the gumilev.txt file. Moreover, if such a file already exists, it will be overwritten. If you do not enter the -n option, then the program, in addition to creating the gum.txt file, will also display the entire contents of the gumilev.txt file.

For working on the command line, it is more convenient to use regular output redirection (> or >>), but in sed scripts, the w command will probably find its use.

r (read) command

This command will not only read the specified file, but also paste its contents into the desired location in the edited file. To select the “right place”, addressing that is already familiar to us is used (by line numbers, by expressions, etc.). Example:

$ echo From Gumilyov's poem: | sed "r gumilev.txt"

From Gumilyov's poem:

What a strange bliss In the early twilight of the morning, In the melting of spring snow, In everything that perishes and is wise.

Team =

Will give the number of the specified line:

$ sed "/snow/=" gumilev.txt What a strange bliss In the early twilight of the morning, 3 In the melting of spring snow, In everything that perishes and is wise.

$ sed -n "/snow/=" gumilev.txt 3

The command accepts only one address, does not accept intervals.

Command y

This command replaces characters from the PATTERN section with characters from the REPLACE section, working like a program tr.

$ echo Car - a legacy of the past | sed "y/Auto/Paro/" Steam car - a legacy of the past

Team y works only if the number of characters in the PATTERN is equal to the number of characters in the REPLACEMENT.

sed program scripts

In order to use the sed editor as a full-fledged text editor, you need to master writing sed scripts. The sed program has its own simple programming language that allows you to write scripts that can work wonders.

This article cannot contain descriptions of sed scripts, just as its author does not set himself the task of mastering the sed programming language. In this article, I focused on using the sed editor on the command line, with an eye to using it as a filter in pipes. For this reason, I have omitted numerous sed commands that are only used in sed scripts.

There are many fans of the sed editor, and many articles on the topic of scripting, including on the RuNet. So for those interested in this wonderful program it will not be difficult to expand their knowledge.

The sed program and Cyrillic characters

As can be seen from the examples in this article, the sed program on a properly Russified system is fluent in the “great and powerful” language.

sed Program Summary

The sed program is a multifunctional data flow editor, indispensable for:

Editing large text arrays
Editing files of any size when the sequence of editing actions is too complex
Editing data as it becomes available, including in real time - that is, in cases where it is difficult or completely impossible to use interactive text editors.

To fully master the sed program, it will take weeks or even months of work, since this requires:

Learn regular expressions
Learn to write sed scripts by mastering the simple programming language used in these scripts

On the other hand, mastering several of the most common commands in the sed editor is no more difficult than any Unix command; I hope this article will help you with this.

Afterword

Until now, in the articles of the HuMan series, I have tried to at least briefly disclose each option, each parameter of the command being described, so that the article could replace mana. In the future I will continue to adhere to this principle.

This article is an exception, as it does not describe all the features of the program. A complete description of them would require not an article, but a book. However, the article allows you to get an idea of the sed editor and get started with this amazing program using its most common commands.

Sed is a lightweight (the binary weighs only 128 kilobytes) and convenient text processing tool.

In this article I will give some simple examples of use sed and tell you about its main capabilities.

Sed takes an input stream or file line by line, edits each line according to the rules defined in the sed script, and then outputs the result. Sed is a Turing complete programming language.

sed command format

The sed command has the format:

sed [ -n ] [ -e script ] [ -f script file ] [ files ]

Flag -n suppresses output
-e- points to a list of instructions given on the command line.
-f- indicates the location of the script file.

Edit Command Format

The script file consists of a set of commands:

[ address [ , address ] ] command [ arguments ]

one on each line.
Addresses are either line numbers, special characters, or a regular expression:

$ - last line
start~N- Each N-th line starting from number Start
/regular expression/- lines that fall under the regular_expression
Examples:

1~2 - Every second line /REGEXP/- all lines in which it occurs /REGEXP/ 10,20 - lines from 10 to 20 10,+10 - lines from 10 to 20 5,~N- lines starting from the 5th and up to the first multiple N 5,/REGEXP/- lines containing /REGEXP/, after the 5th (not including the 5th)

If no address is specified, all lines are processed.
If one address is specified, the corresponding line is processed
If two addresses are specified, then the lines in the specified interval are selected.
!team- performed team, for rows that were not selected by address.

Basic commands

Let's look at the basic commands:

[address] a text- add a new line with text after the specified line

$ cat sed_test sed_test_1 11111 sed_test_2 22222 sed_test_3 33333 $ sed -e "2 a new_line" sed_test sed_test_1 11111 sed_test_2 22222 new_line sed_test_3 33333

[address [, address]] c text- Removes the selected lines and replaces them with text

$ sed -e "2 with new_line" sed_test sed_test_1 11111 new_line sed_test_3 33333 $ sed -e "/3/ with new_line" sed_test sed_test_1 11111 sed_test_2 22222 new_line

[address [, address]] d- Deletes the specified lines.

$ sed -e "2 d" sed_test sed_test_1 11111 sed_test_3 33333 $ sed -e "2!d" sed_test sed_test_2 22222

[address] i text- Insert text in place of the specified line.

$ sed -e "2 i new_line" sed_test sed_test_1 11111 new_text sed_test_2 22222 sed_test_3 33333

[address [, address]] p(with flag -n) displays the found lines.

$ sed -ne "2p" sed_test sed_test_2 22222

[address]q- exit from sed.

[address [, address]] r file- Is reading file and outputs its contents.

[address [, address]] s/regex/replacement/flags- Replaces regular expression on replacement- taking into account flags:

g - in the entire line
i - case insensitive
p - display the result of the replacement

$ sed -ne "s/t/T/g" sed_test sed_TesT_1 11111 sed_TesT_2 22222 sed_TesT_3 33333 $ sed -e "s//d/g" sed_test sed_test_d ddddd sed_test_d ddddd sed_test_d ddddd

[address [, address]] y/line1/line2/- Replaces all occurrences of characters in line1 corresponding symbols from lines2. The lengths of the lines must be the same.

$ sed -ne "y/est/EST/g" sed_test SEd_TEST_1 11111 SEd_TEST_2 22222 SEd_TEST_3 33333

[address[,address]] (commands)- brackets group commands
[address] =- Gives line numbers

Execution Loop

sed works with two data buffers: a main one and an auxiliary one. Initially both buffers are empty.
Working with these buffers is carried out using the commands:\\`h’, `H’, `x’, `g’, `G’ `D’ h- Replace the contents of the auxiliary buffer with the contents of the main one
H- Append a new line to the auxiliary buffer and then append the contents of the main buffer to the contents of the auxiliary
x- Swap the contents of both buffers
g- Replace the contents of the main buffer with the contents of the auxiliary one
G- Append a new line to the main buffer and then append the contents of the auxiliary buffer to the contents of the main one
D- Delete main buffer text up to next newline character
N- Append a new line to the main buffer, then add the next line to be processed there
P- Print the contents of the main buffer up to the next newline character.

More complex examples

The following script swaps the lines of a file (the first lines become the last and vice versa)

$ cat tac.sed #!/usr/bin/sed -nf # starting with the second line, the contents of the buffer (which already contains # all previous lines) is appended to the current line. 1! G # when reaching the last line - print $ p # Write the data into the buffer again h sed -nf tac.sed sed_test sed_test_3 33333 sed_test_2 22222 sed_test_1 11111

We count the lines of the file (we display the number of the last line)

$ cat count.sed #!/usr/bin/sed -nf $=

result

$ sed -nf count.sed sed_test 3

String Reversal

$ cat revers.sed #!/usr/bin/sed -f # skip lines containing one letter /../! b # Reverse the string. Add an empty line before and after the current one. s/%$@~*!G4;:%#`.*$/\ &\ / # Move the first character to the end # the loop works as long as there are characters in the middle line. tx:x s/$\\n.$$.*$$.\\n$/\\3\\2\\1/ tx #remove unnecessary line breaks s/\\n// g

This script moves two letters at a time.

$ sed -f revers.sed sed_test 11111 1_tset_des 22222 2_tset_des 33333 3_tset_des