Understanding sed – Practical Guide

Sed is a powerful stream editor, typically used for editing large amounts of data by providing a simple command. Sed is also used for sophisticated searches, where the Regular Expressions are used.

You can use sed to:

  • Automate editing actions to be performed on one or more files
  • Simplify the task of performing the same edit on multiple files
  • Converting one data format to another

For example the following command:

# sed -i.bup '/^#/d;/^$/d' myfile

Do the following:

  • Edit the file in place
  • Delete lines started with #  (/^#/d)
  • Delete blank lines (/^$/d)
  • Create a backup file with bup extension

To learn sed , we need to understand the syntax and learn by examples

Print command

Print all file

# sed 'p' [file]

The above displays every line twice because it dump the file content to stdout while read it, to remove the stdout add –n:

# sed –n 'p' [file]

Print lines 1-5

# sed -n '1,5p' [file]

(if we remove the –n flag we get all file output to stdout but only the matched line will displayed twice)

Find line started with ‘root’ and print it

# sed -n '/^root/p' /etc/passwd

Print from line 15 to the end:

# sed -n '15,$p' /etc/passwd

Print all except from line 15 to the end:

# sed -n '15,$!p' /etc/passwd

print lines start with ‘a’ followed by number

# sed -n '/^a[0-9]/p' /etc/passwd

print lines with exactly 3 characters

# sed -n '/^...$/p' filename

 

Substitute – Search and replace

General syntax 

# sed ‘ [range] s/<string>/<replacement>/ ‘ filename

simple example:

search for lines started with root and replace the first occurrence with ‘boot’

# sed ' /^root/ s/root/boot/ '

do it for all occurrences

# sed ' /^root/ s/root/boot/g '

 

search and replace default shell for user root

# sed ' /^root/ s@/bin/bash@/bin/sh@ ' /etc/passwd

note that we changed the delimiter because we want to use / in the search term

so this will also works:

# sed ' /^root/ s#/bin/bash#/bin/sh# ' /etc/passwd

 

The range can be regular expression or lines :

# sed ' 4,9 s/nologin/virtual/ ' /etc/passwd

Add -n  to suppress the output and p to display the lines affected

# sed -n ' 4,9 s/nologin/virtual/p ' /etc/passwd

Add -i (without p) to make the changes in place (edit the file)

# sed -i ' 4,9 s/nologin/virtual/ ' /etc/passwd

Add -i.bak to make changes and save the original for backup with bak extenstion

# sed -i.bak ' 4,9 s/nologin/virtual/ ' /etc/passwd

 

Transform Example:

Transform any ‘a’ to ‘_’ and any ‘I’ to ‘*’

# sed 'y/ai/_*/' ./emp

 

Insert, Append and delete

Add line ‘hello’ before lines 2-4:

# sed '2,4 i hello' ./passwd

Add line ‘hello’ after lines 2-4:

# sed '2,4 a hello' ./passwd

Add line ‘hello’ after each line started with s

# sed '/^s/ a hello' ./passwd

Delete lines 2-4

# sed '2,4 d' ./passwd

Delete lines starting with s

# sed '/^s/ d' ./passwd

more regular expressions:

# sed '/^[sm]/ d' ./passwd        # starting with s or m
# sed '/^[smp]/ d' ./passwd       # starting with s or m or p
# sed '/^[^smp]/ d' ./passwd      # not starting with m or s or p

Delete blank lines:

# sed '/^$/d' ./emp_s

 

Multiple sed expressions:

On the command line type:

# sed '{
> 1,3 i hello
> 5,8 d
> 10,12 a bye
> }' ./passwd

write in a file and use -f to run it:

# sed -f ./sedsamp /etc/passwd

in place:

# sed -i.bak -f ./sedsamp ./passwd

 

Substitution Groups

Example:

# sed 's/\([^,]*\)/\U\1/' ./emp

Lets explain it step by step:

We define groups with ( ) but we need to add escape chars it so it should be written ‘\(\)’

[^,] – anything except comma (,)

[^,]* – zero and more anything except comma

so it should be written like this:

\([^,]*\)

replace string:

/\U\1/

\U – command to uppercase

\1 – the first group

So this example uppercase the first field in CSV file:

 

It looks better if we use different command separator:

# sed 's@ \([^,]*\) @ \U\1 @' ./emp

 

using more groups:

# cat ./emp

avi,100,haifa
dani,200,tel aviv
rina,300,aco

Run the substitution 

# sed 's/\([^,]*\),\([^,]*\),\([^,]*\)/\U\1#\2#\U\3/' ./emp

AVI#100#HAIFA
DANI#200#TEL AVIV
RINA#300#ACO

Change order

# sed 's/\([^,]*\),\([^,]*\),\([^,]*\)/\U\3#\2#\U\1/' ./emp

HAIFA#1200#AVI
TEL AVIV#2300#DANI
ACO#3100#RINA

 

Executing commands

# cat ./folders

/lib
/lib32
/lib64

replace the starting line with ‘ls -l ‘

# sed 's/^/ls -l / ' ./folders

ls -l /lib
ls -l /lib32
ls -l /lib64

execute the commands:

# sed 's/^/ls -l / e' ./folders

 

 

 

 

Tagged

1 thought on “Understanding sed – Practical Guide

  1. Well groups command lists which canbe used frequently. Surprised to see that no one commented yet! Thanks for your effort 🙂

Comments are closed.