[!NOTE] references:
CHARACTER REGULAR EXPRESSION EXPLANATION [[:alnum:]][A-Za-z0-9]Alphanumeric characters [[:alpha:]][A-Za-z]Alphabetic characters [[:blank:]][ \t]Space or tab characters only [[:cntrl:]][\x00-\x1F\x7F]Control characters [[:digit:]][0-9]Numeric characters [[:graph:]][!-~]Printable and visible characters [[:lower:]][a-z]Lower-case alphabetic characters [[:print:]][ -~]Printable (non-Control) characters [[:punct:]][!-/:-@[-{-~]`Punctuation characters [[:space:]][ \t\v\f\n\r]All whitespace chars [[:upper:]][A-Z]Upper-case alphabetic characters [[:xdigit:]][0-9a-fA-F]Hexadecimal digit characters
execute multiple sed commands
[!TIP]
-e command Append the editing commands specified by the command argument to the list of commands.references:
example : show only root and nobody in /etc/passwd
-e:$ sed -n -e '/^root/p' -e '/^nobody/p' /etc/passwd nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false root:*:0:0:System Administrator:/var/root:/bin/sh;:$ sed -n -e '/^root/p;/^nobody/p' /etc/passwd nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false root:*:0:0:System Administrator:/var/root:/bin/sh'{}':$ sed -n '{ /^root/p /^nobody/p }' /etc/passwd nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false root:*:0:0:System Administrator:/var/root:/bin/sh
range
specific line
- 2nd line :
N <opt>
until empty line
references:
- Remove everything before a blank line using sed
- Remove empty line before a pattern using sed
- Grep starting from a fixed text, until the first blank line
- Delete unknown number of lines from * until blank line
- remove only the first blank line sed
- Use sed to insert text before two blank lines
- htop output to human readable file
- $HOME/.toprc
$ top -bn1 | sed -n '0,/^\s*$/p'
top - 03:41:45 up 258 days, 19:05, 1 user, load average: 2.33, 0.92, 0.95
Tasks: 856 total, 2 running, 447 sleeping, 0 stopped, 36 zombie
%Cpu(s): 0.3 us, 0.4 sy, 0.0 ni, 99.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 52802012+total, 11152644+free, 24536944 used, 39195673+buff/cache
KiB Swap: 0 total, 0 free, 0 used. 49137280+avail Mem
- or
$ top -bn1 | sed -e '/^$/Q' top - 03:45:55 up 258 days, 19:09, 1 user, load average: 0.17, 0.51, 0.77 Tasks: 857 total, 2 running, 448 sleeping, 0 stopped, 36 zombie %Cpu(s): 0.1 us, 0.4 sy, 0.0 ni, 99.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 52802012+total, 11151089+free, 24546520 used, 39196272+buff/cache KiB Swap: 0 total, 0 free, 0 used. 49136291+avail Mem
n~m range
- n~m lines :
n,m <opt> - n to end lines :
n,$ <opt> - m lines starting with n :
n, +m <opt> - start n skip m via
~:
| pattern | matches | comments |
|---|---|---|
1~2 |
1,3,5,7,... |
start frmo 1, print every 2 lines |
2~2 |
2,4,6,8,... |
start from 2, print every 2 lines |
1~3 |
1,4,7,10,... |
start from 1, print every 3 lines |
2~3 |
2,5,8,11,... |
start from 2, print every 3 lines |
pattern matches range
- between
pattern_1topattern_2:/pattern_1/,/pattern_2/ <opt> - first line to
pattern_2:0,/pattern_2/ <opt>
from pattern to first empty line
$ top -bn1 | sed -n '/^top.*/,/^\s*$/p'
top - 03:49:02 up 258 days, 19:13, 1 user, load average: 0.43, 0.41, 0.68
Tasks: 853 total, 1 running, 448 sleeping, 0 stopped, 36 zombie
%Cpu(s): 0.3 us, 0.4 sy, 0.0 ni, 99.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 52802012+total, 11150942+free, 24543500 used, 39196720+buff/cache
KiB Swap: 0 total, 0 free, 0 used. 49136582+avail Mem
print all lines
- print every line twice
$ sed 'p' employee.txt 101,John Doe,CEO 101,John Doe,CEO 102,Jason Smith,IT Manager 102,Jason Smith,IT Manager 103,Raj Reddy,Sysadmin 103,Raj Reddy,Sysadmin 104,Anand Ram,Developer 104,Anand Ram,Developer 105,Jane Miller,Sales Manager 105,Jane Miller,Sales Manager - print all lines :
$ sed -n 'p' employee.txt
range print
- print the 2nd line :
$ sed -n '2 p' employee.txt n,mrange- print 1~4 lines :
$ sed -n '1,4 p' employee.txt - print all lines since the 2nd line:
$ sed -n '2,$ p' employee.txt
- print 1~4 lines :
~to skip lines- print only odd numbered lines :
sed -n '1~2 p' employee.txt
- print only odd numbered lines :
+(n, +m) :sed -n 'n,+m p' employee.txt
print matched pattern
- find pattern to the end :
$ sed -n '/Raj/,$ p' employee.txt 103,Raj Reddy,Sysadmin 104,Anand Ram,Developer 105,Jane Miller,Sales Manager find pattern and line after the matches line :
$ sed -n '/Raj/, +1 p' employee.txt 103,Raj Reddy,Sysadmin 104,Anand Ram,Developerfind pattern to 4th line :
$ sed -n '/Raj/,4 p' employee.txt 103,Raj Reddy,Sysadmin 104,Anand Ram,Developerfind pattern until find another pattern (
JasontoAnand) :$ sed -n '/Jason/,/Anand/p' employee.txt 102,Jason Smith,IT Manager 103,Raj Reddy,Sysadmin 104,Anand Ram,Developer
print line number of matched pattern
[!NOTE|label:references:]
# original
$ printf "%s\r\n\r\n%s\r\n%s\r\n\r\n%s\r\n" 1 2 3 4 | command cat -A | nl
1 1^M$
2 ^M$
3 2^M$
4 3^M$
5 ^M$
6 4^M$
# result
$ printf "%s\r\n\r\n%s\r\n%s\r\n\r\n%s\r\n" 1 2 3 4 | sed -n '/^\s*\r$/ ='
2
5
# or
$ sed -n '/^\s*\r$/ =' < <(printf "%s\r\n\r\n%s\r\n%s\r\n\r\n%s\r\n" 1 2 3 4)
2
5
# first matched line number
$ printf "%s\r\n\r\n%s\r\n%s\r\n%s\r\n" 1 2 3 4 | sed -n '/^\s*\r$/ =' | head -n1
2
# last matched line number
$ printf "%s\r\n\r\n%s\r\n%s\r\n\r\n%s\r\n" 1 2 3 4 | sed -n '/^\s*\r$/ =' | tail -n1
5
-
# via sed $ sed -n '10,20{=;p}' ## or ## $ nl file.txt | sed -n '10,20p' ## or ## $ sed -n '10,20{=;p}' file.txt | sed '{N; s/\n/ /}' # via awk $ awk 'FNR==10,FNR==20 {print FNR ":" $0}' file.txt ## or ## $ awk 'FNR >= 10 {print FNR ":" $0}; FNR == 20 {exit}' file.txt ## or ## $ awk -v OFS=: 'FNR >= 10 {print FILENAME, FNR, $0}; FNR == 20 {nextfile}' ./*.txt
delete
delete all
$ sed 'd' employee.txt
range delete
- delete the 2nd line :
$ sed '2 d' /path/to/file - delete between
1and4lines :$ sed '1,4 d' /path/to/file
conditional delete
- delete all empty lines:
$ sed '/^$/ d' /path/to/file - delete all comment lines :
$ sed '/^#/ d' /path/to/file
substitute
substitute-flags
| flag | comments |
|---|---|
i |
ignore case flag |
g |
global flag |
1,2,... |
number flag |
p |
print flag |
w |
write flag |
e |
execute flag |
multiple replaces
$ sed '{
s/Developer/IT Manager/
s/Manager/Director/
}' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Director
103,Raj Reddy,Sysadmin
104,Anand Ram,IT Director
105,Jane Miller,Sales Director
get matched pattern
&
When & is used in the replacement-string, it replaces it with whatever text matched the original-string or the regular-expression.
$ sed 's/^[0-9][0-9][0-9]/<&>/g' employee.txt
<101>,John Doe,CEO
<102>,Jason Smith,IT Manager
<103>,Raj Reddy,Sysadmin
<104>,Anand Ram,Developer
<105>,Jane Miller,Sales Manager
substitution grouping
$ sed 's/^\([0-9][0-9][0-9]\).*/<\1>/g' employee.txt
<101>
<102>
<103>
<104>
<105>
or via -r
$ sed -r 's/^([0-9][0-9][0-9]).*/<\1>/g' employee.txt
<101>
<102>
<103>
<104>
<105>
# or
$ sed -nr 's/^([0-9][0-9][0-9])(.*)/<\1>\2/gp' employee.txt
<101>,John Doe,CEO
<102>,Jason Smith,IT Manager
<103>,Raj Reddy,Sysadmin
<104>,Anand Ram,Developer
<105>,Jane Miller,Sales Manager
cheatsheet
get first matching patten ( for CERTIFICATE )
[!TIP]
- sample.crt
$ cat sample.crt -----BEGIN CERTIFICATE----- first paragraph -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- second paragraph -----END CERTIFICATE-----
regular pattern
$ cat sample.crt | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' -----BEGIN CERTIFICATE----- first paragraph -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- second paragraph -----END CERTIFICATE----- # or for short $ cat sample.crt | sed -ne '/-BEGIN/,/-END/p' -----BEGIN CERTIFICATE----- first paragraph -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- second paragraph -----END CERTIFICATE-----get first
[!TIP]
- How to print the text between the first occurence of a pair of strings? [duplicate]
- How to select first occurrence between two patterns including them
more :
# or `-n /../p` # `-n` `p` # | | # v v $ cat sample.crt | sed -n '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p; /-END CERTIFICATE-/q' -----BEGIN CERTIFICATE----- first paragraph -----END CERTIFICATE----- # or `/../!d` # no `-n` `!d` # | | # v v $ cat sample.crt | sed '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/!d; /-END CERTIFICATE-/q' -----BEGIN CERTIFICATE----- first paragraph -----END CERTIFICATE----- # or for short $ cat sample.crt | sed '/-END CERTIFICATE-/q' -----BEGIN CERTIFICATE----- first paragraph -----END CERTIFICATE----- # or $ cat sample.crt | sed '/-END/q' -----BEGIN CERTIFICATE----- first paragraph -----END CERTIFICATE-----
remove both '#' and empty lines
[!NOTE|label:references:]
$ .. | sed -r '/^(#.*)$/d' | sed -r '/^\s*$/d'
# or
$ .. | sed -r '/^(#.*)$/d;/^\s*$/d'
# or
$ .. | sed -r '/(^#.*)|(^\s*)$/d'
- example
$ ldapsearch CN=marslo DN | sed -r '/^(#.*)$/d;/^\s*$/d' dn: CN=marslo,OU=Workers,DC=company,DC=com
remove tailing spaces
[!TIP|label:available patterns]
's/[ \t]*$//''s/[[:blank:]]*$//''s/[[:space:]]*$//'
$ cal | cat -pp -A
····January·2024····␊
Su·Mo·Tu·We·Th·Fr·Sa␊
····1··2··3··4··5··6␊
·7··8··9·10·11·12·13␊
14·15·16·17·18·19·20␊
21·22·23·24·25·26·27␊
28·29·30·31·········␊
····················␊
# remove empty line
$ cal | sed -r '/^(#.*)$/d;/^\s*$/d' | cat -pp -A
····January·2024····␊
Su·Mo·Tu·We·Th·Fr·Sa␊
····1··2··3··4··5··6␊
·7··8··9·10·11·12·13␊
14·15·16·17·18·19·20␊
21·22·23·24·25·26·27␊
28·29·30·31·········␊
# remove trailing spaces
$ cal | sed 's/[[:space:]]*$//' | cat -pp -A
····January·2024␊
Su·Mo·Tu·We·Th·Fr·Sa␊
····1··2··3··4··5··6␊
·7··8··9·10·11·12·13␊
14·15·16·17·18·19·20␊
21·22·23·24·25·26·27␊
28·29·30·31␊
␊
show top summary
[!NOTE] see sed until empty line
contains empty line
$ top -bn1 | sed -n '0,/^\s*$/p' top - 03:41:45 up 258 days, 19:05, 1 user, load average: 2.33, 0.92, 0.95 Tasks: 856 total, 2 running, 447 sleeping, 0 stopped, 36 zombie %Cpu(s): 0.3 us, 0.4 sy, 0.0 ni, 99.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 52802012+total, 11152644+free, 24536944 used, 39195673+buff/cache KiB Swap: 0 total, 0 free, 0 used. 49137280+avail Memwithout empty line
[!TIP] references:
manual:
- The "q" command prints the current line again in less the -n flag was used on the command line and exits the script completely
q[exit-code] (quit) Exit sed without processing any more commands or input. Q[exit-code] (quit) This command is the same as q, but will not print the contents of pattern space. Like q, it provides the ability to return an exit code to the caller.$ top -bn1 | sed -e '/^$/Q' top - 03:45:55 up 258 days, 19:09, 1 user, load average: 0.17, 0.51, 0.77 Tasks: 857 total, 2 running, 448 sleeping, 0 stopped, 36 zombie %Cpu(s): 0.1 us, 0.4 sy, 0.0 ni, 99.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 52802012+total, 11151089+free, 24546520 used, 39196272+buff/cache KiB Swap: 0 total, 0 free, 0 used. 49136291+avail Mem
escape
[!NOTE|label:references:]
-
$ echo "\(\)" | sed 's/\\/\\\\\\/g' \\\(\\\)-
alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\'' # │ │┊┊| │┊┊│ │┊┊│ │┊┊│ # └─STRING──┘┊┊└─STRIN─┘┊┊└─STR─┘┊┊└─STRIN─┘┊┊│ # ┊┊ ┊┊ ┊┊ ┊┊│ # ┊┊ ┊┊ ┊┊ ┊┊│ # └┴─────────┴┴───┰───┴┴─────────┴┘│ # All escaped single quotes │ # │ # ? alias rc='sed '"'"':a;N;$!ba;s/\n/, /g'"'" alias rc='sed '\'':a;N;$!ba;s/\n/, /g'\'tricky
-
add
'or"to strings[!NOTE|label:references:]
$ GIT_OPT="-a -b --c=1 2 3 -d=4 5 6" $ GIT_OPT=$(echo "${GIT_OPT}" | sed -r 's/\s+-/\n-/g' | sed -r "s/^([^=]+)=(.+)$/\1='\2'/g" | sed -e 'N;s/\n/ /' ) $ echo $GIT_OPT -a -b --c='1 2 3' -d='4 5 6' $ GIT_OPT="--c=1 2 3 --d=4 5 6" $ GIT_OPT=$(echo "${GIT_OPT}" | sed -r 's/\s+--/\n--/g' | sed -r "s/^([^=]+)=(.+)$/\1='\2'/g" | sed -e 'N;s/\n/ /' ) $ echo $GIT_OPT --c='1 2 3' --d='4 5 6' $ GIT_OPT="--c=1 2 3 --d=4 5 6" $ GIT_OPT=$(echo "${GIT_OPT}" | sed -r 's/\s+--/\n--/g' | sed -r "s/^([^=]+)=(.+)$/\1=\"\2\"/g" | sed -e 'N;s/\n/ /' ) $ echo $GIT_OPT --c="1 2 3" --d="4 5 6"