Linux users, even beginners are familiar with the simple file utilities like ls, rm, cp etc. If you look at /bin directory in any linux distribution , you will find many tools that can help on a daily use. In this post, I will cover some interesting tools.
The purpose of this post is to introduce the tools only and not as a replacement to the man page – for each tool there are a lot of options in the man page
Find Files
Find c files in the current directory and all sub directories
# find . -name *.c
Find all log files with size greater than 200kb
# find . -name "*.log" -size +200k
Find all tmp files with size greater than 2M that are not belong to user developer and delete them
# find . -name "*tmp" -size +2M ! -user developer -exec rm {} \;
ask for confirmation for each file
# find . -name "*tmp" -size +2M ! -user developer -ok rm {} \;
Grep – Find in files
Find the text ‘hello’ in any text file in the current directory
# grep 'hello' *.txt
Find the text ‘hello’ in any file in the current directory and all sub directories
# grep -r 'hello' .
using regular expression
# grep -r 'id-[0-9]*' .
cut – Select fields or columns from file
select 10 characters from each line
# cut -c1-10 /etc/fstab
Select fields 1,5,6,7 from each line -the separator is ‘:’
# cut -d: -f1,5-7 /etc/passwd
at – Run a program at a specific time
# at 10:37 warning: commands will be executed using /bin/sh at> my script.py [CTRL + D]
bc – Command Line Calculator
# bc 102+240*3.5 942.0 quit #
Use it in a script
x=100 y=20 echo "$x/$y" | bc -l
comm – Compare 2 sorted files
# cat ./students1 avi dani rina zina # cat ./students2 avi dina meni zina # comm ./students1 ./students2 avi dani dina meni rina zina
diff – compare files line by line
# diff ./students1 ./students2 2,3c2,3 < dani < rina --- > dina > meni
df – Report File system disk space usage
# df Filesystem 1K-blocks Used Available Use% Mounted on udev 1994220 0 1994220 0% /dev tmpfs 402816 41768 361048 11% /run /dev/sda1 150041868 77653932 66101296 55% /
du – Estimate file space usage
# du . 548 ./netdrv 152 ./libnetfil 344 ./skbufftest 148 ./netlink 512 ./newsock 68 ./packet 16 ./nludev 28 ./pcaptest 72 ./netsocket 128 ./netfilter 80 ./multiplex 2552 .
Display only summery :
# du -s . 2552 .
file — Determine File type
# file netdrv.ko netdrv.ko: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), BuildID[sha1]=2281471f7eef79a863bfe197f9ec22992eecbabb, not stripped # file netdrv.c netdrv.c: C source, ASCII text # file Makefile Makefile: makefile script, ASCII text
fold – wrap each input line to fit in specified width
# cat ./data1 id-2093384 id-8984773 id-8725536 id-9828835 id-6455351 id-9873773 # fold -w 11 ./data1 id-2093384 id-8984773 id-8725536 id-9828835 id-6455351 id-9873773
head/tail – print first/last line of file
display first/last 2 lines of file:
# head -2 /etc/passwd root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin # tail -2 /etc/passwd cto:x:1002:1003:cto,,,:/home/cto:/bin/bash snort:x:125:135:Snort IDS:/var/log/snort:/bin/false
display first 50 characters from file:
# head -c 50 /etc/passwd root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemo
join lines of two files on a common field
# cat ./s1 avi haifa dani aco rina tel aviv zina ny # cat ./s2 avi 1002 dani 2000 rina 3000 zina 4255 # join -j1 1 -j2 1 ./s1 ./s2 avi haifa 1002 dani aco 2000 rina tel aviv 3000 zina ny 4255
od – dump file in various formats
by byte
# cat ./a1
hello
# od -bc ./a1
0000000 150 145 154 154 157 012
h e l l o \n
0000006
by word
# od -xc ./a1
0000000 6568 6c6c 0a6f
h e l l o \n
0000006
paste – merge lines of files
# cat ./s2 avi 1002 dani 2000 rina 3000 zina 4255 # cat ./s3 20003 france 09388 uk 20019 italy 98377 spain # paste ./s2 ./s3 avi 1002 20003 france dani 2000 09388 uk rina 3000 20019 italy zina 4255 98377 spain
sort – Sorting files
# sort s3 09388 uk 20003 france 20019 italy 98377 spain
split – split a file into pieces
# cat ./s1 avi haifa dani aco rina tel aviv zina ny # split -2 s1 gen_ # cat ./gen_aa avi haifa dani aco # cat ./gen_ab rina tel aviv zina ny
uniq – Remove duplicate lines from a sorted file
# cat ./s2 avi 1002 dani 2000 dani 2000 rina 3000 rina 3000 zina 4255 # uniq ./s2 avi 1002 dani 2000 rina 3000 zina 4255
wc – Print character, word and line count in files
# wc /etc/passwd 47 81 2629 /etc/passwd # wc -l /etc/passwd 47 /etc/passwd # ps aux | wc -l 266