Linux shell commands 102

Chapter 2 介绍关于文件的命令,包括文件命名、文件分割、文件查找、文件条件判断、文件输入、文件内容排序除重、文件内容对比、文件首尾

1 file name

重命名文件

rename *.jpg *.png
rename 's/ /_/g' *

临时文件命名,分割文件,提取文件名

1.1 temp file

$(tempfile) $RANDOM $$ is PID of current script

1.2 splitting file

# generate one test file of 100Kb
dd if=/dev/zero bs=100k count=1 of=data.file
split -b 10k data.file
# -d numeric suffixes, -a suffix length
split -b 10k data.file -d -a 3
# prefix filename for split files
split -b 10k data.file -d -a 3 sfile
# 10 lines in each split
split -l 10 data.file

1.3 slicing filename

提取文件名 ${VAR%.*} 或者文件后缀名 ${VAR#.*}% 为非贪婪模式,从右至左匹配一个 .*# 从左至右匹配一个 .* 。提取方式是贪婪模式是使用两个 %%## 即可。

name=${file%.*}
extn=${file#.*}

2 find

find your files with find.

2.1 list all files with path

# '\n' as the delimitor
find .. -print
# '\0' as the delimitor, useful for filename contains space
find .. -print0

# list only directories
ls -d */
ls -F | grep "/$"
ls -l | grep "^d"
find . -type d -maxdepth 1 -print

If there are blanks, use print0.

2.2 search by name

find / -name "*.txt"print
find . \( -name "*.txt" -o -iname "readme*" \) -print

name to match files, iname to ignore case. path to match whole file paths. regex iregex to match file paths based on regular expression, which specify a pattern.

find . ! -name "*.dll" -print
find devel/source_path \( -name ".git" -prune \) -o \( -type f -print \)
find . -maxdepth 1 -type f -print
find . -mindepth 2 -type f -print

! is nagating argument. It matches file names that not end with .txt. prune exclude portion maxdepth or mindepth specify directory depths.

2.3 search by attributes

atime to last access time mtime to last modify time ctime to last metadata modify time

amin, mmin, cmin are minutes, instead of days in atime. newer specify a reference file to compare timestamp.

$ find . -type f -atime +7 -size +2k -user allen -perm 644 -delete

+, -, / are older, last, exact days or minutes. size to file size, /b, c, w, k, M, G.

user to file owner perm to file permissions

2.4 operations

delete to delete, instead of print exec to execute command on files sorted out.

# find . -type f –user root –exec chown allen {} \;

{} is a match, \; is the end of find.

3 comparison

程序中的条件判断(比较)形成了 流控制

if condition; then
command;
fi
if condition; then
command;
elif condition; then
command
else
command
fi
[[ condition ]] && true_cmd || false_cmd

3.1 math comparison

[ $var -eq 0 ]
-eq
equal
-ne
not equal
-gt
greater than
-lt
less than
-ge
greater or equal
-le
less or equal

3.2 logical comparison

[ $var1 -ne 0 -a $var2 -gt 2 ]
[ $var1 -ne 0 ] && [ $var2 -gt 2 ]
-a
AND
-o
OR

3.3 filesystem conditions

[[ -e ~/.bash_aliases ]] && . ~/.bash_aliases
-f $var
true if $file holds a regular file path
-x $var
true if $var bolds a executable file path
-e $var
true if $var holds an existing file
-d $var
true if $var holds a directory path
-c $var
true if $var holds a char dev file
-b $var
true if $var holds a block dev file
-w $var
true if $var holds a writable file
-r $var
true if $var holds a readable file
-L $var
true if $var holds a symlink

3.4 string conditions / comparisons

[[ $str1 = $str2 ]] # true if $str1 equals $str2
[[ $str1 == $str2 ]] # true if $str1 equals $str2
[[ $str1 != $str2 ]] # true if $str1 mismatches $str2
[[ $str1 > $str2 ]] # true if $str1 alphabetically greater than $str2
[[ -z $str1 ]] # true if $str1 holds an empty string
[[ -n $str1 ]] # true if $str1 holds an non-empty string

4 input

标准输入,或是读取文件,作为脚本输入。输入的其他重要细节之一,是设定分隔符。

4.1 read

读取 n 个字符到 to_var 中
read -n num_chars to_var
输出消息的读取
read -p "message" to_var
屏幕不显示输入内容地读取到 to_var
read -s to_var
设定读取字符的超时
read -t timeout to_var
设定读取字符的分隔符
read -d delim_char to_var
read -n 3 var && echo $var
read -p "Please Enter your name:" var
read -s password
read -t 3 var
read -d ":" var

4.2 IFS

另一个分隔符工具 : internal field separator. $IFS

4.3 xarg

cat file.txt | xargs
cat file.txt | xargs -n 3
cat file.txt | xargs -n 3 -d _
find . -type f -name "*.txt" -print0 | xargs -0 rm -f

5 translate

5.1 将输入中的一组字符串转换成另一组字符串。

tr [option] set1 set2
cat file.txt | tr 'A-Z' 'a-z'
echo 12345 | tr '0-9' '9876543210'

5.2 删除特定字符串

cat file.txt | tr -d '0-9'

5.3 补充特定字符串

cat file.txt | tr -c set1

5.4 集合分类

alnum
alphanumeric characters
alpha
alphabetic characters
cntrl
control characters
digit
numeric characters
xdigit
hexadecimal characters
graph
graphic characters
print
printable characters
punct
punctuation characters
space
whitespace characters
lower
lower-case alpha
upper
upper-case alpha
tr '[:lower:]' '[:upper:]'

6 sort

排序文件内容,并剔除重复

sort file1.txt file2.txt > sorted.txt
sort file1.txt file2.txt -o sorted.txt

# numeric sort && reverse order
sort -n file.txt
sort -r file.txt
# specify key of column 2
sort -k 2 file.txt
# ignore leading blanks && directory order
sort -bd unsorted.txt
# sort by months
sort -M file.txt
# merge sorted files without sorting again
sort -m sorted1 sorted2
sort unsorted.txt | uniq
sort -u unsorted.txt
uniq sorted.txt
# display only unique lines
uniq -u sorted.txt
# display counts
sort unsorted.txt | uniq -c
# display duplicate lines
sort unsorted.txt | uniq -d
# -s skip first N characters
# -w maximum characters to compare
comm sorted1.txt sorted2.txt
comm sorted1.txt sorted2.txt -3
comm sorted1.txt sorted2.txt -1 -2

7 diff

diff file1.txt file2.txt
# Unified output will be like git
diff -u file1.txt file2.txt
# Save differences to file.patch
diff -u file1.txt file2.txt > file.patch
# Apply patch, i.e. having same contents of file2.txt
patch -p1 file1.txt < file.patch
# Revert patch
patch -p1 file1.txt < file.patch -R
# Recursively generating diff
diff -Naur dire1 dire2

8 head & tail

# Print first 10 lines
head file
cat ~/.spacemacs | head
# First 4 lines to be printed
head -n 4 file
# All lines except last 5 lines
head -n -5 file
# Print last 10 lines
tail file
# Last 4 lines to be printed
tail -n 4 file
# All lines except first (5-1) line
tail -n +5
# Reading a growing file, using pid
PID=$(pidof emacs)
tail -f file.txt --pid $PID
# Count number of lines
wc -l file
# print the length of longest line
wc file -L
# Tree view of files and directories
tree ~/Documents
# -P PATTERN highlight only files matched by pattern
tree path -P "*.sh"
# -I PATTERN highlight only files excluding the match pattern
tree path -I "*.org"
# Print size along with them
tree -h
# Generate HTML output at directory PATH
tree PATH -H http://localhost -o out.html