CLI Utilities
Table of Contents
- 1. File Manipulation
- 1.1.
which - 1.2.
whereis - 1.3.
whatis - 1.4.
apropos - 1.5.
locate - 1.6.
find - 1.7.
namei - 1.8.
umask - 1.9.
mknod - 1.10.
split - 1.11.
link - 1.12.
entr - 1.13.
mkfifo - 1.14.
tee - 1.15.
sync - 1.16.
od - 1.17.
dd - 1.18.
shred - 1.19.
stdbuf - 1.20.
truncate - 1.21.
lsof - 1.22.
tar - 1.23.
zip - 1.24.
cpio - 1.25.
stow - 1.26.
cwebpanddwebp - 1.27.
qpdf - 1.28.
pandoc
- 1.1.
- 2. Text Manipulation
- 3. Process Manipulation
- 4. Machine Control
- 5. Networking
- 6. Programming
- 7. Audio Player
- 8. Miscellaneous
1. File Manipulation
1.1. which
Probe what program or alias the following will execute.
1.2. whereis
Show the location of binary, source code (if available), and manpage.
1.3. whatis
Show the manpage entry.
1.4. apropos
Fuzzy-find through the manpages.
1.5. locate
It has been Unix tool for a while.
slocalte(secure locate),mlocate(merging locate),plocate(posting locate) are the implementations in order.locatesymlinks to the specific implementation.
updatedbcommand needs to run to build the databse at/var/lib/plocate/plocate.db.plocate- It uses trigrams to index the database the index is inverted (from trigrams to files)
1.6. find
find [STARTING_POINTS...] EXPRESSION
STARTING POINTis any directory or file- Multiple of them can be given and each one of them is traversed left to right.
- Current directory is assumed when not given.
EXPRESSION: any expression evaluates to either true or false.- Global: Always true
-maxdepth N: 0 means the starting point itself, and 1 means the child of it, and so on.-mindepth N: does not evaluate the expression when depth is less thanN
- Test: true if matched
-name PATTERN: match the file name using the shell file name pattern (*,?)-iname PATTERN: case insensitive match-path PATTERN: match the entire file path to the pattern.-regex PATTERN: use regular expression to match the file name.-type {b|c|f|s|...}match the file type-size N[k|M|G]size ofNkilo/mega/gigabytes.-amin N, accessedNminiutes ago,-atime NaccessedNdays ago+Nor-Ncan be used to indicate "more than" and "less than".-cmin N,-ctime N,-mmin N,-mtime Nfor changes and modified reprectively.
- Action: true on seccess.
-delete-exec COMMAND \;: run the command for the matched file.'{}'is replaced by the file name. Note that\and'are there to prevent the shell parsing it.-print: print the full file name-prune: stop descending further into this directory.-quit: exit.
- Operator
\( EXPR \): force precedence\! EXPR,-not EXPR(not POSIX): true ifEXPRis false.EXPR1 EXPR2,EXPR1 -a EXPR2,EXPR1 -and EXPR2(not POSIX)EXPR1 -o EXPR2,EXPR1 -or EXPR2(not POSIX): it has lower precedence than AND.EXPR1 , EXPR2:EXPR1is evaluated and discared. The value ofEXPR2is the value of the list.
- Global: Always true
-printis implicit after the whole expression, unless-delete,-exec,-print, … is used.
1.7. namei
Take a path pattern and return the information of directories along the path.
1.8. umask
Set the creation file mode for the current environment.
1.9. mknod
mknod [OPTION] NAME TYPE [MAJOR] [MINOR]-m--mode=MODEpermission modeTYPEbbuffered block filecubuffered/unbuffered character filepFIFO
MAJOR MINOR- It must be specified for
bcu 0x0X0== hexadecimal, octal, decimal
- It must be specified for
1.10. split
- Divide a file into smaller ones.
-b: size,-n: division,-lline,--filter 'gzip > $FILE.gz': post-action hook.
The splitted files can be combined with cat on Linux,
copy /b FILE1 + FILE2 OUTFILE on Windows.
7z -v SIZE also does the simple split on the compressed file.
1.11. link
- Creates hard link by directly interacting with the system.
1.12. entr
- 7 Essential Command Line Tools 2022 - YouTube
- Enter.
- Automatically run commands when given file is saved.
1.13. mkfifo
- Creates a named pipe, which can be accessed from multiple sessions.
1.14. tee
- Pipe input to both standard output and a file.
1.15. sync
- Flushes cached data.
1.16. od
- Octal dump. POSIX. Can be used for other bases.
1.17. dd
- The options are given as variables format
bsibsobsbyte sizeifofinput, output filescountskipconvconvert while copying,statusprogress
1.18. shred
Zeroes out the file and then unlink
1.19. stdbuf
Controls the buffer of standard input and output.
1.20. truncate
Modifies file size.
1.21. lsof
List open files
- Note that everything is a file in UNIX-like systems.
-ilist all open network connections?
1.22. tar
tape archive
-ccreate. should be used with-fto set the archive filename-xextract. should be used with-fto select the archive- It detects the compression algorithm automatically, but it can be set manually also.
-a--auto-compressdetect the compression algorithm based on the archive suffix.
-fselect the file or filename.-vverbose- filter through
-zgzip,-Zcompress,-jbzip2,-Jxz,-I COMMANDuseCOMMAND.
1.23. zip
zip OPTIONS ARCHIVE FILES
1.24. cpio
copy in and out. It copies files between archives and directories.
-o--create-i--extract--file--block-size=BLOCK-SIZEset the I/O block size-D--directory=DIR-cold portable archive format,-H--format=FORMATuse theFORMAT, the default format one isnewc?binobsolete binary formatodcold POSIX.1 portable formatnewcnew SVR4 portable formatcrcSVR4 format with checksumtarustarhpbinhpodc
1.25. stow
- NEVER lose dotfiles again with GNU Stow - YouTube
- put dotfiles in one directory, for ease of management and migration.
1.26. cwebp and dwebp
- Compress and decompress into and out of
.webp dwebp WEBP -o PNG/JPGcwebp IMAGE -o WEBP -q [0-100]
1.27. qpdf
Transform PDF files. For example, encryption and decryption, or web optimization.
1.28. pandoc
1.28.1. Extensions
auto_identifiers: Add id field, even if it does not have the id field itself.
1.28.2. Filter
Define a function with a name of a element, that returns a new element.
pandoc.ELEMENTcreate a new element.
2. Text Manipulation
2.1. join
- Relational SQL-like join on two text files.
2.2. fold
- Wrap the text to a specified width.
2.3. pr
- Paginate or columnate text for printing.
2.4. tac
- Reverse strings.
2.5. shuf
- Shuffle input.
2.6. sort
- Sort input.
2.7. sed
- See
2.8. uniq
- Make each word from input unique.
2.9. tr
- Translate(replace) single byte characters.
2.10. jp2a
- Convert JPEG image to ASCII art.
2.11. comm
- Perform set operations.
$ comm -12 A B- Flags
1: Remove elements only in the first set2: Remove elements only in the second set3: Remove elements in both the first and the second set
2.12. seq
seq <initial> <increment> <quantity>
2.13. jq
- jq: A Practical Guide - YouTube
- JSON Queryer
2.14. sed
- Stream Editor
- It processes text line by line.
- The stream version of the
ed, the original UNIX text file editor, which also used similar commands. - The structure is inherited by .
2.14.1. Options
-i- In-place editing. Directly changes the text file.
-z- Replace
\nwithNULL, then^,$,\nmatches withNULL. This enables multi-line editing.
- Replace
-n--quiet--silent- Suppress the automatic printing of pattern space
-e SCRIPT- Each command in the script is separated by a newline.
\w\s.\ddoes not work. Use[0-9]\0,&for the entire match[abc]*\Uto upper,\Lto lower,\Ereturn to normal
-r-E--regexp-extended(abc)\1\NGrouping{N}+?repeat.
-f SCRIPTFILE
2.14.2. Commands
: LABEL#COMMENTADDRESS { COMMANDS }
2.14.2.1. Addresses
NUMBER,FIRST~STEPmatch line numbers$last line/REGEXT/\<C><REGEXP><C>line that matches the0,ADDRESSuntilADDRESSis foundADDRESS,+NNlines follwingADDRESSADDRESS,~NfromADDRESSto the line numberNor its multiples.
2.14.2.2. Zero-, One-Address Commands
=print the line numbera TEXTappend text into the next line.\<ret>is used to insert newlines.i TEXTinsert text into the previous line.qquitr FILENAMEappend text from the file.Rappend one line at a time from the file
2.14.2.3. Address Range Commands
c TEXTddelete pattern spacehcopy pattern space to hold space,Happend to hold spacegcopy hold space to pattern space,Gappend to pattern spacenread the next line into the pattern space,Nappend into the pattern spacepprint the pattern spaces/REGEXP/REPLACEMENT/replace the within pattern space.\1through\9is available, with-r\0or&can be used without-r
s/^/REPLACEMENT/appends in the beginning of a line.\<PATTERN\>matches a word. same as\bPATTERN\b\ddoes not match the number since they are strings.
w FILENAMEwrite the pattern space to a file.Wwrite one line at a timey/SOURCE/DEST/transliterate. See ((669f0999-d1e0-46f6-b0cf-0f57d2148067))b LABELbranch toLABEL. IfLABELis omitted branch to the end of script.t LABELif substitution succeeds, branch toLABEL,T LABELif no substitution succeeds.
2.14.3. Examples
sed -z 's/\ntime: /T/g' -i **/*.mdsed 's/^/# /' # comment out
2.15. awk
- Al *A*ho, Peter *W*einberger, Brian *K*ernighan
- The text file manipulator, powered by
yacc- with a Scripting Language
- The program is reimplemented under GNU project as
gawk, powered by the reimplementation ofyacc,bison.
2.15.1. Syntax
pattern { action }
2.15.1.1. Patterns
BEGIN,ENDspecial pattern that matches the start and end of a programBEGINcan be used for initialization of the environment variables.
/regex/||&&!? :,
2.15.1.2. Actions
print,printf"TEXT"- It can be redirected to a file with
>and>> - It can be piped to external commands with
|, $0is the entire record.$Nis the =N=th field.
- ** Special Variables
ARGCthe number of argumentsARGVarray of argumentsFSfield separator regular expressionNRthe current line numberOFSoutput field separatorFILENAMEthe current input file path
3. Process Manipulation
3.1. pstree
- Show the entire process tree.
3.2. top
- Monitor the processes
- Keybindings
hhelp<>change the sorted columnfchange the properties to displayZchange the color schemeAtoggle multi-window viewVtoggle forest (process trees)vfold a branch
Wsave this presetihide idle processesctoggle full commandsu,ofilter processes- Toggle:
<C-p>namespaces (cgroup: , ipc: , …),<C-n>environment variables,<C-u>supplementary groups (plugdev, wheel, harry),<C-k>full commands,<C-g>control groups (0::/user.slice/…). qexit
- Keybindings
3.3. nproc
- Show the number of processors.
3.4. nice
- Set a
nicevalue to a program so that smaller valued ones finish faster on average.
3.5. fuser
- Show which process is using a file or a device. With
-n tcp <port>flag, it shows which process is using a tcp port.
3.6. timeout
timeout 2h <command>- Terminate the command based on time.
3.7. env
- Set environment variables for the process
3.8. parallel
- GNU Parallel
- Execute multiple jobs of the command given in a special format, each with given arguments.
{}argument,{.}argument without file extension, …::: arguments,< arguments file
3.9. yes
Print y repeatedly, so that it can be piped to another program.
3.10. printenv
Print the current environment variables.
3.11. setsid
Execute command in an existing process.
4. Machine Control
4.1. systemctl soft-reboot
Keep the kernel, reboot the user space.
5. Networking
5.1. tcpdump
Monitor packets
5.2. netstat
from core/net-tools
5.3. ss
dump socket statistics.
6. Programming
6.1. nm
list symbols from object files
6.2. Reverse Engineering
6.2.1. IDA
6.2.2. Ghidra
6.2.3. objdump
7. Audio Player
7.1. mpd
It is the media server that can host various musics. I feel like this is not a good option for personal music player.
7.2. mplayer
It is a CLI tool, that has separate frontend: smplayer, etc.
These tools is not well-maintained.
7.3. mpv
Audio and video player based on MPlayer, mplayer2, and FFmpeg.
mpv-mpris./linux.html#org1602f7c support
7.3.1. OSC Customization
- On Screen Controller (OSC) is the set of GUI elements
- It can be customized with ./lua.html.
- I found this: GitHub - maoiscat/mpv-osc-modern: Another mpv osc script
8. Miscellaneous
8.1. neofetch
- It is configurable
8.2. cowsay
- Animals other than cow is also available.
8.3. fortune
- Random quotes
8.4. figlet
- ASCII text stylizer
8.5. lolcat
8.6. cava
- Audio visualizer
8.7. pipes
- Screensaver
8.8. wtf
8.9. minimodem
- Generate modem signal based on input.
8.10. tldr
- Short version of man page. It provides a quick snippets.
8.11. id
- Show
uid,gidand others of a given user (current user if unprovided).
8.12. fzf
- Fuzzy find command.
8.13. btop
rocm-smi-lib is required to display GPU informations.
8.14. expr
Evaluate the expression. Each component of the expression must be separated by space, and the reserved characters must be escaped.
- String manipulation with
STRING : REGEXP,match STRING REGEXP,length,index,substr STRING POS LEN
- Arithmetic operation with
+-*/%><=>=n | mif n=0 return m; else return n,n & mif n=0 && m=0 return 0; else return n- Precedence is controlled by parenthesis. They need to be escaped in Bash.
8.15. date
- Print datetime
-d STRING--date=STRINGuse this datetime.+FMTformat it-u--utc--universalprint or set UTC--rfc-3339={date|hours|minutes|seconds|ns}-I--iso-8601={date|hours|minutes|seconds|ns}use ISO format
8.16. bc
- Arbitrary precision calculator
This program is too basic to be useful for complex calculation
The syntax is identical to plain C without semicolon and types.
Note that semicolon is still used to separate statements in a
statement list {...; ...}.
Every variable is assumed to be a number, string cannot
be assigned to a variable. String must be surrounded by ".
The function is defined with the keyword define:
define [void] <name> ( <parameters> ) {
<statements>
}
If void keyword is used the function returns nothing instead of zero (the value of print).
Some expressions are builtin.
return (EXPR).length (EXPR)the number of digits before the decimal point,scale (EXPR)the number of digits after the decimal point.read(),scale(EXPR)the precision,sqrt(EXPR)print EXPR1, EXPR2, ...
With math library (-l or --mathlib), six functions are defined:
s (x)sine (in radian)c (x)cosinea (x)arc tangentl (x)natural logarithme (x)exponential functionj (n,x)Bessel function
The exit keyword is quit