LPI Linux Certification/LPIC1 Exam 101/GNU & Unix Commands
![]() |
Section Overview
edit- 1.103.1 Work on the command line
- 1.103.2 Process text streams using filters
- 1.103.3 Perform basic file management
- 1.103.4 Use streams, pipes, and redirects
- 1.103.5 Create, monitor, and kill processes
- 1.103.6 Modify process execution priorities
- 1.103.7 Search text files using regular expressions
- 1.103.8 Perform basic file editing operations using VI
- Unix and Linux
Unix and Linux
editOverview
editUnix is the original Internet Operating System (NFS, TCP/IP, RPC, ...). Linux is a Unix clone written from scratch by Linus Torvalds with assistance from a world wide team of developers across the Net. Properties of both systems are:
- Portable, multi-user, multiprocessing
- Device-independent input/output
- Shared resources, high performance and highly secure
- Line commands
- GUI
Linux aims toward POSIX compliance and supports also other standard API's such as BSD and SVR4. Linux has all the features you would expect in a modern Unix, including true multitasking/multiple-users, virtual memory, shared libraries, demand loading, shared copy-on-write executables, proper memory management and TCP/IP networking.
More and more companies and organizations are selecting Linux because of its performance, low cost, and royalty free license.
Unix and Linux
editUnix is the original Internet Operating System written in 1970 developed at A&T Bell Labs. Distributed freely to government and universities. Different entities did support different distributions.
- System V
- BSD
- SCO
Most varieties of Unix carry copyright licenses.
Linux was released in 1991 by Linus Torvalds. The model of Linux licence is copy left from GPL. Linux implement most of the System V and BSD Unix commands. Use the man pages to determine the version.
Logging In and Out
editThe logging In session is the first step to access a Linux system. You need a login ID and a password to be able to start using the system. The login ID root is the super user ID that has all privileges, normally assigned to the System Administrator. When a log in session is successful it will set some environment variables and start the shell that has been assigned to the login ID account.
Examples of variables set at log in time: $HOME, $SHELL, $PATH, and more. By changing the file /etc/login.defs the administrator can customize some login parameters and variables for all the users.
Examples of some of them:
UID_MIN 500 UID_MAX 60000 UMASK 022 ENV_PATH /usr/local/bin:/usr/bin:/bin
To logout use the exit or logout command.
Password
editNobody can see your password, not even the administrator. To change passwords for users or groups, use passwd.
passwd [options]
Common options:
- -l, -u : Lock and unlock an account.
- -g [| r]: Used for group.
- -g set a password for group and -r removes it.
- -x|n : Maximum and minimum number of days a password remain valid.
As a general rule a password must:
- Have a minimum length
- Not contain any English language words
- Contain non-alphanumeric characters
- Contain a mixture of uppercase and lower case letters.
User information
editFor every user account, there is a line defined in the file /etc/passwd. The encrypted password is stored in /etc/shadow. The format of /etc/passwd is:
root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/bin/bash daemon:x:2:2:Daemon:/sbin:/bin/bash yann:x:500:100:Yann Forget:/home/yann:/bin/bash
The x field is the old location for the password.
The format of /etc/shadow is:
root:IMXweyiV816gg:11996:0:10000:::: bin:!*:8902:0:10000:::: daemon:*:8902:0:10000:::: yann:GoIM8j1S.IuTY:11996:0:99999:7:::
The * for the encrypted password means there is no password defined yet. The ! Before the encrypted password means the account is locked.
Group information
editFor every group of users there is a line in file /etc/group. The encrypted password is stored in /etc/gshadow. The format of /etc/group is:
root:x:0:root bin:x:1:root,bin,daemon daemon:x:2: video:x:33:yann
The format of /etc/gshadow is:
root:*:root:root bin:*:root:root daemon:*:root:root video:*:root:root,yann
Passwords for group are not often implemented since you have to pass to members of a group the password.
Misc. Commands
editTo get information on a logging In session, use id.
id [options] [username]
Common options: -g : Print only the group ID. -u : Print only the user ID.
Example:
$ id uid=0(root) gid=0(root) groups=0(root), 1(bin), 14(uucp), 15(shadow),16(dialout)
To run a shell with another user ID and group ID, use su.
su [options] [username]
Common options: -s : Select another shell.
Example:
$ su toto passwd:
Shell
editA Shell is a command line interpreter and:
- Prompts for commands.
- Interprets command which can include:
- I/O redirection, pipes, filename generation, etc.
- Executes commands.
- Executes scripting language.
There are several popular shells:
- sh: Bourne shell, the standard scripting shell.
- csh: Berkeley shell, job control, command history, alias.
- ksh: Korn shell, Bourne syntax with Berkeley features.
- bash: Bourne Again shell, free, Korn-like shell.
- t csh: Free, Berkeley clone with many extensions.
Most shells let you customize your prompt and environment. For bash the prompt variable is PS1. For bash the file that can be used for the customization of the environment are:
- .bashrc Executed for each new bash session
- .bash_profile Executed at logging In time
- .bash_logout Executed at logging Out time
Exercises
editProcess text streams using filters
editOverview
editDescription: Candidates should should be able to apply filters to text streams. Tasks include sending text files and output streams through text utility filters to modify the output, and using standard UNIX commands found in the GNU textutils package.
Key files terms and utilities include:
cat
cut
expand
fmt
head
join
nl
od
paste
pr
sed
sort
split
tac
tail
tr
unexpand
uniq
wc
Pattern matching and wildcards
editWildcard is a pattern matching mechanism for filenames. The purpose of wildcard is to increase productivity: Locate files that you don't fully remember. Locate files that have something in common. Work with a group of files rather than individual.
The shell interprets these special characters:
! @ # $ % ^ & * ( ) { } [ ] | \ ; ~ ' " ` ?
The characters used for wildcard are:
? * [ ] ~
If you use the wildcard characters the shell will try to generate a file from them. Try the following:
echo all files *
Special wildcard characters
? match any one character. * Any string [abcfghz] One char set [a-z] One char in range [!x-z] Not in set ~ Home directory ~user User home directory
Examples:
? One character filenames only [aA]??? Four characters, starting with a or A. ~toto pathname of toto home directory [!0-9]* All string not starting with a number.
What about these commands?
ls [a-z][A-Z]??.[uk] ls big* ls a???a ls ??*
Shell and wildcards
editA shell command line can be a simple command or more complex.
ls -l [fF]* ls *.c | more ls -l [a-s]* | mail `users`
The first event in the shell is to interpret wildcards. Only the shell interprets unquoted wildcards.
Quoting and Comments
editQuoting
editDo quote to prevent the shell to interpret the special characters and to transform multiple words into one shell word.
- 'string' Nearly everything within the quote is literal:
echo 'He did it, "Why?"' echo 'Because "#@&^:-)"' echo '$VAR='Me
- "string" Like 'string' but interprets $, \, !:
echo "What's happening?" echo "I don't know but check this $ANSWER"
- The backslash \ treats the following character as literal:
echo \$VAR=Me echo What\'s happening\?
- How could we display the backslash? With the following line:
echo \\
Comments
editYou can add comments in a command line or a script. Use the character #. A white space must immediately precede #.
Examples:
echo $HOME # Print my Home directory echo "### PASSED ###" # Only this part is a comment echo The key h#, not g was pressed.
Commands
- cat, tac: Concatenate files and print on the standard output.
- head, tail: Output the first and last part of files.
- nl: Number lines of files.
- wc: Print the number of bytes, words, and lines in files.
- cut: Remove sections from each line of files.
- tr: Translate or delete character.
- expand, unexpand: Convert tabs to spaces and space to tabs.
- paste: Merge lines of files.
- join: Join lines of two files on a common field.
- uniq: Remove duplicate lines from a sorted file.
- split: Split a file into pieces.
- fmt: Simple optimal text formatter.
- pr: Convert text files for printing.
- sort: Sort lines of text files.
- od: Dump files in octal and other formats.
Concatenate files
editTo concatenate files, use cat.
cat [options] [files...] tac [options] [files...]
The results are displayed to the standard output.
Common options:
-s: never more than one single blank line. -n: number all output line.
Examples:
cat file # Display file to the stdout. cat chapter* # Display all chapters to stdout. cat -n -s file # Display file with line number with single blank line.
To concatenate files in reverse order, use tac.
View the beginning and the end of a file
editTo view only few lines at the beginning or at the end of a file, use head or tail.
head [options] [files...] tail [options] [files...]
The results are displayed to the standard output.
Common options:
-n: number of lines to be displayed. (head and tail) -c: number of bytes to be displayed. (head and tail) -f: append output. (tail) -s #: iteration for new data every # sec. (tail)
Examples:
head file # Display the first 10 lines of file. head -n 2 file # Display the first 2 lines of file. tail -c 10 file # Display the last 10 bytes of file. tail -f -s 1 /var/log/messages Display the last 10 lines of messages, block and check for new data every second.
Numbering file lines
editTo add the line number to a file, use nl.
nl [options] [files...]
The results are displayed to the standard output.
Common options:
-i #: increment line number by #. -b: numbering style: a: number all lines t: non empty line n: number no line -n: numbering format: rz: right justified ln: left justified.
Examples:
nl file # Add the line number in each line in the file. nl -b t -n rz file # Add the line number to each non-empty line with zero-completed format.
Counting items in a file
editTo print the number of lines, words and characters of a file, use wc.
wc [options] [files...]
The results are displayed to the standard output.
Common options:
-c: the byte size. -m: the number of character. -w: the number of word. -l: the number of line. -L: length of the longest line.
Examples:
wc *.[ch] # Display the number of lines, words, and characters for all files .c or .h. wc -L file # Display the size of the longest line. wc -w file # Display the number of words.
Cutting fields in files
editTo remove sections from each line of files, use cut.
cut [options] [files...]
The results are displayed to the standard output.
Common options:
-b #: Extract the byte at position #. -f #: Extract the field number #.
Examples:
cut -b 4 file # Extract and display the 4th byte of each line of file. cut -b 4,7 file # Extract and display the 4th and 7th byte of each line. cut -b -2,4-6, 20- file # Extract character 1 and 2, 4 to 6 and 20 to the end of the line for each line of file. cut -f 1,3 -d: /etc/passwd # Extract the username and ID of each line in /etc/passwd.
The default delimiter is TAB but can be specified with -d.
Characters conversion
editTo translate the stdin to stdout, use tr.
tr [options] SET1 SET2
Common options: -d: delete character in SET1. -s: replace sequence of character in SET1 by one.
Examples:
tr ‘a‘ 'A' # Translate lower a with A tr ‘[A-Z]’ ‘[a-z]’ # Translate uppercase to lowercase tr -d ‘ ‘ # Delete all spaces from file
To convert tabs to spaces, use expand and to convert spaces to tabs, use unexpand.
expand file unexpand file
Lines manipulation
editTo paste multiple lines of files, use paste.
paste [options] [files...]
Common options:
-d #: delimiter: Use # for the delimiter. -s: serial: paste one file at the time.
Examples:
paste f1 f2 # Display line of f1 followed by f2. paste -d: file1 file2 # Use ':' for the delimiter.
To join multiple lines of files, use join.
join file1 file2
To remove duplicated line, use uniq.
uniq [options] [files...]
Common options:
-d: only print duplicated lines. -u: only print unique lines.
Examples:
uniq -cd file # Display the number of duplicated line.
Splitting files
To split big files, use split.
split [options] file
Common options:
-l #: split every # lines. -b #: split file in bytes or b for 512 bytes, k for 1Kbytes, m for 1 Mbytes.
Examples:
split -l 25 file # Split file into 25-line files. split -b 512 file # Split file into 512-byte files. split -b 2b file # Split file into 2*512-byte files.
Formatting for printing
editTo format a file, use fmt.
fmt [options] [files...]
Common options: -w #: maximum line width.
Examples:
$ fmt -w 35 file # Display 35-character lines width.
To format a file for a printer, use pr.
pr [options] [files...]
Common options: -d: double space.
Examples:
$ pr -d file # Format file with double-space.
Sort lines of text files
editTo sort the lines of the names files, use sort.
sort [options] file
The results are displayed to the standard output.
Common options:
-r : Reverse -f : Ignore case -n : Numeric -o file: Redirect output to file -u : No duplicate records -t; : Use ';' as delimiter, rather than tab or space.
Examples:
sort file -r sort file -ro result
Binary file dump
editTo dump a binary file, use od.
od [options] file
The results are displayed to the standard output and start with an offset address in octal.
Common options:
-c: each byte as character -x: 2-byte in hex -d: 2-byte in decimal -X: 4-byte in hex. -D: 4-byte in decimal.
Examples:
$ od -cx /bin/ls 0000000 177 E L F 001 001 001 \0 \0 \0 \0 \0 \0 \0 \0 \0 457f 464c 0101 0001 0000 0000 0000 0000 0000020 002 \0 003 \0 001 \0 \0 \0 224 004 \b 4 \0 \0 \0 0002 0003 0001 0000 9420 0804 0034 0000 0000040 ° ² \0 \0 \0 \0 \0 \0 4 \0 \0 006 \0 ( \0 b2b0 0000 0000 0000 0034 0020 0006 0028 0000060 032 \0 031 \0 006 \0 \0 \0 4 \0 \0 \0 4 200 004 \b 001a 0019 0006 0000 0034 0000 8034 0804
Exercises
edit- Use wildcard characters and list all filenames that contain 'x' followed by 'in' in the /etc directory.
- Use wildcard characters and list all filenames that start with any character between 'a' and 'e' that have at least two more characters and do not end with a number.
- Use wildcard characters and list all filenames of exactly 4 characters and all filenames starting with an uppercase letter. Do not descend into any directory found.
- Use wildcard characters and list all files that contains 'sh' in /bin.
- Display you environment variable HOME preceded by the string "$HOME value is:"
- Display the contents of $SHELL with two asterisk characters before and after it.
- How would you display the following string of characters as is with echo using double quote and \.
- @ # $ % ^ & * ( ) ' " \
- Compose echo commands to display the following two strings:
- That's what he said!
- 'Never Again!' he replied.
- Display the number of words in all files that begin with the letter h in the /etc directory.
- How would you send a 2M file with two 1.44M floppy. How would you put back together the pieces of the split file?
- What is the command to translate the : delimiter in /etc/password by #?
Perform basic file management
editOverview
editDescription: Candidates should be able to use the basic UNIX commands to copy, move, and remove files and directories. Tasks include advanced file management operations such as copying multiple files recursively, removing directories recursively, and moving files that meet a wildcard pattern. This includes using simple and advanced wildcard specifications to refer to files, as well as using find to locate and act on files based on type, size, or time.
Key files terms and utilities include:
cp
find
mkdir
mv
ls
rm
rmdir
touch
file globbing
cp: Copy files and directories. ls: List directory contents.
Create and Remove directories
editTo create a directory, use mkdir.
mkdir [options] dir
Common options:
-m mode: set permission mode. Default use umask. -p parent: create parent directory as needed.
Examples:
mkdir -m 0700 bin mkdir -p bin/system/x86
To delete an empty directory, use rmdir.
rmdir [options] dir
Common options:
-p parent: remove empty subdirectories.
Examples:
rmdir tmp rmdir -p bin/system/x86
Copy files and directories
editTo copy one file to another, or to a directory, use cp.
cp [options] source target
Source and target can be a file or a directory.
Common options:
-i interactive: prompt to overwrite -r recursive: copy the subdirectories and contents. Use -R for special files. -f force: force the overwriting
The default is to silently clobber the target file. Does not alter the source.
Examples:
cp *.[a-z] /tmp cp readme readme.orig cp ls /bin cp -ri bin/* /bin
Move & Rename files
editTo rename a file or directory or to move a file or directory to another location, use mv.
mv [options] source target
Source and target can be a file or a directory.
Common options:
-i interactive: prompt to overwrite -f force: force the overwriting -v verbose
The default is to silently clobber the target file.
Examples:
mv *.[a-z] /tmp mv readme readme.orig mv ls /bin mv -fi bin/* /bin
Listing filenames and information
editThe command to list files in the current directory is ls.
ls [options] [filenames]
Common options are:
-l for a long format -F Append a file type character -a All files, including hidden -R Recursive listing of subtree -d Do not descend into directory
The ls is equivalent to the dir command on DOS.
Examples of ls output:
$ ls -l /bin/ls -rwxr-xr-x 1 root root 46784 mar 23 2002 /bin/ls $ ls -ld /bin drwxr-xr-x 2 root root 2144 nov 5 11:55 /bin $ ls -a . .bash_history .bash_profile .bashrc ... $ ls -dF /etc .bashrc /bin/ls .bashrc /bin/ls* /etc/
File types
editThe long format means:
$ ls -l /etc/hosts #List a long format of the file hosts -rw-r—r-- 1 root root 677 Jul 5 22:18 /etc/hosts
File content and location Linux/Unix does not distinguish file by filename extension, like Windows. To determine the file content use file.
$ file /etc .bashrc /bin/ls /dev/cdrom /etc: directory .bashrc: ASCII English text /bin/ls: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped /dev/cdrom: symbolic link to /dev/hdc
To determine if a command is a built-in shell command or a program, use type, and use which to find its location.
$ type cp cd ls which type cp is /bin/cp cd is a shell builtin ls is aliased to `ls $LS_OPTIONS' which is aliased to `type -p' type is a shell builtin $ which cut /usr/bin/cut
Creating and using filenames
editFilenames can be created with:
- I/O redirection
cat chapter1 chapter2 > book
- An editor, such as vi.
vi mynewfile
- Many of the Unix utilities
cp file newfile
- An application
netscape
- The touch command, which creates empty files (or updates the "date modified" of existing files)
touch memo
The valid filename may have:
- Maximum 255 characters per filename
- Case sensitive 'A' and 'a' are different
- Any character except forward '/'
- Recommended alphanumeric characters as well as plus, minus, and underscore characters.
- Characters to avoid
- Hypen character.
- touch my-file -lt
- White space.
touch more drink touch "more drink"
- Most other special characters !@#$%^&*():;"'}{|\<,>.?~`
touch memo*
Remove files or directories
editTo remove files or subtree directories, use rm.
rm [options] files
Files can be a file or a directory.
Common options:
-i interactive: prompt to for each removal -f force: force the overwriting -r recursive: remove subtree directories and contents
There is no unremove or undelete command.
Examples:
rm *.[a-z] rm readme readme.orig rm ls /bin rm -rfi /bin cd; rm -rf * .* # !! this removes all files of in the home directory of the current user, as well as those in the subdirectories !!
Locating files in a subtree directory
editTo search for a file in a subtree directory, use find.
find [subtrees] [conditions] [actions]
The command can take multiple conditions and will search recursively in the subtree.
Some possible conditions are:
-name [FNG] # Search fo the FNG name -type c # Type of file [bcdfl] -size [+-]# # Has a +- size in blocks (c:bytes,k:kilo) -user [name] # Own by user -atime [+-]# # Accessed days ago +n means not been accessed for the last n days -n means been accessed the last ndays. -mtime [+-]# # Modified days ago -perm nnn # Has permision flags nnn
Some possible actions are:
-print # Print the pathname -exec cmd {} \; # Execute cmd on the file -ok cmd {} \; # Same as -exec but ask first
Examples
find . -name '*.[ch]' -print find /var /tmp . -size +20 -print find ~ -type c -name '*sys*' -print find / -type f -size +2c -exec rm -i {} \; find / -atime -3 -print find ~jo ~toto -user chloe -exec mv {} /tmp \;
To locate a binary, source file, or man pages, use whereis.
Whereis [options]
Common options:
-b: Search only for binaries. -m: Search only for manual sections. -s: Search only for sources.
Examples:
$ whereis host host: /usr/bin/host /etc/host.conf /usr/share/man/man1/host.1.gz $ whereis -m host host: /usr/share/man/man1/host.1.gz
To locate a file located somewhere defined by the PATH variable, use which.
$ which -a ls /bin/ls
The -a will look for all possible matches in PATH, not just for the first one.
Exercises
edit- Compose an interactive command to remove all .tmp file in your home directory. Respond y to every prompt.
- List all the files in users home directories ending with .pdf that are bigger than 50 blocks and have not been accessed for a month.
- Create a file file.h that will contain all the filenames ending with .h found in the /usr directory.
- Do a touch on all the c files found in /usr/src/packages directory.
- What are the default permissions when you create a new file and a new directory?
- How would you create a new file or directory that contains a space in the filename? (Example: 'new dir')
- What is the command to remove all the files of type char and block in your home directory?
- How to find the location of the programm find ?
- Delete all files in /tmp witch are not owned by root and have not been accessed for a week.
Use streams, pipes, and redirects
editOverview
editDescription: Candidates should be able to redirect streams and connect them in order to efficiently process textual data. Tasks include redirecting standard input, standard output, and standard error, piping the output of one command to the input of another command, using the output of one command as arguments to another command and sending output to both stdout and a file.
Key files terms and utilities include:
tee
xargs
<
<<
>
>>
|
` `
Standard input and standard output
editFor each command executed in a terminal, there are: a standard input value 0. (default keyboard) a standard output value 1.(default terminal) and a standard output for the errors value 2. (default terminal).
Each channel can also be identified by an address: &0 for input, &1 for output, And &2 for errors.
Each channel [n] can be redirected. [n]< file: Default value of n is 0 and it reads standard input from file. [n]> file: Default value is 1 and it sends standard output to file, overwriting the file if it exists. (clobber) [n]>>file: Default value is 1 and it appends standard output to file. <<word: Read standard input until word is reached. `command`: Substitute the command name by the result.
Examples:
$ pwd > file # out=file in=none error=terminal.
cat chap* >book # out=book in=none error=terminal.
mv /etc/* . 2>error # out=terminal in=none error=error.
echo end of file >> book # out=book in=none error=terminal.
set -o noclobber # Shell does not clobber existing files.
ls > list 2>&1 # ls and errors are redirected to list.
ls 2>&1 > list # Errors are redirected to standard output and ls output is redirected to list.
cat `ls /etc/*.conf` > conffile 2>>/tmp/errors
Concatenate all the configuration files from /etc dir in conffile and append errors in file /tmp/errors.
Redirecting with pipes
Pipes are an efficient way to apply multiple commandes concurrently.
command1 | command2
The standard output of command1 will be piped to the standard input of command2. The standard error is not piped.
Examples:
ls -l /dev | more ls -l /etc/*.conf | grep user | grep 500 ls -l /bin | mail `users`
To redirect the standard output to a file and to the terminal at the same time, use tee.
ls -l /dev | tee file ls -l /etc | tee -a file # Append to the file
Building arguments
The xargs utility constructs an argument list for a command using standard input.
xargs [options] [command]
The xargs command creates an argument list for command from standard input. It is typically used with a pipe.
Common options: -p: prompt the user before executing each command.
Examples:
ls f* | xargs cat # Print to standard output the content of all files starting with f. find ~ -name 'proj1*' print | xargs cat
Search in the Home directory for files starting with proj1 and send it to the standard input of cat.
Use the /dev/null device file to discard output or error messages.
Try the following:
grep try /etc/* grep try /etc/* 2> /dev/null grep try /etc/* > /dev/null 2> /dev/null
Exercises
edit- Create a file list.bin that will contain all the filenames from /bin directory.
- Write a command that will append the list of files from /usr/local/bin to the file named list.bin and discard any error output.
- Split list.bin file into files that are 50 lines long and remove list.bin.
- From the splitted files create back list.bin. but with inversed order.
- Simplify the following commands:
ls *.c | xargs rm ls [aA]* | xargs cat cat `ls *.v` 2>/dev/null
- Use find to do the following command:
more `ls *.c`
- Write a command that will create a file list.sbin with the contents of /sbin and at the same time display it to standard output.
- Create a file that within the filename you include the creation time.
- Create a file that will contain all the filename in reverse order with extension .conf from the /etc directory.
Create, monitor, and kill processes
editOverview
editDescription: Candidates should be able to manage processes. This includes knowing how to run jobs in the foreground and background, bring a job from the background to the foreground and vice versa, start a process that will run without being connected to a terminal and signal a program to continue running after logout. Tasks also include monitoring active processes, selecting and sorting processes for display, sending signals to processes, killing processes and identifying and killing X applications that did not terminate after the X session closed.
Key files terms and utilities include:
&
bg
fg
jobs
kill
nohup
ps
top
Create processes
editA running application is a process. Every processes have: A process ID. A parent process ID. A current directory PWD. A file descriptor table. A program which it is executing. Environment variables, inherited from its parent process. Stdin, stdou, and stderr Other
Bash is a program that when it is executed becomes a process. Each time you execute a command in a shell a new process is created. Except for the buil-in shell command. They run in the shell context. Use type to check if a command is a built-in shell command.
Example type cp ls which type Monitor processes Once the system is up and running from a terminal it is possible to see which processes are running with the ps program. To display a long format of all the processes in the system, do:
ps -Al F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 004 S 0 1 0 0 80 0 - 112 do_sel ? 00:00:04 init 004 S 0 381 1 0 80 0 - 332 do_sel ? 00:00:00 dhcpcd 006 S 0 1000 1 0 80 0 - 339 do_sel ? 00:00:00 inetd 044 R 0 1524 1222 0 79 0 - 761 - pts/3 00:00:00 ps
The ps program will display all the processes running and their PID numbers and other information. To see a long format of the processes in your login session, do a:
ps -l F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 000 S 500 1154 1139 0 80 0 - 724 wait4 pts/1 00:00:00 bash 002 S 500 1285 1283 0 77 0 - 24432 wait_f pts/1 00:00:00 soffice.bin 040 R 500 1442 1435 0 79 0 - 768 - pts/4 00:00:00 ps F: Process Flags 002: being created, 040: forked but didn't exec, 400: killed by a signal. S: Process States: R: runnable, S: sleeping, Z: zompbie UID: User ID, PID: Process ID, PPID: Parent Process ID, C: Scheduler, PRI: priority NI: Nice value, SZ: size of routine, WCHAN: name of routine
Monitor processes
editTo monitor the processes in real-time, use top. top
9:20am up 2:48, 4 users, load average: 0.15, 0.13, 0.09
78 processes: 75 sleeping, 3 running, 0 zombie, 0 stopped CPU states: 15.3% user, 0.3% system, 0.0% nice, 84.2% idle Mem: 254896K av, 251204K used, 3692K free, 0K shrd, 27384K buff Swap: 514072K av, 0K used, 514072K free 120488K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 1517 rarrigon 0 0 40816 39M 17372 R 15.0 16.0 2:59 mozilla-bin 1727 rarrigon 19 0 988 988 768 R 0.3 0.3 0:00 top 1 root 20 0 220 220 188 S 0.0 0.0 0:04 init 2 root 20 0 0 0 0 SW 0.0 0.0 0:00 keventd
RSS: The total amount of physical memory used by the task. SHARE: The amount of shared memory used by the task. %CPU: The task's share of the CPU time. %MEM: The task's share of the physical memory. Once top is running it is also possible to execute interactive commands: Type N to sort tasks by pid. Type A to sort tasks by age (newest first). Type P to sort tasks by CPU usage. Type M to sort tasks by memory usage. Type k to kill a process (prompted for pid).
Kill processes
editThe ps program will display all the processes running and their PID numbers. Once the PID is known, it is possible to send signals to the process. SIGSTOP to stop a process. SIGCONT to continue a stopped process. SIGKILL to kill a process.
The program to send a signal to a process is called kill.
kill -SIGKILL [pid] kill -63 [pid] kill -l
By default a process is started in foreground and it is the only one to receive keyboard input. Use CTRL+Z to suspend it.
To start a process in background use the &.
bash & xeyes &
Job control
In a bash process it is possible to start multiple jobs. The command to manipulate jobs are:
jobs # List all the active jobs bg %job # Resume job in background fg %job # Resume job in foreground kill %job # Kill background job
When bash is terminated all processes that have been started from the session will receive the SIGHUP signal. This will by default terminate the process.
To prevent the termination of a process, the program can be started with the nohup command.
nohup mydaemon
Exercises
editModify process execution priorities
editOverview
editDescription: Candidates should should be able to manage process execution priorities. Tasks include running a program with higher or lower priority, determining the priority of a process and changing the priority of a running process.
Key files terms and utilities include:
nice
ps
renice
top
Priorities
editTo start a command with an adjusted priority, use nice.
nice -n +2 [command]
nice -n -19 [command]
The program nice changes the base time quantum of the scheduler.
i.e. it informs the scheduler how important a process is, which is used as a guide to how much CPU time to give it.
For example if you wanted to listen to music whilst ripping another CD, you could use nice -n +5 oggenc, you shouldn’t get any “hops” in the music playback, as the scheduler “knows” the oggenc process is less important.
The value can go from -19 (highest priority) to +20 (lowest priority).
The default value is 0. Only root can set a value below zero.
To modify the priority of a running program, use renice.
renice +1 -u root
# Change the priority for all root processes.
renice +2 -p 193
# Change the priority for PID 193
Exercises
edit- Which user and root processes are using most of the memory?
- Open two terminals. In one terminal type the following:
while [ 1 ] do echo -n The date is:; date; done
From the other terminal see that you can stop and continue the print out.
3) Same start as 2) but, make the print out to stop for 3[s] and to continue for 1[s] repeatedly. 4) Make a shell script to renice all process called apache to a 19 value. 5) Do a print from ps formated as: “username”, “command”, “nice value” 6) Kill all the process called “bash” and owned by user polto..
Regular Expression
editOverview
editDescription: Candidates should be able to manipulate files and text data using regular expressions. This objective includes creating simple regular expressions containing several notational elements. It also includes using regular expression tools to perform searches through a filesystem or file content.
Key files terms and utilities include:
grep
regexp
sed
Pattern matching
editThere are two kinds of pattern matching:
- Wildcards (File Name Generation)
- Regexp (Regular Expression)
Wildcard characters are mainly applied when they are used in the current directory or subdirectories. When wildcard characters *, ?, [ - ], ~, and ! are used in regexp they no longer generate filenames.
Some of the utilities that use regexp are:
- grep, egrep
- vi
- more
- sed
- Perl
Limited regexp search pattern Used by all utilities using regexp.
- Any 1 char . Ab.a Abla or Abca
- 1 char set [ ] Ab[sd]a Absa or Abda only
- 1 char range [ - ] Ab[a-z]a Abaa or Abba or ...
- Not in set [^ ] Ab[^0-9]a Abaa or Abba or ...
- 0 or more * Ab*a Absala or Aba or ...
- Begin line ^ ^Aba Line starts>Aba
- End line $ Aba$ Aba<line ends
- Literal \ Aba\$ Aba$
Example:
Ab[0-3]s ^Ab\^bA [01]bin$ ^..\\ [^zZ]oro
Combinations of limited regexp Combinations used by all utilities using regexp.
- Any string .* Ab.*a Abrahma or Abaa or ...
- String from [ ]* th[aersti]* There or This or ...
- Multi range [ - - ] Ab[0-2][a-c]a Ab0aa or Ab1aa or ...
- Match \ \\ \\[a-zA-Z]* \Bethoven
Examples:
Ab[0-3][a-z]s ...$ ^[01]\^2 [0-9][a-z] \$ [a-zA-Z]* ^[^c-zC-Z]* ^[a-zA-Z0-9]$
Modifier patterns Replace strings matched by regexp patterns
- Match m \{m\} b[0-9]\{3\} b911
- One or more \{m,\} b[0-9]\{2,\} b52
- Up to n \{m,n\} b[0-9]\{2,4\} b1234
- Beginning of word \< \<wh where
- End of word \> [0-9]\> bin01
grep
editTo find text in a file, use grep.
grep [options] [string] [files]
Best to quote the string to prevent interpretation.
Common options: -i: Ignore case -l: List filename only if at least one matches -c: Display only count of matched lines -n: Also display line number -v: Must not match.
Examples:
grep host /etc/*.conf grep -l '\<mai' /usr/include/*.h grep -n toto /etc/group grep -vc root /etc/passwd grep '^user' /etc/passwd grep '[rR].*' /etc/passwd grep '\<[rR].*' /etc/passwd
sed
editTo apply a command on a stream, use sed.
sed [address1][,address2][!]command[options] [files...]
The program sed will apply a command from address1 to address2 in a file. The address1 and address2 format are regular expressions. The sed program is a noninteractive editing tool.
Examples:
sed '1,3s/aa/bb/g' file # Replace in file from line 1 to 3 aa by bb. sed '/here/,$d' file # Delete line from here to the end. sed '/here/d' file # Delete lines with word here. sed '1,/xxx/p' file # Print lines from 1 to xxx. sed '/ll/,/ff/!s/maison/house/g' file # In file replace words maison by house excluding lines from ll to ff.
Exercises
edit- Process your bookmarks.html file to produces a list containing just the web site titles in a file called mywebsites.txt
- Copy all the files from /etc in your home direcory etc/. Display the contents of all the *.conf files by replacing the word host by machine.
- Display the contents of all the *.conf files that don't contain the root word. What is the command using grep and sed?
- Print out all the group names where root belongs to.
- List all the group names that are 4 or 5 characters long.
- List all the files that contain character lines without spaces (blank).
- List in etc/ directory all the files that contain numerical characters.
- Print with ls only the directory names in /.
- Do “ps -aux” and replace user r_polto by root and print it in to a file called new_process.txt
- List all process called apache and owned by a usernames starting by “p” or “P”..
Perform basic file editing using vi
editOverview
editDescription: Candidates should be able to edit text files using vi. This objective includes vi navigation, basic vi nodes, inserting, editing, deleting, copying, and finding text.
Key files terms and utilities include:
vi
/, ?
h,j,k,l
G, H, L
i, c, d, dd, p, o, a
ZZ, :w!, :q!, :e!
:!
vi
editWhen using X-Windows, you can use mouse-oriented editors such as xedit. In a cross-development environment, users use their favorite editor. On a non-windowing system, you only need a keyboard editor such as vi. The vi editor on Linux is the same as on any Unix systems. vi has two modes:
- Command mode: Anything you type will be interpreted as a command
- Input mode: Anything you type will be inserted into the file
Transition from one mode to another Command mode to Input mode: i, I, a, A, o, O keys Input mode to Command mode: ESC key The default starting mode is the Command mode The file configuration .exrc can be created in your HOME directory to set up some vi behavior.
set ignorecase # No case-sensitive set tabs=3 # 3 space for tab character
Perform basic file editing using vi
Enter Input mode
- i before cursor
- I at start of line
- a after cursor
- A at end of line
- o open line below
- O open line above
Delete
- dw delete word
- dd delete line
- D delete to end of line
- x delete char at cursor
Move cursor
- l one space right
- h one space left
- j one line down
- k one line up
- $ end of line
- ^ start of line
- w next word
- e end of word
Exercices
edit- Use vi from any directory to begin editing an empty buffer.
- Enter a few lines of text into this buffer.
- Save the contents of the buffer to the directory.
- Open the file again with vi.
- Create a new line beneath what you typed earlier. (Don't use i from command mode!)
- Exit vi without saving these changes.