Shell Script - Short Learning for big future.

From a long time, i wanted to write a summarized view of various features available in shell scripting. Finally after reading 2 books of shell scripting, I am trying to summarized the best features so that it can be completed in minimum amount of time by a developer who wants a quick cheat sheet type of page, where one can find all the daily useful commands & remember it for long time. here is the pointed form of the summary:-

1. Cat /etc/shells    -> Lists the shells available

2. echo $SHELL   -> Get current running shell
3. ps $$   -> gives the PID of the shell running in the system.
4. type -a ls -> Gives whether ls is external or internal command, with path
5. Bash Startup Scripts
   a. /etc/profile - The systemwide initialization file, executed for login shells
   b. $HOME/.bash_profile, $HOME/.bash_login, and $HOME/.profile runs second when a user logs in in that order.
   c. $HOME/.bash_logout invoked when you log out.
6. apt-cache search shell -> To find the list of available shell packages under ubuntu
7. which bash -> Will give you the full path of shell command (try Whereis -> this will give the location+man location of this command )
8. vi hello.sh -> Create a new file and write echo "hello world" in it. Execute chmod +x  hello.sh and then execute ./hello.sh to execute the commands under it.

9. #!/bin/bash -> this is the Shebang tha tis to be used at the first line of every script

10. Execute a script also via "bash hello.sh"

11. Debug mode: "bash -xv hello.sh"   or use set -x/+x , -v/+v, -n/+n in script

12. Env or printenv -> list all the system variables

13. Find /home -name *.c -> will find all C files under /home

14. printf "${PATH}\n"   -> Use printf  extensively as like in C.

15. personName="Pankaj Bhatt"
    echo "Person name is ${personName}

16. lastName=${variable:-"bhatt"}    -> If variable is empty it will assign value 'bhatt'

17. Sleep 3  -> Halt the execution of 3 sceonds.

18. Echo *.conf -> Show all the conf files within current directory
19. Export -> This command is used to export variables to all child processes
20. Export -n -> Gives the list of all variables exported to this shell.
21. Unset varname -> Will unset the value of the variable.

22. read -p "Enter your name : "name   -> For reading the value in an Variable

23. Read arguments -t 10 For Timeout input (abort if don’t enter),-s for password

24. ans=$((x +y ))  -> To Do the mathematical computation for two variables, if either x or y is can't converted to INTEGER it should be used as ZERO.

25. declare -i num1=10 -> declare an integer variable. (Assigning else results 0)

26. readonly varName = value -> being used to create a constant OR
    declare -r varname=value

27. varc=${varName:?Error varName is not defined or is empty} -> To check whether varName is defined or not, otherwise  the remaining string will be used to display the error.

28. let sum=no1+no2 :- Let command can be used to do mathmatical operations without using $ in front of variables. Or $[ no1 + no2 ] can also be used and results can be assigned to result.

29. bc comamnd can be used to do mathematical computation. echo "4 * 0.56" | bc

30. Export JAVA_HOME= /usr/bin/java/jdk   -> This command will treat java_home as env variable going forward.

31. Whatis ls -> This command gives a short description of the command from man pages.

32. history -> Command is used to see the history of commands you typed. CTRL-R is used to search the command while typing. ! will execute the command from command history.

33. !! -> This will execute the last command you typed.

34. A {} is used to repeat a pattern. E.g. echo file{1,2,3}.txt will print -> file1.txt file2.txt file3.txt

35. Wildcard (*,?.[..]) supports the above pattern.  E.g. ls *.{c,java}

36. ls [ab]*.sh -> only lists the file starting with a or b. [a-d]*.sh   -> Only a-d starting filenames.

37. ls -d */ -> Gives you a list of all the directories present in the current pwd.

38. Alias -. It list all the alias present in the system.
1. Alias ls ='ls -lah' -> To create alias for any command
2. Unalias ls -> to Remove the Alias
   Alias remove with the session, to store them permanently store them in /{HOME}/.bashrc file

39. STARTUP Scripts
1. /etc/profile -> Linux wide system properties file. Runs first when a user logs in.
2. /etc/profile.d -> directly contains all files that Runs SECOND when user logs in.
3. ${HOME}/.bash_profile -> Runs third when user logs in. Internally this calls .bashrc in HOME directory. Use this for setting anything. Once done type n press "bash" for to refresh the settings placed in .bashrc file.

40. cat command is being used for seeing the contents on the screen.

41. PS1 is the variable that holds the command prompt you use. Set it in .bashrc file, via using export PS1=''${HOME}"

42. CTRL-D is the command, which is being used to come out of the shell.

43. Setting Shell options:-
1. set -o -: This will list all the shell configured variables.
2. set -o variableName :- This will unset a specific variable listed from the above command.
3. set +o variableName :- This will set a specific variable listed from the above command.

44. shopt -> to list some of the variables configured for the environment. For setting it use "shopt -s " and for unset -u option
45. Setting Env Variables :- Modify ~/.bash_rc  and include line EXPORT =""
46. System Vide Shell Options: in /etc/profile.d directory create a file java.sh (with execute permissions)  and write export java commands in it.E.g. export JAVA_HOME=/usr/lib/java/jdk1.6u50 ;     \n export PATH=$PATH:$JAVA_HOME/bin
47. test command is being used for checking the file existence or test values
    test -f /bin/bash && echo "File Exists" || echo "File does not exists."      -> It will print File Exists
48. If [test] CONDITION ->  then  … … …else … … .. -> fi     ( You can also use elif in place of else if  } 
    if [ $myAge -gt 0 ]   -> This is also a valid condiion

49. Exit status of the command -> Use $? To get the status of the earlier command executed

50. Connection command e.g.  > Second set of commands will execute only when first is successful

test! -d /tmp/foo && { read-p "Directory /tmp/foo not found. Hit [Enter] to exit..."enter; exit 1; }

|| Operator -> echo "ram here" >> /dev/null || echo "I M Executing"  -> second will only execute on failure of first.

51. -ge, -eq, lt, -le, -gt, -ne
52. test -z -> to check that the string is not empty &length is not  zero
53. File operation with test :  -a : if file exists, -b : if file exists and block special file , -d : is a directory , -e : if file exists, -f : regular file , -s size
54. Command line arguments: $0, $1, $2, $3, $4 etc.   $*/$@ -> Represents all command line arguments.
55. Parameters set by shell: $*, $@, $#, $-, $?, $$, $!   , ($? -> Represents the status of the last executed command)

56. exit N -> Where N represents the code that is being returned for the script execution status code. 0 means success

57. case $val in \n  "pankaj") echo "got it" ;; \n "abc") echo " This is second change";; \n *) echo "Default Case";;   esac
    The pattern to be matched can contain regular expression characters.
    You can use "shopt -s nocasematch" in the script, to ignore case comparison while case matching in SWITCH case.
58. For $variable in (some kind of list, can be string/numbers etc) \n do \n cmd1;cmd2;cmd3;  done
    C Like for loop also works here. for (( i =1; i <=5; i++ ))

59. While [ $n -eq 0 ] \n do \n cmd1; cmd2; cmd3 ; \n done
    while : \n do cmd1; cmd2; cmd3 \n done -> This is an indefinite loop or (while true ) or (while false)

60. Until [ condition ] \n do \n cmd1; cmd2; cmd3; \n done   -> Reverse of While, until the condition becomes true, it will run.

61. Select var in list \n do \n cmd1;cmd2;cmd3; \n done

62. Break is used to come outside of the innermost loop while break N is to come out of the N levels of loops.

63. continue is also used to keep on going the loop.
64. $(ls) -> This will execute a command enclose in $() and can assign it results in an variable.
65. < :- for input, > :- for output , 2> :- error redirection, &> :- use to redirect both output and error to the same place, 2>> Append error log
66. /dev/null :- All the data written on this is being discarded by the system.

67. bash -c "cd /home; ls -la"  -> This comman will execute the script script inline as it is being passed to the command.

68. cat abc.txt > /dev/lp0 -> print the file to the printer.

69. File Descriptor can be assigned to an input file or to an output file , exec 3< abc.txt  OR exec 3> abc.txt
70. Enclose multiple commands in single block : { cd /home; ls -la ; dir ; }
71. Watch -n 5 "ls -la; df -h ;" -> Watch the execution of this command very 2 seconds. Press CTRL-C to come out.
72. Command & -> To put the command in the background.

73. Pipes are a way to send the output of one command to another one.  e.g cat abc.txt > grep "ram"

74. pstree -> This is the command that will be used to display the tree of process with there childs.

75. pgrep -u pankaj,ram : This is used to list the processes of pankaj & ram user in the system.
76. pkill is a command to kill the processes that belongs to a group/user.
77. trap : trap is the command which is being used to trap (any of the SIGTERM signal) and execute a codes based on that.
78. Functions are being created in shell scripting. Just like in other languages, arguments are being accessible by $0,$1
    Eg.

myFun()

{

echo "Hello, how things are" $1

}

echo "start calling the function"

myFun Argument1
Any outside variable defined within the Function will change its value. To declare any local variable within function use "local nm=$1"
return command can be used to return a value from the function, if no value is being specified hen last command status will be returned.
We can write all the functions in a single script and then can include then file in our main script. E.g. if we have functions.sh at home directory then include it via ". /home/pankaj/functions.sh" . Then you can call every function present in that file. Or another way is to use the source command e.g. "source /home/pankaj/functions.sh".
you can use '&' operator to execute a function in the background. (just like a normal shell command).

79. Subshells are the shells within which your script runs within a shell. You need to EXPORT every variable/function that is defined in the main shell to get it accessed by the Subshell.

80. exec command can be used to replace this shell with specified program without swapping to a new subshell or process.

81. . (dot) command is used to run the script in the current shell, so that any parameters that are defined in the mains shell should be accessible, otherwise they wont be accessible to the script running under a subshell.
82. Compond command is a way to encapsulating multiple commands under a single umbrella. All commands will be executed and generate there output
83. You can also display dialog boxes while running your script. For this "sudo apt-get install dialog" and then you can use it in script
    E.g. dialog --title "hello world" --msgbox "Hello World 6 20"
84. The tty command is used to display the name of device file attached with the terminal. Or (file connected with standard input).

85. Colored output can be print on the console. E.g. echo -e "\e[1;31m This is red text \e[0m"
86. For every process in the system you can get the list of environment varialbles by  cat /proc/{PID}/environ
87. Get Length of a variable via "length=${#varName}
88. Tee command is used to get the data from the last command to store the output in an file and also show it to the screen
    E.g. echo "Ram teri ganga maili" | tee out.txt -> will store the contents in out.txt as well as it store it to the file.
89. /dev/stdout -> refers to the standard output and /dev/stdin -> Refers to the keyboard, /dev/stderr -> To screen, /dev/null -> discard all
90. You can also create  a array in shell scripts via arry_Name=("ram","shyam"); arry_Name[0]=100; echo ${arry_Name[$index]}
    To print all values of an Array Use . ${arry_Name[*]} or ${arry_Name[@]}
    An associate array is being used to use the STRING as the INDEX in the array. e.g. declare -A arry_Name

91. sleep 30 : is a a command that is being used to sleep fo r specific of time.

92. xargs command can be used to take input from STDIN n convert it to the parameters passed to  another command.
93. tr is the command which is used for Translating

94. Md5sum is the command to calculate the md5 of all the filenames that are present in the md5sum command.  Same as sha1sum.

95. Md5deep & sha1deep is used to calculate the md5 n sha1 of the complete directory.  (similarly base64 is also there)

96. Sort command is also being used to sort multiple filenames. Uniq is also used with sort to generate unique file name.
97. Mktemp is a command to create a temp file in the /tmp directory. Use (-d) to create a temp directory.

98. split command is used to break a file in multiple chunks. E.g. split -b 100k data.file  :It will break the chunks in 100K.

99. Csplit is an extended command of split that is used to split files based upon certain conditions (like presence of text etc).
100. rename is the command which is used to rename the files from one to anther and you can use WILDCARD character.
101. /usr/share/dict directory contain the dictionary. Aspell is the command used to find any word there.
102. dd is a command available through which you can generate large size files like 100M etc.

103. /dev/zero is a special character device, which infinitely returns the zero byte (\0).

104. comm is a command available through which we can generate the differences between two files.

105. Chmod is the command which is used the change the file permissions for user/groups/others
     E.g. chmod u=rwx g=rw o=r filename
106. chown command is use to change the ownership of the files. The format is : chown user.group filename
107. /etc/resolv.conf is the file that contains the list of DNS Servers.
108. chattr is the command which is used to make the file immutable. E.g. chattr +I fileName  (you even can't delete it, use -I to reverse this).

109. touch command is used to create the new file or modify the timestamp of the file to the current timestamp.

110. Symbolic  links are being created by  "ln -s  targetFileName  symbolic_link_name"

111. File command is being used to tell the type of file. E.g "file abc.sh"

112. Loopback files can be used to create a file system out of a single file. You can mount the file at a specific mount point.  See details in the LINUX shell Scripting Cookbook for details. Page no 124. You can also create the partition inside loopback images and mount it separately. You can also mount ISO files as loopback files.

113. /dev/zero is a file which will always contain zero, if you read from it.

114. cdrecord command is used to create an ISO into a CD-ROM.

115. Diff command is used to find the difference between two files/directories. A host of options are there, through which you can customize the output.
116. pushd and popd commands are being used to push a specific directory to the stack and then do a CD into it automatically. Dirs is the command that can be used to find the appropriate number of directories in the stack.
117. wc is the command which is used to count the number of lines/words/characters from a text file.
118. tree is the command use to print the complete tree of the complete file system. It does not come bydefault, you have to use it.
119. Grep is the command used to search a test across multiple files. E.g. grep "pattern" file1 file2. egrep is just an extended version of grep command with extended regular expresstions. Its same as "grep -E" Option. -o option is used to print only that line which match the pattern. "-c" is used to count. "-n" prints the line number.  "-R" recursively search over a directory. "-I" ignoring the case. Use  "-e" if there are multiple patterns to match with.  "--include" option is being used to match the files that needs to be included in the search. --include *.{c,cpp} . "--exclude" is used for exclusion of files. "-q" is being used for quiet output, then you can use return status of the command to check whether there are any matches or not.
120. cut is the command which is used to cut the text in a columnar fashion.

121. sed is being used for text replacements. E.g. sed 's/pattern/replace_string/' file. Use -I option to save the changes int eh file after replacements.  Awk is used for advanced text processing.

122. wget is a command which is used to download a file from the internet.

123. Lynx is a command available through which you can get the TEXT content of  a webpage.
124. Curl command used to hit a web page/API and get response. "--cookie" option is used to specify cookies. --cookie "user=ram;a=b". -H is being used to specify headers. curl -H "Host: www.slynux.org" -H "Accept-language: en" URL. -I/-head option to show only response headers.
125. tar -cvf output.tar file1 file2 file3 file4 file5  -> To compress all the files in an tar file
     tar -xf output.tar  -C /path/to/extractdirectory -> To uncompress all the files.
126. gzip is used to zip a file while gunzip is used to unzip the files. Similarly zip/unzip (-r is being used for recursively).
127. rsync is the command to keep in sync two files/directories E.g. rsync -av /opt/jboss/data slynux@192.168.0.6:/home/backups/data

128. To add a scheduler in linux, add a CRON expression based entry in CRONTAB file

129. Ifconfig is the command used to show the network interfaces. This command is used to set the static IP address on a specific network interface. ifconfig wlan0 192.168.0.80 netmask 255.255.255.0. To automatically get the IP from the DHCP use "dhclient eth0"
130. Route, host, nslookup , traceroute ( show all hops to the destination).
131. lsof -I & netstat are being used for port analysis.
132. Df (disk free)/ du (disk usage) . du -a directory : lists size of all files and then of complete directory. Du -c : shows all count sum of all files.         --exclude is used to exclude some file/directory for there usage calculation.

133. time command can be used to measure the amount of time taken by the command. E.g. "time ls -la" . In addtion, this command can be used to find exit status, number of signals received, number of context switches made etc.

134. Users Details: who/w/users/last/

135. logrotate is the command which is useful to configure the log rotation of various files in any directory of the file system. The configuration directory for every process is in /etc/logrotate.d
136. Fsck is the comman for file system scan. E.g. fsck /dev/sdb0. /etc/fstab is the file containing all the file systems.
137. Wall is a command which is used to write a message to all the terminals of all the users who are logged in the system.
138. Gathering system information:  hostname/uname -n/uname -a/ uname -r/uname -m
139. cat /proc/cpuinfo; cat /proc/meminfo/ , cat /proc/partitions; use lshw to get the info about complete file system.