Showing posts with label unix. Show all posts
Showing posts with label unix. Show all posts
UNIX one-liners continued
These one liners may be of help to identify user or group info...
1. To extract only the userid from the /etc/passwd file based on a string sequence in the GECOS field.
cat /etc/passwd | awk -F":" '/\[I]/ { print $1 }'
OR
cat /etc/passwd | awk -F":" '/GB\/I/ { print $1 }'
2. To list records in /etc/passwd of users who are members of the staff group.
grep <groupname> /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd ; rm patFile;
grep staff /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd ; rm patFile;
3. To list userids and home directories of users of the staff group.
grep <groupname> /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | awk -F ":" '{ print $1, $6}'; rm patFile;
grep staff /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | awk -F ":" '{ print $1, $6}'; rm patFile;
4. To list all the groups of which a userid is a member of
grep <userid> /etc/group | awk -F":" '{ print $1 }'
grep sabari /etc/group | awk -F":" '{ print $1 }'
5. To list all users of a group and further grep them by a department signifier in the GECOS field.
grep <groupname> /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | egrep \/I\/ | awk -F":" '{ print $1," ", $5 }'; rm patFile;
grep sys group.file | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | egrep \/I\/ | awk -F":" '{ print $1," ", $5 }'; rm patFile;
6. To print only one particular field from a file
cat <filename> |awk '{ print $5 }'
1. To extract only the userid from the /etc/passwd file based on a string sequence in the GECOS field.
cat /etc/passwd | awk -F":" '/\[I]/ { print $1 }'
OR
cat /etc/passwd | awk -F":" '/GB\/I/ { print $1 }'
2. To list records in /etc/passwd of users who are members of the staff group.
grep <groupname> /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd ; rm patFile;
grep staff /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd ; rm patFile;
3. To list userids and home directories of users of the staff group.
grep <groupname> /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | awk -F ":" '{ print $1, $6}'; rm patFile;
grep staff /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | awk -F ":" '{ print $1, $6}'; rm patFile;
4. To list all the groups of which a userid is a member of
grep <userid> /etc/group | awk -F":" '{ print $1 }'
grep sabari /etc/group | awk -F":" '{ print $1 }'
5. To list all users of a group and further grep them by a department signifier in the GECOS field.
grep <groupname> /etc/group | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | egrep \/I\/ | awk -F":" '{ print $1," ", $5 }'; rm patFile;
grep sys group.file | awk -F":" '{ print$4}' | sed 's/\,/ /g' | xargs -n1 echo > patFile ; grep -f patFile /etc/passwd | egrep \/I\/ | awk -F":" '{ print $1," ", $5 }'; rm patFile;
6. To print only one particular field from a file
cat <filename> |awk '{ print $5 }'
UNIX one liners - Very helpful in daily activities
In our day to day activities, we will be doing most of the routine tasks which needs many commands to run for completeing the task.
Below are few of the one-liner commands that can achieve our tasks.
These one liners may be of help to identify files to be acted upon by xargs.
1. To identify files in dir /tmp dated March with psx2 anywhere in the name :and remove them
find /tmp -type f -name *psx2* -ls | awk '$8=="Mar"' | awk -F" " ' { print $11 }' | xargs rm -rf
2. To identify multiple name formats dated Apr 10 and gzip them ...
cd /logs/of/surpriseme
find `pwd` -type f \( -name user1_log..txt -o -name system3_log.txt \) -ls | grep "Apr 10" | awk -F" " ' { print $11 }' | xargs gzip
3. To identify files dated March 20-31 and remove them.
find `pwd` -type f -ls | awk '$6=="Mar" && $7>19 && $7<32' | awk -F" " ' { print $11 }' | xargs rm -rf
find `pwd` -type f -ls | awk '$8=="Aug" && $9>19 && $9<22'
4. To find the total filespace used by User willy on a system.
find / -user willy -type f -ls 2> /dev/null | awk '{ sum += $7 } END { printf " %7.3f MB \n", (sum / (1024 * 1024)) }'
5. To find 20 biggest files on the /var filesystem showing their owner, group, size in MB and path:
find /var -type f -ls | sort -rn +6 | head -20 | awk -F " " '{ printf " %s \t %s \t %7.3f MB \t %s \n" , $5, $6, ( $2/1024), $11}'
6. To find files older than 12-Jan-2007 and move them to the /tmp dir. First identify a file with that date stamp in the file system. Then
find `pwd` -type f !-newer /absolute_path_to/file_with_cutoffpoint/datestamp - ls | awk -F " " '{ print $11 }' | xargs -I { } mv { } /tmp
7. To rename files names with extension .txt to .log
find $(pwd) -type f -name *.txt | xargs -i mv {} {}.log
Below are few of the one-liner commands that can achieve our tasks.
These one liners may be of help to identify files to be acted upon by xargs.
1. To identify files in dir /tmp dated March with psx2 anywhere in the name :and remove them
find /tmp -type f -name *psx2* -ls | awk '$8=="Mar"' | awk -F" " ' { print $11 }' | xargs rm -rf
2. To identify multiple name formats dated Apr 10 and gzip them ...
cd /logs/of/surpriseme
find `pwd` -type f \( -name user1_log..txt -o -name system3_log.txt \) -ls | grep "Apr 10" | awk -F" " ' { print $11 }' | xargs gzip
3. To identify files dated March 20-31 and remove them.
find `pwd` -type f -ls | awk '$6=="Mar" && $7>19 && $7<32' | awk -F" " ' { print $11 }' | xargs rm -rf
find `pwd` -type f -ls | awk '$8=="Aug" && $9>19 && $9<22'
4. To find the total filespace used by User willy on a system.
find / -user willy -type f -ls 2> /dev/null | awk '{ sum += $7 } END { printf " %7.3f MB \n", (sum / (1024 * 1024)) }'
5. To find 20 biggest files on the /var filesystem showing their owner, group, size in MB and path:
find /var -type f -ls | sort -rn +6 | head -20 | awk -F " " '{ printf " %s \t %s \t %7.3f MB \t %s \n" , $5, $6, ( $2/1024), $11}'
6. To find files older than 12-Jan-2007 and move them to the /tmp dir. First identify a file with that date stamp in the file system. Then
find `pwd` -type f !-newer /absolute_path_to/file_with_cutoffpoint/datestamp - ls | awk -F " " '{ print $11 }' | xargs -I { } mv { } /tmp
7. To rename files names with extension .txt to .log
find $(pwd) -type f -name *.txt | xargs -i mv {} {}.log
Difference Between Linux and UNIX
Most UNIX systems are commercial in nature.
Linux is a UNIX Clone
But if you consider Portable Operating System Interface (POSIX) standards then Linux can be considered as UNIX. To quote from Official Linux kernel README file:
Linux is a Unix clone written from scratch by Linus Torvalds with assistance from a loosely-knit team of hackers across the Net. It aims towards POSIX compliance.
However, "Open Group" do not approve of the construction "Unix-like", and consider it misuse of their UNIX trademark.
Linux Is Just a Kernel
Linux is just a kernel. All Linux distributions includes GUI system + GNU utilities (such as cp, mv, ls,date, bash etc) + installation & management tools + GNU c/c++ Compilers + Editors (vi) + and various applications (such as OpenOffice, Firefox). However, most UNIX operating systems are considered as a complete operating system as everything come from a single source or vendor.
As I said earlier Linux is just a kernel and Linux distribution makes it complete usable operating systems by adding various applications. Most UNIX operating systems comes with A-Z programs such as editor, compilers etc. For example HP-UX or Solaris comes with A-Z programs.
License and cost
Linux is Free (as in beer [freedom]). You can download it from the Internet or redistribute it under GNU licenses. You will see the best community support for Linux. Most UNIX like operating systems are not free (but this is changing fast, for example OpenSolaris UNIX). However, some Linux distributions such as Redhat / Novell provides additional Linux support, consultancy, bug fixing, and training for additional fees.
User-Friendly
Linux is considered as most user friendly UNIX like operating systems. It makes it easy to install sound card, flash players, and other desktop goodies. However, Apple OS X is most popular UNIX operating system for desktop usage.
Security Firewall Software
Linux comes with open source netfilter/iptables based firewall tool to protect your server and desktop from the crackers and hackers. UNIX operating systems comes with its own firewall product (for example Solaris UNIX comes with ipfilter based firewall) or you need to purchase a 3rd party software such as Checkpoint UNIX firewall.
Backup and Recovery Software
UNIX and Linux comes with different set of tools for backing up data to tape and other backup media. However, both of them share some common tools such as tar, dump/restore, and cpio etc.
File Systems
Linux by default supports and use ext3 or ext4 file systems.
UNIX comes with various file systems such as jfs, gpfs (AIX), jfs, gpfs (HP-UX), jfs, gpfs (Solaris).
System Administration Tools
UNIX comes with its own tools such as SAM on HP-UX.
Suse Linux comes with Yast
Redhat Linux comes with its own gui tools called redhat-config-*.
However, editing text config file and typing commands are most popular options for sys admin work under UNIX and Linux.
System Startup Scripts
Almost every version of UNIX and Linux comes with system initialization script but they are located in different directories:
HP-UX - /sbin/init.d
AIX - /etc/rc.d/init.d
Linux - /etc/init.d
End User Perspective
The differences are not that big for the average end user. They will use the same shell (e.g. bash or ksh) and other development tools such as Perl or Eclipse development tool.
System Administrator Perspective
Again, the differences are not that big for the system administrator. However, you may notice various differences while performing the following operations:
Software installation procedure
Hardware device names
Various admin commands or utilities
Software RAID devices and mirroring
Logical volume management
Package management
Patch management
UNIX Operating System Names
A few popular names:
HP-UX
IBM AIX
Sun Solairs
Mac OS X
IRIX
Linux Distribution (Operating System) Names
A few popular names:
Redhat Enterprise Linux
Fedora Linux
Debian Linux
Suse Enterprise Linux
Ubuntu Linux
Common Things Between Linux & UNIX
Both share many common applications such as:
GUI, file, and windows managers (KDE, Gnome)
Shells (ksh, csh, bash)
Various office applications such as OpenOffice.org
Development tools (perl, php, python, GNU c/c++ compilers)
Posix interface
Set backspace in Unix console
Sometimes the backspace key doesn't work for Unix.
we can setup the backspace key to work in the console by typing the command and press backspace key.
stty erase <backspace>
it will be shown as
stty erase ^H
If you have a problem using the backspace it should display you the above output.
If not, it will just perform a backspace.
After doing that press CTRL+c . And now type anything and try backspace, you can see the backspace working...
Unix filesystems explained
A filesystem is a logical collection of files on a partition or disk. A
partition is a container for information and can span an entire hard
drive if desired.
Everything in Unix is considered to be a file, including physical devices such as DVD-ROMs, USB devices, floppy drives, and so forth.
Unix uses a hierarchical file system structure, much like an
upside-down tree, with root (/) at the base of the file system and all
other directories spreading from there.
A UNIX filesystem is a collection of files and directories that has the following properties:
Everything in Unix is considered to be a file, including physical devices such as DVD-ROMs, USB devices, floppy drives, and so forth.
A UNIX filesystem is a collection of files and directories that has the following properties:
- It has a root directory (/) that contains other files and directories.
- Each file or directory is uniquely identified by its name, the directory in which it resides, and a unique identifier, typically called an inode.
- By convention, the root directory has an inode number of 2 and the lost+found directory has an inode number of 3. Inode numbers 0 and 1 are not used. File inode numbers can be seen by specifying the -i option to ls command.
- It is self contained. There are no dependencies between one filesystem and any other.
| Directory | Description |
|---|---|
| / | This is the root directory which should contain only the directories needed at the top level of the file structure. |
| /bin | This is where the executable files are located. They are available to all user. |
| /dev | These are device drivers. |
| /etc | Supervisor directory commands, configuration files, disk configuration files, valid user lists, groups, ethernet, hosts, where to send critical messages. |
| /lib | Contains shared library files and sometimes other kernel-related files. |
| /boot | Contains files for booting the system. |
| /home | Contains the home directory for users and other accounts. |
| /mnt | Used to mount other temporary file systems, such as cdrom and floppy for the CD-ROM drive and floppy diskette drive, respectively |
| /proc | Contains all processes marked as a file by process number or other information that is dynamic to the system. |
| /tmp | Holds temporary files used between system boots |
| /usr | Used for miscellaneous purposes, or can be used by many users. Includes administrative commands, shared files, library files, and others |
| /var | Typically contains variable-length files such as log and print files and any other type of file that may contain a variable amount of data |
| /sbin | Contains binary (executable) files, usually for system administration. For example fdisk and ifconfig utlities. |
| /kernel | Contains kernel files |
Flavors of UNIX
The table below summarizes some of the common UNIX variants and clones. While the table lists about forty different variants, the UNIX world isn't nearly as diverse as it used to be. Some of them are defunct and are listed for historical purposes. Others are on their way out. In some cases, vendors have defected to Microsoft technology. In others, mergers and acquisitions have led to the consolidation of different UNIX implementations. A list of "dead" UNIX implementations would be substantial indeed, consisting of hundreds of variations on the letters "U," "I," and "X" (CLIX, CX/UX, MV/UX, SINIX, VENIX, etc.).
