Cloud DevOps Admin Guide

Follow the below steps when you get this error.

root@yyxxxx4:/dev
# bosboot -ad /dev/ipldevice
0516-602 lslv: Logical volume name not entered.
Usage: lslv [-L] [-l | -m] [-n DescriptorPV] LVname
lslv: [-L] [-n DescriptorPV] -p PVname [LVname]
Lists the characteristics of a logical volume.


0301-168 bosboot: The current boot logical volume, /dev/, does not exist on /dev/hdisk0.

Solution:

lsvg -p rootvg - hdisk0

lslv -m hd5 - hdisk0 on PV1 1st partition

savebase -v - successful

will remove/recreate hd5

rmlv hd5

# mklv -y hd5 -t boot -a e rootvg 1 hdisk0

cd /dev

rm ipldevice

ln /dev/rhdisk0 /dev/ipldevice

bosboot -ad /dev/ipldevice - same error

bootinfo -B hdisk0 - 1

# ln /dev/rhd5 /dev/ipl_blv

cd /dev - same maj/min numbers

bosboot -ad /dev/ipldevice - works!

alt_disk_install was able to proceed now

To view information about a mksysb backup file use:

            Display info about VG backup
            |
# lsmksysb -lf P2_1202_TL7.mk 
             |             |
             |             Mksyb file
             Device:(file,tape,cdrom) 
VOLUME GROUP:           rootvg
BACKUP DATE/TIME:       Tue Feb 21 18:08:29 GMT+01:00 2012
UNAME INFO:             AIX power2s 1 6 00C4489D4C00
BACKUP OSLEVEL:         6.1.7.0
MAINTENANCE LEVEL:      6100-07
BACKUP SIZE (MB):       41216
SHRINK SIZE (MB):       8421
VG DATA ONLY:           no

rootvg:
LV NAME             TYPE       LPs     PPs     PVs  LV STATE      MOUNT POINT
hd6                 paging     16      32      2    open/syncd    N/A
hd5                 boot       1       2       2    closed/syncd  N/A
hd8                 jfs2log    1       2       2    open/syncd    N/A
hd3                 jfs2       4       8       2    open/syncd    /tmp
hd1                 jfs2       1       2       2    open/syncd    /home
hd11admin           jfs2       1       2       2    open/syncd    /admin
livedump            jfs2       2       4       2    open/syncd    /var/adm/ras/livedump
fslv00              jfs2       72      144     2    open/syncd    /usr/sys/inst.images
hd4                 jfs2       22      44      2    open/syncd    /
hd2                 jfs2       34      68      2    open/syncd    /usr
hd9var              jfs2       2       4       2    open/syncd    /var
hd10opt             jfs2       5       10      2    open/syncd    /opt

To get the LPP info from the mksysb use:

            Display LPP info
            |
# lsmksysb -Lf P2_1202_TL7.mk
                   |
                   Mksysb file
Fileset                      Level  State  Type  Description (Uninstaller)
  ----------------------------------------------------------------------------
  ICU4C.rte                  6.1.7.0    C     F    International Components for
                                                   Unicode
  Java5.sdk                5.0.0.430    C     F    Java SDK 32-bit
  Java5_64.sdk             5.0.0.430    C     F    Java SDK 64-bit
  Java6.sdk                6.0.0.280    A     F    Java SDK 32-bit
  Tivoli_Management_Agent.client.rte
                             3.7.1.0    C     F    Management Framework Endpoint
                                                   Runtime"
  X11.adt.bitmaps            6.1.0.0    C     F    AIXwindows Application
                                                   Development Toolkit Bitmap
                                                   Files 
  X11.adt.imake              6.1.6.0    C     F    AIXwindows Application
                                                   Development Toolkit imake
.....

For the list of files contained in the mksysb

# lsmksysb -f P2_1202_TL7.mk
      6666 ./bosinst.data
          11 ./tmp/vgdata/rootvg/image.info
       11861 ./image.data
      187869 ./tmp/vgdata/rootvg/backup.data
           0 ./opt
           0 ./opt/IBM
           0 ./opt/IBM/perfpmr
        2249 ./opt/IBM/perfpmr/Install
        2616 ./opt/IBM/perfpmr/PROBLEM.INFO
        9818 ./opt/IBM/perfpmr/README
       26741 ./opt/IBM/perfpmr/config.sh

A specific file from the mksysb backup can be restored using the restorevgfiles command. In the following example the file will be restored to the current directory (/tmp/restore). Using the -d flag a alternative restore location can be specified.

                                 Path to the mksysb image file
                                 |
(/tmp/restore) # restorevgfiles -f /export2/P2_1202_TL7.mk ./root/j1
                                                             |
                                       The file to be extracted from the mksysb image.
New volume on /export2/P2_1202_TL7.mk:
Cluster size is 51200 bytes (100 blocks).
The volume number is 1.
The backup date is: Tue Feb 21 18:09:12 GMT+01:00 2012
Files are backed up by name.
The user is root.
x            6 ./root/j1
The total size is 6 bytes.
The number of restored files is 1.
==================================================================
(/tmp/restore) # ls -la */*
-rw-r--r--    1 root     system            6 Feb 17 11:16 root/j1

alt_disk_copy:

Required filesets:
    bos.alt_disk_install.boot_images
    bos.alt_disk_install.rte
    bos.msg.en_US.alt_disk_install.rte

alt_disk_copy -d <hdisk to clone rootvg>                 this will clone the rootvg to the specified disk
alt_disk_copy -e /etc/exclude.rootvg -d <hdisk>      this will use the exclude list during the cloning
alt_disk_copy -T -d <hdisk>                                      it will convert jfs to jfs2 on the new target disk (from 6.1 TL4 only)
alt_rootvg_op -X <cloned rootvg to destroy>          this will destroy the cloned rootvg (alt_rootvg_op -X altinst_rootvg)
alt_rootvg_op -W -d <hdisk>                                   this will wake up a disk (cloned filesystems will be mounted with prefix /alt_)
alt_rootvg_op -S -t <hdisk>                                      this will put cloned rootvg to sleep (before that it will do a bosboot)
                                                     (-S: put to sleep earlier "waked up" vg, -t: rebuilds the alt. bootimage before sleep)
alt_rootvg_op -v <new cloned rootvg name> -d <hdisk> this will rename the given cloned rootvg name
                                                     (after wake-up and sleep the cloned vg name will be changed, in this case it is useful)

alt_disk_mksysb -m /mnt/aix1mksysb -d hdisk1 -k      this will resore given mksysb (aix1mksysb) to hdisk1 (-k: keep device configuration)

/var/adm/ras/alt_disk_inst.log                       alt_disk log file
----------------------------------

alt_disk_copy: (copy hdisk0 to hdsik1)
lv names can't be longer than 11 characters (because of alt_ prefix)
do not take out that disk which was used during boot (otherwise there will be problems with bosboot)

-unmirrorvg rootvg hdisk1   
-reducevg rootvg hdisk1       
-bosboot -ad hdisk0       
-bootlist -m normal hdisk0   
-alt_disk_copy -d hdisk1       
-bootlist -m normal hdisk0

after booting from hdisk1:
root@aix11: / # lspv
hdisk0          00cf5d8fe9c88a34                    old_rootvg
hdisk1          00cf5d8fadcaa9a9                    rootvg          active


booting from the old disk:
root@aix11: / # lspv
hdisk0          00cf5d8fe9c88a34                    rootvg          active
hdisk1          00cf5d8fadcaa9a9                    altinst_rootvg


removing the new image (keeping the old one):
-alt_rootvg_op -X altinst_rootvg         <--removing the new image from hdisk1
-chpv -c hdisk1                          <--clear that pv what contained the removed image
-extendvg -f rootvg hdisk1               <--extend the currently used rootvg with the cleared disk (hdisk1)
-mirrorvg -S rootvg hdisk1               <--mirroring rootvg to hdisk1 (checking: lsvg rootvg | grep STALE)(-S: -background sync)
-bosboot -ad hdisk0; bosboot -ad hdisk1  <--recreate the bootimage
-bootlist -m normal hdisk0 hdisk1        <--setup correct bootlist (checking: bootlist -m normal -o)

------------------------------------

Changing lv names (to avoid 11 characters problem):
1. # mkszfile                            <--creates image.data file of rootvg
2. # vi image.data                       <--edit image.data
3. # alt_disk_copy -d hdiskX -i /image.data -B    <--give image.data fie for alt_disk_copy


--------------------------------------

ONLINE UPDATE WITH ALT_DISK_INSTALL:

unmirrorvg rootvg hdisk1                 <--removing mirror ( check: lsvg -p rootvg)
chpv -c hdisk1                           <--clears boot record
reducevg rootvg hdisk1                   <--free up hdisk1
bosboot -ad hdisk0                       <--creates boot record
bootlist -m normal hdisk0                <--sets boot list (check: bootlist -m normal -o)

installp -s                              <--check if anything can be commited
copy new bos.rte.install                 <--will be needed for checking if update will be successful (cd to this directory)
install_all_updates -pYd .               <--preview of new bos.rte.install
install_all_updates -Yd .                <--installs new bos.rte.install

oslevel -sg 5300-09-01-0847              <--shows which fileset is greater than current service pack, it will show bos.rte.install
instfix -i | grep SP                            <--it will show where to update (53-09-020849_SP)
oslevel -sl 53-09-020849                 <--shows which filesets should be update

cd /mnt/5300-09-SP2                      <--go to servicepack dir
install_all_updates -pYd .               <--preview check

alt_disk_copy -d hdisk1 -b update_all -l /mnt/5300-09-SP2     <--this will do the update

shutdown -Fr                                      <--new OS will boot up
smitty commit                                    <--if needed

alt_rootvg_op -X old_rootvg               <--removes cloned old OS
chpv -c hdisk0                                     <--clears bootrecord
extendvg -f rootvg hdisk0                   <--add hdisk0 to rootvg
mirrorvg -S rootvg hdisk0                   <--mirror rootvg (-S: in background)
bosboot -a                                              <--creates boot record
bootlist -m normal hdisk0 hdisk1         <--set bootlist

Below are the commands regarding the paging space in AIX.

To monitor paging space utilization:

lsps –a or lsps -s


To create an additional paging space:

mkps -s <#LPs> <vgname> <disk> or smit mkps


To activate a paging space:

swapon <device file name>


To deactivate a paging space:

swapoff <device file name>


To remove a paging space (must be inactive):

rmps <device file name> or smit mkps


To increase the size of a paging space:

chps –s <#LPs> <paging space name> or smit chps


To decrease the size of a paging space:

chps –d <#LPs> <paging space name> or smit chps


Activate paging space at restart:

chps -a -y <device file name>

Detect orphaned NIM mksysb resources

Here is a handy one-line script that will check all your NIM mksysb resources and tell you if you have any orphaned mksysb resources that don't have a backing file present:

Below are few of the AIX commands which will be useful for AIX admins.

1. To list machines configured in a NIM Server,
# lsnim -c machines

2. To list networks configured in a NIM Server,
# lsnim -c networks

3. To reset a machine (return to ready state)
# nim -Fo reset MachineName

4. To list core file settings for a user,
# lscore user1

The output will look like:
compression: on
path specification: default
corefile location: default
naming specification: off

5. To list the default settings for the system,

# lscore -d

The output will look like:
compression: off
path specification: on
corefile location: /corefiles
naming specification: off

6. To make any process run by root dump compressed core files and restore the location of the core files to the system default,

# chcore -c on -p default root
Note: If no default is specified, cores will dump in the current directory.

7. To enable a default core path for the system, type:

# chcore -p on -l /corefiles -d

8. To scan logical volume lv01, report the status of each partition, and have every block of each partition read to determine whether it is capableof performing I/O operations, type:

# mirscan -l lv01

9. To do the above operation in a PV,

# mirscan -p hdisk1

10. To do the above operation in a VG,

# mirscan -v vg01

11. To determine if the 64-bit kernel extension is loaded,

# genkex grep 64

12. To list all JFS file systems,

# lsjfs

13. To list all JFS2 file systems

# lsjfs2

14. To mirror a terminal1 on terminal2
a. Open terminal 1 and find the pts value (ps -ef grep pts)

b. Open terminal 2 and enter 'portmir -t pts/1'
c. Now you will see commands and outputs from terminal 1 in terminal 2.
This is basically monitor a terminal.
d. Say "portmir -o" to end the mirroring after the use

15. To identify the current run level,

# cat /etc/.init.state

16. To list the available CD ROM drives,

# lsdev -Cc cdrom

17. To find out the speed of your network adapter,

# entstat -d ent0 grep "Media Speed"

18. To find out when your system was last installed/updated

# lslpp -f bos.rte

19. To list the status of your tape drive,

# tctl -f /dev/rmt0 status

20. How to setup anonymous ftp in AIX

Run the below script to setup anon ftp,
# /usr/lpp/tcpip/samples/anon.ftp

21. If telnet takes more time to produce a prompt, do the below checks

a. do nslookup of the client ip from the aix serverb.
b. Check the nameservers in /etc/resolv.confc.
c. Check the 'hosts' entry in /etc/netsvc.conf or NSORDER variable

This issue might be due to the DNS configuration issue. Pointing to a good nameserver should solve the problem.

22. How to shutdown the system to maintenance mode ?

# shutdown -Fm

23. How to log ftp accesses to a file

a. Add the below line in /etc/syslog.confdaemon.debug /tmp/daemon.log
b. # touch /tmp/daemon.log
c. # refresh syslogd
d. Modify your inetd.conf so that ftpd is called with the "-l" flag.

24. How to find a file name from inode number ?

# ncheck -i xxxx /mountpoint
where xxxx -> inode number of the file

25. How to redirect the system console to a file or tty temporarily

# swcons /tmp/console.out

or

# swcons /dev/tty5

26. How to recreate a deleted /dev/null file ?

# /bin/mknod /dev/null c 2 2

27. How to add commands that should get executed during every system shutdown ?

Add them to /etc/rc.shutdown

28. How to reduce the size or do cleanup of /var/adm/wtmp ?

# > /var/adm/wtmp

29. How to find out the fileset a file belongs to ?

# which_fileset command_name

30. In which file, the mapping of file Vs fileset stored ?

# /usr/lpp/bos/AIX_file_list

31. How to set maximum logins for a user in a system ?

Change the value of "maxlogins" under "usw" stanza in /etc/security/login.cfg

32. How to change the initial message that prints while logging in ?

Change the value of "herald" in /etc/security/login.cfg

33. How to set the # of seconds the user is given to enter their password ?

Change the value of "logintimeout" under "usw" stanza in /etc/security/login.cfg

If a boot logical volume is corrupted (for example, bad blocks on a disk might cause a corrupted BLV), a machine will not boot.

To fix this situation, you must boot your machine in maintenance mode, from a CD or tape. If NIM has been set up for a machine, you can also boot the machine from a NIM master in maintenance mode. By the way, that's what you would do on an SP node if an SP node does not boot.

The boot lists are set using the bootlist command or the System Management Services (SMS) program. Some machines support a normal and service boot list. If your model supports this, you will use a function key during bootup to select the appropriate list. Normally, pressing F5 when you hear the first tones during bootup, will force the machine to check for a bootable CD. More on this later.

After booting from CD, tape or NIM an Installation and Maintenance Menu is shown and you can startup the maintenance mode. We will cover this later in this unit. After accessing the rootvg, you can repair the boot logical volume with the bosboot command. You need to specify the corresponding disk device, for example hdisk0:

# bosboot -ad /dev/hdisk0

It is important that you do a proper shutdown. All changes need to be written from memory to disk.

The bosboot command requires that the boot logical volume hd5 exists. If you ever need to re-create the BLV from scratch - maybe it had been deleted by mistake - the following steps should be followed:
1. Boot your machine in maintenance mode (from CD or tape).
2. Create a new hd5 logical volume: one physical partition in size, must be in rootvg. Specify boot as logical volume type.
3. Run the bosboot command as described.
4. Reboot the server run command

# shutdown -Fr

The following is an example of re-creating boot logical volume (BLV)

1. Boot your machine in maintenance mode (from CD, tape, or NIM).

2. Create a new hd5 logical volume: one physical partition in size, must be in rootvg. Specify boot as logical volume type. if needed, remove hd5 using # rmlv -f hd5

# mklv -y hd5 -t boot -a e rootvg 1

3. Run the bosboot command as described below.

# bosboot -ad /dev/hdisk0

4. Shutdown -Fr.

AIX has a large range of Open Source tools, and applications already ported and packaged ready for you to install. This makes it far easier and quick to get Open Source tools and applications working and useful on AIX. All the big popular Open Source stuff is available. My favourites that are in the first URL below are:

• Apache - web server the one that runs the web
• emacs - editor and so much more
• ethereal - network monitoring by packets, protocol, contents
• Ganglia - cluster performance monitoring and graphing
• GNC GCC - compiler collection particularly C and C++
• gimp - image manipulation
• Gnome - desktop
• KDE - desktop
• gzip - file compression utility
• MySQL - the database
• PHP - scripting, good for web servers
• rdist - file distribution
• Samba - Windows filesystem and printing and much more
• rxvt - colour xterm
• squid - proxy server
• vim - improved vi editor for colourised syntax highlighing
• VNC - Virtual Network Computing X windows on any machine (even Windows!!)
• wget - checking and copying websites

Well, your list might be different but there are hundreds of them available. You can, of course, download the original source code and compile it yourself but this means you will need the compilers (most use the GNC compilers), some programing skills and time to read up on the options and build process. then you need to perform some testing to prove it is all working.
Hint: I always compile with the latest GNU compilers to avoid problems.

Below are the commands which are related to the installation in aix.

lslpp -l	To see the details of installed file sets
lslpp -ha bos.net.*	To list the installation history of all file set in bos.net packages
lslpp -f bos.rte	To list the files in the bos.rte package
lslpp -w /etc/hosts	To list the file set which contain /etc/hosts file (parent fileset)
lslpp -p bos.net.nfs.server	To list the pre requisites for bos.net.nfs.server file set
lslpp -d	To show dependancies of fileset
installp -L -d /dev/rmt0.1	To list the installable products on the device rmt0
installp -aX -d /dev/rmt0.1 bos.net	To install all filesets within bos.net and expands file system if it requires
installp -u bos.net	To remove bos.net
installp -r	To reject the applied software
installp -c -f	To commit the applied fileset
installp -C	To cleanup an incomplete installation
lppchk -c	To check the fileset items and verifies that the checksum and filesize are consistent with SWVPD
lppchk -v	verify that all filesets have required requisites and are completely installed
instfix -k IX9999 -d /dev/rmt0.1	To install the file set associated with fix IX9999 from rmt0
instfix -ik IX9999	To verify fix IX9999 installed

Below are the commands related to the paging space in AIX.

lsps -a	To list out all paging spaces
lsps hd6	To display the details of the paging space hd6
chps -a y paging00	To turn on the paging space paging00 on next reboot
chps -a n paging00	To turn off the paging space paging00 on next reboot
chps -s4 paging00	To increase the size of the paging space in 4 LP blocks
mkps -a -n -s4 newvg	To create a paging space on VG newvg of 4 LP size (-s4) and activate it immediately (-n) and activate it at every restarts
rmps paging00	To remove the paging space paging00
swapon -a	To invoke all entries in /etc/swapspaces file
swapon /dev/paging00	To make available swap space paging00
swapoff /dev/paging00	To disable swap space paging00

lsfs	Lists all filesystems in the /etc/filesystems entry
lsfs -a	To list all filesystems (default)
lsfs -q	List all filesystems with detailed info (shows size of FS and LV in it. so we can check whether size of LV=size os FS)
lsfs -l	Specify the output in list format
lsfs -c	Specify the output in column format
lsfs -v jfs	Lists all jfs filesystems
chfs -a size=24576 /test	Change size of FS /test to 24576(blocks)x 512 bytes  (12 MB)
chfs -a size=+24576 /test	Add 24576(blocks)x512 byte  to FS /test
chfs -a size=+64M /test	Add 64 MB to /test
chfs -a size=10G /test	fix size of the FS /test to 10 GB
chfs -m /test /new	Change the mount point from /test to /new
chfs -A /test	To auto mount the filesystem test
chfs -d account /test	Remove account attribute of /test.(from /etc/filesystems file)
chfs -a splitcopy=/backup -a copy=2 /oracle	This will mount the 2nd copy of mirrored filesystem oracle to /backup in read-only mode for backup purpose
crfs -v jfs2 -g newvg -a size=100M -m /test	Creates FS /test of type jfs in VG newvg of size 100 MB with default LV.
crfs -v jfs -d /dev/lv00 -m /test	Create FS /test of type jfs on device /dev/lv00
rmfs /test	Deletes FS /test and associated LV
rmfs -r /test	Deletes FS /test its mount point and associated LV
defragfs /test	To defragment the file system /test
defragfs -q /test	Display the current defrag status of the file system
fsck -y n /dev/lv00	To check the filesystem associated to /dev/lv00 assuming response "yes"
fsck -p /dev/lv00	To restore superblock from backup superblock

If you choose a filesystem other than /tmp, create a directory named ibmsupt in that filesystem and later you will give the snap command the location of this ibmsupt directory with the -d option. The snap.pax.Z file will be located at /thefs/ibmsupt/snap.pax.Z.

# mkdir /thefs/ibmsupt

If your chosen file system is not /tmp, run: #####.###.000.snap.pax.Z
For example, if the PMR number is 12345,678 then name the file 12345.678.000.snap.pax.Z
Ftp the file to testcase.software.ibm.com or use the web interface at
https://testcase.boulder.ibm.com/ to upload the file into the directory /toibm/aix.
# cd /thefs/ibmsupt
# ftp testcase.software.ibm.com
user : anonymous
password : your full email address
> cd /toibm/aix
> bin
> hash
> put 12345.678.000.snap.pax.Z
> bye
Note: If a permission denied error is returned from ftp, first make certain that the ftp current directory is /toibm/aix. If it is, then usually a permissions denied error means an attempt is being made to overwrite an existing file and the permissions on the file will not allow this. In this case, try renaming the file to something like this:
12345.678.000.2.snap.pax.Z