How to Configuration GRID/ASM for single instance using UDEV

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Configuration GRID/ASM for single instance using UDEV

UDEV Linux configuration:

What I’m preparing in the background is a RAC installation using two virtual machine running under VirtualBox. For this I have attached to my first virtual machine four fixed size of 1GB disks like this

 

To attach those disks in another virtual machine go to Virtual Media Manager and change their property to shareable like this

udev02

Once you start your guest OS you should see something:

[root@localhost ~]# fdisk -l
.
.
.
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
 
 
Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
 
 
Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
 
 
Disk /dev/sde: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
.
.
.

In many blog post I have seen creation of a unique primary partition on those disks with:

[root@server3 ~]# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).
 
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
 
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xa4cc3f96.
 
Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-2097151, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2097151, default 2097151):
Using default value 2097151
Partition 1 of type Linux and of size 1023 MiB is set
 
Command (m for help): w
The partition table has been altered!
 
Calling ioctl() to re-read partition table.
Syncing disks.

Or use parted if you want to automate the creation with a shell script:

[root@localhost ~]# parted /dev/sdd mklabel msdos
Information: You may need to update /etc/fstab.
 
[root@localhost ~]# parted /dev/sdd mkpart primary 1m 100%
Information: You may need to update /etc/fstab.
 
[root@localhost ~]# fdisk -l /dev/sdd
 
Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000df279
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1            2048     2097151     1047552   83  Linux
[root@server3 ~]# parted /dev/sdb print
Model: ATA VBOX HARDDISK (scsi)
Disk /dev/sdb: 1074MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
 
Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1074MB  1073MB  primary

But honestly I’m wondering the need of this unique primary partition filling entirely the disk. Apparently it is needed only with ASMLib so as not using it decided to give a try without creating them…

So deleted the one I created (fdisk can also be used):

[root@server3 rules.d]# parted /dev/sde rm 1
Information: You may need to update /etc/fstab

The idea of UDEV is to associate the fixed device id to a more understandable name. On many blog posts you will see the usage of iscsi_id binary. That moved from /sbin in oracle linux5 and 6 to /usr/lib/udev in oracle linux 7.

[root@localhost ~]# for disk in `ls /dev/sd*`
                  do
                    echo $disk
                    /usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=$disk
                  done
/dev/sda
1ATA_VBOX_HARDDISK_VB35905f56-9625bd7c
/dev/sda1
1ATA_VBOX_HARDDISK_VB35905f56-9625bd7c
/dev/sda2
1ATA_VBOX_HARDDISK_VB35905f56-9625bd7c
/dev/sdb
1ATA_VBOX_HARDDISK_VB420a8d94-2a375f5b
/dev/sdc
1ATA_VBOX_HARDDISK_VB63ab3b43-33327cc0
/dev/sdd
1ATA_VBOX_HARDDISK_VBce02798e-7d13dc9a
/dev/sde
1ATA_VBOX_HARDDISK_VB9c27c116-7154ebea

Which can now be replaced (since oracle Linux 6) with udevadm (udevadm info -h for more information):

[root@localhost ~]# udevadm info --query=property /dev/sdb
DEVLINKS=/dev/disk/by-id/ata-VBOX_HARDDISK_VB420a8d94-2a375f5b
DEVNAME=/dev/sdb
DEVPATH=/devices/pci0000:00/0000:00:0d.0/ata4/host3/target3:0:0/3:0:0:0/block/sdb
DEVTYPE=disk
ID_ATA=1
ID_ATA_FEATURE_SET_PM=1
ID_ATA_FEATURE_SET_PM_ENABLED=1
ID_ATA_SATA=1
ID_ATA_SATA_SIGNAL_RATE_GEN2=1
ID_ATA_WRITE_CACHE=1
ID_ATA_WRITE_CACHE_ENABLED=1
ID_BUS=ata
ID_MODEL=VBOX_HARDDISK
ID_MODEL_ENC=VBOX\x20HARDDISK\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20
ID_PART_TABLE_TYPE=dos
ID_REVISION=1.0
ID_SERIAL=VBOX_HARDDISK_VB420a8d94-2a375f5b
ID_SERIAL_SHORT=VB420a8d94-2a375f5b
ID_TYPE=disk
MAJOR=8
MINOR=16
MPATH_SBIN_PATH=/sbin
SUBSYSTEM=block
TAGS=:systemd:
USEC_INITIALIZED=85769

The target is to write UDEV rules to associate our /dev/sdb to /dev/sde disk devices to symbolic names with a taste of ASMLib i.e. /dev/oracleasm/disks/DISKxx. To have a first insight of how to write UDEV rules you can start with man udev command. When using legacy method it gives below rules that you see on many articles, I have decided to put the rules in Oracle suggested file i.e. 99-oracle-asmdevices.rules located in /etc/udev/rules.d directory:

[root@localhost ~]# head -n 1 /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB420a8d94-2a375f5b", SYMLINK+="oracleasm/disks/DISK01", OWNER="oracle", GROUP="dba", MODE="0660"

I rate it a bit cumbersome and hard to read, with newest UDEV rules it can gives something brighter like below:

[root@server3 ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VB420a8d94-2a375f5b", SYMLINK+="oracleasm/disks/DISK01", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VB63ab3b43-33327cc0", SYMLINK+="oracleasm/disks/DISK02", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VBce02798e-7d13dc9a", SYMLINK+="oracleasm/disks/DISK03", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VB9c27c116-7154ebea", SYMLINK+="oracleasm/disks/DISK04", OWNER="oracle", GROUP="dba", MODE="0660"

Easier isn’t it ? Notice the sd? to take all SATA disk, even the system one but as we have not specified its serial id no rule will apply to it. The ENV{ID_SERIAL} match the serial ID of the disk to a generic name, so even if after reboot your devices names have changed the serial ID will not change. The SYMLINK will create a more understandable link to our devices and OWNER, GROUP and MODE gives ownership and permission.

Reload and apply the UDEV rules with:

[root@localhost ~]# udevadm control --reload-rules
[root@localhost ~]# udevadm trigger

Then you can see the link nodes that have been created:

[root@localhost ~]# ll /dev/oracleasm/disks/
total 0
lrwxrwxrwx 1 root root 9 Jul 2 12:33 DISK01 -> ../../sdb
lrwxrwxrwx 1 root root 9 Jul 2 12:33 DISK02 -> ../../sdc
lrwxrwxrwx 1 root root 9 Jul 2 12:33 DISK03 -> ../../sdd
lrwxrwxrwx 1 root root 9 Jul 2 12:33 DISK04 -> ../../sde

You can test your UDEV rules with:

[root@localhost ~]# udevadm test /sys/block/sdb

The rights are set only devices but not on symlinks:

[root@server3 hugepages]# ll /dev/sd*
brw-rw---- 1 root disk 8, 0 Jul 6 13:04 /dev/sda
brw-rw---- 1 root disk 8, 1 Jul 6 13:04 /dev/sda1
brw-rw---- 1 root disk 8, 2 Jul 6 13:04 /dev/sda2
brw-rw---- 1 oracle dba 8, 16 Jul 7 15:45 /dev/sdb
brw-rw---- 1 oracle dba 8, 32 Jul 7 15:46 /dev/sdc
brw-rw---- 1 oracle dba 8, 48 Jul 6 13:27 /dev/sdd
brw-rw---- 1 oracle dba 8, 64 Jul 6 13:27 /dev/sde

ASM configuration with UDEV

Once the UDEV rules have been created you can install Grid Infrastructure and when done use asmca to create your ASM instance and create your first ASM device group on your newly created disks (I have first changed the discovery path to /dev/oracleasm/disks)

I got the “Oracle Grid Infrastructure is not configured properly. ASMCA needs Oracle Grid Infrastructure to configure ASM” error:

What I did not notice when executing Grid Infrastructure root.sh is the below message:

To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user:
/u01/app/12.1.0/grid/perl/bin/perl -I/u01/app/12.1.0/grid/perl/lib -I/u01/app/12.1.0/grid/crs/install /u01/app/12.1.0/grid/crs/install/roothas.pl
 
 
To configure Grid Infrastructure for a Cluster execute the following command as oracle user:
/u01/app/12.1.0/grid/crs/config/config.sh
This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent operation, and the parameters can be passed through the response file that is available in the installation media.

Once script executed the configuration went better. I had few others error message but they were all related to memory allocation of my virtual machine. Oracle Grid Infrastructure in 12cR1 is consuming almost 1 GB of memory so you need a quite big /dev/shm (MEMORY_TARGET parameter).Once the ASM instance has been created even without any partition created on disks everything looks fin

SQL> col PATH FOR a30
SQL> SET lines 200
SQL> SELECT name,total_mb FROM v$asm_diskgroup;
 
NAME                             TOTAL_MB
------------------------------ ----------
DATA                                 3072
 
SQL> SELECT name,PATH,mode_status,os_mb FROM v$asm_disk
 
NAME                           PATH                           MODE_ST      OS_MB
------------------------------ ------------------------------ ------- ----------
                               /dev/oracleasm/disks/DISK01    ONLINE        1024
DATA_0000                      /dev/oracleasm/disks/DISK02    ONLINE        1024
DATA_0002                      /dev/oracleasm/disks/DISK04    ONLINE        1024
DATA_0001                      /dev/oracleasm/disks/DISK03    ONLINE        1024

10 thoughts on “How to Configuration GRID/ASM for single instance using UDEV”

  1. The very next time I read a blog, Hopefully it does not
    disappoint me just as much as this one. After all, Yes, it was my choice to read through, but I actually believed
    you would probably have something useful to talk about.
    All I hear is a bunch of crying about something
    that you could possibly fix if you were not too busy
    looking for attention.

  2. Do đó dù triển khai hệ thống wifi như thế
    nà o cÅ©ng phải giải quyết bà i toán mang đến má»�i ngÆ°á»�i cùng truy cập.

  3. Elke week is het een mogelijkheid om uw palm met water te besproeien, en dat voornamelijk
    in de herfst of lente, en in de avonden van de zomer.

  4. Hello, you used to write fantastic, but the last few posts have been kinda boring?
    I miss your tremendous writings. Past several posts are just a little bit out of track!

    come on!

Leave a Reply

Your email address will not be published. Required fields are marked *