及负载均衡搭建手册.doc

资料内容仅供您学习参考，如有不当之处，请联系改正或者删除。
Oracle 11G R2 RAC安装手册
一、 基础知识
1. RAC是什么
RAC, 全称real application clusters, 译为”实时应用集群”, 是Oracle新版数据库中采用的一项新技术, 是高可用性的一种, 也是Oracle数据库支持网格计算环境的核心技术。
2. RAC的优缺点
l 优点
Oracle RAC支持Oracle9i、 10g、 11g版本, 能够支持24 x 7 有效的数据库应用系统, 在低成本服务器上构建高可用性数据库系统, 而且自由部署应用, 无需修改代码。在Oracle RAC环境下, Oracle集成提供了集群软件和存储管理软件, 为用户降低了应用成本。当应用规模需要扩充时, 用户能够按需扩展系统, 以保证系统的性能。
(1) 多节点负载均衡;
(2) 提供高可用: 故障容错和无缝切换功能, 将硬件和软件错误造成的影响最小化;
(3) 经过并行执行技术提高事务响应时间----一般见于数据分析系统;
(4) 经过横向扩展提高每秒交易数和连接数----一般对于联机事务系统;
(5) 节约硬件成本, 能够用多个廉价PC服务器代替昂贵的小型机或大型机, 同时节约相应维护成本;
(6) 可扩展性好, 能够方便添加删除节点, 扩展硬件资源。
l 缺点
(1) 相对单机, 管理更复杂, 要求更高;
(2) 可能会增加软件成本
3. Oracle RAC原理
在一个应用环境当中, 所有的服务器使用和管理同一个数据库, 目的是为了分散每一台服务器的工作量, 硬件上至少需要两台以上的服务器, 而且还需要一个共享存储设备。同时所有服务器上的OS都应该是同一类OS,根据负载均衡的配置策略, 当一个客户端发送请求到某一台服务的listener后, 这台服务器根据我们的负载均衡策略, 会把请求发送给本机的RAC组件处理也可能会发送给另外一台服务器的RAC组件处理, 处理完请求后, RAC会经过集群软件来访问共享存储设备.
逻辑结构上看:
l 每一个参加集群的节点有一个独立的instance, 这些instance访问同一个数据库。
l 每一个节点的linstance都有自己的SGA。
l 每一个节点的linstance都有自己的background process。
l 每一个节点的linstance都有自己的redo logs。
l 每一个节点的instance都有自己的undo表空间。
l 所有节点都共享一份datafiles和controlfiles。
三类Resource:
l VIP - 虚拟IP地址(Virtual IP)
l OCR - Oracle Cluster Registry(集群注册文件),记录每个节点的相关信息
l Voting Disk,仲裁机制用于仲裁多个节点向共享节点同时写的行为, 这样 做是为了避免发生冲突。
二、 RAC环境设计
1. 服务器配置
操作系统
主机名
内存
网卡
Redhat Enterprise Linux 5.4
rac1
2G
eth0( Public)
eth1( Prive)
Redhat Enterprise Linux 5.4
rac2
2G
eth0( Public)
eth1( Prive)
2. 用户组规划
GroupName
GroupID
说明
HOST
oinstall 
501
Oracle清单和软件所有者
rac1, rac2
asmadmin 
504
Oracle自动存储管理组
rac1, rac2
asmdba
506
ASM数据库管理员组
rac1, rac2
asmopen
507
ASM操作员组
rac1, rac2
dba
502
数据库管理员
rac1, rac2
3. 用户规划
UserID
UserName
Group
说明
HOST
口令
501
grid
oinstall
dba
asmadmin
asmdba
asmoper
Oracle Cluster用户
rac1
rac2
grid123
502
oracle
oinstall
dba
asmdba
Oracle数据库管理员
rac1
rac2
oracle123
4. IP规划
主机名
公用IP
私有IP
虚拟IP
rac1
172.16.1.211
10.10.10.211
172.16.1.111
rac2
172.16.1.212
10.10.10.212
172.16.1.112
ScanIP
172.16.1.100
5. 存储规划
存储组件
Block Device
ASMlib Name
Size
OCR
/dev/sdb
OCR_VOL1
1G
/dev/sdc
OCR_VOL2
1G
/dev/sdd
OCR_VOL3
1G
数据文件
/dev/sde
DATA_VOL1
10G
文件备份
/dev/sdf
BACK_VOL1
5G
三、 Oracle RAC搭建
1. 配置共享存储( ASM依赖)
l 方法一: vmware workstation 9虚拟共享存储
(1) 在VM安装目录\VMware Workstation下执行命令:
#vmware-vdiskmanager.exe -c -s 1GB -a lsilogic -t 2 D:\racDisk\db01.vmdk
#vmware-vdiskmanager.exe -c -s 1GB -a lsilogic -t 2 D:\racDisk\db02.vmdk
#vmware-vdiskmanager.exe -c -s 1GB -a lsilogic -t 2 D:\racDisk\db03.vmdk
#vmware-vdiskmanager.exe -c -s 10GB -a lsilogic -t 2 D:\racDisk\db04.vmdk
#vmware-vdiskmanager.exe -c -s 5GB -a lsilogic -t 2 D:\racDisk\db05.vmdk
(2) 修改rac1和rac2下的.vmx文件, 在最后一行添加如下内容:
diskLib.dataCacheMaxSize=0
diskLib.dataCacheMaxReadAheadSize=0
diskLib.dataCacheMinReadAheadSize=0
diskLib.dataCachePageSize=4096
diskLib.maxUnsyncedWrites = "0"
disk.locking = "FALSE"
scsi1.sharedBus = "virtual"
scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"
scsi1:0.mode = "independent-persistent"
scsi1:0.deviceType = "disk"
scsi1:0.present = "TRUE"
scsi1:0.fileName = "D:\racDisk\db01.vmdk"
scsi1:1.mode = "independent-persistent"
scsi1:1.deviceType = "disk"
scsi1:1.present = "TRUE"
scsi1:1.fileName = "D:\racDisk\db02.vmdk"
scsi1:2.mode = "independent-persistent"
scsi1:2.deviceType = "disk"
scsi1:2.present = "TRUE"
scsi1:2.fileName = "D:\racDisk\db03.vmdk"
scsi1:3.mode = "independent-persistent"
scsi1:3.deviceType = "disk"
scsi1:3.present = "TRUE"
scsi1:3.fileName = "D:\racDisk\db04.vmdk"
scsi1:4.mode = "independent-persistent"
scsi1:4.deviceType = "disk"
scsi1:4.present = "TRUE"
scsi1:4.fileName = "D:\racDisk\db05.vmdk"
(3) 重新打开虚拟机, 并启动。

(4) 磁盘分区, 在rac1执行
#fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or
OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sdb: 1048 MB, bytes
64 heads, 32 sectors/track, 1000 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1000, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1000, default 1000):
Using default value 1000

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
#fdisk /dev/sdc …重复上面操作
#fdisk /dev/sdd …重复上面操作
#fdisk /dev/sde …重复上面操作
#fdisk /dev/sdf …重复上面操作
l 方法二: Linux mount挂载( 待解决)
2. 安装依赖包
#yum install -y expect* iscsi* binutils* compat-libstdc++* expat* elfutils-libelf* gcc* glibc* ksh* libgcc*
#yum install -y pdksh* libgomp* compat-db* libXp* libaio* cpp* setarch* sysstat* libstdc++* make-3*
#yum install -y unixODBC
#yum install -y unixODBC-devel
3. 查看依赖包安装状况
#rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE}(%{ARCH})\n' binutils \
elfutils-libelf \
elfutils-libelf \
expat \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
glibc-headers \
ksh \
libaio \
libaio-devel \
libgcc \
libstdc++ \
libstdc++-devel \
make \
pdksh \
sysstat \
unixODBC \
unixODBC-devel
执行结果…
binutils-2.17.50.0.6-26.el5(x86_64)
elfutils-libelf-0.137-3.el5(x86_64)
elfutils-libelf-0.137-3.el5(i386)
elfutils-libelf-0.137-3.el5(x86_64)
elfutils-libelf-0.137-3.el5(i386)
expat-1.95.8-11.el5_8(x86_64)
expat-1.95.8-11.el5_8(i386)
gcc-4.1.2-54.el5(x86_64)
gcc-c++-4.1.2-54.el5(x86_64)
glibc-2.5-118.el5_10.2(i686)
glibc-2.5-118.el5_10.2(x86_64)
glibc-common-2.5-118.el5_10.2(x86_64)
glibc-devel-2.5-118.el5_10.2(x86_64)
glibc-devel-2.5-118.el5_10.2(i386)
glibc-headers-2.5-118.el5_10.2(x86_64)
ksh- 0621-18.el5(x86_64)
libaio-0.3.106-5(x86_64)
libaio-0.3.106-5(i386)
libaio-devel-0.3.106-5(i386)
libaio-devel-0.3.106-5(x86_64)
libgcc-4.1.2-54.el5(x86_64)
libgcc-4.1.2-54.el5(i386)
libstdc++-4.1.2-54.el5(x86_64)
libstdc++-4.1.2-54.el5(i386)
libstdc++-devel-4.1.2-54.el5(x86_64)
libstdc++-devel-4.1.2-54.el5(i386)
make-3.81-3.el5(x86_64)
pdksh-5.2.14-37.el5_8.1(x86_64)
sysstat-7.0.2-12.el5(x86_64)
unixODBC-2.2.11-10.el5(i386)
unixODBC-2.2.11-10.el5(x86_64)
unixODBC-devel-2.2.11-10.el5(x86_64)
unixODBC-devel-2.2.11-10.el5(i386)
4. 网络配置
(1) 修改rac1和rac2的network
#vi /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=rac1 #此处为主机名
GATEWAY=172.16.1.254
#vi /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=rac2
GATEWAY=172.16.1.254
(2) 修改rac1和rac2的hosts
#vi /etc/hosts
###eth0-Public IP###
172.16.1.211 rac1
172.16.1.212 rac2
###eth1–PRIVATE IP###
10.10.10.211 rac1-priv
10.10.10.212 rac2-priv
###VIP###
172.16.1.111 rac1-vip
172.16.1.112 rac2-vip
###SCAN###
172.16.1.100 rac-cluster-scan
(3) 重启rac1和rac2网络服务
#service network restart
(4) 配置虚拟IP
#ifconfig eth0:0 172.16.1.111 netmask 255.255.0.0 up
#ifconfig eth0:0 172.16.1.112 netmask 255.255.0.0 up
*注: 虚拟IP在重启网络服务会被卸载, 需要重新设置。
5. 配置NTPD服务
NTPD服务为Grid的检查项, 提供时间同步服务。
(1) 修改NTPD参数文件( rac1, rac2)
#vi /etc/sysconfig/ntpd
SYNC_HWCLOCK=yes
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"
(2) 修改rac1的ntpd.conf文件, 删除原有内容录入以下内容
#cp /etc/ntp.conf /etc/ntp.conf.bak
#vi /etc/ntp.conf
restrict 0.0.0.0 mask 0.0.0.0 nomodify
server 127.127.1.0
fudge 127.127.1.0 stratum 10
restrict 127.127.1.0
driftfile /var/lib/ntp/drift
broadcastdelay 0.008
authenticate no
keys /etc/ntp/keys
(3) 清空rac1的ntpservers原有内容
#cp /etc/ntp/ntpservers /etc/ntp/ntpservers.bak
#vi /etc/ntp/ntpservers
(4) 编辑rac1的step-tickers, 添加一行
#cp /etc/ntp/step-tickers /etc/ntp/step-tickers.bak
#vi /etc/ntp/step-tickers
127.127.1.0
(5) 修改rac2的ntpd.conf文件, 删除原有内容录入以下内容
#cp /etc/ntp.conf /etc/ntp.conf.bak
#vi /etc/ntp.conf
restrict 0.0.0.0 mask 0.0.0.0 nomodify
server 127.127.1.0
server 172.16.1.211
fudge 127.127.1.0 stratum 10
driftfile /var/lib/ntp/drift
broadcastdelay 0.008
authenticate no
keys /etc/ntp/keys
(6) 清空rac2的ntpservers原有内容
#cp /etc/ntp/ntpservers /etc/ntp/ntpservers.bak
#vi /etc/ntp/ntpservers
(7) 编辑rac2的step-tickers, 添加一行
#cp /etc/ntp/step-tickers /etc/ntp/step-tickers.bak
#vi /etc/ntp/step-tickers
172.16.1.211 #rac1的ip地址
(8) rac1节点重启NTPD服务
#service ntpd restart
(9) rac2节点停止NTPD服务, rac1重启NTPD服务后5、 6分钟再执行
#service ntpd stop
#ntpdate rac1
#service ntpd start
(10) NTPD启用自动加载( rac1, rac2)
#chkconfig ntpd on
6. 关闭服务
关闭rac1和rac2无用的sendmail和cups服务, 加快开机速度。
#chkconfig sendmail off
#chkconfig cups off
7. 用户、 用户组
(1) 创立用户和用户组
#groupadd -g 501 oinstall
#groupadd -g 502 dba
#groupadd -g 504 asmadmin
#groupadd -g 506 asmdba
#groupadd -g 507 asmoper
#useradd -u 501 -g oinstall -G dba,asmadmin,asmdba,asmoper grid
#useradd -u 502 -g oinstall -G dba,asmdba oracle
#id oracle
#id grid
(2) 修改用户密码
#passwd oracle
#passwd grid
8. 文件夹及权限
#mkdir -p /u01/app/grid/
#mkdir -p /u01/app/11.2.0/grid/product/db_1
#mkdir -p /u01/app/oraInventory
#chown -R grid:oinstall /u01/app
#mkdir -p /u01/app/oracle/product/11.2.0/db_1
#chown -R oracle:oinstall /u01/app/oracle
#chmod -R 775 /u01
9. 修改系统参数
(1) 修改rac1和rac2的limits.conf
#vi /etc/security/limits.conf
###ORACLE SETTING
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
(2) 修改rac1和rac2的/etc/pam.d/login
#vi /etc/pam.d/login
###ORACLE SETTING
session required pam_limits.so
(3) 修改rac1和rac2的/etc/sysctl.conf
#vi /etc/sysctl.conf
###ORACLE SETTING
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax =
kernel.shmmni = 4096
kernel.sem = 250 3 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
(4) 执行命令(rac1、 rac2)
#sysctl -p
(5) 修改rac1和rac2的profile, 在末尾添加
#vi /etc/profile
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
(6) 执行命令(rac1、 rac2)
#source /etc/profile
(7) 修改rac1和rac2的/etc/csh.login
#vi /etc/csh.login
if ( $USER = "oracle" || $USER = "grid" ) then
limit maxproc 16384
limit descriptors 65536
endif
EOFCSH
(8) 修改grid用户的环境变量
#su - grid
$cd /home/grid/
$vi .bash_profile
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/11.2.0/grid/product/db_1; export ORACLE_HOME
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
THREADS_FLAG=native; export THREADS_FLAG
PATH=$ORACLE_HOME/bin:$PATH; export PATH
THREADS_FLAG=native; export THREADS_FLAG
PATH=$ORACLE_HOME/bin:$PATH; export PATH
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
#su - grid
$cd /home/grid/
$vi .bash_profile
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_SID=+ASM2; export ORACLE_SID
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/11.2.0/grid/product/db_1; export ORACLE_HOME
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
THREADS_FLAG=native; export THREADS_FLAG
PATH=$ORACLE_HOME/bin:$PATH; export PATH
THREADS_FLAG=native; export THREADS_FLAG
PATH=$ORACLE_HOME/bin:$PATH; export PATH
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
(9) 执行命令(rac1、 rac2)
$source .bash_profile
(10) 修改oracle用户的环境变量
#su - oracle
$cd /home/oracle/
$vi .bash_profile
# Oracle Settings oracle
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME
ORACLE_SID=orcl1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
PATH=/usr/sbin:$PATH; export PATH
PATH=$ORACLE_HOME/bin:$PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
#su - grid
$cd /home/grid/
$vi .bash_profile
# Oracle Settings oracle
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME
ORACLE_SID=orcl2; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
PATH=/usr/sbin:$PATH; export PATH
PATH=$ORACLE_HOME/bin:$PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
(11) 执行命令(rac1、 rac2)
$source .bash_profile
10. 配置用户等效性
(1) 使用Oracle用户, 在所有节点执行
#su - oracle
$ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Created directory '/home/oracle/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
be:8b:34:fd:c9:34:a7:2f:d2:f7:d9:30:1f:2e:48:d6 oracle@rac1
$ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
4f:bc:4b:9a:a6:cd:1c:15:e7:67:92:94:a2:cd:50:14 oracle@rac1
(2) 使用Oracle用户, 在节点rac1执行
$cat /home/oracle/.ssh/*.pub >> /home/oracle/.ssh/authorized_keys
$ssh