標籤彙整: FreeBSD

Setting up a redundant MySQL with HAST and CARP

Common Address Redundancy Protocol ( CARP )

  1. 角色說明
    主要會有兩種角色, 一種是Host(  Primary ), 一種是Provider ( Secondary )
    Host 主要就是我們提供服務的機器
    Provider 就是當Host 掛掉後, 接手服務的機器
  2. 參數說明 ( 這部份看原文吧, 這是我看OpenBSD上找來的, FreeBSD上敘述的比較少, 請大家搭配著一起服用 )
    vhid
    The Virtual Host ID. This is a unique number that is used to identify the redundancy group to other nodes on the network. Acceptable values are from 1 to 255.
    password
    The authentication password to use when talking to other CARP-enabled hosts in this redundancy group. This must be the same on all members of the group.
    carpdev
    This optional parameter specifies the physical network interface that belongs to this redundancy group. By default, CARP will try to determine which interface to use by looking for a physical interface that is in the same subnet as the ipaddress and mask combination given to the carp(4) interface.
    advbase
    This optional parameter specifies how often, in seconds, to advertise that we’re a member of the redundancy group. The default is 1 second. Acceptable values are from 1 to 255.
    advskew
    This optional parameter specifies how much to skew the advbase when sending CARP advertisements. By manipulating advskew, the master CARP host can be chosen. The higher the number, the less preferred the host will be when choosing a master. The default is 0. Acceptable values are from 0 to 254.
    state
    Force a carp(4) interface into a certain state. Valid states are init, backup, and master.
    group, -group
    Add or remove a carp(4) interface to a certain interface group. By default all carp(4) interfaces are added to the carp group. Each group has a carpdemote counter affecting all carp(4) interfaces belonging to that group. As described below, it can be useful to group certain interfaces together for failover purposes.
    ipaddress
    This is the shared IP address assigned to the redundancy group. This address does not have to be in the same subnet as the IP address on the physical interface (if present). This address needs to be the same on all hosts in the group, however.
    mask
    The subnet mask of the shared IP.
  3. 實作
    要用CARP的功能有兩個方法
    一個是 rebuild kernel

    device carp

    另一個是load Kernel Module, 在/boot/loader.conf加入下面這行

    if_carp_load="YES"

    我個人是傾向用Kernel Module, 這以用freebsd-update 昇級時比較方便.

    接下來先在hasta ( Host ) 這台機器中, 編輯/etc/rc.conf 加入下面的設定

    hostname="hasta.example.org"
    ifconfig_fxp0="inet 192.168.1.51 netmask 255.255.255.0"
    cloned_interfaces="carp0"
    ifconfig_carp0="vhid 1 pass testpass 192.168.1.50/24"

    再編輯 hastb ( Provider ) 的 /etc/rc.conf

    hostname="hastb.example.org"
    ifconfig_fxp0="inet 192.168.1.52 netmask 255.255.255.0"
    cloned_interfaces="carp0"
    ifconfig_carp0="vhid 1 advskew 100 pass testpass 192.168.1.50/24"

    然後Host重開機, 等個幾秒再重開 Provider, 不重開機的話, 也可以試試

    # kldload if_carp.ko
    # ifconfig carp0 create
    # ifconfig carp0 down && ifcofig carp0 up ( 兩台都要, 先Host, 再Provider )

    這樣Host 應該就會拿到192.168.1.50的IP了, 從同網段的機器ping 一下, 看是不是有成功

  4. failback
    在 Host 那台執行

    # sysctl net.inet.carp.preempt=1

    就可以了, 若搭配了HAST, 會滿不建議這樣處理的, 最好還是人工去看狀況怎麼樣, 再做調整及切換

Highly Available Storage  ( HAST )

  1. 角色說明
    primary ( carp 中的 Host ) => 會把資料傳給slave 叫他乖乖的寫進去
    secondary ( carp 中的 provider ) => 乖乖的收 master 的資料來同步
  2. 實作
    編輯兩台機器上的/etc/hast.conf, 加入以下資料

    resource test {
        on hasta {
            local /dev/ad6 ( 看你那顆HD要來做同步 )
            remote 192.168.1.52
        }
        on hastb {
            local /dev/ad6
            remote 192.168.1.51
        }
    }

    及執行下列指令

    # hastctl create test
    # /etc/rc.d/hastd onestart

    在 “primay” 機器上

    hastctl role primary test

    在 “secondary” 機器上

    # hastctl role secondary test

    接下來在 primary 就newfs, mount

    # newfs -U /dev/hast/test
    # mkdir -p /hast/test
    # mount /dev/hast/test /hast/test

    最後在編輯兩台機器/etc/rc.conf

    # hastd_enable="YES"

    這樣系統Boot後HAST就會自已啟動了…

  3. Failover
    在之前的Wiki上是要搭配ifstated一起服用
    新的文件上則是透過devd來做
    作法如下
    編輯兩台機器的 /etc/devd.conf

    notify 30 {
        match "system" "IFNET";
        match "subsystem" "carp0";
        match "type" "LINK_UP";
        action "/usr/local/sbin/carp-hast-switch master";
    };
    
    notify 30 {
        match "system" "IFNET";
        match "subsystem" "carp0";
        match "type" "LINK_DOWN";
        action "/usr/local/sbin/carp-hast-switch slave";
    };

    詳細的意思可以man devd.conf
    接著重啟devd

    # /etc/rc.d/devd restart

    再來就是編輯/usr/local/sbin/carp-hast-switch 了( 兩台都要有 )

    #!/bin/sh
    
    # Original script by Freddie Cash <fjwcash@gmail.com>
    # Modified by Michael W. Lucas <mwlucas@BlackHelicopters.org>
    # and Viktor Petersson <vpetersson@wireload.net>
    
    # The names of the HAST resources, as listed in /etc/hast.conf
    resources="test"
    
    # delay in mounting HAST resource after becoming master
    # make your best guess
    delay=3
    
    # logging
    log="local0.debug"
    name="carp-hast"
    
    # wait_count
    wait_count=7
    
    # end of user configurable stuff
    
    case "$1" in
        master)
            logger -p $log -t $name "Switching to primary provider for ${resources}."
            sleep ${delay}
    
            # Wait for any "hastd secondary" processes to stop
            for disk in ${resources}; do
                while $( pgrep -lf "hastd: ${disk} (secondary)" > /dev/null 2>&1 && [ $wait_count -gt 0 ] ); do
    								logger -p $log -t $name "countdown => ${wait_count}."
    								wait_count=`expr $wait_count - 1`
                    sleep 1
                done
    						wait_count=7
    
                # Switch role for each disk
                hastctl role primary ${disk}
                if [ $? -ne 0 ]; then
                    logger -p $log -t $name "Unable to change role to primary for resource ${disk}."
                    exit 1
                fi
            done
    
            # Wait for the /dev/hast/* devices to appear
            for disk in ${resources}; do
                for I in $( jot 60 ); do
                    [ -c "/dev/hast/${disk}" ] && break
                    sleep 0.5
                done
    
                if [ ! -c "/dev/hast/${disk}" ]; then
                    logger -p $log -t $name "GEOM provider /dev/hast/${disk} did not appear."
                    exit 1
                fi
            done
    
            logger -p $log -t $name "Role for HAST resources ${resources} switched to primary."
    
            logger -p $log -t $name "Mounting disks."
            for disk in ${resources}; do
                mkdir -p /hast/${disk}
                fsck -p -y -t ufs /dev/hast/${disk}
                mount /dev/hast/${disk} /hast/${disk}
    	    ## start mysql server
    	    logger -p $log -t $name "start mysql"
    	    /bin/sh /usr/local/etc/rc.d/mysql-server start
            done
    
        ;;
    
        slave)
            logger -p $log -t $name "Switching to secondary provider for ${resources}."
    
            # Switch roles for the HAST resources
            for disk in ${resources}; do
                if ! mount | grep -q "^/dev/hast/${disk} on "
                then
                else
    	        ## stop mysql server
                    logger -p $log -t $name "stop mysql"
    	        /bin/sh /usr/local/etc/rc.d/mysql-server stop
    	        sleep 0.5
    	        ## umount
                    logger -p $log -t $name "umount ${disk}."
                    umount -f /hast/${disk}
                fi
                sleep $delay
                hastctl role secondary ${disk} 2>&1
                if [ $? -ne 0 ]; then
                    logger -p $log -t $name "Unable to switch role to secondary for resource ${disk}."
                    exit 1
                fi
                logger -p $log -t $name "Role switched to secondary for resource ${disk}."
            done
        ;;
    esac

    這個script我有改了一小部份, 加入了mysql 的啟動, 和最大等待次數

  4. 測試方法
    # ifconfig carp0 down && ifconfig carp0 up

    然後用 hastctl status test 觀看吧

  5. 錯誤回復
    確定那一台的資料比較新, 在舊的那台執行

    # hastctl role init <resource>
    # hastctl create <resource>
    # hastctl role secondary <resource>

    然後觀看 primary 的 HAST status 應該會發現 dirty: 的部份會重跑

MySQL

  1. 請把DB 的 dir 設到HAST 上面
    我的/etc/rc.conf 設定如下

    ## MYSQL
    mysql_enable="YES"
    mysql_dbdir="/hast/test/mysql"
    mysql_args="--bind-address=192.168.1.50 --skip-name-resolve"

    然後把/usr/local/etc/rc.d/mysql-server 的權限設成000
    怕mysql 在開機後自已啟了

注意事項

  1. 不要在 dirty: 還有值的情況下去切換 primary, secondary 會爆炸的
  2. 請打開 promiscuous model 不然完全ping不到vip
  3. 可以把 secondary 這台機器的開機加上delay, 以防整個機房跳電, primary, secondary 同時開機造成split-brain
    編輯/boot/loader.conf

    ## delay secondary boot
    autoboot_delay="240"

參考資料

  1. http://www.freebsd.org/doc/handbook/carp.html
  2. http://www.freebsd.org/doc/handbook/disks-hast.html
  3. http://www.openbsd.org/faq/pf/carp.html
  4. http://developer.51cto.com/art/200509/3863.htm
  5. man hast
  6. man hastctl
  7. man carp
  8. man devd.conf
  9. /usr/src/sys/netinet/ip_carp.c ( 因為我man carp 怎麼樣都沒看到他說 net.inet.carp.preempt 設成1 就會failback……, 所以就開這個來看, 裡面的 http://paste.plurk.com/show/394406/ 有寫到 )

debug

  1. /var/log/message
  2. /var/log/debug.log

截圖
pic

FreeBSD branches支援時限

The current supported branches and expected EoL dates are:

+———————————————————————+
| Branch | Release | Type | Release date | Estimated EoL |
|———–+————+——–+—————–+—————–|
|RELENG_6 |n/a |n/a |n/a |November 30, 2010|
|———————————————————————|
|RELENG_6_4 |6.4-RELEASE |Extended|November 18, 2008|November 30, 2010|
|———————————————————————|
|RELENG_7 |n/a |n/a |n/a |last release + 2y|
|———–+————+——–+—————–+—————–|
|RELENG_7_1 |7.1-RELEASE |Extended|January 4, 2009 |January 31, 2011 |
|———–+————+——–+—————–+—————–|
|RELENG_7_2 |7.2-RELEASE |Normal |May 4, 2009 |June 30, 2010 |
|———–+————+——–+—————–+—————–|
|RELENG_7_3 |7.3-RELEASE |Extended|March 23, 2010 |March 31, 2012 |
|———–+————+——–+—————–+—————–|
|RELENG_8 |n/a |n/a |n/a |last release + 2y|
|———–+————+——–+—————–+—————–|
|RELENG_8_0 |8.0-RELEASE |Normal |November 25, 2009|November 30, 2010|
|———–+————+——–+—————–+—————–|
|RELENG_8_1 |8.1-RELEASE |Extended|not yet |release + 2 years|
+———————————————————————+

FreeBSD Customized CD(Make your own release)

1 make buildworld
1.1 請先做cvsup 更新src(記得先裝cvsup-with-gui 等等會用到)
[sourcecode language=’bash’]
# cd /usr/ports/net/cvsup-without-gui; make install clean
[/sourcecode]
1.2
[sourcecode language=’bash’]
# cd /usr/src; make buildworld
[/sourcecode]
2 make release
2.1
[sourcecode language=’bash’]
# cp /usr/share/examples/cvsup/cvs-supfile /etc
[/sourcecode]
2.2
[sourcecode language=’bash’]
# mkdir /home/ncvs /home/releng
[/sourcecode]
2.3
[sourcecode language=’bash’]
# vi /etc/cvs-supfile
*default host=cvsup.tw.FreeBSD.org
*default prefix=/home/ncvs
[/sourcecode]
2.4 csup 只能check,所以要裝cvsup
[sourcecode language=’bash’]
# cvsup /etc/cvs-supfile
[/sourcecode]
2.5
[sourcecode language=’bash’]
# cd /usr/src/release;
[/sourcecode]
2.6 開始做release
[sourcecode language=’bash’]
# make release RELEASETAG=RELENG_7_1 PORTSRELEASETAG=HEAD
BUILDNAME=7.1-RELEASE-p3 CHROOTDIR=/home/releng
CVSROOT=/home/ncvs NODOC=yes
[/sourcecode]
指令說明:
RELEASETAG:
The CVS tag corresponding to the release that is to be built. If undefined, the release will be built from the HEAD of the CVS tree (a “-CURRENT snapshot”).,
PORTSRELEASETAG:
The CVS tag to use when checking out the ports tree. Usually, the head of the ports tree is used by default. If RELEASETAG specifies a release tag, then the associated release version is used as the default instead.

BUILDNAME:
The name of the release to be built. This is used to set the RELEASE value in sys/conf/newvers.sh, which affects the output of uname(1). If not set, a name with the timestamp and the “-SNAP” suffix will be generated.

CHROOTDIR:
The directory to be used as the chroot(8) environment for the entire release build. This file system should have at least 3.2 gigabytes of free space on the i386 architecture.

CVSROOT:
The location of the FreeBSD CVS repository. This path name is in reference to the real system root, not the root of the chroot(8) directory tree.

NODOC:
If defined, the SGML-based documentation from the FreeBSD Documentation Project will not be built. However, the “doc” distribution will still be created with the minimal documentation set providedin src/share/doc.

2.7 放入你想要額外用的檔案
[sourcecode language=’bash’]
# cp xxx.tgz /home/ releng/R/cdrom/dvd1/7.1-RELEASE-p3
[/sourcecode]
3 加入install.cfg 至 mfsroot
3.1
[sourcecode language=’bash’]
# cd /home/releng/R/cdrom/dvd1/boot
[/sourcecode]
3.2
[sourcecode language=’bash’]
# gunzip /home/releng/R/cdrom/dvd1/boot/mfsroot.gz
[/sourcecode]
3.3
[sourcecode language=’bash’]
# mdconfig -a -t vnode -f /home/ releng/R/cdrom/dvd1/boot/mfsroot -u 0
[/sourcecode]
3.4
[sourcecode language=’bash’]
# mount /dev/md0 /mnt
[/sourcecode]
3.5
[sourcecode language=’bash’]
# cp /usr/src/usr.sbin/sysinstall/install.cfg /mnt
[/sourcecode]
3.6
[sourcecode language=’bash’]
# vi /mnt/install.cfg (依照每台機器修改)
[/sourcecode]
加入or修改以下資訊

[sourcecode language=’bash’]
# This is a sample installation configuration file for my test machine,
# crate.cdrom.com.
# It is included here merely as a sort-of-documented example.
#
# $FreeBSD: src/usr.sbin/sysinstall/install.cfg,v 1.12.6.1 2008/11/25 02:59:29 kensmith Exp $
# Turn on extra debugging.
debug=yes

##partition 互動調整硬碟
diskInteractive=yes
diskPartitionEditor

##label互動式調整硬碟
diskInteractive=yes
diskLabelEditor

###############################
# Select which distributions we want. 設定為最小安裝
##dists=base doc manpages info src sbase ssys kernels GENERIC
distSetMinimum

# media 從CD-ROM安裝
mediaSetCDROM

# OK, everything is set. Do it!
installCommit

#解壓 xxx.tgz
command=touch /tmp/ok
system
[/sourcecode]

指令說明:
command 不能斷行,不然在跑指令的時後會有error
command在安裝時的debug輸出訊息會是這樣
setting variable ‘command=我們打的command的內容’
所以在command 須要用到引號的部份最好都用雙引號

3.7 製作ISO檔
[sourcecode language=’bash’]
sh /usr/src/release/i386/mkisoimages.sh -b FreeBSD7 /home/7.1-RELEASE-p3.iso
/home/releng/R/cdrom/dvd1
[/sourcecode]
3.8 參考資料
# man release
# man sysinstall
http://romana.now.ie/writing/customfreebsdiso.html
http://www.sfc.wide.ad.jp/~watari/FreeBSD/boot.html
http://www.bsdunix.ch/public/docu/freebsd_sysinstall.html

這個是寫滿久了….忙一直忘了放上來….

FreeBSD src更新—-指正!!

23:51 <@chxxxn> johnpupu: 那個要 make buildworld 跟 make installworld 啊,
23:51 <@chxxxn> 如何升級到7.0-p1
23:51 <@chxxxn> http://phorum.study-area.org/index.php/topic,51783.0.html
23:52 <@chxxxn> “你沒跳大版號…應該不用make world ..”……看了看問題標題:『如何升級到7.0-p1』
23:53 * chxxxn 不常逛 www forum, 也懶得那麼多站一個一個都註冊..以免又收到一些奇怪信件
23:54 <@chxxxn> johnpupu: 某長輩說的好:『至於有沒有大版號, 要不要 buildworld? 這是看到底是為甚麼改版號吧, 有的是 user land library 做 security patch, 關 kernel 鳥事, 只 make kernel 是把頭埋進沙裡的行為.』
Day changed to 04 Jun 2008
00:00 <@jxxxn> security issue 都寫的很清楚啊
00:03 <@chxxxn> 然後日後就會有人在批踢踢上面發問 [問題] 網路寫說『關於 security patch 方面,你沒跳大版號…應該不用make world ..』……

昨天在irc 上 長輩的指正…
記一下..免的以後忘記…
感謝長輩的指正………..
從以前都昨天………….update 都沒buildworld , installworld………….
錯了大蓋一年多了……..
Orz

Keyboard doesn’t work for FreeBSD 6.2R on IBM x3105

在用6.2R bootonly 裝IBM X3105 時
開完機進安裝畫面後…
鍵盤就不動了….
我試了一下,發現其實是有反應的…
就是當你不管按了什麼後….只要再按num block 這個按鍵…
他就會動了………..
很suck …
我也用了這樣的方式裝好了6.2R
一步一步 的按num block ::(:
裝好了沒事就算了…..
結果 …
一開機進os後….發現
就算安裝完成…問題還是在..
不過遠端登入,就沒這個問題
但是整個系統都怪怪的
top 出來的畫面很慢才會refresh
然後不能reboot
會卡住要一直按num block
這個按鍵……….
後來換了
FreeBSD 5.5R 就沒這個問題….
有遇到的人….若是解不掉的人
可以換一下Release 版本…
::(:

誤刪 /var/db/pkg 的解救方法!!!

剛剛在irc 看到的~~備忘一下
(17:54:34) weithenn: 請教長輩手賤把 /var/db/pkg 砍了如何 rebuild?
(17:55:18) S3pPoor: 到 /usr/ports/ 裡面開始重新裝… XD
(17:55:49) weithenn: S3pPoor:需要的在重裝一次?
(17:55:54) tys0n: … //pat
(17:55:59) ***S3pPoor 雙手合十,喃喃唸著「阿彌陀佛」。
(17:56:21) S3pPoor: weithenn: /var/db/pkg 裡面的東西沒有第二份啊, 所以就…
(17:56:44) Jedi_: 怎麼辦喔,就從備援機上面 co 回來
(17:56:46) weithenn: S3pPoor:了解了,手賤是要付出代價的
(17:56:48) ***Jedi_ flee
(17:57:14) weithenn: Jedi_:大師說的是啊
(17:57:37) Jedi_: 不過我自己也沒有備援機,嗚
(17:57:49) Jedi_: 我只有松果有做 A/B Backup
(17:57:56) ***weithenn 這台也沒有
(17:58:02) ***weithenn
(17:58:18) Jedi_: 所以我們還是需要 versioned fs
(17:58:38) ***S3pPoor ╮(╯OO╰)╭
(17:59:06) ***weithenn 下次手要先洗乾淨
(18:02:41) chinsan: 先裝上 portupgrade ,然後 mkdir -p /var/db/pkg ; pkgdb -Fu
(18:04:17) plasmaball: chinsan: 好強
(18:06:19) weithenn: chinsan:我試試在來回報
(18:06:30) chinsan: plasmaball: 漿伯長輩,您想太多 @_@
(18:09:07) psilotum 離開聊天室 (quit: “leaving”)。
(18:35:10) candyz [~candyz@dns.kandix.idv.tw] 進入聊天室。
(18:35:59) ***weithenn 回來不少,有些還是漏了
(18:36:07) ***weithenn 在觀察看看吧
:(h):