Layer7 IMQ Route Multipath Loadbalance Debian Lenny 2.6.28
Debian Lenny, Clarkconnect Enterprise 4.3 Kernel 2.6.28.10, iptables 1.4.3 IMQ Route_Multipath patch contribute and test by [email protected] June 13 2009( Update 19 July 2009 )
วิธีการด้านล่างนี้ทั้งหมดทั้งสิ้นอาจมีข้อผิดพลาดซึ้งไม่รับประกันการใช้งานใด้ๆทั้งสิ้น
ด้านล่างนี้เป็นการคอนฟิกหลักบน Debian Lenny, Clarkconnect 4.3 ( Kernel 2.6.28 + Layer7 + IMQ + Multipath Route ) ซึ่งสามารถทำงานบน Virtual interface และ Loadbalance Wan interface ได้
Contents
Debian
[root@gateway ~]# apt-get install gzip unzip bzip2 [root@gateway ~]# apt-get install debhelper screen fakeroot zlib1g-dev build-essential libncurses5-dev kernel-package
Clarkconnect 4.3
[root@gateway ~]# apt-get install cc-devel
ดาวโหลด Package
[root@gateway ~]# wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.28.10.tar.bz2 [root@gateway ~]# wget http://ufpr.dl.sourceforge.net/sourceforge/l7-filter/netfilter-layer7-v2.22.tar.gz [root@gateway ~]# wget http://ufpr.dl.sourceforge.net/sourceforge/l7-filter/l7-protocols-2009-05-28.tar.gz [root@gateway ~]# wget http://www.ssi.bg/~ja/routes-2.6.28-16.diff [root@gateway ~]# wget http://www.linuximq.net/patchs/linux-2.6.28.9-imq-test2.diff [root@gateway ~]# wget http://www.linuximq.net/patchs/iptables-1.4.3.2-imq_xt.diff [root@gateway ~]# wget http://www.netfilter.org/projects/iptables/files/iptables-1.4.3.2.tar.bz2
แตกไฟล์ออกมา
[root@gateway ~]# tar xjfv linux-2.6.28.10.tar.bz2 [root@gateway ~]# tar xjfv iptables-1.4.3.2.tar.bz2 [root@gateway ~]# tar zxvf netfilter-layer7-v2.22.tar.gz [root@gateway ~]# tar xzfv l7-protocols-2009-05-28.tar.gz
สร้าง Symbol Link
[root@gateway ~]# ln -s /usr/src/linux-2.6.28.10 /usr/src/linux
Patch Kernel ด้วย patch file
[root@gateway ~]# cd linux [root@gateway ~]# patch -p1 </usr/src/netfilter-layer7-v2.22/kernel-2.6.25-2.6.28-layer7-2.22.patch [root@gateway ~]# patch -p1 </usr/src/routes-2.6.28-16.diff [root@gateway ~]# patch -p1 </usr/src/linux-2.6.28.9-imq-test2.diff
Config Kernel
[root@gateway ~]# make menuconfig
<source lang=bash> Networking support > Networking options > Network packet filtering framework (Netfilter) > Core Netfilter Configuration. [ ] layer7 match support
[*] select all [M] select all
<M> "IMQ" target support <M> "layer7" match support [*] "Layer7" debugging output
Networking support > Networking options > Network packet filtering framework (Netfilter) > IP: Netfilter Configuration.
[*] select all [M] select all <M> Full NAT
</source> Exit Save .config
คอมไพล์และติดตั้งมันซะ Deb + CC4.3 ( Options 1 )
[root@gateway ~]# make && make modules && make modules_install && make install
ทำให้มันบูท Kernel ใหม่ ( Debian Only ) CC4.3 ไม่ต้องทำ
[root@gateway ~]# cd /boot [root@gateway ~]# mkinitramfs -o initrd.img-2.6.28.10 2.6.28.10 [root@gateway ~]# update-grub [root@gateway ~]# reboot
ถ้าต้องการ Compile และสร้าง .deb ด้วย ใช้คำสั่ง ( Options 2 )
[root@gateway ~]# make clean && make mrproper [root@gateway ~]# make menuconfig
<source lang=bash> Networking support > Networking options > Network packet filtering framework (Netfilter) > Core Netfilter Configuration. [ ] layer7 match support
[*] select all [M] select all
<M> "IMQ" target support <M> "layer7" match support [*] "Layer7" debugging output
Networking support > Networking options > Network packet filtering framework (Netfilter) > IP: Netfilter Configuration.
[*] select all [M] select all <M> Full NAT </source> Exit Save .config
[root@gateway ~]# make-kpkg clean [root@gateway ~]# fakeroot make-kpkg --initrd --append-to-version=-mscomputel7mpath kernel_image kernel_headers [root@gateway ~]# cd /usr/src [root@gateway ~]# dpkg -i linux-image-* [root@gateway ~]# dpkg -i linux-headers-* [root@gateway ~]# reboot
ขี้เกียจคอมไพล์ ดาวโหลด
แก้ไข /boot/grub/menu.lst ( CC4.3 Only )
[root@gateway ~]# nano /boot/grub/menu.lst
<source lang=bash>
- grub.conf generated by anaconda
- Note that you do not have to rerun grub after making changes to this file
- NOTICE: You have a /boot partition. This means that
- all kernel and initrd paths are relative to /boot/, eg.
- root (hd0,0)
- kernel /vmlinuz-version ro root=/dev/sda3
- initrd /initrd-version.img
- boot=/dev/sda
default=1 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Linux (2.6.28)
root (hd0,0)
kernel /vmlinuz-2.6.28 ro root=LABEL=/ video=vesafb vga=0x313
initrd /initrd-2.6.28.img
- title Linux (2.6.18-93.cc4)
- root (hd0,0)
- kernel /vmlinuz-2.6.18-93.cc4 ro root=LABEL=/ video=vesafb vga=0x313
- initrd /initrd-2.6.18-93.cc4.img
- title Linux Safe Mode (2.6.18-93.cc4)
- root (hd0,0)
- kernel /vmlinuz-2.6.18-93.cc4 ro root=LABEL=/
- initrd /initrd-2.6.18-93.cc4.img
</source> Reboot เครื่อง
[root@gateway ~]# reboot
ตรวจสอบ Kernel Version CC4.3
[root@gateway ~]# uname -a Linux gateway.clarkconnect.lan 2.6.28 #1 SMP Fri Jun 19 13:17:45 ICT 2009 i686 i686 i386 GNU/Linux
ตรวจสอบ Kernel Version Debian Lenny
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Fri Jun 19 12:02:27 2009 from 125.24.196.166.adsl.dynamic.totbb.net gw:~# uname -a Linux l7.mscompute.com 2.6.28 #2 SMP Sat Jun 13 18:19:43 ICT 2009 i686 GNU/Linux
คอมไพล์ iptables v1.4.3.2 เพื่อให้รองรับกับ layer7
[root@gateway ~]# cd /usr/src/iptables-1.4.3.2 [root@gateway ~]# patch -p1 </usr/src/iptables-1.4.3.2-imq_xt.diff [root@gateway ~]# cp /usr/src/netfilter-layer7-v2.22/iptables-1.4.3forward-for-kernel-2.6.20forward/* /usr/src/iptables-1.4.3.2/extensions/ [root@gateway ~]# ./configure --with-kernel=/usr/src/linux [root@gateway ~]# make [root@gateway ~]# make install [root@gateway ~]# cd /usr/src/l7-protocols-2009-05-28 [root@gateway ~]# make install [root@gateway ~]# cp /usr/local/sbin/iptables /sbin/ [root@gateway ~]# modprobe xt_layer7
แก้ไข /etc/rc.local
[root@gateway ~]# nano /etc/rc.local
<source lang=text> modprobe xt_rateest modprobe xt_helper modprobe xt_dccp modprobe xt_TPROXY modprobe xt_NFLOG modprobe xt_limit modprobe xt_tcpmss modprobe xt_connbytes modprobe xt_owner modprobe xt_sctp modprobe xt_DSCP modprobe xt_MARK modprobe xt_IMQ modprobe xt_statistic modprobe xt_quota modprobe xt_layer7 modprobe xt_TCPOPTSTRIP modprobe xt_recent modprobe xt_NOTRACK modprobe xt_iprange modprobe xt_CONNSECMARK modprobe xt_multiport modprobe xt_CONNMARK modprobe xt_RATEEST modprobe xt_policy modprobe xt_dscp modprobe xt_pkttype modprobe xt_length modprobe xt_CLASSIFY modprobe xt_physdev modprobe xt_SECMARK modprobe xt_connlimit modprobe xt_tcpudp modprobe xt_TRACE modprobe xt_realm modprobe xt_conntrack modprobe xt_string modprobe xt_hashlimit modprobe xt_mac modprobe xt_time modprobe xt_mark modprobe xt_comment modprobe xt_u32 modprobe xt_NFQUEUE modprobe xt_TCPMSS modprobe xt_socket modprobe xt_esp modprobe xt_state modprobe xt_connmark modprobe nf_conntrack_ftp modprobe nf_conntrack modprobe nf_nat_ftp modprobe nf_nat </source>
[root@gateway ~]# reboot
ตรวจสอบ xt_layer7
[root@gateway ~]# lsmod | grep xt_layer7 xt_layer7 14356 0 nf_conntrack 64392 14 xt_layer7,xt_CONNMARK,xt_state,nf_nat_pptp,nf_nat_irc,nf_nat_ftp,ipt_MASQUERADE,nf_conntrack_pptp,nf_conntrack_proto_gre,nf_conntrack_irc,nf_conntrack_ftp,iptable_nat,nf_nat,nf_conntrack_ipv4 x_tables 18188 10 xt_layer7,xt_CONNMARK,xt_mark,xt_tcpudp,xt_state,ipt_MASQUERADE,ipt_REJECT,ipt_LOG,iptable_nat,ip_tables
ทดสอบ layer7
[root@gateway ~]# iptables -A FORWARD -m layer7 --l7proto bittorrent -j DROP [root@gateway ~]# iptables -A FORWARD -m layer7 --l7proto msnmessenger -j DROP [root@gateway ~]# iptables -A FORWARD -m layer7 --l7proto fasttrack -j DROP
[root@gateway ~]# iptables -nvL | grep LAYER
<source lang=bash>
533 50633 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto bittorrent state NEW
30091 2183K DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto edonkey state NEW
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto fasttrack state NEW 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto gnutella state NEW
74468 6939K DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto bittorrent state NEW
325K 24M DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto edonkey state NEW 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto fasttrack state NEW 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto gnutella state NEW 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto bittorrent state NEW
17392 1161K DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto edonkey state NEW
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto fasttrack state NEW 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto gnutella state NEW
</source>
ทดสอบ IMQ
http://www.linuximq.net/usage.html
Load Balance
Script ด้านล่างเป็นการใช้งานแบบ 3 Link Loadbalance เบื้องต้นจำเป็นต้องเซ็ทอัพ Interfaces ทั้งหมดก่อน
นำ Script ด้านล่างไว้ที่ /etc/init.d/
<source lang=bash>
- !/bin/bash
- IP Route Script ###
- mscompute.com ###
- 12 July 2009 ###
- IP Route Balance script on virtual interface
- requirement edit /etc/network/interface 3 virtual interface
RT_TABLES=/etc/iproute2/rt_tables TABLE_PREFIX=router
- Internet Link 1
EXT_IF1=eth0 EXT_IP1=192.168.101.2 EXT_NETWORK1=192.168.101.0/24 EXT_GW1=192.168.101.1
- Internet Link 2
EXT_IF2=eth0:1 EXT_IP2=192.168.102.2 EXT_NETWORK2=192.168.102.0/24 EXT_GW2=192.168.102.1
- Internet Link 3
EXT_IF3=eth0:2 EXT_IP3=192.168.103.2 EXT_NETWORK3=192.168.103.0/24 EXT_GW3=192.168.103.1
- Check pre-defined tables name
rt_tables=`cat ${RT_TABLES} | grep "200 ${TABLE_PREFIX}0" | wc -l` if [ $rt_tables -eq 0 ]; then
## Create the route tables name from prefix
echo "" >> ${RT_TABLES}
echo "#" >> ${RT_TABLES}
echo "# Added by PPP Load Balancer" >> ${RT_TABLES}
echo "#" >> ${RT_TABLES}
for (( i = 0; i < 10; i++ )); do
echo "20${i} ${TABLE_PREFIX}${i}" >> ${RT_TABLES}
done
fi
- Link 1
ip route add $EXT_NETWORK1 dev $EXT_IF1 src $EXT_IP1 table 201 ip route add default via $EXT_GW1 table 201
ip rule add from $EXT_IP1 table 201 ip rule add fwmark 0x1 table 201
- Link 2
ip route add $EXT_NETWORK2 dev $EXT_IF2 src $EXT_IP2 table 202 ip route add default via $EXT_GW2 table 202
ip rule add from $EXT_IP2 table 202 ip rule add fwmark 0x2 table 202
- Link 3
ip route add $EXT_NETWORK3 dev $EXT_IF3 src $EXT_IP3 table 203 ip route add default via $EXT_GW3 table 203
ip rule add from $EXT_IP3 table 203 ip rule add fwmark 0x3 table 203
- For 2 Link
- ip route add default scope global equalize nexthop via $EXT_GW1 dev $EXT_IF1 weight 1 nexthop via $EXT_GW2 dev $EXT_IF2 weight 1
- For 3 Link
ip route add default scope global equalize nexthop via $EXT_GW1 dev $EXT_IF1 weight 1 nexthop via $EXT_GW2 dev $EXT_IF2 weight 1 nexthop via $EXT_GW3 dev $EXT_IF3 weight 1
- ip route add 192.168.1.0/24 mpath rr nexthop via $EXT_GW1 weight 1 nexthop via $EXT_GW2 weight 1 nexthop via $EXT_GW3 weight 1
- iptables -t nat -A POSTROUTING -m statistic --mode nth --every 3 --packet 0 -j SNAT --to-source $EXT_IP1
- iptables -t nat -A POSTROUTING -m statistic --mode nth --every 3 --packet 1 -j SNAT --to-source $EXT_IP2
- iptables -t nat -A POSTROUTING -m statistic --mode nth --every 3 --packet 2 -j SNAT --to-source $EXT_IP3
- End Script ##
</source> Test it !!!!
olo:/usr/src# ip route
192.168.101.0/24 dev eth0 proto kernel scope link src 192.168.101.2
192.168.102.0/24 dev eth1 proto kernel scope link src 192.168.102.2
192.168.103.0/24 dev eth1 proto kernel scope link src 192.168.103.2
192.168.1.0/24 dev eth2 proto kernel scope link src 192.168.1.1
default equalize
nexthop via 192.168.101.1 dev eth0 weight 1
nexthop via 192.168.102.1 dev eth0 weight 1
nexthop via 192.168.103.1 dev eth0 weight 1
olo:/usr/src# ping -I eth0 mscompute.com PING mscompute.com (122.155.1.103) from 192.168.101.2 eth0: 56(84) bytes of data. 64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=1 ttl=52 time=26.6 ms 64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=2 ttl=52 time=28.0 ms 64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=3 ttl=52 time=26.4 ms ^C --- mscompute.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 26.410/27.035/28.078/0.765 ms olo:/usr/src# ping -I eth1 mscompute.com PING mscompute.com (122.155.1.103) from 192.168.102.2 eth1: 56(84) bytes of data. 64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=1 ttl=52 time=25.7 ms 64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=2 ttl=52 time=26.2 ms 64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=3 ttl=52 time=27.0 ms 64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=4 ttl=52 time=27.9 ms
Vuurmuur Firewall Rule For Dualwan+Dota
[x] 1 Accept ntp any any
[x] 2 Accept ssh any any
[x] 3 Accept dns any any nfmark="1"
[x] 4 Accept ftp any any nfmark="1"
[x] 5 Accept https any any nfmark="1"
[x] 6 Accept http any any nfmark="1"
[x] 7 Accept Game any any nfmark="2"
[x] 8 Redirect ftp Lan.Local All.ExtDual redirectport="2121"
[x] 9 Redirect http Lan.Local All.ExtDual redirectport="8888"
[x] 10 Snat any All.ExtDual Lan.Local -
[x] 11 Snat any Lan.Local All.ExtDual -
[x] 12 Snat any Lan.Local PCMASTER.Lan.Local -
References
http://www.ssi.bg/~ja/#routes-2.6
http://l7-filter.sourceforge.net/protocols
Protocal Definition /etc/l7-protocols
