View source for Layer7 IMQ Route Multipath Loadbalance Debian Lenny 2.6.28

'''Debian Lenny, Clarkconnect Enterprise 4.3 Kernel 2.6.28.10, iptables 1.4.3 IMQ Route_Multipath patch contribute and test by phat@mscompute.com June 13 2009( Update 19 July 2009 ) '''<br>
วิธีการด้านล่างนี้ทั้งหมดทั้งสิ้นอาจมีข้อผิดพลาดซึ้งไม่รับประกันการใช้งานใด้ๆทั้งสิ้น
ด้านล่างนี้เป็นการคอนฟิกหลักบน Debian Lenny, Clarkconnect 4.3  ( Kernel 2.6.28 + Layer7 + IMQ + Multipath Route ) ซึ่งสามารถทำงานบน Virtual interface และ Loadbalance Wan interface ได้

===== Debian  =====
<pre>[root@gateway ~]# apt-get install gzip unzip bzip2
[root@gateway ~]# apt-get install debhelper screen fakeroot zlib1g-dev build-essential libncurses5-dev kernel-package
</pre> 
===== Clarkconnect 4.3  =====
<pre>[root@gateway ~]# apt-get install cc-devel</pre>
ดาวโหลด Package 
<pre>[root@gateway ~]# wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.28.10.tar.bz2
[root@gateway ~]# wget http://ufpr.dl.sourceforge.net/sourceforge/l7-filter/netfilter-layer7-v2.22.tar.gz
[root@gateway ~]# wget http://ufpr.dl.sourceforge.net/sourceforge/l7-filter/l7-protocols-2009-05-28.tar.gz
[root@gateway ~]# wget http://www.ssi.bg/~ja/routes-2.6.28-16.diff
[root@gateway ~]# wget http://www.linuximq.net/patchs/linux-2.6.28.9-imq-test2.diff
[root@gateway ~]# wget http://www.linuximq.net/patchs/iptables-1.4.3.2-imq_xt.diff
[root@gateway ~]# wget http://www.netfilter.org/projects/iptables/files/iptables-1.4.3.2.tar.bz2</pre>
แตกไฟล์ออกมา
<pre>[root@gateway ~]# tar xjfv linux-2.6.28.10.tar.bz2
[root@gateway ~]# tar xjfv iptables-1.4.3.2.tar.bz2
[root@gateway ~]# tar zxvf netfilter-layer7-v2.22.tar.gz
[root@gateway ~]# tar xzfv l7-protocols-2009-05-28.tar.gz
</pre> 
สร้าง Symbol Link 
<pre>[root@gateway ~]# ln -s /usr/src/linux-2.6.28.10 /usr/src/linux
</pre> 
Patch Kernel ด้วย patch file 
<pre>[root@gateway ~]# cd linux 
[root@gateway ~]# patch -p1 </usr/src/netfilter-layer7-v2.22/kernel-2.6.25-2.6.28-layer7-2.22.patch
[root@gateway ~]# patch -p1 </usr/src/routes-2.6.28-16.diff
[root@gateway ~]# patch -p1 </usr/src/linux-2.6.28.9-imq-test2.diff
</pre> 
===== Config Kernel  =====
<pre>
[root@gateway ~]# make menuconfig
</pre>

<source lang=bash>
Networking support > Networking options > Network packet filtering framework (Netfilter) > Core Netfilter Configuration.
[ ] layer7 match support

[*] select all
[M] select all

<M>   "IMQ" target support
<M>   "layer7" match support
[*]   "Layer7" debugging output

Networking support > Networking options > Network packet filtering framework (Netfilter) > IP: Netfilter Configuration.

[*] select all
[M] select all
<M>   Full NAT

</source> 
Exit Save .config

'''คอมไพล์และติดตั้งมันซะ Deb + CC4.3''' ( Options 1 )
<pre>[root@gateway ~]# make &amp;&amp; make modules &amp;&amp; make modules_install &amp;&amp; make install</pre> 
ทำให้มันบูท Kernel ใหม่ ( Debian Only ) CC4.3 ไม่ต้องทำ 
<pre>[root@gateway ~]# cd /boot
[root@gateway ~]# mkinitramfs -o initrd.img-2.6.28.10 2.6.28.10
[root@gateway ~]# update-grub
[root@gateway ~]# reboot
</pre>

'''ถ้าต้องการ Compile และสร้าง .deb ด้วย ใช้คำสั่ง''' ( Options 2 )
<pre>
[root@gateway ~]# make clean && make mrproper
[root@gateway ~]# make menuconfig
</pre>
<source lang=bash>
Networking support > Networking options > Network packet filtering framework (Netfilter) > Core Netfilter Configuration.
[ ] layer7 match support

[*] select all
[M] select all

<M>   "IMQ" target support
<M>   "layer7" match support
[*]   "Layer7" debugging output

Networking support > Networking options > Network packet filtering framework (Netfilter) > IP: Netfilter Configuration.

[*] select all
[M] select all
<M>   Full NAT
</source> 
Exit Save .config
<pre>
[root@gateway ~]# make-kpkg clean
[root@gateway ~]# fakeroot make-kpkg --initrd --append-to-version=-mscomputel7mpath kernel_image kernel_headers
[root@gateway ~]# cd /usr/src
[root@gateway ~]# dpkg -i linux-image-*
[root@gateway ~]# dpkg -i linux-headers-*
[root@gateway ~]# reboot
</pre>
ขี้เกียจคอมไพล์ [http://rapidshare.com/files/254621616/2.6.28-l7multiroute_Debian_Lenny.tar.bz2 ดาวโหลด]

แก้ไข /boot/grub/menu.lst ( CC4.3 Only ) 
<pre>
[root@gateway ~]# nano /boot/grub/menu.lst
</pre>

<source lang=bash>
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda3
#          initrd /initrd-version.img
#boot=/dev/sda
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Linux (2.6.28)
        root (hd0,0)
        kernel /vmlinuz-2.6.28 ro root=LABEL=/ video=vesafb vga=0x313
        initrd /initrd-2.6.28.img
#title Linux (2.6.18-93.cc4)
#       root (hd0,0)
#       kernel /vmlinuz-2.6.18-93.cc4 ro root=LABEL=/ video=vesafb vga=0x313
#       initrd /initrd-2.6.18-93.cc4.img
#title Linux Safe Mode (2.6.18-93.cc4)
#       root (hd0,0)
#       kernel /vmlinuz-2.6.18-93.cc4 ro root=LABEL=/
#       initrd /initrd-2.6.18-93.cc4.img
</source> 
Reboot เครื่อง 
<pre>[root@gateway ~]# reboot
</pre>

===== ตรวจสอบ Kernel Version CC4.3  =====
<pre>[root@gateway ~]# uname -a
Linux gateway.clarkconnect.lan 2.6.28 #1 SMP Fri Jun 19 13:17:45 ICT 2009 i686 i686 i386 GNU/Linux
</pre> 
===== ตรวจสอบ Kernel Version Debian Lenny  =====
<pre>Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Jun 19 12:02:27 2009 from 125.24.196.166.adsl.dynamic.totbb.net
gw:~# uname -a
Linux l7.mscompute.com 2.6.28 #2 SMP Sat Jun 13 18:19:43 ICT 2009 i686 GNU/Linux
</pre> 
===== คอมไพล์ iptables v1.4.3.2 เพื่อให้รองรับกับ layer7  =====
<pre>[root@gateway ~]# cd /usr/src/iptables-1.4.3.2
[root@gateway ~]# patch -p1 </usr/src/iptables-1.4.3.2-imq_xt.diff
[root@gateway ~]# cp /usr/src/netfilter-layer7-v2.22/iptables-1.4.3forward-for-kernel-2.6.20forward/* /usr/src/iptables-1.4.3.2/extensions/
[root@gateway ~]# ./configure --with-kernel=/usr/src/linux
[root@gateway ~]# make
[root@gateway ~]# make install
[root@gateway ~]# cd /usr/src/l7-protocols-2009-05-28
[root@gateway ~]# make install
[root@gateway ~]# cp /usr/local/sbin/iptables /sbin/
[root@gateway ~]# modprobe xt_layer7
</pre>
แก้ไข /etc/rc.local 
<pre>[root@gateway ~]# nano /etc/rc.local</pre>
<source lang=text>
modprobe xt_rateest
modprobe xt_helper
modprobe xt_dccp
modprobe xt_TPROXY
modprobe xt_NFLOG
modprobe xt_limit
modprobe xt_tcpmss
modprobe xt_connbytes
modprobe xt_owner
modprobe xt_sctp
modprobe xt_DSCP
modprobe xt_MARK
modprobe xt_IMQ
modprobe xt_statistic
modprobe xt_quota
modprobe xt_layer7
modprobe xt_TCPOPTSTRIP
modprobe xt_recent
modprobe xt_NOTRACK
modprobe xt_iprange
modprobe xt_CONNSECMARK
modprobe xt_multiport
modprobe xt_CONNMARK
modprobe xt_RATEEST
modprobe xt_policy
modprobe xt_dscp
modprobe xt_pkttype
modprobe xt_length
modprobe xt_CLASSIFY
modprobe xt_physdev
modprobe xt_SECMARK
modprobe xt_connlimit
modprobe xt_tcpudp
modprobe xt_TRACE
modprobe xt_realm
modprobe xt_conntrack
modprobe xt_string
modprobe xt_hashlimit
modprobe xt_mac
modprobe xt_time
modprobe xt_mark
modprobe xt_comment
modprobe xt_u32
modprobe xt_NFQUEUE
modprobe xt_TCPMSS
modprobe xt_socket
modprobe xt_esp
modprobe xt_state
modprobe xt_connmark
modprobe nf_conntrack_ftp
modprobe nf_conntrack
modprobe nf_nat_ftp
modprobe nf_nat
</source>
<pre>[root@gateway ~]# reboot
</pre> 
ตรวจสอบ xt_layer7 
<pre>[root@gateway ~]# lsmod | grep xt_layer7
xt_layer7              14356  0
nf_conntrack           64392  14 xt_layer7,xt_CONNMARK,xt_state,nf_nat_pptp,nf_nat_irc,nf_nat_ftp,ipt_MASQUERADE,nf_conntrack_pptp,nf_conntrack_proto_gre,nf_conntrack_irc,nf_conntrack_ftp,iptable_nat,nf_nat,nf_conntrack_ipv4
x_tables               18188  10 xt_layer7,xt_CONNMARK,xt_mark,xt_tcpudp,xt_state,ipt_MASQUERADE,ipt_REJECT,ipt_LOG,iptable_nat,ip_tables
</pre> 
===== ทดสอบ layer7  =====
<pre>
[root@gateway ~]# iptables -A FORWARD -m layer7 --l7proto bittorrent -j DROP
[root@gateway ~]# iptables -A FORWARD -m layer7 --l7proto msnmessenger -j DROP
[root@gateway ~]# iptables -A FORWARD -m layer7 --l7proto fasttrack -j DROP
</pre>
<pre>[root@gateway ~]# iptables -nvL | grep LAYER 
</pre>
<source lang=bash>
  533 50633 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto bittorrent state NEW
30091 2183K DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto edonkey state NEW
    0     0 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto fasttrack state NEW
    0     0 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto gnutella state NEW
74468 6939K DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto bittorrent state NEW
 325K   24M DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto edonkey state NEW
    0     0 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto fasttrack state NEW
    0     0 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto gnutella state NEW
    0     0 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto bittorrent state NEW
17392 1161K DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto edonkey state NEW
    0     0 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto fasttrack state NEW
    0     0 DROP       all  --  *      *       0.0.0.0/0       0.0.0.0/0       LAYER7 l7proto gnutella state NEW
</source>

===== ทดสอบ IMQ  =====
http://www.linuximq.net/usage.html

===== Load Balance =====
Script ด้านล่างเป็นการใช้งานแบบ 3 Link Loadbalance เบื้องต้นจำเป็นต้องเซ็ทอัพ Interfaces ทั้งหมดก่อน

[http://mscompute.com/wiki/index.php/Multi_interface_Alias_Debian_Ubuntu Setup interfaces]

นำ Script ด้านล่างไว้ที่ /etc/init.d/

----

<source lang=bash>
#!/bin/bash
##################################################
###             IP Route Script                ###
###              mscompute.com                 ###
###              12 July 2009                  ###
##################################################
# IP Route Balance script on virtual interface
#
# requirement edit /etc/network/interface 3 virtual interface

RT_TABLES=/etc/iproute2/rt_tables
TABLE_PREFIX=router

# Internet Link 1
EXT_IF1=eth0
EXT_IP1=192.168.101.2
EXT_NETWORK1=192.168.101.0/24
EXT_GW1=192.168.101.1

# Internet Link 2
EXT_IF2=eth0:1
EXT_IP2=192.168.102.2
EXT_NETWORK2=192.168.102.0/24
EXT_GW2=192.168.102.1

# Internet Link 3
EXT_IF3=eth0:2
EXT_IP3=192.168.103.2
EXT_NETWORK3=192.168.103.0/24
EXT_GW3=192.168.103.1

# Check pre-defined tables name
rt_tables=`cat ${RT_TABLES} | grep "200	${TABLE_PREFIX}0" | wc -l`
if [ $rt_tables -eq 0 ]; then
  ## Create the route tables name from prefix
  echo ""  >> ${RT_TABLES}
  echo "#" >> ${RT_TABLES}
  echo "# Added by PPP Load Balancer" >> ${RT_TABLES}
  echo "#" >> ${RT_TABLES}

  for (( i = 0; i < 10; i++ )); do
    echo "20${i}	${TABLE_PREFIX}${i}" >> ${RT_TABLES}
  done
fi

#Link 1
ip route add $EXT_NETWORK1 dev $EXT_IF1 src $EXT_IP1 table 201
ip route add default via $EXT_GW1 table 201

ip rule add from $EXT_IP1 table 201
ip rule add fwmark 0x1 table 201


#Link 2
ip route add $EXT_NETWORK2 dev $EXT_IF2 src $EXT_IP2 table 202
ip route add default via $EXT_GW2 table 202

ip rule add from $EXT_IP2 table 202
ip rule add fwmark 0x2 table 202

#Link 3
ip route add $EXT_NETWORK3 dev $EXT_IF3 src $EXT_IP3 table 203
ip route add default via $EXT_GW3 table 203

ip rule add from $EXT_IP3 table 203
ip rule add fwmark 0x3 table 203


#For 2 Link
#ip route add default scope global equalize nexthop via $EXT_GW1 dev $EXT_IF1 weight 1 nexthop via $EXT_GW2 dev $EXT_IF2 weight 1

#For 3 Link
ip route add default scope global equalize nexthop via $EXT_GW1 dev $EXT_IF1 weight 1 nexthop via $EXT_GW2 dev $EXT_IF2 weight 1 nexthop via $EXT_GW3 dev $EXT_IF3 weight 1

#ip route add 192.168.1.0/24 mpath rr nexthop via $EXT_GW1 weight 1 nexthop via $EXT_GW2 weight 1 nexthop via $EXT_GW3 weight 1

#iptables -t nat -A POSTROUTING -m statistic --mode nth --every 3 --packet 0 -j SNAT --to-source $EXT_IP1
#iptables -t nat -A POSTROUTING -m statistic --mode nth --every 3 --packet 1 -j SNAT --to-source $EXT_IP2
#iptables -t nat -A POSTROUTING -m statistic --mode nth --every 3 --packet 2 -j SNAT --to-source $EXT_IP3

## End Script ##
</source>
Test it !!!!
<pre>
olo:/usr/src# ip route
192.168.101.0/24 dev eth0  proto kernel  scope link  src 192.168.101.2
192.168.102.0/24 dev eth1  proto kernel  scope link  src 192.168.102.2
192.168.103.0/24 dev eth1  proto kernel  scope link  src 192.168.103.2
192.168.1.0/24 dev eth2  proto kernel  scope link  src 192.168.1.1
default equalize
        nexthop via 192.168.101.1  dev eth0 weight 1
        nexthop via 192.168.102.1  dev eth0 weight 1
        nexthop via 192.168.103.1  dev eth0 weight 1
</pre>
<pre>
olo:/usr/src# ping -I eth0 mscompute.com
PING mscompute.com (122.155.1.103) from 192.168.101.2 eth0: 56(84) bytes of data.
64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=1 ttl=52 time=26.6 ms
64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=2 ttl=52 time=28.0 ms
64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=3 ttl=52 time=26.4 ms
^C
--- mscompute.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 26.410/27.035/28.078/0.765 ms
olo:/usr/src# ping -I eth1 mscompute.com
PING mscompute.com (122.155.1.103) from 192.168.102.2 eth1: 56(84) bytes of data.
64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=1 ttl=52 time=25.7 ms
64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=2 ttl=52 time=26.2 ms
64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=3 ttl=52 time=27.0 ms
64 bytes from ns1-1551103.dragonhispeed.com (122.155.1.103): icmp_seq=4 ttl=52 time=27.9 ms
</pre>

Vuurmuur Firewall Rule For Dualwan+Dota
<pre>
[x]  1  Accept    ntp                             any                    any             
                                     
[x]  2  Accept    ssh                             any                     any          
                                    
[x]  3  Accept    dns                             any                     any                  nfmark="1"

[x]  4  Accept    ftp                             any                     any                  nfmark="1"  

[x]  5  Accept    https                           any                     any                  nfmark="1"  

[x]  6  Accept    http                            any                     any                  nfmark="1"            

[x]  7  Accept    Game                            any                    any                   nfmark="2"           

[x]  8  Redirect  ftp                             Lan.Local              All.ExtDual           redirectport="2121"      

[x]  9  Redirect  http                            Lan.Local              All.ExtDual           redirectport="8888"         

[x] 10  Snat      any                             All.ExtDual            Lan.Local                    -   
                                  
[x] 11  Snat      any                             Lan.Local              All.ExtDual                  -   
                                  
[x] 12  Snat      any                             Lan.Local              PCMASTER.Lan.Local           -          
             
</pre>
----
'''References'''
----


http://www.ssi.bg/~ja/#routes-2.6

http://l7-filter.sourceforge.net/protocols 

Protocal Definition /etc/l7-protocols

http://www.linuximq.net/

http://www.howtoforge.com/kernel_compilation_debian_etch

http://www.linuximq.net/