Solaris 11 - Beginners Guide to Network Administration / Link Aggregation / IPMP
Last post we have gone through some basics of Network administration. In this post, we will be targeting some Advance features of Network administrations like Link Aggregation, IPMP (IP Multipathing) features. Before going in deep we can see some basic concepts of networking.
Basic Concepts:
- Bandwidth: Bandwidth is bit rate for sending and receiving data over the network.
- Failover: Capability to have a backup component for the one becomes unavailable.
- Load balancing: Distribution of workload to achieve best utilization.
Link Aggregation
Link Aggregation is a combination of two or three physical interfaces on the system which are considered as a single logical unit (aggr) that provides the best bandwidth, automatic fail over features, load balancing, and redundancy benefits. Link Aggregation’s Link should be in the same Speed and same mode. We can administrate the link aggregation by using dladm command by adding, modifying, deleting links.
Before starting the aggregation, we have to make sure the physical interface should have the same mode as full-duplex and same speed. By executing dladm show-link. Creating aggregartion link with using two interface net1 and net2
# dladm create-aggr -l net1 -l net2 aggr0
#
# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2
#
# dladm show-aggr
LINK MODE POLICY ADDRPOLICY LACPACTIVITY LACPTIMER
aggr0 trunk L4 auto off short
#
To Add a another Link (net3) to an Aggregation (aggr0)
# dladm add-aggr -l net3 aggr0
# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2 net3
#
To assigning IP to the aggregation link
# ipadm create-addr -T static -a 192.168.113.142 aggr0
ipadm: cannot create address: No such interface
#
Opps……getting error says that No such interface, that means we have to probe the interface first
# ipadm create-ip aggr0
# ipadm create-addr -T static -a 192.168.113.142 aggr0
aggr0/v4
#
Yes…we have done the aggr0 aggregation link with using net1 net2 net3 physical link and assigned Ip address too.
# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2 net3
#
# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
aggr0/v4 static ok 192.168.113.142/24
lo0/v6 static ok ::1/128
#
To delete/remove the link from an Aggregation link
# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 up --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2 net3
# dladm remove-aggr -l net3 aggr0
#
# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net3 phys 1500 unknown --
net2 phys 1500 up --
aggr0 aggr 1500 up net1 net2
#
IP Multipathing (IPMP)
IP Multipathing (IPMP) provides physical interface failure detection, Load balancing of packets, and transparent redundancy features. IPMP makes it possible to assign the IP address to a group of the network interface. if anyone of the interface fails, the group can bind the IP address to another interface from the same IPMP group as transparent redundancy. IPMP can be configured for both IPv4 and IPv6. IPMP group will come up with ACTIVE - ACTIVE and ACTIVE- STANDBY
Advantage of IPMP
- Fault tolerance
- Spreading the load
- Increased bandwidth
- Transparent redundancy
- Works even if its different switches
Components of IPMP
- IPMP daemon : in.mapthd
- IPMP service : svc:/network/ipmp:default
- IPMP conf file : /etc/default/mpathd
- IPMP Commands : ipadm, ipmpstat
Let we start Creating an IPMP group and Adding IP address to an IPMP group and Deleting and disable the IPMP group. Creating ipmp0 group with using “ipadm create-ipmp” command:
# ipadm create-ip net1
# ipadm create-ip net2
# ipadm create-ip net3
# ipadm create-ipmp ipmp0
# ipadm add-ipmp -i net1 -i net3 ipmp0
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net3 net1
#
Assigning IP to that ipmp0 group:
# ipadm create-addr -T static -a 192.168.113.145 ipmp0/v4addr1
# ipadm create-addr -T static -a 192.168.113.146 ipmp0/v4addr2
# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
ipmp0/v4addr1 static ok 192.168.113.145/24
ipmp0/v4addr2 static ok 192.168.113.146/24
lo0/v6 static ok ::1/128
#
Remove the interface from IPMP group:
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net1 net3
#
# ipadm remove-ipmp -i net1 ipmp0
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net3
#
Adding one Interface to the IPMP group:
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net3
# ipadm add-ipmp -i net2 ipmp0
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net2 net3
#
To disable an IPMP group:
# ipadm disable-if -t ipmp0
# ipmpstat -g
# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
lo0/v6 static ok ::1/128
ipmp0/v4addr1 static disabled 192.168.113.145
ipmp0/v4addr2 static disabled 192.168.113.146
#
To delete the IPMP group:
# ipadm delete-ipmp ipmp0
# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4test static ok 192.168.113.139/24
lo0/v6 static ok ::1/128
#
Common Issues and Troubleshooting
Encountered some issues and fixes.
ERROR: ipadm: cannot delete IPMP interface ipmp0: IPMP group is not empty.
# ipadm delete-ipmp ipmp0
ipadm: cannot delete IPMP interface ipmp0: IPMP group is not empty
#
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok -- net1
# ipmpstat -i
INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE
net1 yes ipmp0 --mbM-- up disabled ok
# ipadm delete-ip net1
# ipadm delete-ipmp ipmp0
# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
net0 ip ok yes --
net3 ip disabled no --
net2 ip disabled no --
#
ERROR: ipadm: cannot create interface net2: Operation not supported on disabled object ERROR: ipadm: persistent operation not supported for enable-if
# ipadm create-ip net2
ipadm: cannot create interface net2: Operation not supported on disabled object
#
# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
net0 ip ok yes --
net3 ip disabled no --
net2 ip disabled no --
#
# ipadm enable-if net3
ipadm: persistent operation not supported for enable-if
#
# ipadm enable-if -t net3
# ipadm enable-if -t net2
#
# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
net0 ip ok yes --
net2 ip down no --
net3 ip down no --
#