Lab-38: Linux bridge with Linux containers (lxc)

by sunnynetworkFebruary 6, 2017February 8, 2017

This is a fun lab which involves Linux bridges, Linux containers and spanning tree protocol. I will create L2 switch network using three Linux bridges and three containers connected to bridges. It is going to be a ring network so we can test some basic basic spanning tree function.

Prerequisite:

Install Linux containers using procedure in Lab-36.

In this lab I am using Centos 7. We will be using brctl command to create bridge so Install bridge-utils if not already installed

#yum install bridge-utils

Procedure:

Create three Linux bridges, br0,br1 & br2 using brctl command.

sudo brctl addbr <bridge name>

[root@localhost]# sudo brctl addbr br0
[root@localhost]# sudo brctl addbr br1
[root@localhost]# sudo brctl addbr br2

//bring bridge up
[root@localhost]# ifconfig br0 up
[root@localhost]# ifconfig br0
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c0f6:91ff:fed2:38b0  prefixlen 64  scopeid 0x20
        ether c2:f6:91:d2:38:b0  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6  bytes 508 (508.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost]# ifconfig br2 up
[root@localhost]# ifconfig br1 up

Create virtual ethernet (veth) links and bring them up. veth created in pair, these will serve as trunk ports between bridges

//create veth with pair i.e. veth0 paired with veth1
[root@localhost ~]# ip link add veth0 type veth peer name veth1

[root@localhost ~]# ifconfig veth0 up
[root@localhost ~]# ifconfig veth1 up

//add veth to respective bridges (see topology diagram)
[root@localhost ~]# sudo brctl addif br0 veth0
[root@localhost ~]# sudo brctl addif br1 veth1
[root@localhost ~]# sudo brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.e274c68ffca6       no              veth0
br1             8000.d28623e04c88       no              veth1

//create veth2 paired with veth3, veth4 paired with veth5
[root@localhost ~]# ip link add veth2 type veth peer name veth3
[root@localhost ~]# ip link add veth4 type veth peer name veth5

//bring veth up
[root@localhost ~]# ifconfig veth2 up
[root@localhost ~]# ifconfig veth3 up
[root@localhost ~]# ifconfig veth4 up
[root@localhost ~]# ifconfig veth5 up

//add veth to respective bridges
[root@localhost ~]# sudo brctl addif br1 veth2
[root@localhost ~]# sudo brctl addif br2 veth3
[root@localhost ~]# sudo brctl addif br2 veth4
[root@localhost ~]# sudo brctl addif br0 veth5
//as can be seen veth interfaces are attached to bridges
[root@localhost ~]# sudo brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.c2f691d238b0       no              veth0
                                                        veth5
br1             8000.d28623e04c88       no              veth1
                                                        veth2
br2             8000.9a127ad7bf76       no              veth3
                                                        veth4

//check bridge and veth interfaces
[root@localhost ~]# ip addr
52: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether c2:f6:91:d2:38:b0 brd ff:ff:ff:ff:ff:ff
53: br1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether d2:86:23:e0:4c:88 brd ff:ff:ff:ff:ff:ff
54: veth1@veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br1 state UP qlen 1000
    link/ether d2:86:23:e0:4c:88 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d086:23ff:fee0:4c88/64 scope link
       valid_lft forever preferred_lft forever
55: veth0@veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP qlen 1000
    link/ether e2:74:c6:8f:fc:a6 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e074:c6ff:fe8f:fca6/64 scope link
       valid_lft forever preferred_lft forever
56: veth3@veth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br2 state UP qlen 1000
    link/ether 9a:12:7a:d7:bf:76 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::9812:7aff:fed7:bf76/64 scope link
       valid_lft forever preferred_lft forever
57: veth2@veth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br1 state UP qlen 1000
    link/ether ee:bf:b0:92:54:a0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ecbf:b0ff:fe92:54a0/64 scope link
       valid_lft forever preferred_lft forever
58: veth5@veth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP qlen 1000
    link/ether c2:f6:91:d2:38:b0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::c0f6:91ff:fed2:38b0/64 scope link
       valid_lft forever preferred_lft forever
59: veth4@veth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br2 state UP qlen 1000
    link/ether fe:fe:da:e0:3a:09 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fcfe:daff:fee0:3a09/64 scope link
       valid_lft forever preferred_lft forever
60: br2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 9a:12:7a:d7:bf:76 brd ff:ff:ff:ff:ff:ff
[root@localhost ~]#

Create three Linux containers, c1,c2 & c3

#lxc-create -t ubuntu -n c1
#lxc-create -t ubuntu -n c2
#lxc-create -t ubuntu -n c3

[root@localhost ~]# lxc-create -t ubuntu -n c1
Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ...
Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/lxc/c1/rootfs ...
Copying rootfs to /var/lib/lxc/c1/rootfs ...
Generating locales...
  en_US.UTF-8... up-to-date
Generation complete.
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
Timezone in container is not configured. Adjust it manually.

##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##

[root@localhost ~]# lxc-create -t ubuntu -n c2
Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ...
Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/lxc/c2/rootfs ...
Copying rootfs to /var/lib/lxc/c2/rootfs ...
Generating locales...
  en_US.UTF-8... up-to-date
Generation complete.
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
Timezone in container is not configured. Adjust it manually.

##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##

[root@localhost ~]# lxc-create -t ubuntu -n c3
Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ...
Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/lxc/c3/rootfs ...
Copying rootfs to /var/lib/lxc/c3/rootfs ...
Generating locales...
  en_US.UTF-8... up-to-date
Generation complete.
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
Timezone in container is not configured. Adjust it manually.

##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##

Edit container config file to create veth network types. Container config files for my system is located at /var/lib/lxc/<container name>/config

Container c1 attached to bridge br0, c2 to br1 and c3 to br2

Below sample container c1 config file /var/lib/lxc/c1/config

# Network configuration
lxc.network.type = veth
lxc.network.hwaddr = 00:16:3e:80:81:03
lxc.network.flags = up
lxc.network.link = br0
lxc.network.ipv4 = 192.168.2.1/24

below sample container c2 config file /var/lib/lxc/c2/config

# Network configuration
lxc.network.type = veth
lxc.network.hwaddr = 00:16:3e:6e:70:a7
lxc.network.flags = up
lxc.network.link = br1
lxc.network.ipv4 = 192.168.2.2/24

Below sample container c3 config file /var/lib/lxc/c3/config

# Network configuration
lxc.network.type = veth
lxc.network.hwaddr = 00:16:3e:18:56:d5
lxc.network.flags = up
lxc.network.link = br2
lxc.network.ipv4 = 192.168.2.3/24

Check container network interface. Use lxc-attach command to list ip interfaces

#lxc-attach -n <container name> /sbin/ip addr

//container c1 assigned with ip address 192.168.2.1
[root@localhost]# lxc-attach -n c1 /sbin/ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
67: eth0@if68: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 00:16:3e:80:81:03 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.1/24 brd 192.168.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe80:8103/64 scope link
       valid_lft forever preferred_lft forever
//container c2 assigned ip address 192.168.2.2
[root@localhost]# lxc-attach -n c2 /sbin/ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
69: eth0@if70: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 00:16:3e:6e:70:a7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.2/24 brd 192.168.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe6e:70a7/64 scope link
       valid_lft forever preferred_lft forever

//container c3 assigned ip address 192.168.2.3
[root@localhost]# lxc-attach -n c3 /sbin/ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
71: eth0@if72: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 00:16:3e:18:56:d5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.3/24 brd 192.168.2.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe18:56d5/64 scope link
       valid_lft forever preferred_lft forever

Check bridge status. As can be seen three virtual ethernet interfaces are attached to each bridge

[root@localhost]# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.c2f691d238b0       no              veth0
                                                        veth5
                                                        vethOC0NPG
br1             8000.d28623e04c88       no              veth1
                                                        veth2
                                                        vethSQUCH3
br2             8000.9a127ad7bf76       no              veth3
                                                        veth4
                                                        veth577WF3

At this point we have created this topology. Containers c1,c2 & c3 attached to respective bridges and all bridges connected in a ring. Our L2 network is ready for testing

stp_1

Check spanning tree (stp) status on bridges. As can be seen stp is disabled on all bridges as every bridge is a designated root bridge

[root@localhost]# brctl showstp br0
br0
 bridge id              8000.c2f691d238b0
 designated root        8000.c2f691d238b0
 root port                 0                    path cost                  0
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               1.66                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                  53.31
 flags


veth0 (1)
 port id                8001                    state                forwarding
 designated root        8000.c2f691d238b0       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.66
 flags

veth5 (2)
 port id                8002                    state                forwarding
 designated root        8000.c2f691d238b0       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.66
 flags

vethOC0NPG (3)
 port id                8003                    state                forwarding
 designated root        8000.c2f691d238b0       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.66
 flags

[root@localhost]# brctl showstp br1
br1
 bridge id              8000.d28623e04c88
 designated root        8000.d28623e04c88
 root port                 0                    path cost                  0
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               1.04                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                  45.06
 flags


veth1 (1)
 port id                8001                    state                forwarding
 designated root        8000.d28623e04c88       path cost                  2
 designated bridge      8000.d28623e04c88       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.04
 flags

veth2 (2)
 port id                8002                    state                forwarding
 designated root        8000.d28623e04c88       path cost                  2
 designated bridge      8000.d28623e04c88       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.04
 flags

vethSQUCH3 (3)
 port id                8003                    state                forwarding
 designated root        8000.d28623e04c88       path cost                  2
 designated bridge      8000.d28623e04c88       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.04
 flags

[root@localhost]# brctl showstp br2
br2
 bridge id              8000.9a127ad7bf76
 designated root        8000.9a127ad7bf76
 root port                 0                    path cost                  0
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               1.07                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                  16.98
 flags


veth3 (1)
 port id                8001                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.07
 flags

veth4 (2)
 port id                8002                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.07
 flags

veth577WF3 (3)
 port id                8003                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.07
 flags

Login to container c2 and ping to container c1 (192.168.2.1). Because of loop in the topology network will experience broadcast storm and create congestion. Ping will intermittently pass.

You can confirm the broadcast storm by checking packet counts on bridges. You will see packet counts increment at very high rate

ubuntu@c1:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:80:81:03
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe80:8103/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:87 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:648 (648.0 B)  TX bytes:18666 (18.6 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:56 errors:0 dropped:0 overruns:0 frame:0
          TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:5440 (5.4 KB)  TX bytes:5440 (5.4 KB)

ubuntu@c1:~$ ip route
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1
ubuntu@c1:~$ ping 192.168.2.2
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_req=13 ttl=64 time=4009 ms
64 bytes from 192.168.2.2: icmp_req=19 ttl=64 time=3007 ms
64 bytes from 192.168.2.2: icmp_req=20 ttl=64 time=2005 ms

[root@localhost ~]# ifconfig br0
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c0f6:91ff:fed2:38b0  prefixlen 64  scopeid 0x20<link>
        ether c2:f6:91:d2:38:b0  txqueuelen 1000  (Ethernet)
        RX packets 233826705  bytes 49096357638 (45.7 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost ~]# ifconfig br0
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c0f6:91ff:fed2:38b0  prefixlen 64  scopeid 0x20<link>
        ether c2:f6:91:d2:38:b0  txqueuelen 1000  (Ethernet)
        RX packets 233921147  bytes 49127334614 (45.7 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Stop ping and check bridge packet count again. As can be seen packets counts are are still incrementing at a very high rate. Packets are circulating in the network due to topology loop

[root@localhost]# ifconfig br0
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c0f6:91ff:fed2:38b0  prefixlen 64  scopeid 0x20
        ether c2:f6:91:d2:38:b0  txqueuelen 1000  (Ethernet)
        RX packets 158174506  bytes 28354814180 (26.4 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost]# ifconfig br0
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::c0f6:91ff:fed2:38b0  prefixlen 64  scopeid 0x20
        ether c2:f6:91:d2:38:b0  txqueuelen 1000  (Ethernet)
        RX packets 159069415  bytes 28530733852 (26.5 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost]# ifconfig br1
br1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::d086:23ff:fee0:4c88  prefixlen 64  scopeid 0x20
        ether d2:86:23:e0:4c:88  txqueuelen 1000  (Ethernet)
        RX packets 160435925  bytes 28800879432 (26.8 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3  bytes 258 (258.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost]# ifconfig br1
br1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::d086:23ff:fee0:4c88  prefixlen 64  scopeid 0x20
        ether d2:86:23:e0:4c:88  txqueuelen 1000  (Ethernet)
        RX packets 161046481  bytes 28923084308 (26.9 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3  bytes 258 (258.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost]# ifconfig br2
br2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::9812:7aff:fed7:bf76  prefixlen 64  scopeid 0x20
        ether 9a:12:7a:d7:bf:76  txqueuelen 1000  (Ethernet)
        RX packets 162073378  bytes 29129182172 (27.1 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost]# ifconfig br2
br2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::9812:7aff:fed7:bf76  prefixlen 64  scopeid 0x20
        ether 9a:12:7a:d7:bf:76  txqueuelen 1000  (Ethernet)
        RX packets 162616331  bytes 29238047188 (27.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Enable spanning tree on all bridges, br0,br1 & br2.

As can be seen bridge br2 is elected as designated root bridge by spanning tree protocol. This is because the mac address of br2 is lower then br0 & br1

Bridge br1 port veth1(1) blocked by stp to break the loop and now there is no broadcast storm. The new topology after stp enabled should look like this

You ask why veth1 blocked this is because spanning tree packets received on this port has higher path cost than packet received on veth2 which is directly connected to root bridge

stp_2

Another logical view

stp_3

[root@localhost]# brctl stp br0 on
[root@localhost]# brctl stp br1 on
[root@localhost]# brctl stp br2 on

[root@localhost]# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.c2f691d238b0       yes             veth0
                                                        veth5
                                                        vethOC0NPG
br1             8000.d28623e04c88       yes             veth1
                                                        veth2
                                                        vethSQUCH3
br2             8000.9a127ad7bf76       yes             veth3
                                                        veth4
  
                                                      veth577WF3
//all ports in bridge br0 are in forwarding state
[root@localhost]# brctl showstp br0
br0
 bridge id              8000.c2f691d238b0
 designated root        8000.9a127ad7bf76
 root port                 2                    path cost                  2
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               0.00                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                  62.44
 flags                  TOPOLOGY_CHANGE


veth0 (1)
 port id                8001                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.00
 flags

veth5 (2)
 port id                8002                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer         18.79
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

vethOC0NPG (3)
 port id                8003                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.00
 flags

//port veth1(1) is in blocking state
[root@localhost]# brctl showstp br1
br1
 bridge id              8000.d28623e04c88
 designated root        8000.9a127ad7bf76
 root port                 2                    path cost                  2
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               0.00                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                  73.66
 flags


veth1 (1)
 port id                8001                    state                  blocking
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer         19.64
 designated port        8001                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.00
 flags

veth2 (2)
 port id                8002                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer         19.64
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

vethSQUCH3 (3)
 port id                8003                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.d28623e04c88       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.57
 flags

//all ports in bridge br2 are in forwarding state
[root@localhost]# brctl showstp br2
br2
 bridge id              8000.9a127ad7bf76
 designated root        8000.9a127ad7bf76
 root port                 0                    path cost                  0
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               1.68                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                  66.65
 flags


veth3 (1)
 port id                8001                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.68
 flags

veth4 (2)
 port id                8002                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.68
 flags

veth577WF3 (3)
 port id                8003                    state                forwarding
 designated root        8000.9a127ad7bf76       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.68
 flags

Let’s try to change the root bridge selection. The easiest way to do is by changing bridge priority

Change bridge br2 priority to make it root bridge. Bridge with lower priority become root. In this case I am reducing bridge priority just by one 7999.

As can be seen port veth5(2) on bridge br0 transition to block state to break loop. Bridge br1 elected as root bridge

[root@localhost ~]# sudo brctl setbridgeprio br1 7999

[root@localhost]# brctl showstp br1
br1
 bridge id              1f3f.d28623e04c88
 designated root        1f3f.d28623e04c88
 root port                 0                    path cost                  0
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               1.30                 tcn timer                  0.00
 topology change timer    19.86                 gc timer                 243.09
 flags                  TOPOLOGY_CHANGE TOPOLOGY_CHANGE_DETECTED


veth1 (1)
 port id                8001                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      1f3f.d28623e04c88       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.30
 flags

veth2 (2)
 port id                8002                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      1f3f.d28623e04c88       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.30
 flags

vethSQUCH3 (3)
 port id                8003                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      1f3f.d28623e04c88       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.30
 flags

[root@localhost ns]# brctl showstp br0
br0
 bridge id              8000.c2f691d238b0
 designated root        1f3f.d28623e04c88
 root port                 1                    path cost                  2
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               0.00                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                 223.34
 flags                  TOPOLOGY_CHANGE


veth0 (1)
 port id                8001                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      1f3f.d28623e04c88       message age timer         19.57
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

veth5 (2)
 port id                8002                    state                  blocking
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer         19.57
 designated port        8002                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.00
 flags

vethOC0NPG (3)
 port id                8003                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.55
 flags

[root@localhost]# brctl showstp br2
br2
 bridge id              8000.9a127ad7bf76
 designated root        1f3f.d28623e04c88
 root port                 1                    path cost                  2
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               0.00                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                   0.32
 flags                  TOPOLOGY_CHANGE


veth3 (1)
 port id                8001                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      1f3f.d28623e04c88       message age timer         18.34
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

veth4 (2)
 port id                8002                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.00
 flags

veth577WF3 (3)
 port id                8003                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.00
 flags

The new topology should look like this

stp_4

Let’s change the root port on br0 from veth0 to veth5. The easiest way to do that is by changing the port cost on veth0. Edit port cost on veth0 to 10. This will cause veth5 to move to forwarding state and veth0 to blocking.

As can be seen port veth0 transitioned to blocking state

[root@localhost ~]# brctl setpathcost br0 veth0 10
[root@localhost ~]# brctl showstp br0
br0
 bridge id              8000.c2f691d238b0
 designated root        1f3f.d28623e04c88
 root port                 2                    path cost                  4
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               0.00                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                 226.87
 flags                  TOPOLOGY_CHANGE


veth0 (1)
 port id                8001                    state                  blocking
 designated root        1f3f.d28623e04c88       path cost                 10
 designated bridge      1f3f.d28623e04c88       message age timer         18.81
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

veth5 (2)
 port id                8002                    state                 listening
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer         18.81
 designated port        8002                    forward delay timer       12.41
 designated cost           2                    hold timer                 0.00
 flags

vethOC0NPG (3)
 port id                8003                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           4                    hold timer                 0.78
 flags

[root@localhost ~]# brctl showstp br0
br0
 bridge id              8000.c2f691d238b0
 designated root        1f3f.d28623e04c88
 root port                 2                    path cost                  4
 max age                  20.00                 bridge max age            20.00
 hello time                2.00                 bridge hello time          2.00
 forward delay            15.00                 bridge forward delay      15.00
 ageing time             300.00
 hello timer               0.00                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                  32.26
 flags


veth0 (1)
 port id                8001                    state                  blocking
 designated root        1f3f.d28623e04c88       path cost                 10
 designated bridge      1f3f.d28623e04c88       message age timer         19.07
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.00
 flags

veth5 (2)
 port id                8002                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.9a127ad7bf76       message age timer         19.01
 designated port        8002                    forward delay timer        0.00
 designated cost           2                    hold timer                 0.00
 flags

vethOC0NPG (3)
 port id                8003                    state                forwarding
 designated root        1f3f.d28623e04c88       path cost                  2
 designated bridge      8000.c2f691d238b0       message age timer          0.00
 designated port        8003                    forward delay timer        0.00
 designated cost           4                    hold timer                 0.00
 flags

Other useful command is to check mac table

[root@localhost ~]# brctl showmacs br0
port no mac addr                is local?       ageing timer
  2     00:16:3e:18:56:d5       no                10.04
  2     00:16:3e:6e:70:a7       no                12.00
  3     00:16:3e:80:81:03       no                17.73
  2     c2:f6:91:d2:38:b0       yes                0.00
  1     e2:74:c6:8f:fc:a6       yes                0.00
  1     e2:74:c6:8f:fc:a6       yes                0.00
  3     fe:0e:18:bc:7e:e3       yes                0.00
  3     fe:0e:18:bc:7e:e3       yes                0.00
  2     fe:fe:da:e0:3a:09       no                 0.81

You can dump packets on virtual interfaces and bridge

[root@localhost ~]# tcpdump -i veth5 -XX
tcpdump: WARNING: veth5: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth5, link-type EN10MB (Ethernet), capture size 65535 bytes
22:03:48.738230 STP 802.1d, Config, Flags [none], bridge-id 8000.9a:12:7a:d7:bf:76.8002, length 35
        0x0000:  0180 c200 0000 fefe dae0 3a09 0026 4242  ..........:..&BB
        0x0010:  0300 0000 0000 1f3f d286 23e0 4c88 0000  .......?..#.L...
        0x0020:  0002 8000 9a12 7ad7 bf76 8002 0001 1400  ......z..v......
        0x0030:  0200 0f00                                ....
22:03:50.738236 STP 802.1d, Config, Flags [none], bridge-id 8000.9a:12:7a:d7:bf:76.8002, length 35
        0x0000:  0180 c200 0000 fefe dae0 3a09 0026 4242  ..........:..&BB
        0x0010:  0300 0000 0000 1f3f d286 23e0 4c88 0000  .......?..#.L...
        0x0020:  0002 8000 9a12 7ad7 bf76 8002 0001 1400  ......z..v......
        0x0030:  0200 0f00

Lab-37: Linux container (lxc) networking

by sunnynetworkJanuary 31, 2017February 6, 2017

I briefly touched Linux container network type in Lab-36. In this lab I will go over various methods to virtualize Linux container network. Linux containers support these types of network virtualization:

empty
phys
veth
vlan
macvlan supports these modes:
1. bridge
2. vepa
3. private

You can read more about Linux container manual pages here

Prerequisite

Using Lab-36 install Linux container. In this lab I am using Centos 7

Linux container (lxc) configuration file

lxc support two types of configuration files. Since we are working on network I will concentrate on network setting only:

Default configuration file: In my OS this files is located in this path /etc/lxc/default.conf. This file contains default network types. Container inherits default network types from this file. default network setting in my system

#cat default.conf
lxc.network.type = veth
lxc.network.link = virbr0
lxc.network.flags = up

Container configuration file: Each container has its own configuration file, in my OS this file is located in this path /var/lib/lxc/<container name>/conf. When container starts it uses this config file to setup container run time environment including network. This file inherits network setting from default.conf file. This is my container network file looks like for container lxc_cn1

# Template used to create this container: /usr/share/lxc/templates/lxc-ubuntu
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)

# Common configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf

# Container specific configuration
lxc.rootfs = /var/lib/lxc/lxc_cn1/rootfs
lxc.mount = /var/lib/lxc/lxc_cn1/fstab
lxc.utsname = lxc_cn1
lxc.arch = amd64


# Network configuration
lxc.network.type = veth
lxc.network.hwaddr = 00:16:3e:43:aa:ba
lxc.network.flags = up
lxc.network.link = virbr0

In my exercises I will be updating this file to create different network types.

I will perform these connectivity tests where ever applicable

Ping test from container to container
Ping test from host to container and container to host
Ping to external system

Below lab topology. I am using Linux servers connected thru physical interface enp0s29f7u1. I will create Linux containers on machine named host machine.

linux_container_6

lxc.network.type= empty

In this mode no network interface created inside container. This option can be used when container is standalone and don’t need network connectivity.

create a container

#lxc-create -t ubuntu -n empty_cn1

Update container configuration file /var/lib/lxc/empty_cn1/config

# Network configuration
lxc.network.type = empty
lxc.network.flags = up

Start the container and check network setting

[root@localhost]# lxc-start -n empty_cn1 -d

[root@localhost]# lxc-info -n empty_cn1
Name:           empty_cn1
State:          RUNNING
PID:            11825
CPU use:        0.20 seconds
BlkIO use:      0 bytes
Memory use:     3.07 MiB
KMem use:       0 bytes
Link:           virbr0
 TX bytes:      3.01 MiB
 RX bytes:      7.31 KiB
 Total bytes:   3.01 MiB


//check container network setting. No interface created other than loopback
[root@localhost]# lxc-attach -n empty_cn1 /sbin/ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever

lxc.network.type= phys

This option links host interface specified in lxc.network.link to container. To run this option you need an unused interface which can be assigned to container. Once interface assigned to container it disappears from host interface list.

linux_container_4

Create a container

[root@localhost ~]# lxc-create -t ubuntu -n phys_cn1
Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ...
Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/lxc/phys_cn1/rootfs ...
Copying rootfs to /var/lib/lxc/phys_cn1/rootfs ...
Generating locales...
  en_US.UTF-8... up-to-date
Generation complete.
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
Timezone in container is not configured. Adjust it manually.

##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##

Edit container config file /var/lib/lxc/phys_cn1/config. Note: I have updated fields in bold italic

lxc.network.link = enp0s29f7u1
lxc.network.type = phys
lxc.network.ipv4 = 192.168.2.1/24

# Network configuration
lxc.network.type = phys
lxc.network.hwaddr = 00:16:3e:bf:a5:4e
lxc.network.flags = up
lxc.network.link = enp0s29f7u1
lxc.network.ipv4 = 192.168.2.1/24

Start container and check network setting

[root@localhost]# lxc-start -n phys_cn1 -d
[root@localhost]# lxc-info -n phys_cn1
Name:           phys_cn1
State:          RUNNING
PID:            23805
IP:             192.168.2.1
CPU use:        0.21 seconds
BlkIO use:      0 bytes
Memory use:     3.10 MiB
KMem use:       0 bytes
Link:           enp0s29f7u1
 Total bytes:   0 bytes
[root@localhost phys_cn1]#

//network interface disappeared from host
[root@localhost ~]# ifconfig enp0s29f7u1
enp0s29f7u1: error fetching interface information: Device not found

[root@localhost]# lxc-console -n phys_cn1

//interface enp0s29f7u1 linked to container. IP address and mac assigned from 
container configuration file
ubuntu@phys_cn1:~$ ifconfig
enp0s29f7u1 Link encap:Ethernet  HWaddr 00:16:3e:bf:a5:4e
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:febf:a54e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9422 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12999 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:934818 (934.8 KB)  TX bytes:2522162 (2.5 MB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)


//ping to external system passed
ubuntu@phys_cn1:~$ ping 192.168.2.101
PING 192.168.2.101 (192.168.2.101) 56(84) bytes of data.
64 bytes from 192.168.2.101: icmp_req=1 ttl=64 time=0.699 ms
64 bytes from 192.168.2.101: icmp_req=2 ttl=64 time=0.217 ms
64 bytes from 192.168.2.101: icmp_req=3 ttl=64 time=0.204 ms
^C
--- 192.168.2.101 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.204/0.373/0.699/0.230 ms

ubuntu@phys_cn1:~$ ip route
192.168.2.0/24 dev enp0s29f7u1  proto kernel  scope link  src 192.168.2.1
ubuntu@phys_cn1:~$

Ping container from host

//ping to container from host failed
[root@localhost ~]# ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
^C
--- 192.168.2.1 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3004ms

Stop container

//stop container
[root@localhost phys_cn1]# lxc-stop -n phys_cn1

//host interface reappeared 
[root@localhost phys_cn1]# ifconfig enp0s29f7u1
enp0s29f7u1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.100  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::250:b6ff:fe19:4165  prefixlen 64  scopeid 0x20
        ether 00:50:b6:19:41:65  txqueuelen 1000  (Ethernet)
        RX packets 9427  bytes 935162 (913.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 13025  bytes 2525292 (2.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lxc.network.type = vlan

In this mode a vlan interface is linked with the interface specified by the lxc.network.link and assigned to the container. The vlan identifier is specified by the option lxc.network.vlan.id

In this mode container can communicate with external system but can’t communicate with host

linux_container_5

Create a container vlan_cn1

[root@localhost]# lxc-create -t ubuntu -n vlan_cn1
Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ...
Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/lxc/vlan_cn1/rootfs ...
Copying rootfs to /var/lib/lxc/vlan_cn1/rootfs ...
Generating locales...
  en_US.UTF-8... up-to-date
Generation complete.
Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
Timezone in container is not configured. Adjust it manually.

##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##

[root@localhost]# lxc-info -n vlan_cn1
Name:           vlan_cn1
State:          STOPPED
[root@localhost]#

Edit container configuration file. Note: I have updated fields in bold and italic

lxc.network.type= vlan
lxc.network.link = enp0s29f7u1
lxc.network.vlan.id = 100
lxc.network.ipv4 = 192.168.2.1/24

# Network configuration
lxc.network.type = vlan
lxc.network.vlan.id = 100
lxc.network.hwaddr = 00:16:3e:e2:d6:19
lxc.network.flags = up
lxc.network.link = enp0s29f7u1
lxc.network.ipv4 = 192.168.2.1/24

Start container

[root@localhost]# lxc-start -n vlan_cn1 -d
[root@localhost]# lxc-info -n vlan_cn1
Name:           vlan_cn1
State:          RUNNING
PID:            24472
IP:             192.168.2.1
CPU use:        0.21 seconds
BlkIO use:      0 bytes
Memory use:     3.70 MiB
KMem use:       0 bytes
Link:           enp0s29f7u1
 TX bytes:      913.24 KiB
 RX bytes:      2.41 MiB
 Total bytes:   3.30 MiB
[root@localhost vlan_cn1]#

To test connectivity we need to create vlan interface on external system. To create vlan interface you need to Install vconfig

#yum install epel-release
#yum install vconfig

[root@localhost ~]# yum install epel-release
[root@localhost ~]# yum install vconfig

//add vlan interface with vlan.id=100 on external system
[root@localhost ~]# vconfig add enp0s29f7u1 100
WARNING:  Could not open /proc/net/vlan/config.  Maybe you need to load the 8021q module, or maybe you are not using PROCFS??
Added VLAN with VID == 100 to IF -:enp0s29f7u1:-

//a vlan inteface with vlan.id=100 created
[root@localhost ~]# ip addr
5: enp0s29f7u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:b6:19:41:63 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.101/24 scope global enp0s29f7u1
       valid_lft forever preferred_lft forever
    inet6 fe80::250:b6ff:fe19:4163/64 scope link
       valid_lft forever preferred_lft forever
6: enp0s29f7u1.100@enp0s29f7u1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether 00:50:b6:19:41:63 brd ff:ff:ff:ff:ff:ff

//remove ip from base interface 
[root@localhost ~]# ip addr del 192.168.2.101/24 dev enp0s29f7u1

//add ip to vlan interface
[root@localhost ~]# ip addr add 192.168.2.101/24 dev enp0s29f7u1.100@enp0s29f7u1 

[root@localhost ~]# ifconfig enp0s29f7u1.100@enp0s29f7u1 
enp0s29f7u1.100: flags=4098<BROADCAST,MULTICAST>  mtu 1500         
inet 192.168.2.101  netmask 255.255.255.0  broadcast 0.0.0.0         
ether 00:50:b6:19:41:63  txqueuelen 0  (Ethernet)         
RX packets 0  bytes 0 (0.0 B)         
RX errors 0  dropped 0  overruns 0  frame 0         
TX packets 0  bytes 0 (0.0 B)         
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 

[root@localhost ~]# ifconfig enp0s29f7u1.100@enp0s29f7u1 up  

[root@localhost ~]# ifconfig enp0s29f7u1.100@enp0s29f7u1 
enp0s29f7u1.100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500         
inet 192.168.2.101  netmask 255.255.255.0  broadcast 0.0.0.0         
inet6 fe80::250:b6ff:fe19:4163  prefixlen 64  scopeid 0x20         
ether 00:50:b6:19:41:63  txqueuelen 0  (Ethernet)         
RX packets 0  bytes 0 (0.0 B)         
RX errors 0  dropped 0  overruns 0  frame 0         
TX packets 0  bytes 0 (0.0 B)         
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 

[root@localhost ~]#ip addr 
5: enp0s29f7u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 
qdisc pfifo_fast state UP qlen 1000     
link/ether 00:50:b6:19:41:63 brd ff:ff:ff:ff:ff:ff     
inet6 fe80::250:b6ff:fe19:4163/64 scope link        
valid_lft forever preferred_lft forever 

6: enp0s29f7u1.100@enp0s29f7u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 
qdisc noqueue state UP     link/ether 00:50:b6:19:41:63 brd ff:ff:ff:ff:ff:ff     
inet 192.168.2.101/24 scope global enp0s29f7u1.100        
valid_lft forever preferred_lft forever     
inet6 fe80::250:b6ff:fe19:4163/64 scope link        
valid_lft forever preferred_lft forever

ubuntu@vlan_cn1:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:e2:d6:19
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fee2:d619/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:268 errors:0 dropped:0 overruns:0 frame:0
          TX packets:914 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21956 (21.9 KB)  TX bytes:81868 (81.8 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:709 errors:0 dropped:0 overruns:0 frame:0
          TX packets:709 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:78800 (78.8 KB)  TX bytes:78800 (78.8 KB)

//ping to external system passed
ubuntu@vlan_cn1:~$ ping 192.168.2.101
PING 192.168.2.101 (192.168.2.101) 56(84) bytes of data.
64 bytes from 192.168.2.101: icmp_req=1 ttl=64 time=0.387 ms
64 bytes from 192.168.2.101: icmp_req=2 ttl=64 time=0.393 ms
64 bytes from 192.168.2.101: icmp_req=3 ttl=64 time=0.325 ms
64 bytes from 192.168.2.101: icmp_req=4 ttl=64 time=0.223 ms
^C
--- 192.168.2.101 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.223/0.332/0.393/0.068 ms
ubuntu@vlan_cn1:~$

Check incoming packets on external system. As can be seen packets are received with vlan.id=100

//see underlined for vlan.id
[root@localhost ~]# tcpdump -i enp0s29f7u1 -XX
tcpdump: WARNING: enp0s29f7u1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s29f7u1, link-type EN10MB (Ethernet), capture size 65535 bytes
23:51:38.940412 IP 192.168.2.1 > 192.168.2.101: ICMP echo request, id 305, seq 115, length 64
        0x0000:  0050 b619 4163 0016 3ee2 d619 8100 0064  .P..Ac..>......d
        0x0010:  0800 4500 0054 5423 4000 4001 60cf c0a8  ..E..TT#@.@.`...
        0x0020:  0201 c0a8 0265 0800 6ddb 0131 0073 0250  .....e..m..1.s.P
        0x0030:  9558 0000 0000 2305 0f00 0000 0000 1011  .X....#.........
        0x0040:  1213 1415 1617 1819 1a1b 1c1d 1e1f 2021  ...............!
        0x0050:  2223 2425 2627 2829 2a2b 2c2d 2e2f 3031  "#$%&'()*+,-./01
        0x0060:  3233 3435 3637                           234567
23:51:38.940438 IP 192.168.2.101 > 192.168.2.1: ICMP echo reply, id 305, seq 115, length 64
        0x0000:  0016 3ee2 d619 0050 b619 4163 8100 0064  ..>....P..Ac...d
        0x0010:  0800 4500 0054 9f4b 0000 4001 55a7 c0a8  ..E..T.K..@.U...
        0x0020:  0265 c0a8 0201 0000 75db 0131 0073 0250  .e......u..1.s.P
        0x0030:  9558 0000 0000 2305 0f00 0000 0000 1011  .X....#.........
        0x0040:  1213 1415 1617 1819 1a1b 1c1d 1e1f 2021  ...............!
        0x0050:  2223 2425 2627 2829 2a2b 2c2d 2e2f 3031  "#$%&'()*+,-./01
        0x0060:  3233 3435 3637                           234567

Ping container from host

//ping from host to container failed
[root@localhost ~]# ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
From 166.127.6.242 icmp_seq=1 Time to live exceeded
From 166.127.6.242 icmp_seq=2 Time to live exceeded
From 166.127.6.242 icmp_seq=3 Time to live exceeded
From 166.127.6.242 icmp_seq=4 Time to live exceeded
From 166.127.6.242 icmp_seq=5 Time to live exceeded
^C
--- 192.168.2.1 ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4006ms

[root@localhost ~]#

lxc.network.type= veth

In this mode a virtual ethernet pair device is created with one side assigned to the container and the other side attached to a bridge specified by the lxc.network.link option.

In this mode containers can communicate with other, host can communicate with containers and containers can communicate with external system.

Below our topology diagram host machine is connected to external Linux server via physical interface enp0s29f7u1. This interface linked to bridge lxcbr0. No IP needed on this interface enp0s29f7u1

linux_container_3

First we need to create a Linux bridge (lxcbr0), assign IP address to bridge (in the same subnet as containers) and link host machine physical interface (enp0s29f7u1) to it. I am using brctl command to perform these tasks

//create bridge lxcbr0
[root@localhost]# sudo brctl addbr lxcbr0
 
//assign IP address to bridge
[root@localhost ~]# ip addr add 192.168.2.100/24 dev lxcbr0

[root@localhost ~]# ifconfig lxcbr0
 lxcbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet 192.168.2.100  netmask 255.255.255.0  broadcast 0.0.0.0
         inet6 fe80::9c7e:c7ff:fe64:be71  prefixlen 64  scopeid 0x20
         ether fe:01:92:6f:88:21  txqueuelen 1000  (Ethernet)
         RX packets 1554  bytes 505536 (493.6 KiB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 19  bytes 3403 (3.3 KiB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
//link physical host interface to bridge
[root@localhost ~]# brctl addif lxcbr0 enp0s29f7u1

//bridge show interface associated wth it
[root@localhost ~]# brctl show
 bridge name     bridge id               STP enabled     interfaces
 lxcbr0          8000.0050b6194165       no              enp0s29f7u1
 virbr0          8000.000000000000       no

//no IP needed on physical interface
[root@localhost ~]# ifconfig enp0s29f7u1
enp0s29f7u1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:50:b6:19:41:65  txqueuelen 1000  (Ethernet)
        RX packets 9422  bytes 934818 (912.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12589  bytes 2384139 (2.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Create two conatiners named veth_cn1 & veth_cn2

#lxc-create -t ubuntu -n veth_cn1
#lxc-create -t ubuntu -n veth_cn2

Edit container configuration files. container configuration files are located in this path /var/lib/lxc//config. Note: Remember I am using Centos, path on your machine may differ if you are not using Centos

container veth_cn1 configuration file. Note: I have updated fields in bold italic

# Network configuration
 lxc.network.type = veth
 lxc.network.hwaddr = 00:16:3e:0a:3b:cc
 lxc.network.flags = up
 lxc.network.link = lxcbr0
 lxc.network.ipv4 = 192.168.2.1/24

Container veth_cn2 configuration file. Note: I have updated fields in bold italic

# Network configuration
 lxc.network.type = veth
 lxc.network.hwaddr = 00:16:3e:9b:66:cf
 lxc.network.flags = up
 lxc.network.link = lxcbr0
 lxc.network.ipv4 = 192.168.2.2/24

Start both containers

[root@localhost]# lxc-start -n veth_cn1 -d

[root@localhost]# lxc-start -n veth_cn2 -d

//as you see two new veth interfaces created and attached to bridge (lxcbr0). The
other end of these interfaces are connected to containers
[root@localhost]# sudo brctl show
bridge name     bridge id               STP enabled     interfaces
lxcbr0          8000.0050b6194165       no              enp0s29f7u1
                                                        veth2GN030
                                                        vethMFY0GP
virbr0          8000.000000000000       no

//check veth interfaces
[root@localhost ~]# ifconfig veth2GN030
veth2GN030: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fc01:92ff:fe6f:8821  prefixlen 64  scopeid 0x20
        ether fe:01:92:6f:88:21  txqueuelen 1000  (Ethernet)
        RX packets 882  bytes 295072 (288.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 937  bytes 307987 (300.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost ~]# ifconfig vethMFY0GP
vethMFY0GP: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fca6:3ff:fe2d:cbb5  prefixlen 64  scopeid 0x20
        ether fe:a6:03:2d:cb:b5  txqueuelen 1000  (Ethernet)
        RX packets 899  bytes 302950 (295.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 897  bytes 297597 (290.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost ~]# lxc-console -n veth_cn1

ubuntu@veth_cn1:~$ ifconfig
 eth0      Link encap:Ethernet  HWaddr 00:16:3e:0a:3b:cc
           inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
           inet6 addr: fe80::216:3eff:fe0a:3bcc/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:814 errors:0 dropped:0 overruns:0 frame:0
           TX packets:767 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:273205 (273.2 KB)  TX bytes:260226 (260.2 KB)
 
 lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Mask:255.0.0.0
           inet6 addr: ::1/128 Scope:Host
           UP LOOPBACK RUNNING  MTU:65536  Metric:1
           RX packets:16 errors:0 dropped:0 overruns:0 frame:0
           TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1
           RX bytes:1184 (1.1 KB)  TX bytes:1184 (1.1 KB)
 
//ping to bridge interface ip
ubuntu@veth_cn1:~$ ping 192.168.2.100
 PING 192.168.2.100 (192.168.2.100) 56(84) bytes of data.
 64 bytes from 192.168.2.100: icmp_req=1 ttl=64 time=0.098 ms
 64 bytes from 192.168.2.100: icmp_req=2 ttl=64 time=0.040 ms
 64 bytes from 192.168.2.100: icmp_req=3 ttl=64 time=0.043 ms
 64 bytes from 192.168.2.100: icmp_req=4 ttl=64 time=0.041 ms
 ^C
 --- 192.168.2.100 ping statistics ---
 4 packets transmitted, 4 received, 0% packet loss, time 3000ms
 rtt min/avg/max/mdev = 0.040/0.055/0.098/0.025 ms

//ping to container veth_cn2
ubuntu@veth_cn1:~$ ping 192.168.2.2
 PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
 64 bytes from 192.168.2.2: icmp_req=1 ttl=64 time=0.092 ms
 64 bytes from 192.168.2.2: icmp_req=2 ttl=64 time=0.045 ms
 ^C
 --- 192.168.2.2 ping statistics ---
 2 packets transmitted, 2 received, 0% packet loss, time 1000ms
 rtt min/avg/max/mdev = 0.045/0.068/0.092/0.024 ms

//ping to external system
ubuntu@veth_cn1:~$ ping 192.168.2.101
 PING 192.168.2.101 (192.168.2.101) 56(84) bytes of data.
 64 bytes from 192.168.2.101: icmp_req=1 ttl=64 time=0.565 ms
 64 bytes from 192.168.2.101: icmp_req=2 ttl=64 time=0.326 ms
 64 bytes from 192.168.2.101: icmp_req=3 ttl=64 time=0.378 ms
 ^C
 --- 192.168.2.101 ping statistics ---
 3 packets transmitted, 3 received, 0% packet loss, time 1999ms
 rtt min/avg/max/mdev = 0.326/0.423/0.565/0.102 ms
 ubuntu@veth_cn1:~$

Ping from host to containers

//ping to first container veth_cn1
[root@localhost ~]# ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=0.053 ms
64 bytes from 192.168.2.1: icmp_seq=2 ttl=64 time=0.036 ms
64 bytes from 192.168.2.1: icmp_seq=3 ttl=64 time=0.036 ms
^C
--- 192.168.2.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.036/0.041/0.053/0.010 ms


//ping to second container veth_cn2
[root@localhost ~]# ping 192.168.2.2
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_seq=1 ttl=64 time=0.082 ms
64 bytes from 192.168.2.2: icmp_seq=2 ttl=64 time=0.039 ms
64 bytes from 192.168.2.2: icmp_seq=3 ttl=64 time=0.037 ms
^C
--- 192.168.2.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.037/0.052/0.082/0.022 ms
[root@localhost ~]#

Ping from external system to container veth_cn1 and veth_cn2

//ping to first container veth_cn1
[root@localhost ~]# ping 192.168.2.1
 PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
 64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=0.417 ms
 64 bytes from 192.168.2.1: icmp_seq=2 ttl=64 time=0.327 ms
 ^C
 --- 192.168.2.1 ping statistics ---
 2 packets transmitted, 2 received, 0% packet loss, time 1000ms
 rtt min/avg/max/mdev = 0.327/0.372/0.417/0.045 ms
 
//ping to second container veth_cn2
[root@localhost ~]# ping 192.168.2.2
 PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
 64 bytes from 192.168.2.2: icmp_seq=1 ttl=64 time=0.614 ms
 64 bytes from 192.168.2.2: icmp_seq=2 ttl=64 time=0.393 ms
 64 bytes from 192.168.2.2: icmp_seq=3 ttl=64 time=0.341 ms
 ^C
 --- 192.168.2.2 ping statistics ---
 3 packets transmitted, 3 received, 0% packet loss, time 2000ms
 rtt min/avg/max/mdev = 0.341/0.449/0.614/0.119 ms

lxc.network.type= macvlan

macvlan supports these three modes

bridge
VEPA (Virtual Ethernet Port Aggregator
private

Let’s try these modes one by one

macvlan.mode = bridge

In macvlan bridge mode containers can communicate with each other, containers can communicate with external system and external system can communicate with containers. But containers do not have network access to host. In this mode you are creating logical network interfaces on host physical interface

Below topology diagram to test macvlan bridge mode. External system is a Linux server directly connected to host physical interface and configured in same subnet as containers.

linux_container_7

Create two containers

lxc-create -t ubuntu -n macvlan_cn1
lxc-create -t ubuntu -n macvlan_cn2

Edit configuration file for containers.

macvlan_br1 container file: /var/lib/lxc/macvlan_cn1/config. Note: I have updated fields in bold italic

# Network configuration
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.hwaddr = 00:16:3e:56:95:80
lxc.network.flags = up
lxc.network.link = enp0s29f7u1
lxc.network.ipv4 = 192.168.2.1/24

macvlan_br2 container file:/var/lib/lxc/macvlan_cn2/config. Note: I have updated fields in bold italics

# Network configuration
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.hwaddr = 00:16:3e:cb:f5:71
lxc.network.flags = up
lxc.network.link = enp0s29f7u1
lxc.network.ipv4 = 192.168.2.2/24

Start both containers

#lxc-start -n macvlan_cn1 -d
#lxc-start -n macvlan_cn2 -d

[root@localhost]# lxc-info -n macvlan_cn1
Name:           macvlan_cn1
State:          RUNNING
PID:            8140
IP:             192.168.2.1
CPU use:        0.20 seconds
BlkIO use:      0 bytes
Memory use:     3.95 MiB
KMem use:       0 bytes
Link:           enp0s29f7u1
 TX bytes:      1.11 KiB
 RX bytes:      98.12 KiB
 Total bytes:   99.23 KiB

[root@localhost]# lxc-info -n macvlan_cn2
Name:           macvlan_cn2
State:          RUNNING
PID:            8353
IP:             192.168.2.2
CPU use:        0.20 seconds
BlkIO use:      0 bytes
Memory use:     3.72 MiB
KMem use:       0 bytes
Link:           enp0s29f7u1
 TX bytes:      1.11 KiB
 RX bytes:      98.87 KiB
 Total bytes:   99.98 KiB

[root@localhost ~]# lxc-console -n macvlan_cn1

ubuntu@macvlan_cn1:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:56:95:80
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe56:9580/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:12 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4272 (4.2 KB)  TX bytes:5094 (5.0 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ubuntu@macvlan_cn1:~$ ip route
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1

//ping to container 2 passed
ubuntu@macvlan_cn1:~$ ping 192.168.2.2
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_req=1 ttl=64 time=0.086 ms
64 bytes from 192.168.2.2: icmp_req=2 ttl=64 time=0.038 ms
64 bytes from 192.168.2.2: icmp_req=3 ttl=64 time=0.038 ms
^C
--- 192.168.2.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.038/0.054/0.086/0.022 ms

ubuntu@macvlan_cn2:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:cb:f5:71
          inet addr:192.168.2.2  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fecb:f571/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:26 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7868 (7.8 KB)  TX bytes:8208 (8.2 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

//ping to first container ip passed
ubuntu@macvlan_cn2:~$ ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_req=1 ttl=64 time=0.055 ms
64 bytes from 192.168.2.1: icmp_req=2 ttl=64 time=0.038 ms
64 bytes from 192.168.2.1: icmp_req=3 ttl=64 time=0.038 ms
64 bytes from 192.168.2.1: icmp_req=4 ttl=64 time=0.037 ms
^C
--- 192.168.2.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.037/0.042/0.055/0.007 ms

ubuntu@macvlan_cn2:~$ ip route
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.2

Ping test from host to containers

//ping from host to container 1 failed
[root@localhost ~]# ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
^C
--- 192.168.2.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms

//ping from host to container 2 failed
[root@localhost ~]# ping 192.168.2.2
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
^C
--- 192.168.2.2 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms

Ping to external machine. Login to a container and perform ping test to external system IP address

//ping from second container to external system pass
ubuntu@macvlan_br2:~$ ping 192.168.2.101
PING 192.168.2.101 (192.168.2.101) 56(84) bytes of data.
64 bytes from 192.168.2.101: icmp_req=1 ttl=64 time=0.460 ms
64 bytes from 192.168.2.101: icmp_req=2 ttl=64 time=0.433 ms
64 bytes from 192.168.2.101: icmp_req=3 ttl=64 time=0.316 ms
^C
--- 192.168.2.101 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.316/0.403/0.460/0.062 ms
ubuntu@macvlan_br2:~$

//ping from first container to external system
ubuntu@macvlan_br1:~$ ping 192.168.2.101
PING 192.168.2.101 (192.168.2.101) 56(84) bytes of data.
64 bytes from 192.168.2.101: icmp_req=1 ttl=64 time=0.774 ms
64 bytes from 192.168.2.101: icmp_req=2 ttl=64 time=0.329 ms
64 bytes from 192.168.2.101: icmp_req=3 ttl=64 time=0.218 ms
^C
--- 192.168.2.101 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.218/0.440/0.774/0.240 ms
ubuntu@macvlan_br1:~$

tcpdump on external machine

[root@]# tcpdump -i enp0s29f7u1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s29f7u1, link-type EN10MB (Ethernet), capture size 65535 bytes
23:50:13.477433 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:16:3e:56:95:80 (oui Unknown), length 300
23:50:14.071784 IP 192.168.2.1 > 192.168.2.101: ICMP echo request, id 306, seq 9, length 64
23:50:14.071806 IP 192.168.2.101 > 192.168.2.1: ICMP echo reply, id 306, seq 9, length 64
23:50:15.071807 IP 192.168.2.1 > 192.168.2.101: ICMP echo request, id 306, seq 10, length 64
23:50:15.071828 IP 192.168.2.101 > 192.168.2.1: ICMP echo reply, id 306, seq 10, length 64

macvlan.mode = vepa (virtual ethernet port aggregation)

In this mode containers have access to only external network. No access to host and other container network.

linux_container_8

Edit configuration file for our containers (macvlan_cn1 & macvlan_cn2)

macvlan_cn1 container file: /var/lib/lxc/macvlan_cn1/config

# Network configuration
lxc.network.type = macvlan
lxc.network.macvlan.mode = vepa
lxc.network.hwaddr = 00:16:3e:56:95:80
lxc.network.flags = up
lxc.network.link = enp0s29f7u1
lxc.network.ipv4 = 192.168.2.1/24

macvlan_cn2 container file:/var/lib/lxc/macvlan_cn2/config

# Network configuration
lxc.network.type = macvlan
lxc.network.macvlan.mode = vepa
lxc.network.hwaddr = 00:16:3e:cb:f5:71
lxc.network.flags = up
lxc.network.link = enp0s29f7u1
lxc.network.ipv4 = 192.168.2.2/24

Start both containers

#lxc-start -n macvlan_cn1 -d
#lxc-start -n macvlan_cn2 -d

[root@localhost ~]# lxc-info -n macvlan_cn1
Name:           macvlan_cn1
State:          RUNNING
PID:            19204
IP:             192.168.2.1
CPU use:        0.21 seconds
BlkIO use:      0 bytes
Memory use:     5.98 MiB
KMem use:       0 bytes
Link:           enp0s29f7u1
 TX bytes:      891.69 KiB
 RX bytes:      2.00 MiB
 Total bytes:   2.87 MiB
[root@localhost ~]#

[root@localhost ~]# lxc-info -n macvlan_cn2
Name:           macvlan_cn2
State:          RUNNING
PID:            19382
IP:             192.168.2.2
CPU use:        0.20 seconds
BlkIO use:      0 bytes
Memory use:     3.71 MiB
KMem use:       0 bytes
Link:           enp0s29f7u1
 TX bytes:      891.69 KiB
 RX bytes:      2.00 MiB
 Total bytes:   2.87 MiB
[root@localhost ~]#

As you see ping test between container failed

[root@localhost ~]# lxc-console -n macvlan_cn1

ubuntu@macvlan_cn1:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:56:95:80
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe56:9580/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:5094 (5.0 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ubuntu@macvlan_cn1:~$ ip route
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1

//ping from container 1 to container 2 failed
ubuntu@macvlan_cn1:~$ ping 192.168.2.2
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
From 192.168.2.1 icmp_seq=1 Destination Host Unreachable
From 192.168.2.1 icmp_seq=2 Destination Host Unreachable
From 192.168.2.1 icmp_seq=3 Destination Host Unreachable
From 192.168.2.1 icmp_seq=4 Destination Host Unreachable
^C
--- 192.168.2.2 ping statistics ---
5 packets transmitted, 0 received, +4 errors, 100% packet loss, time 4001ms
pipe 4

Perform ping test from container macvlan_cn2 to macvlan_cn1. As you can see ping test failed

ubuntu@macvlan_cn2:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:cb:f5:71
          inet addr:192.168.2.2  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fecb:f571/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:8856 (8.8 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:1184 (1.1 KB)  TX bytes:1184 (1.1 KB)

ubuntu@macvlan_cn2:~$ ip route
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.2

//ping test from container 2 to container 1 failed
ubuntu@macvlan_cn2:~$ ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
From 192.168.2.2 icmp_seq=1 Destination Host Unreachable
From 192.168.2.2 icmp_seq=2 Destination Host Unreachable
From 192.168.2.2 icmp_seq=3 Destination Host Unreachable
From 192.168.2.2 icmp_seq=4 Destination Host Unreachable
^C
--- 192.168.2.1 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3000ms
pipe 4

Ping from container to external system.

As you can see ping test pass

[root@localhost ~]# lxc-console -n macvlan_cn2

Connected to tty 1
Type <Ctrl+a q> to exit the console, <Ctrl+a Ctrl+a> to enter Ctrl+a itself

//ping to external system passed
ubuntu@macvlan_cn2:~$ ping 192.168.2.101
PING 192.168.2.101 (192.168.2.101) 56(84) bytes of data.
64 bytes from 192.168.2.101: icmp_req=1 ttl=64 time=0.373 ms
64 bytes from 192.168.2.101: icmp_req=2 ttl=64 time=0.271 ms
64 bytes from 192.168.2.101: icmp_req=3 ttl=64 time=0.384 ms
64 bytes from 192.168.2.101: icmp_req=4 ttl=64 time=0.265 ms
64 bytes from 192.168.2.101: icmp_req=5 ttl=64 time=0.400 ms
64 bytes from 192.168.2.101: icmp_req=6 ttl=64 time=0.398 ms
64 bytes from 192.168.2.101: icmp_req=7 ttl=64 time=0.347 ms
64 bytes from 192.168.2.101: icmp_req=8 ttl=64 time=0.381 ms
^C
--- 192.168.2.101 ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 6999ms
rtt min/avg/max/mdev = 0.265/0.352/0.400/0.053 ms

Ping from container to host. As you can see ping failed

[root@localhost macvlan_cn1]# ifconfig enp0s29f7u1
enp0s29f7u1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.100  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:50:b6:19:41:65  txqueuelen 1000  (Ethernet)
        RX packets 9392  bytes 932640 (910.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11896  bytes 2158529 (2.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

//ping from container to host failed
ubuntu@macvlan_cn2:~$ ping 192.168.2.100
PING 192.168.2.100 (192.168.2.100) 56(84) bytes of data.
From 192.168.2.2 icmp_seq=1 Destination Host Unreachable
From 192.168.2.2 icmp_seq=2 Destination Host Unreachable
From 192.168.2.2 icmp_seq=3 Destination Host Unreachable
From 192.168.2.2 icmp_seq=4 Destination Host Unreachable
^C
--- 192.168.2.100 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3000ms
pipe 4

Save

Lab-36: Linux containers (lxc)

by sunnynetworkJanuary 28, 2017January 31, 2017

Last three labs Lab-33,Lab-34 & Lab-35 talk about Docker container. As said Docker container in the background uses Linux container. So what is Linux containers?. In this lab I will give a brief introduction of Linux container. How to spin Linux container etc

More reading on Linux containers here

Prerequisite:

I am using Linux Centos 7.2 for this lab

My machine specification

//OS release
[root@localhost ~]# cat /etc/*release
CentOS Linux release 7.3.1611 (Core)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"

REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.3.1611 (Core)
CentOS Linux release 7.3.1611 (Core)
[root@localhost ~]#

//Check kernel version
[root@localhost ~]# uname -r
3.10.0-514.2.2.el7.x86_64
[root@localhost ~]#

Install Linux container (lxc) and dependencies

$yum update -y
$yum install epel-release

//Install dependencies
$yum install debootstrap perl libvirt bridge-utils
$yum install lxc lxc-templates

//start service
$systemctl start lxc.service
$systemctl start libvirtd

Check Linux container service status

[root@localhost ~]# systemctl status lxc.service
● lxc.service - LXC Container Initialization and Autoboot Code
   Loaded: loaded (/usr/lib/systemd/system/lxc.service; disabled; vendor preset: disabled)
   Active: active (exited) since Fri 2017-01-27 18:05:26 EST; 16h ago
  Process: 4452 ExecStart=/usr/libexec/lxc/lxc-autostart-helper start (code=exited, status=0/SUCCESS)
  Process: 4444 ExecStartPre=/usr/libexec/lxc/lxc-devsetup (code=exited, status=0/SUCCESS)
 Main PID: 4452 (code=exited, status=0/SUCCESS)

Jan 27 18:05:26 localhost.localdomain systemd[1]: Starting LXC Container Initialization and Autoboot Code...
Jan 27 18:05:26 localhost.localdomain lxc-devsetup[4444]: Creating /dev/.lxc
Jan 27 18:05:26 localhost.localdomain lxc-devsetup[4444]: /dev is devtmpfs
Jan 27 18:05:26 localhost.localdomain lxc-devsetup[4444]: Creating /dev/.lxc/user
Jan 27 18:05:26 localhost.localdomain lxc-autostart-helper[4452]: Starting LXC autoboot containers:  [  OK  ]
Jan 27 18:05:26 localhost.localdomain systemd[1]: Started LXC Container Initialization and Autoboot Code.

Check lxc configuration. Important things to note here are:

Namespaces are enabled
Veth pair device: enabled. We will sing default veth network type
Control groups are enabled

//Check lxc configuration
[root@localhost ~]# lxc-checkconfig
Kernel configuration not found at /proc/config.gz; searching...
Kernel configuration found at /boot/config-3.10.0-514.2.2.el7.x86_64
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled

--- Control groups ---
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

--- Misc ---
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
Bridges: enabled
Advanced netfilter: enabled
CONFIG_NF_NAT_IPV4: enabled
CONFIG_NF_NAT_IPV6: enabled
CONFIG_IP_NF_TARGET_MASQUERADE: enabled
CONFIG_IP6_NF_TARGET_MASQUERADE: enabled
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled

--- Checkpoint/Restore ---
checkpoint restore: enabled
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: enabled
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: enabled
CONFIG_NETLINK_DIAG: enabled
File capabilities: enabled

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /bin/lxc-checkconfig

[root@localhost ~]#

Procedure:

Create container. Let’s create a container with Ubuntu system

lxc-create -t ubuntu -n ubuntu_container

-n: Name of the container

-t:Container template

lxc comes with ready made templates for various systems. You can check templates on your system here

[root@localhost]# ls /usr/share/lxc/templates
lxc-alpine    lxc-archlinux  lxc-centos  lxc-debian    lxc-fedora  lxc-openmandriva  lxc-oracle  lxc-sshd    lxc-ubuntu-cloud
lxc-altlinux  lxc-busybox    lxc-cirros  lxc-download  lxc-gentoo  lxc-opensuse      lxc-plamo   lxc-ubuntu

//create container using ubuntu system. Container user/password ubuntu/ubuntu
[root@localhost ~]# lxc-create -t ubuntu -n ubuntu_container

<truncated>
##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##

Check container status using lxc-status

[root@localhost ~]# lxc-info --name ubuntu_container
Name:           ubuntu_container
State:          STOPPED
[root@localhost ~]#

Start the container.

lxc-start -n <container name> -d

-d: for running container in background

Make sure Linux bridge virbr0 created if not create the bridge because by default Linux container uses Linux bridge virbr0. Check network setting for the container we created in file /var/lib/lxc/ubuntu_container/conf. This is what my container network setting looks like

lxc.network.type = veth
lxc.network.link = virbr0
lxc.network.flags = up

lxc.network.type: lxc supports these network types:

veth – virtual ethernet network interface created inside container and other side of this interface attached to bridge specified in lxc.network.link
empty – In this case only loopback interface created inside container
vlan – a vlan interface is linked with the interface specified bythe lxc.network.link and assigned to the container. The vlan identifier is specified with the option lxc.network.vlan.id
phy – assign a physical interface to container
macvlan – mac address based vlan

In this lab I am using default network type veth.

Note:If you like to change the default network behavior you can do that by changing default file /etc/lxc/default.conf

[root@localhost lxc]# cat /etc/lxc/default.conf
lxc.network.type = veth
lxc.network.link = virbr0
lxc.network.flags = up

[root@localhost lxc]# ifconfig virbr0
virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.10  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::a8e8:3fff:fe5d:c8f0  prefixlen 64  scopeid 0x20<link>
        ether fe:e7:4b:69:2c:0f  txqueuelen 1000  (Ethernet)
        RX packets 2376  bytes 775152 (756.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 56  bytes 7486 (7.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

//start lxc container -d mean run in the background
[root@localhost ~]# lxc-start -n ubuntu_container -d
[root@localhost ~]# lxc-info -n ubuntu_container
Name:           ubuntu_container
State:          RUNNING
PID:            5317
CPU use:        0.21 seconds
BlkIO use:      0 bytes
Memory use:     3.75 MiB
KMem use:       0 bytes
Link:           vethXMQ8QW
 TX bytes:      1.57 KiB
 RX bytes:      578 bytes
 Total bytes:   2.13 KiB
[root@localhost ~]#

//container network interface vethXMQ8QW attached to Linux bridge virbr0 
[root@localhost ~]# sudo brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.fee74b692c0f       no              vethXMQ8QW

[root@localhost ~]# ifconfig vethXMQ8QW
vethXMQ8QW: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fce7:4bff:fe69:2c0f  prefixlen 64  scopeid 0x20<link>
        ether fe:e7:4b:69:2c:0f  txqueuelen 1000  (Ethernet)
        RX packets 16  bytes 3384 (3.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@localhost lxc]# ps -ef | grep ubuntu_container
root      5307     1  0 Jan27 ?        00:00:00 lxc-start -n ubuntu_container -d
root      7859  7011  0 10:40 pts/6    00:00:00 grep --color=auto ubuntu_container
[root@localhost lxc]#

lxc creates configuration for each container. Let’s check configuration file of our container (ubuntu_container)

[root@localhost]# cat /var/lib/lxc/ubuntu_container/config
# Template used to create this container: /usr/share/lxc/templates/lxc-ubuntu
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)

# Common configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf

# Container specific configuration
lxc.rootfs = /var/lib/lxc/ubuntu_container/rootfs
lxc.mount = /var/lib/lxc/ubuntu_container/fstab
lxc.utsname = ubuntu_container
lxc.arch = amd64

# Network configuration
lxc.network.type = veth
lxc.network.hwaddr = 00:16:3e:9b:3c:8d
lxc.network.flags = up
lxc.network.link = virbr0
[root@localhost ubuntu_container]#

You can attach to your container and run commands. This is a quick way to check your container

lxc-attach -n <container name> [<command>]

[root@localhost]# lxc-attach -n ubuntu_container /sbin/ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
16: eth0@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 00:16:3e:9b:3c:8d brd ff:ff:ff:ff:ff:f
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe9b:3c8d/64 scope link
       valid_lft forever preferred_lft forever
[root@localhost ubuntu_container]#

Connect to container using lxc-console command, user/password ubuntu/ubuntu. To terminate the session try ctrl+a, q

lxc-console -n ubuntu_container

[root@localhost ~]# lxc-console -n ubuntu_container

Connected to tty 1
Type <Ctrl+a q> to exit the console, <Ctrl+a Ctrl+a> to enter Ctrl+a itself


ubuntu@ubuntu_container:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:9b:3c:8d
          inet6 addr: fe80::216:3eff:fe9b:3c8d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9492 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:648 (648.0 B)  TX bytes:3242376 (3.2 MB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:25 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:2080 (2.0 KB)  TX bytes:2080 (2.0 KB)

Freeze and unfreeze to stop and start process inside container

[root@localhost ~]# lxc-info -n ubuntu_container
Name:           ubuntu_container
State:          RUNNING
PID:            5317
CPU use:        21.22 seconds
BlkIO use:      171.33 MiB
Memory use:     91.39 MiB
KMem use:       0 bytes
Link:           vethXMQ8QW
 TX bytes:      3.12 MiB
 RX bytes:      648 bytes
 Total bytes:   3.12 MiB

[root@localhost ~]# lxc-freeze -n ubuntu_container
[root@localhost ~]# lxc-info -n ubuntu_container
Name:           ubuntu_container
State:          FROZEN
PID:            5317
CPU use:        21.22 seconds
BlkIO use:      171.33 MiB
Memory use:     91.39 MiB
KMem use:       0 bytes
Link:           vethXMQ8QW
 TX bytes:      3.12 MiB
 RX bytes:      648 bytes
 Total bytes:   3.12 MiB

[root@localhost ~]# lxc-unfreeze -n ubuntu_container
[root@localhost ~]# lxc-info -n ubuntu_container
Name:           ubuntu_container
State:          RUNNING
PID:            5317
CPU use:        21.22 seconds
BlkIO use:      171.33 MiB
Memory use:     91.39 MiB
KMem use:       0 bytes
Link:           vethXMQ8QW
 TX bytes:      3.12 MiB
 RX bytes:      648 bytes
 Total bytes:   3.12 MiB

Delete container using lxc-destroy. You must stop running container before you issue delete command

lxc-stop -n <container name>
lxc-destroy -n <container name>

[root@localhost ~]# lxc-info -n ubuntu_container
Name:           ubuntu_container
State:          RUNNING
PID:            5317
CPU use:        21.24 seconds
BlkIO use:      171.33 MiB
Memory use:     91.39 MiB
KMem use:       0 bytes
Link:           vethXMQ8QW
 TX bytes:      3.12 MiB
 RX bytes:      648 bytes
 Total bytes:   3.12 MiB

[root@localhost ~]# lxc-stop -n ubuntu_container
[root@localhost ~]# lxc-info -n ubuntu_container
Name:           ubuntu_container
State:          STOPPED

[root@localhost ~]# lxc-destroy -n ubuntu_container
[root@localhost ~]# lxc-info -n ubuntu_container
ubuntu_container doesn't exist
[root@localhost ~]#

//container interface removed from bridge
[root@localhost ~]# sudo brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       no
[root@localhost ~]#

	Arco on Lab-1:create vlan based Openfl…
	Lab-40: Container or… on Lab-41: Create Multi-node infr…
	Kannan on Lab-6:Render CLI from Con…
	Lab-4:Configuring OV… on Lab-2: Openday light (ODL) con…
	Lab-13:Using and Cre… on Lab-8:Python for network autom…

Welcome to cloudNet

cloud networking, IoT,Openstack,SDN, NFV,IoT,OVS,Netconf, Yang

linux containers (lxc)

Lab-38: Linux bridge with Linux containers (lxc)

Prerequisite:

Procedure:

Lab-37: Linux container (lxc) networking

lxc.network.type= empty

lxc.network.type= phys

lxc.network.type = vlan

lxc.network.type= veth

lxc.network.type= macvlan

macvlan.mode = bridge

macvlan.mode = vepa (virtual ethernet port aggregation)

Lab-36: Linux containers (lxc)

Prerequisite:

Procedure: