Index

How to use jails with epair with FreeBSD 9.0

January 2012.

Introduction

System used:


FreeBSD testjails 9.0-RELEASE FreeBSD 9.0-RELEASE #1: Thu Jan 19 10:47:03 CET 2012     root@testjails:/usr/obj/usr/src/sys/JAILKERNEL  i386

Build and install a kernel including VIMAGE, epair and briged interfaces

Configure the kernel: (/usr/src/sys/i386/conf/JAILKERNEL)


include GENERIC

cpu             I686_CPU
ident           JAILKERNEL

# Virtual networking for jail
options         VIMAGE
device          epair
device          if_bridge

# The nullFS to mount local directory
options         NULLFS

Make and install the kernel:


cd /usr/src
make buildkernel KERNCONF=JAILKERNEL
make installkernel KERNCONF=JAILKERNEL
reboot

Create the jail's environement


mkdir /jails/somejail/
cd /usr/src
make buildworld
make installworld DESTDIR=/jails/somejail/
make distribution DESTDIR=/jails/somejail/
echo 'hostname="testjail.example.com"' >> /jails/somejail/etc/rc.conf
echo 'nameserver 8.8.8.8"' >> /jails/somejail/etc/resolv.conf
mkdir /jails/somejail/usr/ports

Start and use the jail manually

Start the jail:


# jail -c vnet name=testjail host.hostname=testjail path=/jails/somejail persist
# jls
   JID  IP Address      Hostname                      Path
     1  -               testjail                      /jails/somejail

Create the virtual ethernet cable:


# ifconfig epair0 create
epair0a
# ifconfig epair0b vnet testjail

Create the bridge:


# ifconfig bridge0 create
# ifconfig bridge0 addm em0 addm epair0a
# ifconfig bridge0 up

Configure the network on both sides:


# ifconfig em0 192.168.1.1/24
# ifconfig
em0: flags=8943 metric 0 mtu 1500
        options=98
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=29
        media: Ethernet autoselect (1000baseT )
        status: active
epair0a: flags=8943 metric 0 mtu 1500
        options=8
        nd6 options=29
        media: Ethernet 10Gbase-T (10Gbase-T )
        status: active
bridge0: flags=8843 metric 0 mtu 1500
        nd6 options=29
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: epair0a flags=143
                ifmaxaddr 0 port 5 priority 128 path cost 14183
        member: em0 flags=143
                ifmaxaddr 0 port 1 priority 128 path cost 20000
# jexec testjail ifconfig epair0b 192.168.1.2/24
# jexec testjail ifconfig
lo0: flags=8008 metric 0 mtu 16384
        options=3
        nd6 options=21
epair0b: flags=8843 metric 0 mtu 1500
        options=8
        inet6 fe80::c0:a4ff:fe00:60b%epair0b prefixlen 64 scopeid 0x2
        inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=21
        media: Ethernet 10Gbase-T (10Gbase-T )
        status: active

Test the connection:


# jexec testjail ping -c 1 192.168.1.1
PING 192.168.1.1 (192.168.3.223): 56 data bytes
64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.795 ms

--- 192.168.1.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.795/0.795/0.795/0.000 ms

# jexec testjail route add default 192.168.1.250
add net default: gateway 192.168.1.250

# jexec testjail ping -c 1 example.net
PING example.net (192.0.43.10): 56 data bytes
64 bytes from 192.0.43.10: icmp_seq=0 ttl=243 time=92.568 ms

--- example.net ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 92.568/92.568/92.568/0.000 ms

Stop the jail:


# jail -r testjail
enceinte# jls
   JID  IP Address      Hostname                      Path

Clean the network:


# ifconfig epair0a destroy
# ifconfig bridge0 destroy

Start the jail automatically

As of 2011/08/01, /etc/rc.d/jail has problems using the new jail command (v2), so you must patch it if you want to use vnets:
http://www.freebsd.org/cgi/query-pr.cgi?pr=142972
Also, you need this patch to be able to put the epair interface in the jail's vnet after it is created, but before starting the jail's rc.script (via earlypoststart entries in rc.conf).

See also:
http://lists.freebsd.org/pipermail/freebsd-stable/2012-January/065556.html

My original /etc/rc.d/jail script was revision 1.47.

Here is the modified patch that worked for me:
fbsd9_rcdjail147_vnet_patch

Then, in /etc/rc.conf:


#
# Jails networking
#
cloned_interfaces="bridge0"
ifconfig_bridge0="addm em0"

#
# Jails configuration
#
jail_enable="YES"
jail_v2_enable="YES"
jail_list="testjail"

jail_testjail_name="testjail"
jail_testjail_hostname="testjail.example.net"
jail_testjail_devfs_enable="YES"
jail_testjail_rootdir="/jails/somejail"
jail_testjail_vnet_enable="YES"
jail_testjail_exec_prestart0="ifconfig epair0 create"
jail_testjail_exec_prestart1="ifconfig bridge0 addm epair0a"
jail_testjail_exec_prestart2="ifconfig epair0a up"
jail_testjail_exec_earlypoststart0="ifconfig epair0b vnet testjail"
jail_testjail_exec_afterstart0="ifconfig lo0 127.0.0.1"
jail_testjail_exec_afterstart1="ifconfig epair0b 192.168.1.2/24 up"
jail_testjail_exec_afterstart2="route add default 192.168.1.250"
jail_testjail_exec_afterstart3="/bin/sh /etc/rc"
jail_testjail_exec_poststop0="ifconfig bridge0 deletem epair0a"
jail_testjail_exec_poststop1="ifconfig epair0a destroy"

Then: