Extra WAN interfaces / Virtual circuits

The external port of the unit may be set up to act as more than one full WAN IP interface. One may think of the WAN interface as being split into several “channels”. There are various reasons for using more than one WAN: different types of IP traffic may be sent/received on each channel and treated with different priority, or the service provider may want to use private subnets for certain traffic. Typically, there is one “ordinary” internet channel, behaving much the same as if it was the single WAN, but on top of that one might want to have one or several WAN channels that are particularly “good”, prioritized or “private”. One example is the “Triple Play”, where data (“surf”, mail etc.), TV and telephony use separate WAN channels.

The multiple-channel feature, here called Extra WAN Interfaces, could be set up in the web interface by the page Extra WAN Interfaces, a link to this is found on the Advanced network page. Setting up extra WAN interfaces is for more advanced usage, in practise only to be used in cooperation and guidance by your Internet provider.

The way multiple WAN interfaces work is depending on the underlying technology for the WAN port, being ADSL/ATM or Ethernet.

Read more: WAN, Extra WAN Interfaces configuration page

If “LINE” is selected as WAN interface, the unit uses ADSL and ATM technology, normally on one single PVC (Permanent Virtual Circuit). A single PVC corresponds to one set of VPI/VCI values which determines its routing in the ATM world. In more advanced cases, one may use more than one PVC, to send different IP traffic on each of them, virtually concurrently.

For each PVC, an IP interface is set up on the web page Extra WAN interfaces/ Virtual circuits, each IP interface with the possibility to use a manual IP address or an IP address that is acquired by DHCP or PPPoE/PPPoA. The first “channel” (or rather “interface”) is always the ordinary Internet interface, called “LINE”, with its VPI/VCI values entered on the Network Configuration page. As seen on the Extra WAN interfaces/ Virtual circuits web page, the LINE interface cannot be removed, but other interfaces, called “ipwan2”, “ipwan3” etc. can be added (or removed), by using their respective In use checkbox.

The VPI and VCI (VPI=Virtual Path Identifier, VCI=Virtual Channel Identifier) values must be set to unique values, or rather, there must not be two interfaces (rows) with the same combination of VPI and VCI. The VPI and VCI values together determine their channel identification in the ATM processing, in the central site modem (DSLAM) and further up the network. As with almost all settings concerning Extra WAN interfaces, these values must be set by specification from the Internet service provider.

ATM has the ability to specify resources on a per-PVC basis, thus using the bandwidth in an optimal way for different types of traffic, for example ensuring a minimal delay for certain critical traffic. The fields for setting these service parameters on the Extra WAN interfaces/ Virtual circuits page are ATM QoS class, PCR, SCR, BT and MBS. The ATM QoS class is by default set to UBR (Unspecified Bit Rate), implying that the traffic on this channel takes what bandwidth is available. One may think of UBR as the “best effort”. Other classes can be used to give traffic more priority. In particular, VBR-real time (VBR=Variable Bit Rate) is used for traffic sensitive to delays, for example voice streams. Further ATM settings that may be specified are:

• PCR – Peak Cell Rate, in number of cells per second (one cell is 48 octets)
• SCR – Sustainable Cell Rate, in cells/s
• BT – Burst Tolerance, in number of cell times (one cell time is the time to transmit 48 octets)
• MBS – Maximum Burst Size, in number of cells.

Only one of BT or MBS should be set, they are different ways of specifying the maximum burst that can be sent at a peak rate (before other channels are allowed to send). The SCR and BT/MBS values need only to be set for VBR types of ATM QoS class.

It should be remembered that these ATM QoS settings (as the QoS routing described later) essentially describes the upstream behaviour. The incoming downstream flow of traffic is simply processed as the packets arrive, without any real possibility to do any prioritization at all.

Read more: ATM, PVC, VPI, VCI, ATM traffic types

On the Network Configuration web page, one may select to use “ET0” (on older 4-port models “ET4”) as the WAN interface instead of “LINE”. After clicking “Change” and “Apply” one is asked to restart the unit, thus setting it up for using an Ethernet port, the ET0, as the normal Internet interface. The ADSL interface will no longer be active. (Some models may not have an ADSL interface at all, and will consequently already be set up for using ET0 as WAN). In the Ethernet case, there are some support for using more than one WAN interface, utilizing two different technologies:

1. VLAN-tagged ethernet WAN packets
2. Untagged WAN traffic, using different network subnets on the WAN port

VLAN-tagged traffic is not supported on the inside (LAN) interfaces. (More on VLAN-taggging below.)

Like the ADSL case, there is a Extra WAN interfaces configuration page, a link to this is found on the Advanced network page. The first time this page is visited, one must enable the multiple-WAN feature by selecting “enabled” in the Extra WAN Interfaces drop-down box. Clicking “Apply” will then enforce a new restart, and after that the web page will contain the full set of configurations, in Ethernet mode.

Many of the column headings are the same as for the ADSL case. In the Ethernet case, there is obviously no need for any ATM values. Instead, there is two columns to the far right of the page named VLAN Id and PCP. (older versions of Internet Gate may not support the VLAN tagging, having no such column. One can also check the Help: Device Information from the top menu, under the “Hardware” box the switch chip type should be IP175D to support VLANs.)

The VLAN (Virtual LAN, Virtual Local Area Network) tagging according to specification IEEE Std 802.1Q is a way to let logical groups of stations communicate as if they were on the same LAN. One such group have a certain value, a VLAN Id in a tag inserted into the ethernet packet, thus enabling switches and routers differentiating between groups and route the packets as they would have been, were they on physically separated network segments. When using VLAN tags for the WAN connection, the idea is to separate traffic for some reason, for example using different IP subnets and possibly different priority or routing behaviour in the service provider’s network.

By filling in a value in the VLAN Id field one enables VLAN tagging. Packets sent out on that interface (e.g. “ipwan2”) will have a VLAN tag inserted. Packets coming in will have their VLAN tags inspected, the packet appearing as received on the WAN interface corresponding to the VLAN Id of the tag (e.g. “ipwan2”), irrespective of IP or MAC destination address. A mixture of tagged and non-tagged traffic on the WAN side is supported in only one special case: The use of a single non-VLAN-tagged channel along with one ore more tagged VLANs. The single non-VLAN-tagged interface should then be set to the (faked) VLAN Id “4096”.

One may also, for each VLAN used, fill in a PCP value. This is the three bit Priority Code Point value, sometimes called CoS (Class of Service), according to IEEE 802.1p standard. It is used for marking outgoing traffic with a priority value (0-7) that may be treated specially in upstream switches and routers. If the PCP field is left empty (which is common) the priority value defaults to 0.

Read more: VLAN, IEEE 802.1Q standard, IEEE 802.1p, PCP, CoS

VLAN tags fulfil the same purpose as the PVC:s in the ADSL case: to get a full separation between IP traffic at a lower lever, as if the IP packets were operating on physically separated network segments. However, one can also use multiple WAN ethernet interfaces without VLAN tags, by simply not filling in any of the VLAN Id fields. (As mentioned above, some products may not even support the VLAN tagging, having no VLAN Id fields at all). In essence, this will be like tying several IP interfaces together on one single ethernet port. How could there be any channel separation in this case?

The solution is a built-in feature of the ethernet driver in the unit. Each IP interface is internally related to one virtual ethernet port, with its own MAC address. Thus, as long as incoming packets have unicast MAC destination address they know to which WAN IP interface (“ET0”, “ipwan2” etc.) they will go, even without any VLAN tag. For multicast and broadcast packets this is not true, so there are special treatments of those. Broadcast (incoming) packets are copied onto all WAN interfaces, in fact as if the interfaces were connected together in a bridge (switch). An incoming ARP request, for example, will be copied to all WAN IP interfaces, but only the interface with the requested IP address will answer with its underlying MAC address. If the WAN IP interfaces use DHCP, the DHCPDISCOVER/DHCPREQUEST packets sent out will request the server to use unicast (the broadcast bit in the DHCP message flags cleared), so consequently the DHCP server must support unicast (as broadcast DHCP messages from the server would be impossible to route to the correct IP interface). Multicast (incoming) packets are treated specially, they are sent to the interface having its IGMP Proxy checkbox checked (more on IGMP below).

Read more: ARP, DHCP, MAC address

All the multiple IP interfaces described above are essentially ‘outside’. This means that they will have ‘outside’ rules on them, same as the normal WAN (Internet) interface. The Firewall Rules status page displays the rules that are applied, one grey box of firewall rules for each of “ET0” (or “LINE”), “ipwan2” etc. Since these interfaces are ‘outside’, the NAT (Network Address Translation) is applied throughout.

When incoming packets on one of the multiple WAN interfaces have passed the firewall and having their destination IP addresses NAT:ed they are subjected to routing to any of the local (‘inside’) ports. At this stage, there is no longer any information in the packet by which WAN interface it came in (such as VLAN tag). One should not try to build a one-to-one relationship between a particular WAN “channel” and a particular local port, a packet coming in on any WAN can go out on any LAN port, all controlled by the IP address/routing only. For the upstream direction there is similar NAT and firewall treatment, here the QoS routing (described below) or ordinary destination address routing are the mechanisms that make a packet select its WAN “channel”.

Read more: NAT

The DHCP or PPP clients will work independently on the multiple WAN IP interfaces, in the same way as in the single WAN case. If the DHCP/PPP client succeeds in acquiring an IP address, the addresses are filled in on the respective field on the Extra WAN Interfaces page (for the “ET0” or “LINE” also on the Network Configuration page), and the successful event is reported in the system log. Empty fields, or “0.0.0.0” means that no IP address has yet been assigned on that interface. For DHCP there is a Renew and a Release button on the Network Configuration page. These buttons are common to all WAN interfaces. For example, pressing Release will produce a DHCPRELEASE packet sent out simultaneously on all WAN IP interfaces (that have their Access type set to “DHCP”). A Renew may not succeed to renew all interfaces’ IP address and will report if partly failed. A failure is not “for ever”, the DHCP client keeps constantly trying on all the WAN IP interfaces that haven’t yet got their IP address leases.

Read more: DHCP, PPP, PPPoE, PPPoA

QoS routing relates to the use of extra WAN interfaces. It could be regarded as a way to force certain types of traffic into particular WAN “channels” (=IP interfaces). It is only applicable to upstream traffic, the downstream being sorted by other means (VPI/VCI values, VLAN tags or MAC address). The router in the Internet Gateway does the QoS routing job, apart from using its ‘normal’ routing strategies (destination addresses etc.).

For this purpose, each packet coming in on the LAN or produced by the box itself, is marked by QoS ‘tag’ value (not to be confused with a VLAN tag value). Those tag values are the numbers that are visible for each choice in a QoS routing dropdown box on the web page (the Extra WAN Interfaces page). The values may seem unlogical at a first glance, this has to do with their internal use in internal queueing mechanisms and should not in themselves be bothered about. The important thing is to regard each value as a representation of a certain type of traffic. In particular, the values 4,7, 261,262 are values set for SIP/telephony traffic. If those values are chosen for a WAN interface (for example the “ipwan2”), the router will sort the SIP (and its media) upstream traffic into that channel. Other pre-assigned values are IP-TV (value=3) and TR069 provisioning (value=259). Selecting a QoS routing value for an interface (“channel”) is simply a way to let that interface ‘grab’ the upstream traffic of that type. Normally, the internet interface “ET0” or “LINE” should have no QoS tag value at all, or more precisely, its tag having a zero value. This tag value is also the default for any packet that has not been marked to belong to a certain type of traffic.

Does the QoS routing means that the actual destination IP address of a packet is unimportant? The answer is no. The router does as follows, for a (upstream) packet:

1. Examine the destination address of the packet.
2. If there is any IP interface with a corresponding IP address/subnet mask, send the packet out on that interface.
3. Examine any QoS tag value that has possibly been marked for this packet. If no such tag, go to step 5.
4. If there is an IP interface that is configured to ‘grab’ this traffic type (i.e. packets with this QoS tag), send it to that interface. Here, the destination IP address does not lie within the interface’s subnet, so send out the packet to the Gateway configured for that interface. (If the destination address would have matched the interface’s subnet it would already have been sent out in step 1).
5. If the packet hasn’t been sent out anywhere so far, send it to the default gateway, that is, the gateway of the internet interface (“ET4” or “LINE”).

It is important to understand the different gateways mentioned here. A gateway should be configured for each extra WAN interface (in the Gateway field on the page), or could be acquired automatically (by DHCP/PPP). It is sometimes called ‘next hop’ and is the address of the next router up the (provider’s) network, that will handle the packet’s further destiny. Each WAN IP interface must have its own next hop (gateway). However, there could be only one ‘default gateway’ in the box, this is the ‘next hop’ address for the internet interface “ET0” or “LINE”. It is the router where to send all packets that have no other obvious IP interface to use. In the routing table (on the web page “Status: Network”, “Advanced”) the default gateway is represented by the line starting with “0.0.0.0” (=Destination). The ‘next hop’ addresses for the extra WAN interfaces are non-default gateways, also visible in the routing table. Needless to say, these entries in the routing table will not show up until the respective IP interface has got its IP address (by DHCP, PPP or manually).

Read more: Routing table, Default gateway, Next hop

The QoS tag value is an out-of-packet data set by a classifier operating on the incoming packets on the LAN interfaces (such as “ET1”, “AIR”). It could also be set by agents (clients/servers) running in the box, when they produce output packets to be sent upstream. A special case is the SIP/voice traffic. The SIP Proxy takes care of all the QoS marking that needs to be done for sending such traffic into the WAN interface that has been configured for it.

Furthermore, one could also separate SIP traffic (and media streams) for a particular domain into its own designated WAN IP interface, leaving SIP traffic to other domains to another WAN (e.g. the internet WAN). This could be useful if the service provider wish to route SIP calls differently if they are destined to anyone within the providers ‘domain’ or not. The QoS tag value set on packets for the special domain has the value 4 and is called SIP, Specific in the QoS routing dropdown selection on the Extra WAN interfaces page. To be able to use this special SIP route, one must also check a checkbox on the SIP Settings web page. The checkbox is found in under the Outbound Proxy heading, where one must of course also enter the domain name (or an IP address) and filter values for the kind of SIP traffic that should go into the SIP specific-marked WAN interface.

For clarity, it should also be mentioned that related to SIP QoS are also some settings on the Advanced SIP Settings page. Under Upstream prioritization (QoS) there is a checkbox Enable. Checking this box (it is checked by default in the ADSL case) will give priority to media packets as they are competing for bandwidth with other traffic on the same IP interface. This may be helpful for the voice quality if using one single WAN interface, but does not affect the QoS routing that has been described here. In fact, if SIP and media streams are QoS-routed to a particular (extra) WAN IP interface (by the mechanisms described above), and that IP interface only handles SIP/media, the settings of this checkbox is not critical, but may be left checked.

There are also, on the SIP Advanced web page, settings called Set DiffServ bits in all SIP media streams. These only affect the contents of the packets but do not affect the routing strategies.

Apart from the built-in support for SIP and its QoS-routing tagging, there are possibilities to specify other types of traffic that should have special QoS tags. As already mentioned, the decision to tag a packet (thus forcing it to a particular WAN interface) is done by a classifier seeing the packet coming in on a LAN interface. The tag marking may be done based on the packet’s destination or source IP address, the destination or source ports, the packet’s protocol or its DiffServ (TOS) bits. On the Extra WAN interfaces page there is a box of settings to set up this profile, much like a packet filter.

For example, if one sets the Destination IP address to 200.10.11.0, the Destination Subnet Mask to 255.255.255.0 and the QoS routing dropdown to IPTV, the packets coming in from the LAN that are directed to a “IP-TV” server on the IP address 200.10.11.34 will be QoS-tagged (classified) and specially treated by the router. If “ipwan3” have been set up to receive IPTV upstream traffic (by selecting “IPTV” in the QoS routing dropdown for “ipwan3”), those upstream packets will go into the “ipwan3” channel. (The reason for this may be to forward provisioning requests or other boot-up traffic from a set top box on the LAN to a IP-TV server that doesn’t sit on the public internet). There is also a way to mark those packets by a particular DiffServ bits setting, by writing the bit pattern into the Set DiffServ bits field on the same line. This action does not affect the QoS routing or any other treatment in the box, but may perhaps be important for handling further up in the network.

One of the WAN IP interfaces may be used for receiving multicast traffic, typically data streams for video and IP-TV. The task for a router is to distribute this traffic to whatever clients on the local side (LAN) that are listening. It is of course preferred not to send all multicast packets to all LAN clients at any time. The IGMP (Internet Group Management Protocol) is therefore used to manage membership in groups, where (local) hosts can receive multicast streams by “applying” for membership in such groups. A router periodically sends out queries to the local hosts to get the membership reports. The Internet Gate is equipped with an IGMP Proxy, that on the WAN side acts as a ‘host’, to receive and answer queries that come from further up the network. On the local side it acts as the ‘router’ that sends queries and keeps track of what hosts are members of which groups.

The IGMP Proxy needs to know which is the upstream interface, i.e. from where it will expect incoming IGMP queries and to where it should send back reports of membership additions/removals in groups. Normally, when using only one WAN interface this is of course no problem. When using extra WAN interfaces, however, one must decide which of the multiple WAN “channels” that should be regarded as the upstream interface. Needless to say, this should be the subnet where the multicast server (e.g. an IP-TV server) is reached. For that purpose there is a checkbox IGMP Proxy on the Extra WAN interfaces page, one checkbox for each IP interface. Only one of them or none should be checked. If none is checked, the internet IP interface (“ET0” or “LINE”) will be used as the IGMP upstream interface.

If multicast streams and IGMP will be used, one must also enable the IGMP/Multicast on the Security Settings for the profile (Hi, Lo or AC) that is used. There is a checkbox under Applications from inside that should be checked to allow the IGMP traffic through the firewall. Under “General Settings” on that page there are also settings for the IGMP Proxy itself (enabled by default).

To reduce LAN traffic the Internet Gate uses link-layer unicast when sending multicast packets to single IGMP hosts. Thus if only one host has applied for a multicast channel the router sends the multicast packets to it with the host’s own MAC address as target ethernet address – allowing switches to send those packets to that host only. No need to send them to other LAN units not interested in multicast packets. If there are two or more hosts on the LAN listening to the same multicast channel the multicast packets are transmitted on the LAN in normal, link-layer multicast manner, using special multicast MAC addresses.

Read more: IGMP, proxy, Multicast