Getting BorderManager (NetWare) to Work with Bridged Internet Connections - Feb 25, 2005

For years, I had been seeing a problem with NetWare servers connected to bridged Internet connections not being able to work with secondary IP addresses. The primary address was fine, but you could not access any secondary IP address bound to the network card.

This affects people with bridged connections to the ISP, and people hooking up BorderManager directly to a cable modem. It only affects servers with more than one public IP address assigned to the server.

The problem itself is an ARP issue with how the NetWare TCP stack is designed to respond to ARP requests. Novell TID 10077130 tells you that this is working by design, but it sure causes problems.

If you suspect that you have this problem, here is how you can confirm it and see what is going on:

1. SET TCP ARP DEBUG=ON
2. Look at the logger screen. Have someone on the Internet ping a secondary IP address assigned to your server. You should be seeing ARPRECEIVE requests. Note that for requests to the secondary IP addresses assigned to your server, there are no ARPSEND replies going out.
3. Look closely at the sending IP address of the ARP requestor, as well as the target IP address. If the target IP address is the secondary address on the public side of the server, this is the data of interest to you. The sending IP address will be of a server located at the ISP.

The problem occurs when the sending IP address is NOT in the same subnet as your public IP address on the NetWare server. For instance, you have a 12.1.2.194 (255.255.255.240) IP address assigned to your public NIC, giving you 13 usable IP addresses for your server, and your default gateway is 12.1.2.193. You would expect that ARP requests would be coming from 12.1.2.193, and if you had a routed Internet connection, you would see ARP requests from there. But in this case you see them from a completely different subnet - for example 10.40.0.208.

Per the Novell TID mentioned above, NetWare's TCP/IP stack will not respond to ARP requests coming from another subnet. You see this in the ARP debug data showing up as ARPRECEIVE's that never have an ARPSEND reply.

A Workaround

It took me years to think of this, as some nice simple ideas do sometimes, but it has worked for me. Since the problem is that the server will not respond to requests from another subnet, we need to make sure that the requests are seen to be coming from a local-attached subnet. AND we need respond to requests on one subnet with ARP data for another one, which means using Proxy ARP. So we bind another public IP address, in the same subnet as the sending router at the ISP, and enable Proxy ARP on it. You have to be careful in case the router at the ISP changes addresses, especially if it changes to the same one you assign to the server, but unless that happens this trick should work.

EXAMPLE: (taken from a real customer experience)

ARPRECEIVE coming from 10.40.0.208, for addresses on a 12.x.x.x (255.255.255.240) subnet.

1. In INETCFG, add another public binding 10.40.0.200 (255.255.255.0). (The ARP requester address must be within this subnet).
2. In the new binding, go to Configure TCP/IP Bind Options, Expert TCP/IP Bind Options, ARP Options
3. Set Force Proxy ARP to Enabled
4. Reinitialize System and test. The secondary addresses should now work.

There are potential issues with this, mostly involving conflicts with internal subnets matching whatever might be in use at the ISP. You could try using a 255.255.255.252 or .248 mask on the new public subnet to limit the exposure, but in my experience, bridged connections to the Internet tend to involve small clients with not many internal subnets. (And you could probably change internal IP addresses if necessary to get around this issue).



Return to the Main Page