A sysadmin often has to facilitate requests to provide access from the network
they manage to resources hosted by an external party. If the target resource
cannot be accessed, they have to establish whether this is due to the traffic
being blocked by a device on their network or by that of the other party.
One way to determine this is for the sysadmin to examine the logs of any
relevant software firewalls and routing devices (routers, layer three switches,
firewalls, etc) on their network. The purpose of this post is to demonstrate
some alternative techniques which are particularly useful to someone without
access to such logs. It will focus specifically on diagnosing TCP connection
failures to IPv4 endpoints using route tracing tools.
For the purpose of this post I’ll be using a fictitious scenario. A computer
within a corporate network with private IP 192.168.0.5 requires SSH (TCP port
22) access to an external endpoint with public IP 203.0.113.1.
Route tracing tools provide details of the path across an IP network to a
specified endpoint. Each routing device that’s traversed is listed along with
associated statistics. This post is only concerned with the path taken, therefore
I won’t be elaborating on the significance of the statistics.
traceroute (Unix-like OSs)
On Unix-like systems, route tracing can be achieved with the traceroute command.
Its default behaviour is to send a series of UDP packets with destination port
numbers ranging from 33434 to 33534 which is of no use in this scenario. What is
of use is an alternative mode of operation which is to send TCP SYN packets to a
TCP port of your choice.
The output shows that the traffic successfully traversed the
only routing device within the corporate network.
Windows ships with tracert, a route tracing tool with only one mode of
operation which is to send ICMP echo requests. This is of no use since the
scenario in this post is to determine the point at which traffic is being
blocked for a specific TCP port. Fortunately there is a suitable third party
tool for Windows, tracetcp.
tracetcp operates in the same way as traceroute by sending TCP SYN packets
on a TCP port of your choice.
When testing against Topology A I kept getting strange
output like this:
on the r/sysadmin subreddit seems to confirm
that such behaviour occurs when a routing device routes to another routing device
within the same network segment (which is exactly how Topology A is designed).
This Github issue seems to
imply that the behaviour is a bug, which at the time of writing has not been fixed.
To work around this I had to use the -g parameter to specify that the firewall
should be used as the gateway, thus bypassing the layer three switch:
Invoking the -g parameter to use the firewall as the gateway means that
topologies A and B are rendered equivalent. Therefore what follows applies to
both Topology A and Topology B.
The output shows that the traffic successfully traversed the only
routing device within the corporate network.
Unless a static route has been defined on a computer, traffic bound for an
external party should traverse the default gateway as the first hop. Something
to be aware of is that if the gateway is a virtual/logical address then the IP
that’s recorded as the first hop by a route tracing tool may differ.
A virtual/logical address may be used for load balancing purposes, such as with
VRRP (Virtual Router Redundancy Protocol) or HSRP (Hot Standby Router Protocol).
In such cases the route tracing tool will show the first physical address of the