XDM has the annoying behavior to start an Xserver which is listening on port 6000 for incoming TCP connections. If you want to turn this feature off, you need to edit the xdm config files. On FreeBSD (8), this file can be found at

/usr/local/lib/X11/xdm/Xservers

Change

:0 local /usr/local/bin/X :0

to

:0 local /usr/local/bin/X :0 -nolisten tcp