How can I harden the TCP/IP stack on my Windows 2003 server against a Denial of Service (DOS) attack?

TCP/IP Registry Values That Harden the TCP/IP Stack

The following list explains the TCP/IP-related registry values that you can configure to harden the TCP/IP stack on computers that are directly connected to the Internet. All of these values should be created under the following registry key, unless otherwise noted:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services

NOTE: All values are in hexadecimal unless otherwise noted.
• Value name: SynAttackProtect
Key: Tcpip\Parameters
Value Type: REG_DWORD
Valid Range: 0,1
Default: 0

This registry value causes Transmission Control Protocol (TCP) to adjust retransmission of SYN-ACKS. When you configure this value, the connection responses time out more quickly during a SYN attack (a type of denial of service attack).

The following parameters can be used with this registry value:
• 0 (default value): Set SynAttackProtect to 0 for typical protection against SYN attacks.
• 1: Set SynAttackProtect to 1 for better protection against SYN attacks. This parameter causes TCP to adjust the retransmission of SYN-ACKS. When you set SynAttackProtect to 1, connection responses time out more quickly if the system detects that a SYN attack is in progress. Windows uses the following values to determine whether an attack is in progress:
• TcpMaxPortsExhausted
• TCPMaxHalfOpen
• TCPMaxHalfOpenRetried

• Value name: EnableDeadGWDetect
Key: Tcpip\Parameters
Value Type: REG_DWORD
Valid Range: 0, 1 (False, True)
Default: 1 (True)

The following list explains the parameters that you can use with this registry value:
• 1: When you set EnableDeadGWDetect to 1, TCP is permitted to perform dead-gateway detection. When dead-gateway detection is enabled, TCP may ask the Internet Protocol (IP) to change to a backup gateway if a number of connections are experiencing difficulty. Backup gateways are defined in the Advanced section of the TCP/IP configuration dialog box in the Network tool in Control Panel.
• 0: Microsoft recommends that you set the EnableDeadGWDetect value to 0. If you do not set this value to 0, an attack may force the server to switch gateways and cause it to switch to an unintended gateway.
• Value name: EnablePMTUDiscoveryKey: Tcpip\ParametersValue Type: REG_DWORDValid Range: 0, 1 (False, True)Default: 1 (True)

The following list explains the parameters that you can use with this registry value:
• 1: When you set EnablePMTUDiscovery to 1, TCP tries to discover either the maximum transmission unit (MTU) or the largest packet size over the path to a remote host. TCP can remove fragmentation at routers along the path that connect networks with different MTUs by discovering the path MTU and limiting TCP segments to this size. Fragmentation adversely affects TCP throughput.
• 0: Microsoft recommends that you set EnablePMTUDiscovery to 0. When you do so, an MTU of 576 bytes is used for all connections that are not hosts on the local subnet. If you do not set this value to 0, an attacker may force the MTU value to a very small value and overwork the stack.

Important Setting EnablePMTUDiscovery to 0 negatively affects TCP/IP performance and throughput. Even though Microsoft recommends this setting, it should not be used unless you are fully aware of this performance loss.
• Value name: KeepAliveTimeKey: Tcpip\ParametersValue Type: REG_DWORD-Time in millisecondsValid Range: 1-0xFFFFFFFFDefault: 7,200,000 (two hours)

This value controls how frequently TCP tries to verify that an idle connection is still intact by sending a keep-alive packet. If the remote computer is still reachable, it acknowledges the keep-alive packet. Keep-alive packets are not sent by default. You can use a program to configure this value on a connection. The recommended value setting is 300,000 (5 minutes).
• Value name: NoNameReleaseOnDemand
Key: Netbt\Parameters
Value Type: REG_DWORD
Valid Range: 0, 1 (False, True)
Default: 0 (False)

This value determines whether the computer releases its NetBIOS name when it receives a name-release request. This value was added to permit the administrator to protect the computer against malicious name-release attacks. Microsoft recommends that you set the NoNameReleaseOnDemand value to 1.