Fortigate Firewall Diagnostics Pocket Guide [PDF]

  • 0 0 0
  • Suka dengan makalah ini dan mengunduhnya? Anda bisa menerbitkan file PDF Anda sendiri secara online secara gratis dalam beberapa menit saja! Sign Up
File loading please wait...
Citation preview

What this Book ​is all about This book is a follow up to “Fortigate Admin Pocket Guide“ and “Fortigate Security Pocket Guide” : Following The basic administration and applying security profiles, we will now start to diagnose our FortiGate for troubleshooting purposes and analyzing what is really happening on your network “Fortigate Diagnostics pocket guide”​, will walk you through the most important diagnostics commands in different use cases Every chapter includes hands-on practices It is written for​ beginners and intermediate users



The following book will help you to diagnose and analyze sessions, policies, interface issues, network congestions, and more



Table Of​ ​Contents What this Book is all about



1



Table Of Contents



3



General Connectivity issues



5



Ping and Traceroute



6



Packet Capture



14



TCP Dump Way



20



Using Filters



25



Examine TCP with the SYN flag on



31



Interfaces, Sessions, Services



35



DNS Settings



41



DHCP Settings



45



Diagnose Your ARP table



52



Diagnosing the Routes



55



Performance Issues



58



System and user Processes



63



Terminating Processes



65



CPU High Usage Stitch



71



Conserve Mode



77



Diagnose Sessions



80



Session List



84



Reduce session time



89



Debug Packet Flow



91



Final Words



98



Knowing how to diagnose your FortiGate is probably one of the most important tools that you can acquire as a FortiGate professional. It will make you aware of what is happening on your network, on your FortiGate kernel, services, and much more. this skill set is unique and the mindset that you will acquire will serve you not only on your firewall We will start with a low-level view of our FortiGate traffic, moving on to General network issues, system performance, and from there to sessions and packet flow view



General Connectivity ​issues



Ping and ​Traceroute



Before we enter packet capturing and debugging the packet flow, one of the first steps, that you will want to do in case of connectivity issues, the following 2 commands “exe ping “exe traceroute The two will verify connectivity between 2 points, but will also help you to verify that the correct route is used. so let’s look at them On your CLI, type: ​“execute ping google.com”



Whenever a Ping is being from your Fortigate, The default settings are: ● One second interval between Pings ● A two-second timeout ● 5 Pings are sent ● The size of the is 56 bytes, not including 8-byte headers, which makes it 64 bytes Our output shows us that the packets arrive at their destination if there is a packet loss, how many packets were sent, the size of the ping packet, we can also, see that the latency ( which can become an issue, if it is too high ) is O.K



But let’s assume, that we aren’t getting an answer, from our Ping request, the first thing, that should come to your mind is “ IS there a DNS resolve issue” To check that, we will use a known IP address of Google's DNS server



If the packets do arrive at their destination, then yes, you have a DNS issue, if not, the problem is at another place You can with different Ping options using the command “execute ping-options” ​followed by a question mark



You can play around with the different options, you can view the current settings using the view-settings



Let's look at the most important ones: Interval​ - Once an ICMP response, a new ICMP packet is sent. Currently, the interval between the two is 1 second Adaptive​ - on the contrary to the interval, once you choose ​adaptive​, then your ICMP packets will be sent immediately as the last returns. so if we use adaptive, you will see, a major change in the pace of pings sent



Now let’s change our interface and send All ICMP packets, from the LAN interface towards our gateway For that, we will use the source option and IP address of our interface . ​that can help to check connectivity issues in different interfaces on your FortiGate



Using​ ​Traceroute To track real-time paths ( routes ) from source to destination, we use traceroute, a known network diagnostic tool, that is available on your FortiGate CLI Traceroute uses ICMP packets to make sure that the traffic is following the right path and that the correct route is being used. it will report the different IP addresses of the different routers it pinged, and will also report, the time it took for each hop So let's open our CLI and start by typing ​“execute traceroute-options”



Let's look at the current settings



As you can see, there aren't many settings as in the ping command, but you can modify the Source IP or interface for a traceroute So let's set the source to 10.0.5.7 and look again at the settings



So now our source IP for traceroute is my Linux device on my LAN



Traceroute will help you to discover any DNS resolve issues. If you enter a domain name, it will try to resolve, if it won’t succeed, then you will know that you have a DNS issue So let’s traceroute our ​Fortigate​ to cnn.com



If traceroute is having issues with packets that do not return within the expected timeout window, then an ​asterisk (*) character ​ will show up, which needs to be checked



Packet ​Capture One of the coolest features of your FortiGate is its ability to packet capture any traffic that flows from different sources towards different destinations. Packet capture will help you to analyze your network and troubleshoot issues as : ● Routing issues ● Is traffic arriving at your FortiGate on the expected port? ● Missing packets ● Broadcast storm ● Failure in ARP resolution - look for hosts that don’t respond to your FortiGate arp requests I have choose Packet Capture as the first technique to troubleshoot and diagnose your network, as it is will become one of the most valuable tools, that you will use often



You can capture packets using two ways. The first one is using the Graphical User Interface. Navigate to network---packet capture



Create New



And here you can select the interface that you wish to capture the packets that flow through, you can use different filters, such as : ● Specific host on that interface ● Specific ports ( as ​Port 53​ for DNS Traffic, Port 443 for HTTPS, and so on )



● VLAN’s on that physical interface ● And the protocol, which you wish to capture ( TCP, UDP…)



Choosing Protocol, is done using the different Protocol numbers, the main ones are : ● TCP is protocol number 6 ● UDP is protocol number 17 ● ICMP is protocol number 1



So let’s capture traffic on our WAN interface, and open it using Wireshark Press OK and you will get back to the packet capture page, where the message not running appear



Right-click on the “not Running and a pop-up menu will list possible options



Click on ​“Start”



Wait a few seconds, and click again on ​Stop You will see the message



Right-click again and you will have the option to ​download​ a Pcap ( packet capture ) file



Double click on the Pcap file and it will open in ​Wireshark



If you want to practice and understand the underneath of TCP, HTTP, and other protocols and services That are used extensively on your network, then Wireshark is a must​. www.wireshark.org



TCP​ ​Dump Way



Up until now, we have Captured the traffic, downloaded the Pcap file, and opened it in Wireshark. we can now look at the output and start to analyze what is happening. The second way of doing so is through the CLI. The syntax is very similar to ​TCP Dump i​ n Linux, I personally find it much quicker to analyze . Let's open up our CLI The basic syntax for capturing packets is the following Diagnose sniffer packet



Following the ​“diagnose sniffer packet “​ which is the standard syntax that you will use, every time that you wish to capture packets, we can choose a specific interface ( as Port 2 or any other ) or use the ​“any”​ which means’ look for the traffic on all interfaces



We can also use a ​filter​ on our command, we will do that very soon The following number after the​ filter value ​is the​ verbosity ​level from 1-4, each number represents the amount of data that will be shown Diagnose sniffer packet



● Number 1 ​Inputs the Plain Packets headers



In the screenshot, we can see the source IP​ ( 192.168.1.44 ) ​connection



to the ​192.168.184​ Destination IP address using port 80. The output also includes the different TCP flags, indicating the connection status. TCP sequence numbers are also present Following a few lines, we can see that ​192.168.1.88​ is sending an ARP request asking everyone in the broadcast domain, ​“who is 192.168.1.198”​, looking for its MAC address To stop the capture press ​CTRL



● Number 2 ​Inputs header and data from IP packets Here we can see much more information, including the data from the IP packets, again represented in hex format



● Number 3 ​Inputs header and data from Ethernet packets ( in hex )



● Number 4 ​Inputs header of packets with interface ( Port ) name Next to each interface, you will also see, the direction of the traffic, which tells you if the packet is entering or leaving the interface



Using ​Filters The filter allows us much more granularity and defines what to look for in the output as: ● Hosts ● Protocols ● Source ● Destination ● Ports Let's examine the filter functionality and look for ICMP packets, from a host with the ​10.0.5.7​ IP address that is connected to our LAN interface



Let’s ping from our ​Host



And now let's use the filter option



We can use different filters, but to capture traffic coming from 10.0.5.7 and ICMP protocol, the syntax is Diagnose sniffer packet port2 ‘src 10.0.5.7 and icmp’ 1



Now set the count of packets in the output, let's use 10 packets ​ will appear after the verbosity level



You can play around with the different filters, but you're most likely to use Source and Destination on your daily FortiGate routines You can also combine protocols together, using the ​or​ and the ​and​ statement. “or”​ will display at least one of our choices “and”​ will show both so let's do that



We can see ​TCP and ICMP​ traffic only on our output Let's continue, with​ DNS traffic​ only on port 53



We can trace ​ 192.168.1.44​ on port 1 connecting Google’s DNS server on port 53 over UDP and the answer coming in from ​8.8.8.8. everything looks good



We can even use​ hexadecimal byte position ​to look for specific data, such as ARP packets



Or do it the regular way, using the arp keyword in our filter



Examine TCP with the ​SYN flag on Another cool thing that you can do on your command line, is to view only TCP Packets with the ​SYN​ flag on



This command is good if you wish to watch packets that are in the 3-way handshake establishment process ( as ​TCP​ with the ​SYN​ flag ) or that are terminating the session ( ​TCP​ Packet with the FIN flag )



In our filter, you can see that we are using TCP followed by the number ​13​ in square parentheses. the number 13 indicates the specific SYN flag position in the TCP packet ( octet number 13 which is equal to the decimal value 2 )



Lateral ​Movement



Our last packet capture will be done towards a device that is trying to probe our Gateway using ​Nmap Packet capture is not usually used to detect such behavior, for that you can use the IPS signatures in the FortiGate DoS Policy, but it gives us a glimpse, of what can be analyzed Lateral movement behavior is when 2 devices on your local area network, connect often, it is not so common, unless, there is a good purpose to do so. Another reason could be that a compromised PC, is trying to get credentials from an administrator PC So on our ubuntu device, we will scan our gateway using ​Nmap full TCP scan Which will initiate a full 3-way handshake with the destination target



And when we use the following syntax on our packet capture



We can see that ​10.0.5.7​ is sending syn request towards almost any available port of our gateway ( on a full TCP scan, it actually sends probes to the most 1000 common ports ) and finding out which one is​ Open or Closed



Interfaces, ​Sessions, Services



If you are having issues with traffic that is not flowing from your FortiGate, then this could be a hardware issue, routing issue, policy, and so on. Let’s try to pinpoint the solution, and checkout of everything is O.K with our interfaces



The first thing is to check that our interface is up, we will check our WAN interface at port 1 config systme interface Edit port1 Set status up



If you find out that the status is down, that may be an issue of an unplugged cable, so check your hardware physical connections



Next let’s look at our interface settings, using the “show” command The​ show​ command will list the default configuration, you can also use the ​get command to list all the available settings Show system interface



It seems that our IP address is in place, everything looks O.K The next Diagnostic command is the get hardware Nic diagnose hardware deviceinfo nic



This command is important, and it will list interface errors (as bad frames, dropped frames, collisions ... )



Things to look at : ● MTU​ ( the largest packet size in bytes ) size​ -​ if not configured on purpose, then your ​MTU​ size should be 1500 bytes ● Do you have any dropped packets in your interface? ● Is the status up or down? ● Are you working a full-duplex or maybe half-duplex? ● What is the speed of your interface? ●



Look for errors in the ​Rx CRC field​, You might have Checksum value errors







If you are witnessing a missed Packet count, then you will see it in the Rx_Missed_Errors field



DNS ​Settings Now let’s check our DNS Settings DNS can be the bottleneck of your network, you might witness latency and performance issues, or worse than that, traffic might not get out, due to a wrong configuration The first place you go is the​ Network---DNS



This is where you will find your ​System DNS​ settings To the right, you will also find The Latency of each DNS server If you are using an internal DNS server, check your LOCAL Domain nameis it correct? Now, we will check our ​DNS setting​ and performance using the: “diag test application dnsproxy” ​command Write it down and press enter



The different options indicate specific things you can check or do Number 1 . will dump or clear your DNS cache​ - sometimes, you will have issues connecting to a web site or a web application, that their IP address changed, yet your DNS cache saved the older IP address Number 2​ is good to indicate your DNS performance as latency and TTL



Using the ​Number 3​ option, you can check your DNS settings and the ongoing connections



Looking at our DNS servers configuration, we can also see, their latency ‘ labeled as rt



Are they using ​TLS​ ( secure connection )



TLS=1 = secure DNS connection TLS=0- regular, not encrypted Moving on to our DNS cache settings



Here we can learn 2 important things 1. The number of entries in your ​DNS cache​- currently we are set to 5000



2. The second is the Time To Live setting ​ ​ for cached entries. currently, we are set on 1800 seconds ( default setting ). Depending on your setting, you can enjoy a quicker response, once your DNS entries are cached for a longer time.



DHCP ​Settings Your DHCP server is a crucial part of your network, and its settings can indicate a lot of what is happening on your network, for example, a host that is not receiving an IP address We will start by navigating to our ​LAN interface Edit DHCP Server



Let's look at our DHCP through the CLI “config system dhcp server” “show” This command will list all available DHCP servers. Our DHCP that serves the 10.0.5.0/24 network is number 2



You can look at the IP ranges, that your DHCP leases, sometimes, it is configured for a small range of IP address, and new clients will not get an IP address . ​so make sure, that you have enough IP addresses in your Pool



If you want to disable your DHCP server and then enable it, you could do that using ( it is not recommended, but it is a safer way to restart your DHCP server operation than to kill its Process )



If you need to clear all DHCP leases



You can monitor your DHCP using the graphical user interface. I am using fortiOS 6.4.4. , so this may change in earlier versions On your Dashboard, click on the Plus sign ​(+) ​to create a new Dashboard



Name your DHCP Dashboard



Click O.K Here you will be asked to select a widget, we will choose DHCP



The following technique can be used to add up different widgets, that will give you a detailed overview of different aspects of your FortiGate, like routing, CPU utilization, and more Press on the widget, that you have just created



Here you see your Different IP addresses that are connected to the different DHCP servers. you can also, see the lease time expiration date Right-click on each IP address, will allow you to take different actions



You can create new IP reservations for specific devices, using their MAC address



And you can also revoke ( delete ) IP addresses



Diagnose Your ​ARP ​table Arp s used to determine the MAC address of your hosts, one of the first things that your hosts do on the local area network, is to send ARP requests asking all the other hosts on the LAN if they know who has a particular IP address, and if so, send the MAC address associated with it



If you suspect that there is an IP address conflict, or that an IP address has been assigned to the wrong interface, then you should look a the ARP Table On a packet capture sniffer, as we used earlier in the book, an ARP request looks like that



Our Gateway (​192.168.1.1​) sends a broadcast request to all the different interfaces, asking anyone “who has the 192.168.1.126 IP address “ 192.168.1.126​ who listens to the broadcast request will reply with its MAC address and it will be saved in the ARP cache table To look at your FortiGate ARP table, you can use the following command’s “Get system arp”



As you can see, the ARP table consists of a : ● IP address ● MAC address ● Interface name A more detailed view can be seen using the “Diag IP arp list”



Diagnosing the ​Routes Whenever you witness, that packets are not arriving at their destination, or that in general, there is no connectivity one of the first things, that you will want to do, is to check: ● If there is a route( static, default ) for the destination ● Are there routes with higher distance than others ● Is the correct route being used? Understanding your routing table will help you to get an overview of the roads outside and inside your network To view the routing table, we will start with the following command : “Get router info routing-table all” This command will list the active routes only. to list all the routes, we will use another command



On the left, we can see the type of route, the most used are:



● s=static ● c=connected ● o=ospf ● b=bgp Moving to the right, you will see the destination route, so let’s analyze the static route



The destination is the default route, which is at the ​0.0.0.0/0​ address, the



following in the square brackets, is the distance, which is ​10​ ( the lower the better, so if you have several routes, to the same destination, the one that will take precedence is the lower distance route ), next to it is the priority (the default is 0, the lower the better ) Following is the next hop to the destination, which is ​192,168.1.1,​ that is our Gateway ( my ISP modem ), and finally the port number If you want to list all the routes, including, those that are not active, you will use the following command “get router info routing-table database “



Here we can see, that we have another default route, but this time, the next hop is 10.0.5.4 The asterisk ​(*)​ sign next to our second static route, tells us that although the 2 routes, he is the active one



Performance ​Issues



Performance can be a big issue on your FortiGate if one or more operations are consuming too much memory, that may lead to dropped packets and overall bad performance. you can even get into what is know n as a conserve mode, where your FortiGate will stop performing security scans or even worse You can look at system performance using many commands, lets analyze what is happening with the following “get system performance status”



This command tells you a lot of your Fortigate performance At the top output, you will see how many CPUs your FortiGate supports CPU usage of user and kernel processes



Following that is Memory, your RAM usage. The first thing that you should look at is the memory usage ​(used)​, if it is over 70%, there could be issues with your memory, either memory leaks, or overloaded with traffic, processes that are running, it can even be too much memory in the cache that can be freed



The amount of cache memory that is freeable from the cache is shown next



And then your network sessions as the number of average sessions per minute



In order for you to recognize, if something is wrong, either CPU, Memory Usage, Amount of sessions, you will need to have a ​baseline​ of your network.



The baseline is nothing more than your normal network behavior in terms of: ● CPU utilization ● Memory Utilization ● Bandwidth ● Traffic Volume Those metrics can be gathered through your Logs, using the ​Log and Report menu



or your FortiView, where you can look and analyze, your network performance over time



The best practice is to use “get system performance status “ several times throughout an hour. this could give you an understanding of traffic spikes in your network, which can be seen by the total number of sessions, and CPU utilization



System and user ​Processes Another way to get a quick and detailed view of Memory and CPU utilization is to use the Top command on your FortiGate, the same top command that is used on Linux To list processes that are running on your FortiGate and find the memory that is allocated to the process instance, you can use 2 commands “diag sys top” “diag sys top summary”



Let’s analyze the output On the right of the table, you can see the process name. if you are seeing a duplicate of the process name, then it is probably due to the fact that its instance is running several times



Following the process name is its​ ID​, it is good, in case that you wish to Kill the process The third column is the state of the process : ● Running ● Sleep ● Zombie ● Disk sleep



Moving to the last 2 column’s, we can see that the “​snmpd”​ process, which is responsible for SNMP monitoring is consuming 0.5% of CPU capacity To the right, our last column shows the amount of memory that the process is running. memory usage can range from 0.0 to 5.5. And even higher



As you can see, my processes are in a sleep state, but on a daily basis, you will see different processes, that consumes your FortiGate CPU and Memory resources and that should be taken into account



Terminating ​Processes So you're using high-level encryption in your VPN or using IPS to scan different patterns and anomalies, all of that consumes lots and lots of CPU resources and memory. Let's look at the diag system top command again, add a few more parameters, and finally terminate a process We can play with our output refresh interval and the number of processes, that it will show So we will display 10 processes in an interval of 20 seconds “Diag sys top ”



Now we can also list the most demanding processes using 2 characters M- to list the most demanding in terms of memory



P - to list the most demanding in terms of CPU



The most memory-demanding process in our output is the “​cmdbsvr” ( the command database server application ) ​Usually, you will find other processes as the IPS and “​scanunitd”​ ( antivirus ) that will consume resources, You can also find the most CPU demanding processes, using the following command “Get system performance top”



Note - Killing Processes is not recommended, since it can interrupt your network operation. The following should be done, only as of the last option, knowing exactly what you are doing.



To Kill A processes we will use the ​“diag sys kill”​ command



Following that, you will enter what is called a signal that is a term that comes from Linux and Unix, which is actually a polite way to ask your system to stop the process. We can use different signal numbers, we will use 15, which is an aggressive way to tell your system to kill that process. “Diag sys kill 15”



We will also enter the process ​ID​ in our case it is ​9665 We have just killed that process. And here we can see that the DNS proxy process is actually being terminated.



CPU High Usage ​Stitch To be notified, when your FortiGate CPU utilization is getting high and exceeds a specific threshold, you can create a script, known also as an automation stitch that will trigger an email towards you Open your CLI and enter: “config system global” “set cpu-use-thershold ”



Here we will enter the percentage of total CPU utilization. Let’s choose 85%



Next, we will enter our Automation script actions, which are mostly debug CLI commands:



Don’t forget to start and close your script with double quotes



The next step is to set your Automation stitch trigger



I have named it “CPU_HIGH_AUT” but you can name it anything The event type should be set to ​high-cpu Next, we will create the stitch itself



As you can see, we are using the trigger that we have just configured Following that, let’s move to our admin FortiGate page security-fabric-automation Here you will find a new stitch named ​“high CPU”



Double click on that stitch



Select email next to our CLI script action and set your email for notification



Conserve ​Mode One symptom of Memory issues is when your FortiGate gets into a “conserve mode“ Getting into a conserve mode are determined by thresholds that are pre-configured, but you can change them ● Red - The first threshold​ is when your FortiGate enters a conserve mode. The threshold is configured using the percentage of memory that goes behind the total RAM ● Green - The second Threshold​ is when Your Fortigate exits from conserve mode ● Extreme - The third Threshold​ is when the sessions are being dropped, you're entering an extreme threshold. When your Fortigate enters Conserve Mode, it will stop accepting new configuration up to becoming unstable and drop new sessions. so be aware of Memory and CPU utilization on your machine To get into the conserve mode setting, we will use global settings “config system global” “set memory-use-threshold-green



Using the TAB key, you can scroll between the 3 thresholds percentage



So let's set the red mode to ​85%



If you want to check if you're in the conserve mode, or not, you can just use the “diag hardware sysinfo conserve”



You can see that you're currently not in a conserve mode. you can also see your total RAM and the memory that is being used, how much is freeable, and the threshold for the Green, red and extreme states



Diagnose ​Sessions



Our Network settings seem quite good, so let’s move and look at the session itself. We can trace the session route, and look at the multiple sessions that are happening on our network



A higher number of sessions lead to higher memory usage Let’s start by looking at the number of system sessions “Get system session status”



We currently have a small number of sessions We can also look at our sessions and their state using the: Diag sys session full-stat



Here we can see again the number of sessions, but also: ● Session table size ● State of our sessions - 8 TCP sessions in the established state ( already running), 1 in the SYN state, which means, that it is starting the 3-way handshake ● Clashes To view a list of all the running sessions on your FortiGate, you can use the​ “get sys session list “



The output shows: ●



sessions from source to destination



● Protocols used ● Time of Expiration ● Source and destination NAT “Get system session list “​ is a great way to list active sessions, and to have a high-level view of the running sessions



Session​ ​List Probably the best command to list sessions on your FortiGate is the ​“diag sys session list”



The output shows all the sessions running on your FortiGate session table, and it can be overwhelming, so we can use filters to define exactly what we want to see So let’s filter out our host which is ​10.0.5.7​ and analyze the output, we will use the src filter, but you can play around with the different filter options using the TAB key



And here is our output



We will get different sessions that are related to our host. On the last output, you will also see the total number of sessions which is ​4



Now let’s analyze the first session



proto​= the protocol being used. protocols are mapped using numbers, number ​6 is TCP



State​ = a TCP session has different states: 1= established 2- SYN sent



3=SYN and SYN-ACK 4=FIN_Wait 5=Time_Wait 6=close 7=Close_wait 8=Last ACK 9=Listen Each state has its own expiration time, but if we look at the output, we can see that our session state is 1, which means, that it is already established Are we using traffic shapers on that session?



We can also see the source NAT and the destination



Is it moving through an acceleration chip or the regular path?



In the session state, we see that it is may dirty



When it is ​dirty​, the session needs to be validated again. When your FortiGate receives the first packet, it evaluates the packet according to the policy. the evaluation is usually done on the first packet unless the firewall policy changes, routing changes, or network configuration has been done If the packet follows the policies, then it is labeled ​“may dirty” When a policy changes or any other condition, then the state for the following packets changes to “dirty “



Reduce ​session time One method to overcome performance issues ​( CPU )​, is to reduce session timers. Using a smaller Timer will free your FortiGate resources faster, as it will handle fewer sessions



TCP sessions are actually comprised of different steps, each with its own timer, but you can reduce the timer, so let’s move to our CLI



“Config system global” “Set tcp-halfclose-timer ”



The​ half-close timer​ is in when one side finishes sending the data, throughout a session. It sends a TCP packet with a FIN flag, but it still wants, to get the data that is sent from the other side, so it has a timer, which by default is 120 seconds. Example applications that need half-close timers are Databases



You can also change the half-open timer, that is the case where one side of the TCP connection actually crashes and doesn’t send data anymore. The other will keep the session for a specif time that you can configure



Debug ​Packet Flow



So you browse the internet and everything seems just okay. Let's just head over to cnn.com. See what's new. it uploads, Now, let's go to bbc.com. And yeah, everything seems okay.



But now you decide that you want to​ Ping ( send ICMP Echo request packet ) somewhere just to check that there are no connectivity issues. ​And it seems that you do not have a DNS resolve.



So let's stop that. And let's try to ping directly to Google's DNS server.



And yet again, you have an issue with ICMP. So how do you get around with that? And how do you diagnose the issue behind it?



So let's get back to our FortiGate. Let's log in. And one of the first things that I would do is to use ​“diagnose sniffer packet”​ with my host as the filter let's just write down the IP address of our host. And see what happens.



It seems that I have an echo request. But I'm not getting any echo-response​ back. So that's not enough. packet capture of my traffic shows me the ​symptom​ but doesn't give me ​medicine​.



So the next thing to do is to work at the CPU level and see how your Fortigate handles the packets step by step For that, we will use the ​“diag debug flow” let's just get back to our CLI. and write the following: “diag debug enable” “diag debug flow filter”-​. Let's filter our source, which is the 10.0.5.7



So that will be​ “diag debug flow filter saddr 10.0.5.7” and let's write “diag debug flow trace start”



And press enter



I'm actually debugging the source address 10.0.5.7, when your press the TAB key following the Filter word, you will see that you can also filter : ● Destination address ● VDOM ● Port ● Source port ● Destination port ● Protocol



Following the trace, you can decide how many packets are to be traced



So let’s see if we have any clues, that will help us to resolve our Ping issue



And yes, there it is​ “denied by forward policy check (policy number two)” There are several reasons for the “denied by forward policy check” : ● There is no firewall policy that matches the traffic ● The traffic matches Firewall Deny Policy ● A firewall policy is in place, but the user hasn’t accepted the disclaimer page



So we actually have a policy that denies ICMP. Let's check our FortiGate, and yes, there it is



We could have looked at our policy page, but in production, where your FortiGate handles hundreds of different policies that we do not have time to check, using the diag debug flow, can actually make troubleshooting faster Other messages that you might get is the ​ "reverse path check fail, drop" That is when your FortiGate drops packets based on its source IP address



Your Fortigate uses an RPF ( reverse path forward ) mechanism, that helps him to handle spoofing attacks. the Packet source IP is checked against the routing table for a route back to the source IP. if it doesn’t have one, the packet can be dropped. RPF has several configurations options, you can set to accept the packets or to drop them



Final ​Words You have just Finished “Fortigate Diagnostics Pocket Guide “ Part 3 I hope that you enjoyed the journey. My aim was to give you a head start on Troubleshooting and diagnostics, on one of the best next-generation firewalls in the market.



Sincerely yours Ofer Shmueli