Menu

Fortigate CPU utilization

a few times i came across a Fortinet firewall with a stuck ips process, it mostly occurring as a bug when working with the policy base, when this happens there are two ways of solving the issue,
first - reboot the machine.
second - find the process the causing the problem and "Kill" it, this may take a few minutes of work but more appropriate for production networks that cant afford the long down-time caused by a full reboot,
Seeing as a fortinet firewall is based on a linux OS, there is a simple way of monitoring witch process is running and witch "eats" the most of your CPU resources at a specific moment.

note that This type of debug is done via the command line of the Forti,
in order to view the status of the firewall all we need is to run
FGT# diagnose sys top

for example :
FGT # diagnose sys top
Run Time:  5 days, 12 hours and 11 minutes
31U, 14S, 54I; 249T, 73F, 54KF
          newcli     1175      R       0.9     2.8
       ipsengine     1065      S <     0.0    13.4
       ipsengine     1064      S <     0.0    10.3

To stop one of the processes use
FGT# diagnose sys kill 11 (pid)
If we need to stop one of the IPS engine process it will be
FGT# diagnose sys kill 11 1065    

There is also a way to restart the IPS engine, to do so use the
FGT # diagnose test application ipsengine 99

After the restart the memory will jump to full usage to fix it clear the restart log.
FGT # diagnose test application ipsengine 4


P.S, IPS Engine Test Usage:
1: Display IPS engine information
2: Toggle IPS engine enable/disable status
3: Display restart log
4: Clear restart log
5: Toggle bypass status
6: Submit attack characteristics now
99: Restart all IPS engines and monitor

Hope this post was helpful, If it was please consider a donation:
BTC Address: 1CnyMpjd1RntRDxSus2hu2aDMyzL4Kj29N
LTC Address: LUqrKbzGihTU2GEnL3EwsuuLHCsxCJMdtR

5 comments:

  1. Thanks for the diag test app command! Much better to restart a process than the whole firewall.

    ReplyDelete
    Replies
    1. Agreed, I always prefer solving the actual problem rather than causing a long downtime for a temporary one..

      Delete
  2. For me it was ipsmonitor who had to be restarted, but thanks for the tip! :D

    ReplyDelete
  3. Thanks, but how do I find out the process that eaten up the utilization?

    ReplyDelete
  4. I got it, http://docs.fortinet.com/fgt/handbook/40mr3/fortigate-troubleshooting-40-mr3.pdf thanks.

    ReplyDelete