Category: Troubleshooting

CheckPoint Managment Server logs space utilization

Posted on

Today I encountered an issue related to CheckPoint Managment server: it started to report incorrect file system space utilization on the logs’ mount point. The output of dfcommand reports the mount point almost full:

Instead, the sum of all files and dir present under /var/log is around 37 Gb.

This is a strange behavior, especially because the mount point is used for logging. Comparing both commands, it’s easy to understand that more than half of the total disk space is wrongly marked as used, so I would like to figure out where the problem lay. I think the unlinked open files may be the problem and I’m checking for this using lsof:

As I expected, the lsof output reports lots of open file entries, all associated to cplmd daemon. Based on CheckPoint knowledge base (here):

“In order to get the data that should be presented in SmartView Tracker, FWM spawns a child process CPLMD, which reads the information from the log file and performs unification (if necessary). Upon receiving an answer from CPLMD, FWM transfers it to SmartView Tracker.”

The easy way to fix this is to restart the daemon in order to allow it to “flush” all deleted files still hanging:

This solution fix the problem. Now, df command reports the right space utilizzation.

and also all orphan files are disappeared.

tcpdump: a brief review

Posted on

tcpdump is a useful command-line packet sniffer running on libpcap, usually present on *nix systems and on most network appliances, such as Juniper, F5 loadbalancer or CheckPoint firewall (both based on *nix). Also, with the support of WinPcap library, it could be installed on Windows systems in order to made troubleshooting easier.

On shell or CLI, to simply running tcpdump, you may type:

tcpdump support a lots of options and filters, let’s see it assuming to use eth0 and VLAN 150 as a example.

TCPDUMP OPTIONS

To capture all eth0 traffic:

To capture traffic of a specific vlan passing through eth0:

Prepare full .pcap file for viewing on Wireshark

#the option -s specify the length of data to be capture, -s0 means all packet data. Instead -s30 means to capture the first 30 byte of the packet.

View MAC address on a capture over eth0

Do not resolve hostname / service on a capture over eth0

TCPDUMP FILTERS

There are four mainly filters:

– the host filter used to capture packets based on single ip (eg. 192.168.10.4)
– the net filter used to capture packets based on network address (eg. 10.0.0.0/16)
– the port filter used to capture packets based on port (eg. 22 – ssh services)
– the protocol filter like icmp , arp, tcp, udp to capture only certain of packets

Filters can be combined all together using the logical operator AND. Also, for the host, net and port filters you may specify the if it could be as source (src) or destination (dst)

To capture only packets containing ip 192.168.10.4 as source or destination on eth0

To capture only packets containing network 10.0.0.0/16 as source or destination on eth0

To capture only packets containing port 22 as source or destination on eth0

To capture all ssh packets (port 22) to host 192.168.10.4 form network 10.0.0.0/16 on eth0

To capture all icmp packets to host 192.168.10.4 on eth0

Finally, you can combine options and filters as you need. For example, with the following line you will capture arp request from network 192.168.10.0/24 passing trough eth0 and including MAC address. Also, host name will be not resolved:

The same example, can be also exported as Wireshark file:

tcpdump is a very powerfull, there are a lot of other options an filters to speedup your troubleshooting and go deep in the analysis. You can get more information on the official website tcpdump.org.