From: Matthew Davis <matthewndavis () gmail com>
Date: Sat, 20 Aug 2022 17:46:59 -0500
Hello. The project I work on is a Windows-only, 64-bit-only application that collects and processes application layer messages across various system-specific protocols. Our legacy application used the old Windows raw socket mechanism (before migrating to WinPcap 4.1.3) to read the packets off the wire. Our revamped application is using the OEM version of Npcap with MUCH better success. However... A recent survey of our log files from the field indicates that we are missing packets. Specifically, converting our log files to .pcapng and opening them in Wireshark, we see about 1% of the packets showing the [TCP Previous segment not captured] message. Due to the nature of this data, this 1% loss is unacceptable to our users. As expected, this loss gets more dramatic with additional network traffic. Testing in the lab shows that Wireshark v3.6.7 captures the packets from a stress test with no apparent packet loss, so I know the problem is on our end. I think there are some application-level improvements we can make, but I'm interested in knowing if we are configuring and executing our packet capture in the most efficient way possible. Here is a brief summary of how we are doing things currently. - We call pcap_open() with a snap length of 65536, promiscuous mode enabled, and a read timeout of 500ms. - We call pcap_setbuff() to increase the size of the kernel buffer to 16MB. - We then call pcap_next_ex() inside a for loop to get the next capture. - Upon successful return, we allocate a byte array using pkt_header.caplen, copy the pkt_data into the byte array, and add the byte array to a pre-allocated list. - We execute this for loop until the pre-allocated list is filled (to avoid reallocation) or a predetermined timeout is exceeded on the application side. - When either of these conditions is satisfied, we hand the pre-allocated list off to another thread, allocate a new list, and do the loop again. Here are my questions. 1. Is pcap_next_ex() the most efficient way of transferring captures to the application? It looks like pcap_loop() or pcap_dispatch() might allow multiple captures to be returned via a single callback. Is that correct? And if so, would that be the recommended way to get the captures in a high data rate environment? 2. Our understanding is that the kernel buffer *IS* the ring buffer that must be read from at least as fast as the data is received in order to minimize/eliminate the occurrence of dropped packets. We understand the size of the buffer won't prevent dropped packets if the application can't keep up (it merely delays the moment when that occurs). But a bigger ring buffer can accommodate data spikes, allowing the application to catch up during data lulls. To this end, how big can we make the kernel buffer via pcap_setbuff()? Is there a practical or rule-of-thumb limit? Is the ring buffer associated with each handle??? If we collect simultaneously with Npcap on multiple NICs, does the size of each ring buffer need to be limited in any way? Should we use pcap_set_buffer_size() instead of pcap_setbuff()? Can pcap_set_buffer_size() be called after pcap_open() like pcap_setbuff() can? Or do we need to use the create-set-activate pattern? 3. We are confused about the difference between the kernel buffer and the user buffer. How does the user buffer work with pcap_next_ex()? Since pcap_next_ex() only returns a single packet at a time, does the user buffer even matter? Perhaps it comes into play with pcap_loop() or pcap_dispatch() being able to return more data in the callback? 4. How does pcap_mintocopy() work with pcap_next_ex()? Again, since pcap_next_ex() only returns a single packet at a time, does this even apply? Perhaps with pcap_loop() or pcap_dispatch()? 5. Finally, can stats be enabled for the same handle that is capturing the data? If we want to monitor, for example, the number of dropped packets seen during a capture, how do we do that with the pcap_t returned by pcap_open()? If we need two pcap_t handles, one for capture and one for stats, does that imply a single ring buffer under the hood for a given NIC? As stated earlier, we are using Windows 64-bit only with Npcap OEM 1.70. I apologize for the length, but any help/guidance is GREATLY appreciated. And hopefully this will help others out as well. Matt Davis
_______________________________________________ Sent through the dev mailing list https://nmap.org/mailman/listinfo/dev Archived at https://seclists.org/nmap-dev/
Current thread:
- Live Capture Performance to Rival Wireshark Matthew Davis (Sep 12)
