Inputs and Filters

• 1: Filters and masks
• 2: Input options

1 - Filters and masks

There are a few ways to manipulate incoming packets in various steps of dnsmonster pipeline. They operate in different levels of stack and have different performance implications.

BPF

BPF is by far the most performant way to filter incoming packets. It’s only supported on live capture (--devName). It uses the tcpdump’s pcap-filter language to filter out the packets. There are plans to potentially move away from this method and accept base64-encoded bpf bytecode in the future.

Sample Ratio

Sample ratio (--sampleRatio) is an easy way to reduce the number of packets being pushed to the pipeline purely by numbers. the default value is 1:1 meaning for each 1 incoming packet, 1 gets pushed to the pipeline. you can change that if you have a huge number of packets or your output is not catching up with the input. Checkout performance guide for more detail.

De-duplication

The experimental de-duplication (--dedup) feature is implemented to provide a rudimentary packet de-duplication capability. The functionality of de-duplication is very simple. It uses a non-cryptography hashing function (FNV-1) on the raw packets and generates a hash table of incoming packets as the come in. Note that the hashing function happens before stripping 802.1q, vxlan, ethernet layers so the de-duplication happens purely on the packet bytes.

There’s also the option --dedupCleanupInterval to specify cleanup time for the hash table. around the time of cleanup, there could be a few duplicate packets since the hash table is not time-bound on its own. It gets flushed completely at the interval.

Applied after Sample Ratio for each packet.

Port

There’s an additional filter specifying the port (--port) of each packet. since the vast majority of the DNS packets are served out of port 53, this parameter shouldn’t have any effect by default. note that this filter will not be applied to fragmented packets.

IP Masks

While processing the packets, the source and destination IPv4 and IPv6 packets can be masked by a specified number of bytes (--maskSize4 and --maskSize6 options). Since this step happens after de-duplication, there could be seemingly duplicate entries in the output purely because of the fact that IP prefixes appear the same.

Allow and Skip Domain list

These two filters specify an allowlist and a skip list for the domain outputs. --skipDomainsFile is used to avoid writing noisy, repetitive data to your Output. The skip domain list is a csv-formatted file (or a URL containing the file), with only two columns: a string representing part or all of a FQDN, and a logic for that particular string. dnsmonster supports three logics for each entry: prefix, suffix and fqdn. prefix and suffix means that only the domains starting/ending with the mentioned string will be skipped from being sent to output. Note that since the process is being done on DNS questions, your string will most likely have a trailing . that needs to be included in your skip list row as well (take a look at skipdomains.csv.sample for a better view). You can also have a full FQDN match to avoid writing highly noisy FQDNs into your database.

--allowDomainsFile provides the exact opposite of skip domain logic, meaning your output will be limited to the entries inside this list.

both --skipDomainsFile and --allowDomainsFile have an automatic refresh interval and re-fetch the FQDNs using --skipDomainsRefreshIntervaland --allowDomainsRefreshInterval options.

For each output type, you can specify which of these tables are used. Check the output section for more detail regarding the output modes.

2 - Input options

Let’s go through some examples of how to set up dnsmonster inputs

Live interface

To start listening on an interface, simply put the name of the interface in the --devName= parameter. In unix-like systems, the ip a command or ifconfig gives you a list of interfaces that you can use. In this mode, dnsmonster needs to run with higher privileges.

In Windows environments, to get a list of interfaces, open cmd.exe as Administrator and run the following: getmac.exe. You’ll see a table with your interfaces’ MAC address and a Transport Name column with something like this: \Device\Tcpip_{16000000-0000-0000-0000-145C4638064C}.

Then you simply replace the word Tcpip_ with NPF_ and use it as the --devName parameter. Like so

dnsmonster.exe --devName \Device\NPF_{16000000-0000-0000-0000-145C4638064C}

Pcap file

To analyze a pcap file, you can simply use the --pcapFile= option. You can also use the value - or /dev/stdin to read the pcap from stdin. This can be used in pcap-over-ip and zipped pcaps that you would like to analyze on the fly. For example, this example will read the packets as they’re getting extracted without saving the extracted pcap on the disk

lz4cat /path/to/a/hug/dns/capture.pcap.lz4 | dnsmonster --pcapFile=- --stdoutOutputType=1

Pcap-over-Ip

dnsmonster doesn’t support pcap-over-ip directly, but you can achieve the same results by combining a program like netcat or socat with dnsmonster to make pcap-over-ip work.

to connect to a remote pcap-over-ip server, use the following

while true; do
  nc -w 10 REMOTE_IP REMOTE_PORT | dnsmonster --pcapFile=- --stdoutOutputType=1
done

to listen on pcap-over-ip, the following code can be used

while true; do
  nc -l -p REMOTE_PORT | dnsmonster --pcapFile=- --stdoutOutputType=1
done

if pcap-over-ip is a popular enough option, the process of building a native capability to support it shouldn’t be too difficult. Feel free to open a topic in the discussion page or simply an issue on the repo if this is something you care about.

dnstap

dnsmonster can listen on a dnstap TCP or Unix socket and process the dnstap logs as they come in just like a network packet, since dnstap’s specification is very close to the packet itself. to learn more about dnstap, visit their website here.

to use dnstap as a TCP listener, use --dnstapSocket with a syntax like --dnstapSocket=tcp://0.0.0.0:5555. If you’re using a Unix socket to listen for dnstap packets, you can use unix:///tmp/dnstap.sock and set the socket file permission with --dnstapPermission option.

Currently, the dnstap in client mode is unsupported since the use case of it is very rare. in case you need this function, you can use a tcp port proxy or socat to convert the TCP connection into a unix socket and read it from dnsmonster.