GRC DNS Benchmark vs direct measurement on router-based dnsmasq

[argel1200]

Ran into a discrepancy I can't explain and I'm curious whether it reproduces for others.


SETUP

ASUS RT-AX88U Pro on Asuswrt-Merlin, dnsmasq forwarding to OpenDNS.

Business fiber, static IP, no VPN in path.

DNS Benchmark Release 5 (latest).


WHAT GRC REPORTS

Router (192.168.0.1):

Avg 0.056-0.060
Cached 0.062-0.064
Min 0.017

Three runs across different conditions: one with a PS5 saturating the downlink, two idle, one with a custom server list.

Router numbers were identical each time.

Public resolvers in the same runs varied normally with conditions.


WHAT DIRECT MEASUREMENT SHOWS

From a Windows client:

Measure-Command { Resolve-DnsName -Server 192.168.0.1 -Name google.com -DnsOnly }

Run 1: 215ms (cold + cmdlet startup)
Run 2: 3.8ms
Run 3: 10.4ms
Run 4: 3.3ms
Run 5: 2.8ms

So roughly 3ms measured against roughly 62ms reported.


DNSMASQ'S OWN STATS

cache size 10000, 0/24445 cache insertions re-used unexpired cache entries

server 208.67.222.222#53: queries sent 344, retried 3, failed 0,
nxdomain replies 6, avg. latency 30ms

Zero evictions after 24k insertions. Zero upstream failures.


WHAT I'VE RULED OUT

My understanding is that GRC primes the cache before timing the "Cached Name" pass, so this shouldn't be measuring a forward to upstream.

But there's a floor around 17ms I can't account for.

- Merlin sets no-negcache, so every NXDOMAIN goes upstream. If GRC's
test set includes bogus names, some of the average is uncacheable by
design.

- DNSSEC counters are all zero, so validation isn't running and isn't
the cost.

- Zero evictions, so no cache pressure.


THE QUESTION

Has anyone compared GRC's local-forwarder numbers against direct measurement on other hardware?

Trying to work out whether this is something about dnsmasq specifically, something in my config, or expected behavior when the tool is pointed at a forwarder rather than a recursive resolver.


CONFIG (Merlin defaults, only cache-size changed)

bind-dynamic
no-resolv
servers-file=/tmp/resolv.dnsmasq
no-negcache
cache-size=10000
edns-packet-max=1232

 

[peterblaise]

Though "from our current point of view" is inherently unavoidable,
DNSBench is intented to measure and report on DNS servers, not
"our current point of view".

Does that direct where to look for why any other measurement
scheme does not match DNSBench - that they are "looking" at,
measuring, and reporting different things?

If you are testing only IPV4, how do the numbers from DNSBench 1
compare to the numbers from DNSBench 2, for you, from your "point
of view"?

 

[argel1200]

Only IPv4. I don''t have v1. The question is why is it so far off on the router's numbers? Guessing maybe some NXDOMAIN tests are going on?

 

[ColbyBouma]

That's weird. I have a pfSense router running Unbound and I get 2 ms for cached.

 

[peterblaise]

Cool.

I get 12 to 11 ms ( 7 to 26 ms ) on my IPv4 DNS servers from my
worst hell-hole Windows 7x64 Pro computer, and though
192.168.0.1 is my router, I've stopped using my ISP servers, opting
for ANYTHING else:

UTC: 2026-08-16, from 01:07:44 to 01:07:59, for 00:15.212

1. 0. 0. 1 | Min | Avg | Max |Std.Dev| Jitter|
----------------+-------+-------+-------+-------+-------+
- Average Delay | 0.007 | 0.024 | 0.277 | 0.007 | 0.292 |
- Cached Name | 0.007 | 0.012 | 0.024 | 0.003 | 0.250 |
- Uncached Name | 0.010 | 0.046 | 0.277 | 0.070 | 1.522 |
- DotCom Lookup | 0.010 | 0.016 | 0.029 | 0.003 | 0.188 |
---<-------->---+-------+-------+-------+-------+-------+
one.one.one.one
APNIC AND CLOUDFLARE DNS RESOLVER PROJECT


149.112.112.112 | Min | Avg | Max |Std.Dev| Jitter|
----------------+-------+-------+-------+-------+-------+
- Average Delay | 0.007 | 0.026 | 0.300 | 0.008 | 0.308 |
- Cached Name | 0.007 | 0.011 | 0.026 | 0.003 | 0.273 |
- Uncached Name | 0.009 | 0.052 | 0.300 | 0.073 | 1.404 |
- DotCom Lookup | 0.010 | 0.017 | 0.041 | 0.005 | 0.294 |
---<---O--O->---+-------+-------+-------+-------+-------+
dns.quad9.net
QUAD9


9. 9. 9. 9 | Min | Avg | Max |Std.Dev| Jitter|
----------------+-------+-------+-------+-------+-------+
- Average Delay | 0.007 | 0.027 | 0.338 | 0.009 | 0.333 |
- Cached Name | 0.007 | 0.011 | 0.021 | 0.003 | 0.273 |
- Uncached Name | 0.008 | 0.054 | 0.338 | 0.081 | 1.500 |
- DotCom Lookup | 0.009 | 0.016 | 0.036 | 0.004 | 0.250 |
---<-----O-->---+-------+-------+-------+-------+-------+
dns9.quad9.net
QUAD9


192.168. 0. 1 | DNS queries are not answered at this IP.
---<-------->---+-------+-------+-------+-------+-------+
Non-routable local internet address
Local Network DNS server


UTC: 2026-08-16, from 01:07:44 to 01:07:59, for 00:15.212

What are we looking for, again?

 

[argel1200]

What are we looking for, again?

Using dnsmasq on the router - benchmarks seem off. I do IT infra for a living - trying not to take work home with me, though have to admit unbound is tempting.

 

[peterblaise]

What is dnsmasq?

dnsmasq is a lightweight, low-footprint networking software package that provides local Domain Name System (DNS) caching, a Dynamic Host Configuration Protocol (DHCP) server, and Router Advertisement/TFTP capabilities. It is the default DNS and DHCP management engine for almost all custom and commercial embedded router firmwares, including OpenWrt, DD-WRT, Tomato, Asuswrt, and Pi-hole.

How dnsmasq Works on a Router​

On a router, dnsmasq acts as an intermediate DNS proxy between local network devices (LAN) and external upstream DNS servers (such as Cloudflare, Google, or your ISP).

1. Query Interception: When a connected device requests the IP address for a domain (e.g., example.com), the query goes directly to the router's internal IP address (e.g., 192.168.1.1).
2. Cache Inspection: dnsmasq checks its local RAM cache.
   • Cache Hit: If the record exists and its Time-To-Live (TTL) hasn't expired, dnsmasq instantly returns the IP address to the client without sending any traffic out to the internet.
   • Cache Miss: If the record is missing or expired, dnsmasq forwards the query to the configured WAN DNS servers, returns the answer to the client, and stores the result in RAM for future requests.
3. Local Hostname Mapping: Integrates directly with its DHCP lease table and local /etc/hosts file to resolve local network names (e.g., nas.local, printer.lan) without sending internal queries out to public DNS servers.

Key Features and Benefits​

• Ultra-Fast Local Caching: Eliminates internet round-trip latency for frequently visited domains.
• Minimal Resource Footprint: Requires less than 5 MB of RAM and negligible CPU usage, making it ideal for low-power hardware.
• Network-Wide Ad/Malware Blocking: Can load blocklists to map thousands of tracking/ad domains to 0.0.0.0, blocking unwanted traffic for every device on the LAN.
• DNS Wildcards & Redirection: Supports custom domain overrides and split-horizon DNS routing.
• DHCP/BOOTP/PXE Support: Assigns static and dynamic IP addresses while facilitating network booting for diskless clients.

Measurement Results: With vs. Without dnsmasq​

When testing DNS performance using tools like GRC DNS Benchmark (v1 / v2) or CLI utilities, adding dnsmasq changes the metrics drastically depending on whether the query is cached or uncached.

Metric / Scenario	Without dnsmasq (Direct Upstream)	With dnsmasq (Router Caching)
Response Time (Cached)	10 ms – 60+ ms (Requires full WAN round-trip)	0.1 ms – 0.8 ms (Instant LAN loopback response)
Response Time (Uncached)	10 ms – 60+ ms	10 ms – 60+ ms (+ ~0.2 ms forwarding overhead)
Upstream Bandwidth	Every domain query generates WAN traffic	Repeated queries generate zero WAN traffic
Reliability Under WAN Drop	Fails completely if WAN flickers	Serves all cached domains and local IPs during outages

Observations in GRC DNS Benchmark (v1 & v2)​

• Cached Queries (Green Bar): Testing 192.168.1.1 with dnsmasq enabled yields a near-zero response time bar (sub-millisecond). It consistently outperforms every public DNS server on the planet for cached domains.
• Uncached Queries (Blue Bar): Tests unique, unvisited domains. The results directly reflect the speed of your chosen upstream servers (e.g., 1.1.1.1 or 8.8.8.8). dnsmasq adds a virtually imperceptible forwarding delay (~0.1 to 0.3 ms).
• DotCom Queries (Red Bar): Measures root-level .com name lookup responsiveness. Shows identical performance to the upstream provider unless top-level domain names are cached in RAM.
• GRC DNSBench v2 Specifics: Version 2 places heavier emphasis on socket exhaustion, concurrent connection stability, and multi-threaded stress. With dnsmasq, testing reveals whether the router’s CPU throttles under high query volume or if its maximum concurrent connection limit (dns-forward-max, default 150) requires tuning.

Other Tools for Measuring DNS Performance and Reliability​

• dig (Domain Information Grok): Standard CLI tool for Linux/macOS/Windows (via BIND utilities). Displays exact query duration in milliseconds (Query time: 0 msec indicates a local cache hit).
• dnsperf / resperf: Industrial-grade CLI benchmarking tools built by DNS-O-Matic/Nominum to simulate massive query loads and measure DNS server throughput, capacity, and packet drop rates.
• Namebench: An open-source benchmark tool that compares your current resolver (including local caches) against major public resolvers using your actual web browsing history.
• PowerShell Resolve-DnsName: Built-in Windows cmdlet to measure lookup timing and inspect query headers (Measure-Command { Resolve-DnsName example.com }).
• dog / doggo: Modern CLI DNS clients that output structured JSON/colored text with precise timing metrics and support for DNS-over-HTTPS (DoH) and DNS-over-TLS (DoT).

Try:

DNSBENCH /pause 9999