NetworkManager with dnsmasq caching NXDOMAIN response of router

Discussion:

(too old to reply)

David Ayers

2024-07-07 23:20:01 UTC

Hello everyone!

My Debian 12/bookworm laptop uses DHCP with NetworkManager which
produce anÂ /etc/resolv.conf containing:
# Generated by NetworkManager
```
search home
nameserver 192.168.1.254
```

I've setup NetworkManager to use its local dnsmasq instance to add
additional name resolution for libvirt and a VPN, according to [1].
My /etc/NetworkManager/conf.d/localdns.conf contains:
```
[main]
dns=dnsmasq
```
and my /etc/NetworkManager/dnsmasq.d/local_dnsmasq.conf contains
```
server=/virt/192.168.122.1
server=/122.168.192.in-addr.arpa/192.168.122.1
server=/vpn/10.70.71.1
server=/71.70.10.in-addr.arpa/10.70.71.1
log-queries
```

The name resolutions (and reverse resolutions) for the the *.vpn and
*.virt work just fine. But I'm having issues with the *.home domain as
soon as set the dns=dnsmasq in /etc/NetworkManager/conf.d/localdns.conf
independent of any entries in
/etc/NetworkManager/dnsmasq.d/local_dnsmasq.conf

After starting (or restarting) NetworkManager either with
```
sudo systemctl reload NetworkManager.service
```
or with
```
sudo nmcli general reload dns-full
```
the name resolution works twice for anyhost.home in the local domain
but subsequently fails with NXDOMAIN.

Here ist the output of the log-queries output for a successful
```
ping -c 1 nas-server.home
PING nas-server.home (192.168.1.103) 56(84) bytes of data.
64 bytes from nas-server.home (192.168.1.103): icmp_seq=1 ttl=64 time=7.47 ms
```
with the corresponding
```
sudo tail -f /var/log/syslog
TS HOST systemd[1]: Reloaded NetworkManager.service - Network Manager.
TS HOST dnsmasq[169260]: query[A] nas-server.home from 127.0.0.1
TS HOST dnsmasq[169260]: forwarded nas-server.home to 192.168.1.254
TS HOST dnsmasq[169260]: query[AAAA] nas-server.home from 127.0.0.1
TS HOST dnsmasq[169260]: forwarded nas-server.home to 192.168.1.254
TS HOST dnsmasq[169260]: reply nas-server.home is 192.168.1.103
TS HOST dnsmasq[169260]: reply nas-server.home is NXDOMAIN
TS HOST dnsmasq[169260]: query[PTR] 103.1.168.192.in-addr.arpa from 127.0.0.1
TS HOST dnsmasq[169260]: forwarded 103.1.168.192.in-addr.arpa to 192.168.1.254
TS HOST dnsmasq[169260]: reply 192.168.1.103 is nas-server.home
```
Notice the IPv6 AAAA query and the two replies with the FQDN.

The first subsequent query succeeds again with:
```
ping -c 1 nas-server.home
ping: nas-server.home: Name or service not known
```
with the corresponding
```
sudo tail -f /var/log/syslog
TS HOST dnsmasq[171213]: query[A] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[AAAA] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[A] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: forwarded nas-server to 192.168.1.254
TS HOST dnsmasq[171213]: query[AAAA] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: forwarded nas-server to 192.168.1.254
TS HOST dnsmasq[171213]: reply nas-server is 192.168.1.103
TS HOST dnsmasq[171213]: reply nas-server is NXDOMAIN
TS HOST dnsmasq[171213]: query[PTR] 103.1.168.192.in-addr.arpa from 127.0.0.1
TS HOST dnsmasq[171213]: forwarded 103.1.168.192.in-addr.arpa to 192.168.1.254
TS HOST dnsmasq[171213]: reply 192.168.1.103 is nas-server.home
```
Notice that the FQDN caches with NXDOMAIN are followed up with just the
host name and the same two replies, one with the IP and the other with
NXDOMAIN.

But all subsequent queries will fail with:
```
ping -c 1 nas-server.home
ping: nas-server.home: Name or service not known
```
with the corresponding
```
TS HOST dnsmasq[171213]: query[A] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[AAAA] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[A] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server is NXDOMAIN
TS HOST dnsmasq[171213]: query[AAAA] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server is NXDOMAIN
```

Once I restart/reload NetworkManager (i.e. clear the cache) I get two
successful name resolutions with subsequent requests failing again.

I do notice that when querying external domains, they seem to return
NODATA-IPv6 instead of NXDOMAIN for what I assume are the AAAA queries.
But I have no control of that my ZTE based ISP suppired router will
reply for the AAAA queries. I suppose, that the router is returning
the wrong reply for its own local domain for AAAA queries.

So I guess my question is, can I tell dnsmasq somehow not to cache
NXDOMAIN or interpret it as NODATA-IPv6 for queries to the *.home
domain?

Any other suggestions are also welcome!

And in case this isn't it, where is the correct mailing list, to pose
suche a question?

Thanks, anyone!
David

[1] https://networkmanager.dev/docs/api/latest/NetworkManager.conf.html

--
David Ayers

Supporting:
Free Software Foundation Europe [] (http://www.fsfe.org)
Become a supporter of the FSFE! [][][]
Your donation powers important work! || (http://fsfe.org/donate)

jeremy ardley

2024-07-08 03:50:02 UTC

Permalink

Post by David Ayers
Hello everyone!
My Debian 12/bookworm laptop uses DHCP with NetworkManager which
# Generated by NetworkManager
```
search home
nameserver 192.168.1.254
```
I've setup NetworkManager to use its local dnsmasq instance to add
additional name resolution for libvirt and a VPN, according to [1].
```
[main]
dns=dnsmasq
```
and my /etc/NetworkManager/dnsmasq.d/local_dnsmasq.conf contains
```
server=/virt/192.168.122.1
server=/122.168.192.in-addr.arpa/192.168.122.1
server=/vpn/10.70.71.1
server=/71.70.10.in-addr.arpa/10.70.71.1
log-queries
```
The name resolutions (and reverse resolutions) for the the *.vpn and
*.virt work just fine. But I'm having issues with the *.home domain as
soon as set the dns=dnsmasq in /etc/NetworkManager/conf.d/localdns.conf
independent of any entries in
/etc/NetworkManager/dnsmasq.d/local_dnsmasq.conf
After starting (or restarting) NetworkManager either with
```
sudo systemctl reload NetworkManager.service
```
or with
```
sudo nmcli general reload dns-full
```
the name resolution works twice for anyhost.home in the local domain
but subsequently fails with NXDOMAIN.
Here ist the output of the log-queries output for a successful
```
ping -c 1 nas-server.home
PING nas-server.home (192.168.1.103) 56(84) bytes of data.
64 bytes from nas-server.home (192.168.1.103): icmp_seq=1 ttl=64 time=7.47 ms
```
with the corresponding
```
sudo tail -f /var/log/syslog
TS HOST systemd[1]: Reloaded NetworkManager.service - Network Manager.
TS HOST dnsmasq[169260]: query[A] nas-server.home from 127.0.0.1
TS HOST dnsmasq[169260]: forwarded nas-server.home to 192.168.1.254
TS HOST dnsmasq[169260]: query[AAAA] nas-server.home from 127.0.0.1
TS HOST dnsmasq[169260]: forwarded nas-server.home to 192.168.1.254
TS HOST dnsmasq[169260]: reply nas-server.home is 192.168.1.103
TS HOST dnsmasq[169260]: reply nas-server.home is NXDOMAIN
TS HOST dnsmasq[169260]: query[PTR] 103.1.168.192.in-addr.arpa from 127.0.0.1
TS HOST dnsmasq[169260]: forwarded 103.1.168.192.in-addr.arpa to 192.168.1.254
TS HOST dnsmasq[169260]: reply 192.168.1.103 is nas-server.home
```
Notice the IPv6 AAAA query and the two replies with the FQDN.
```
ping -c 1 nas-server.home
ping: nas-server.home: Name or service not known
```
with the corresponding
```
sudo tail -f /var/log/syslog
TS HOST dnsmasq[171213]: query[A] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[AAAA] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[A] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: forwarded nas-server to 192.168.1.254
TS HOST dnsmasq[171213]: query[AAAA] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: forwarded nas-server to 192.168.1.254
TS HOST dnsmasq[171213]: reply nas-server is 192.168.1.103
TS HOST dnsmasq[171213]: reply nas-server is NXDOMAIN
TS HOST dnsmasq[171213]: query[PTR] 103.1.168.192.in-addr.arpa from 127.0.0.1
TS HOST dnsmasq[171213]: forwarded 103.1.168.192.in-addr.arpa to 192.168.1.254
TS HOST dnsmasq[171213]: reply 192.168.1.103 is nas-server.home
```
Notice that the FQDN caches with NXDOMAIN are followed up with just the
host name and the same two replies, one with the IP and the other with
NXDOMAIN.
```
ping -c 1 nas-server.home
ping: nas-server.home: Name or service not known
```
with the corresponding
```
TS HOST dnsmasq[171213]: query[A] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[AAAA] nas-server.home from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server.home is NXDOMAIN
TS HOST dnsmasq[171213]: query[A] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server is NXDOMAIN
TS HOST dnsmasq[171213]: query[AAAA] nas-server from 127.0.0.1
TS HOST dnsmasq[171213]: cached nas-server is NXDOMAIN
```
Once I restart/reload NetworkManager (i.e. clear the cache) I get two
successful name resolutions with subsequent requests failing again.
I do notice that when querying external domains, they seem to return
NODATA-IPv6 instead of NXDOMAIN for what I assume are the AAAA queries.
But I have no control of that my ZTE based ISP suppired router will
reply for the AAAA queries. I suppose, that the router is returning
the wrong reply for its own local domain for AAAA queries.
So I guess my question is, can I tell dnsmasq somehow not to cache
NXDOMAIN or interpret it as NODATA-IPv6 for queries to the *.home
domain?
Any other suggestions are also welcome!
And in case this isn't it, where is the correct mailing list, to pose
suche a question?
Thanks, anyone!
David
[1] https://networkmanager.dev/docs/api/latest/NetworkManager.conf.html

There is also the file /etc/nsswitch.conf. That gives you fine grained
control over name services and the order they are consulted

If you have a very small .home network you can create static entries in
/etc/hosts and if configured in nsswitch.conf will be used before any
call to an external dns provider.

jeremy ardley

2024-07-08 04:10:01 UTC

Permalink

Post by jeremy ardley

There is also the file /etc/nsswitch.conf. That gives you fine grained
control over name services and the order they are consulted
If you have a very small .home network you can create static entries in
/etc/hosts and if configured in nsswitch.conf will be used before any
call to an external dns provider.

I also forgot to mention my usual warning:

NetworkManager is *not* stable and if you do anything complex with it
you can expect trouble.

Personally I use systemd-networkd as that seems much more stable and
predictable and is easier to congigure

David Ayers

2024-07-08 10:00:01 UTC

Permalink

Post by jeremy ardley
There is also the file /etc/nsswitch.conf. That gives you fine
grained control over name services and the order they are consulted
If you have a very small .home network you can create static entries
in /etc/hosts and if configured in nsswitch.conf will be used before
any call to an external dns provider.
NetworkManager is *not* stable and if you do anything complex with it
you can expect trouble.
Personally I use systemd-networkd as that seems much more stable and
predictable and is easier to congigure

Thanks, and yes, /etc/hosts is what I have been juggling until now but
even though the network is "ÃŒberschaubar" (small) it is volatile.
After reboots of the router the printer/scanner, nas all get new
IPs/DHCP leases and editing /etc/hosts is becoming cumbersome.

NetworkManager is the default... I assume that the defaults is what is
the most stable. If it is not, there should be a process to exchange
the default. I'd really like to avoid straying from what most people
use. But I don't really believe that this is a NetworkManager issue.
Do me it seems to be a ZTE router, which I don't control, or hopefully
a dnsmasq configuration issue, which I do control.

Thanks!
David

PS: it seems I'm not receiving mails via the list subscription so
please keep my CC:ed if you will. Thank you!

--
David Ayers

Supporting:
Free Software Foundation Europe [] (http://www.fsfe.org)
Become a supporter of the FSFE! [][][]
Your donation powers important work! || (http://fsfe.org/donate)

Detlef Vollmann

2024-07-10 13:20:01 UTC

Permalink

Post by David Ayers

Post by jeremy ardley
NetworkManager is *not* stable and if you do anything complex with it
you can expect trouble.
Personally I use systemd-networkd as that seems much more stable and
predictable and is easier to congigure

NetworkManager is the default... I assume that the defaults is what is
the most stable. If it is not, there should be a process to exchange
the default. I'd really like to avoid straying from what most people
use.

Only responding to this part.

NetworkManager is the "default" (whatever "default" means) as it serves
well the need of many users to connect a client machine to the Internet.

I doubt that many users use NetworkManager for what you're trying to do.
So you probably are "straying from what most people use".

Detlef

Greg Wooledge

2024-07-10 13:20:01 UTC

Permalink

Post by Detlef Vollmann
NetworkManager is the "default" (whatever "default" means) as it serves
well the need of many users to connect a client machine to the Internet.

It's only the "default" if a Desktop Environment is installed. On a
Standard Debian installation (e.g. one you'd use for servers), NM is
not installed at all.

Worry less about what the "default" is, because the entire concept of
defaults is extremely vague in Debian. Debian offers a multitude
of choices. Worry more about how to solve *your* own problems, using
whichever solution is most suitable.

songbird

2024-07-10 23:00:01 UTC

Permalink

Post by Greg Wooledge

Post by Detlef Vollmann
NetworkManager is the "default" (whatever "default" means) as it serves
well the need of many users to connect a client machine to the Internet.

It's only the "default" if a Desktop Environment is installed. On a
Standard Debian installation (e.g. one you'd use for servers), NM is
not installed at all.

when it got installed i masked it out and continued my
manual ways as i don't want an automatic network connection
coming up. i don't always do things on-line so i don't want
a connection just sitting there doing nothing but getting
probes and messages from the bozo's doing whatever on the
local ISP network.

Post by Greg Wooledge
Worry less about what the "default" is, because the entire concept of
defaults is extremely vague in Debian. Debian offers a multitude
of choices. Worry more about how to solve *your* own problems, using
whichever solution is most suitable.

yes. :) and then try to remember what you've done that
isn't quite normal... some years later you may forget and
i don't write all my stuff down. next time i do a clean
install i'll have to keep a better log of my local changes.
i have a lot of things masked out from systemd that i don't
use.

songbird

Michael Kjörling

2024-07-08 07:50:01 UTC

Permalink

Post by David Ayers
Hello everyone!
My Debian 12/bookworm laptop uses DHCP with NetworkManager which
# Generated by NetworkManager
```
search home
nameserver 192.168.1.254
```

Note that .home is somewhat of a special snowflake with regards to
TLDs. It was suggested as the default for HNCP in 2016 (RFC 7788
section 8 <https://www.rfc-editor.org/rfc/rfc7788#section-8>);
rejected as a gTLD in 2018
<https://www.icann.org/en/board-activities-and-meetings/materials/approved-board-resolutions-regular-meeting-of-the-icann-board-04-02-2018-en#2.c>;
and then the usage from RFC 7788 was effectively superceded by the
recommendation and assignment for non-unique use of home.arpa a few
months later in RFC 8375 <https://www.rfc-editor.org/rfc/rfc8375>.

This may or may not have anything to do with your issues; but in
general, making up own TLDs and hoping that they will never conflict
with public ones is a bad idea these days. Just look at how many
internal names suddenly started having issues after Google was
assigned .dev in 2019; to say nothing of that they made it a
preloaded-HSTS TLD.

It's better to either use .home.arpa (which is specifically reserved
for the purpose) or to actually register a domain (even if the name
server delegations are bogus so it never meaningfully resolves on the
public Internet).

--
Michael Kjörling 🔗 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

David Ayers

2024-07-08 10:30:01 UTC

Permalink

Post by Michael KjÃ¶rling
Note that .home is somewhat of a special snowflake with regards to
TLDs. It was suggested as the default for HNCP in 2016 (RFC 7788
section 8 <https://www.rfc-editor.org/rfc/rfc7788#section-8>);
rejected as a gTLD in 2018

<https://www.icann.org/en/board-activities-and-meetings/materials/approved-board-resolutions-regular-meeting-of-the-icann-board-04-02-2018-en#2.c

Post by Michael KjÃ¶rling
;
and then the usage from RFC 7788 was effectively superceded by the
recommendation and assignment for non-unique use of home.arpa a few
months later in RFC 8375 <https://www.rfc-editor.org/rfc/rfc8375>.
This may or may not have anything to do with your issues; but in
general, making up own TLDs and hoping that they will never conflict
with public ones is a bad idea these days. Just look at how many
internal names suddenly started having issues after Google was
assigned .dev in 2019; to say nothing of that they made it a
preloaded-HSTS TLD.
It's better to either use .home.arpa (which is specifically reserved
for the purpose) or to actually register a domain (even if the name
server delegations are bogus so it never meaningfully resolves on the
public Internet).

Thank you for the insight!

I just clicked through the routers DHCP configuration options (note
there are no explicit DNS options). This is a ZTS ZXHN H268N Router
provided with a custom Firmware A1 WLAN Box 027_42w2_MU from my
provider that claims "The firmware of your device is the latest."...

... and I haven't found a way to configure the domain.

But note, if I do _not_ configure
/etc/NetworkManager/conf.d/localdns.conf
dns=dnsmasq

but leave the default, then DNS resolves fine.

It's only when I add dnsmasq to handle the .vpn and .virt domains that
the .home domain starts caching the NXDOMAIN responses and causes
issues. So I'm still crossing my fingers that this can be resolved
with some dnsmasq configuration which I haven't understood yet.

Thanks!
David

PS: forgive me for repeating: it seems I'm not receiving mails via the
list subscription so please keep my CC:ed if you will. Thank you!

--
David Ayers

Supporting:
Free Software Foundation Europe [] (http://www.fsfe.org)
Become a supporter of the FSFE! [][][]
Your donation powers important work! || (http://fsfe.org/donate)

Thomas Schmitt

2024-07-08 10:40:01 UTC

Permalink

Hi,

Post by David Ayers
PS: it seems I'm not receiving mails via the list subscription so
please keep my CC:ed if you will. Thank you!

The "X-Spam-Status:" header of your mail does not show "LDOSUBSCRIBER".
So i assume that ***@fsfe.org is not known to the list server as a
subscribed e-mail address.

It would be interesting to see what happens if you try to unsubscribe
and after getting e-mail feedback and confirmation subscribe again.
List-Subscribe:
<mailto:debian-user-***@lists.debian.org?subject=subscribe>
List-Unsubscribe:
<mailto:debian-user-***@lists.debian.org?subject=unsubscribe>

Have a nice day :)

Thomas