What is it? #
Your server resolves names constantly: package repositories, APIs it calls, database hostnames, mail servers.
The resolution path on Linux involves several pieces: the /etc/hosts file, the resolver configuration, and often a local caching resolver such as systemd-resolved.
Understanding the order matters, because a stale entry in /etc/hosts overrides everything and produces the most confusing category of "it works on the other server" problem.
Diagnosing DNS is fast once you know three commands and the difference between what your application sees and what an external resolver returns.
Think of it like this #
Looking up a phone number. You check the note stuck to your monitor first, then your personal address book, then you ring directory enquiries.
A wrong note on the monitor beats the correct entry in every other source, which is exactly what a stale hosts file does.
Simple example #
Your application cannot reach an internal API by name. You need to establish whether the name resolves at all, what it resolves to, whether the answer is cached, and whether the problem is DNS or the network.
Code #
Resolution order (typical)
1. /etc/hosts static entries, always win
2. local caching resolver systemd-resolved, or dnsmasq
3. upstream resolvers from /etc/resolv.conf or DHCP
4. the DNS hierarchy root → TLD → authoritative# What resolvers is this machine using?
cat /etc/resolv.conf
resolvectl status # systemd-resolved: per-interface resolvers
# Static overrides
cat /etc/hosts
# 127.0.0.1 localhost
# 10.0.1.20 db-internal db-internal.local
# Which file order applies?
cat /etc/nsswitch.conf | grep hosts
# hosts: files dns → /etc/hosts is consulted before DNS# Debugging, in order
dig +short api.internal # what does the configured resolver return?
dig @1.1.1.1 +short api.internal # what does an external resolver say?
dig +trace api.internal # the whole resolution path
getent hosts api.internal # what the SYSTEM resolves, including /etc/hosts
resolvectl query api.internal # systemd-resolved's view, with cache info
# Clear the local cache
sudo resolvectl flush-cachesReading the difference between tools
dig queries DNS directly — it ignores /etc/hosts
getent uses the full system resolution path, including /etc/hosts
If dig returns the right answer but the application does not work,
check /etc/hosts and nsswitch.conf. That mismatch is the usual cause.# Is it DNS or the network?
dig +short api.internal # resolves to 10.0.1.20
nc -zv 10.0.1.20 443 # can we reach it by IP?
# resolves but unreachable → network or firewall
# does not resolve → DNSHow it works #
/etc/nsswitch.conf defines the order of sources. The common setting hosts: files dns means /etc/hosts is consulted before DNS, which is why a static entry silently overrides a corrected DNS record.
/etc/resolv.conf lists upstream resolvers. On systems using systemd-resolved it is often a symlink to a generated file, and editing it directly is overwritten on the next network change — which explains why a manual fix disappears after a reboot.
dig talks to DNS directly and ignores /etc/hosts. getent hosts uses the same path the application will, including the hosts file. Comparing the two is the fastest way to find a static override.
Querying an external resolver with @1.1.1.1 separates "the record is wrong" from "our resolver has a stale or broken answer".
Caching explains delayed changes. A record updated at the authoritative server is not visible locally until the cached answer expires, which is what resolvectl flush-caches resolves for testing.
The final block separates the two categories of failure cleanly. If the name resolves but the port is unreachable, DNS is fine and the problem is network, firewall or the service itself.
Real-world use #
Hosts file entries added during testing and never removed cause a specific and frustrating class of bug: one server behaves differently from the rest for no visible reason.
In containers, DNS is provided by the runtime, and service names resolve through it. A container that cannot resolve a service name usually has a network configuration problem rather than a DNS server problem.
Internal service discovery in cloud environments is DNS-based, so understanding search domains and resolution order matters when a short name works in one place and not another.
A resolver outage looks like a total outage, because nothing can reach anything by name. Configuring two upstream resolvers is basic resilience.
When a deployment starts failing to reach an external API, checking DNS first is often faster than checking the application, and dig plus nc answers it in two commands.
Common mistakes #
- Leaving test entries in /etc/hosts that override real DNS.
- Editing /etc/resolv.conf directly on a systemd-resolved system, so changes are overwritten.
- Using dig alone and missing a hosts file override that affects the application.
- Assuming a name failure is DNS without checking reachability by IP.
- Configuring only one upstream resolver.
Practice #
On a test machine, add an entry to /etc/hosts pointing a real domain at 127.0.0.1. Compare the output of dig and getent for that name, then remove the entry and flush the cache. Finally, diagnose a name that resolves but is unreachable and state which layer is at fault.