Kubernetes 之 Nameserver limits were exceeded
1、问题描述
最近查看kubernetes 的events,发现了有两个节点经常出现下面的信息:
DNSConfigForming Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 192.96.0.10 10.2.5.1 10.2.5.2
这个是kubelet打出来的,字面意思很好理解,就是nameserver 超出限制,超出的将被忽略。那限制是多少呢?so带着这个疑问就去github kubernetes源码一探究竟。
首先确定知道这个是kubelet里面的events,找到dns的相关源码:kubernetes/pkg/kubelet/network/dns/dns.go
func (c *Configurer) formDNSNameserversFitsLimits(nameservers []string, pod *v1.Pod) []string {
if len(nameservers) > validation.MaxDNSNameservers {
nameservers = nameservers[0:validation.MaxDNSNameservers]
log := fmt.Sprintf("Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: %s", strings.Join(nameservers, " "))
c.recorder.Event(pod, v1.EventTypeWarning, "DNSConfigForming", log)
klog.Error(log)
}
return nameservers
}
找到上面的函数,有一个validation.MaxDNSNameservers,所以kubelet读取resolv.conf里面的nameserver是有数量限制的,最大值就是validation.MaxDNSNameservers,我们现在去找validation的源码。在上面的import里面
找到k8s.io/kubernetes/pkg/apis/core/validation,找到validation这个package的源码:
const (
// Limits on various DNS parameters. These are derived from
// restrictions in Linux libc name resolution handling.
// Max number of DNS name servers.
MaxDNSNameservers = 3
// Max number of domains in search path.
MaxDNSSearchPaths = 6
// Max number of characters in search path.
MaxDNSSearchListChars = 256
)
原来默认的支持最大nameserver是3个。然后去两台抛出DNSConfigForming的服务器上面检查/etc/resolv.conf里面的nameserver发现果然超过了三个。
二、解决问题
当然一般被忽略掉的那个nameserver不影响服务使用的话,可以不作为紧急处理。
可以在kubelet设置一个kubermetes专用的resolv.conf文件,保证kubernetes使用到的nameserver不超过三个,这样就可以解决。
在/var/lib/kubelet路径下,修改config.yaml
resolvConf: /etc/resolv.conf
重启kubelet生效。