LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Containers (https://www.linuxquestions.org/questions/linux-containers-122/)
-   -   Error occurred while creating a cluster using Kubespray (https://www.linuxquestions.org/questions/linux-containers-122/error-occurred-while-creating-a-cluster-using-kubespray-4175722992/)

e@d79 03-13-2023 06:45 AM

Error occurred while creating a cluster using Kubespray
 
I want to create a Kubernetes cluster using Kubespary. I have created three nodes. I am using the official documentation: https://kubernetes.io/docs/setup/pro...ols/kubespray/
I have three remote virtual machines running openSUSE, serving as nodes.
After running the following command:
Code:

ansible-playbook -i inventory/local/hosts.yaml -u root --become --become-user=root cluster.yml
I get the result. Error:
Code:

TASK [etcd : Configure | reload systemd] *******************************************************************************************************************************************************************************************************************************************************************************
ok: [node1]
ok: [node3]
ok: [node2]
Monday 13 March 2023  07:18:30 +0000 (0:00:01.168)      0:07:40.914 **********

TASK [etcd : Configure | Ensure etcd is running] ***********************************************************************************************************************************************************************************************************************************************************************
ok: [node1]
ok: [node2]
ok: [node3]
Monday 13 March 2023  07:18:30 +0000 (0:00:00.558)      0:07:41.472 **********
Monday 13 March 2023  07:18:30 +0000 (0:00:00.055)      0:07:41.528 **********
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (4 retries left).
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (3 retries left).
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (2 retries left).
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (1 retries left).

TASK [etcd : Configure | Wait for etcd cluster to be healthy] **********************************************************************************************************************************************************************************************************************************************************
fatal: [node1]: FAILED! => {"attempts": 4, "changed": false, "cmd": "set -o pipefail && /usr/local/bin/etcdctl endpoint --cluster status && /usr/local/bin/etcdctl endpoint --cluster health 2>&1 | grep -v 'Error: unhealthy cluster' >/dev/null", "delta": "0:00:05.031251", "end": "2023-03-13 03:19:17.721758", "msg": "non-zero return code", "rc": 1, "start": "2023-03-13 03:19:12.690507", "stderr": "{\"level\":\"warn\",\"ts\":\"2023-03-13T03:19:17.717-0400\",\"logger\":\"etcd-client\",\"caller\":\"v3@v3.5.6/retry_interceptor.go:62\",\"msg\":\"retrying of unary invoker failed\",\"target\":\"etcd-endpoints://0xc00001e000/192.168.122.233:2379\",\"attempt\":0,\"error\":\"rpc error: code = DeadlineExceeded desc = context deadline exceeded\"}\nFailed to get the status of endpoint https://192.168.122.120:2379 (context deadline exceeded)", "stderr_lines": ["{\"level\":\"warn\",\"ts\":\"2023-03-13T03:19:17.717-0400\",\"logger\":\"etcd-client\",\"caller\":\"v3@v3.5.6/retry_interceptor.go:62\",\"msg\":\"retrying of unary invoker failed\",\"target\":\"etcd-endpoints://0xc00001e000/192.168.122.233:2379\",\"attempt\":0,\"error\":\"rpc error: code = DeadlineExceeded desc = context deadline exceeded\"}", "Failed to get the status of endpoint https://192.168.122.120:2379 (context deadline exceeded)"], "stdout": "https://192.168.122.233:2379, 4dc4060cd0d7d06, 3.5.6, 20 kB, false, false, 3, 10, 10, ", "stdout_lines": ["https://192.168.122.233:2379, 4dc4060cd0d7d06, 3.5.6, 20 kB, false, false, 3, 10, 10, "]}

NO MORE HOSTS LEFT *****************************************************************************************************************************************************************************************************************************************************************************************************

PLAY RECAP *************************************************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0 
node1                      : ok=518  changed=26  unreachable=0    failed=1    skipped=611  rescued=0    ignored=0 
node2                      : ok=484  changed=26  unreachable=0    failed=0    skipped=528  rescued=0    ignored=0 
node3                      : ok=437  changed=22  unreachable=0    failed=0    skipped=506  rescued=0    ignored=0 

Monday 13 March 2023  07:19:17 +0000 (0:00:47.097)      0:08:28.625 **********
===============================================================================
etcd : Configure | Wait for etcd cluster to be healthy --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 47.10s
bootstrap-os : Install python-xml ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 44.31s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 34.17s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 21.84s
container-engine/containerd : download_file | Download item ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 17.07s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 15.86s
container-engine/crictl : download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 12.04s
etcd : Gen_certs | Write etcd member/admin and kube_control_plane client certs to other etcd nodes ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 11.50s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 11.48s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.40s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 9.20s
container-engine/nerdctl : download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.90s
container-engine/runc : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.79s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 6.39s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 5.97s
etcd : Configure | Check if etcd cluster is healthy ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.51s
network_plugin/calico : Get current calico cluster version ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 4.08s
etcd : Gen_certs | Gather etcd member/admin and kube_control_plane client certs from first etcd node ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 3.59s
bootstrap-os : Install python3-cryptography --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.25s
container-engine/containerd : containerd | Unpack containerd archive -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.07s
qa@qa:~/kubespray$

Here is my host.yaml:

Code:

all:
  hosts:
    node1:
      ansible_host: 192.168.122.233
      ip: 192.168.122.233
      access_ip: 192.168.122.233
    node2:
      ansible_host: 192.168.122.120
      ip: 192.168.122.120
      access_ip: 192.168.122.120
    node3:
      ansible_host: 192.168.122.242
      ip: 192.168.122.242
      access_ip: 192.168.122.242
  children:
    kube_control_plane:
      hosts:
        node1:
        node2:
    kube_node:
      hosts:
        node1:
        node2:
        node3:
    etcd:
      hosts:
        node1:
        node2:
        node3:
    k8s_cluster:
      children:
        kube_control_plane:
        kube_node:
    calico_rr:
      hosts: {}

There is communication between the hosts.
Additional info :-
VMs - openSUSE
ssh & sudo - working fine
Kubespray version- v2.21.0-64
Client Version: v1.26.2
Kustomize Version: v4.5.7

e@d79 03-14-2023 08:09 AM

Closed issue!
 
Quote:

Originally Posted by e@d79 (Post 6417415)
I want to create a Kubernetes cluster using Kubespary. I have created three nodes. I am using the official documentation: https://kubernetes.io/docs/setup/pro...ols/kubespray/
I have three remote virtual machines running openSUSE, serving as nodes.
After running the following command:
Code:

ansible-playbook -i inventory/local/hosts.yaml -u root --become --become-user=root cluster.yml
I get the result. Error:
Code:

TASK [etcd : Configure | reload systemd] *******************************************************************************************************************************************************************************************************************************************************************************
ok: [node1]
ok: [node3]
ok: [node2]
Monday 13 March 2023  07:18:30 +0000 (0:00:01.168)      0:07:40.914 **********

TASK [etcd : Configure | Ensure etcd is running] ***********************************************************************************************************************************************************************************************************************************************************************
ok: [node1]
ok: [node2]
ok: [node3]
Monday 13 March 2023  07:18:30 +0000 (0:00:00.558)      0:07:41.472 **********
Monday 13 March 2023  07:18:30 +0000 (0:00:00.055)      0:07:41.528 **********
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (4 retries left).
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (3 retries left).
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (2 retries left).
FAILED - RETRYING: [node1]: Configure | Wait for etcd cluster to be healthy (1 retries left).

TASK [etcd : Configure | Wait for etcd cluster to be healthy] **********************************************************************************************************************************************************************************************************************************************************
fatal: [node1]: FAILED! => {"attempts": 4, "changed": false, "cmd": "set -o pipefail && /usr/local/bin/etcdctl endpoint --cluster status && /usr/local/bin/etcdctl endpoint --cluster health 2>&1 | grep -v 'Error: unhealthy cluster' >/dev/null", "delta": "0:00:05.031251", "end": "2023-03-13 03:19:17.721758", "msg": "non-zero return code", "rc": 1, "start": "2023-03-13 03:19:12.690507", "stderr": "{\"level\":\"warn\",\"ts\":\"2023-03-13T03:19:17.717-0400\",\"logger\":\"etcd-client\",\"caller\":\"v3@v3.5.6/retry_interceptor.go:62\",\"msg\":\"retrying of unary invoker failed\",\"target\":\"etcd-endpoints://0xc00001e000/192.168.122.233:2379\",\"attempt\":0,\"error\":\"rpc error: code = DeadlineExceeded desc = context deadline exceeded\"}\nFailed to get the status of endpoint https://192.168.122.120:2379 (context deadline exceeded)", "stderr_lines": ["{\"level\":\"warn\",\"ts\":\"2023-03-13T03:19:17.717-0400\",\"logger\":\"etcd-client\",\"caller\":\"v3@v3.5.6/retry_interceptor.go:62\",\"msg\":\"retrying of unary invoker failed\",\"target\":\"etcd-endpoints://0xc00001e000/192.168.122.233:2379\",\"attempt\":0,\"error\":\"rpc error: code = DeadlineExceeded desc = context deadline exceeded\"}", "Failed to get the status of endpoint https://192.168.122.120:2379 (context deadline exceeded)"], "stdout": "https://192.168.122.233:2379, 4dc4060cd0d7d06, 3.5.6, 20 kB, false, false, 3, 10, 10, ", "stdout_lines": ["https://192.168.122.233:2379, 4dc4060cd0d7d06, 3.5.6, 20 kB, false, false, 3, 10, 10, "]}

NO MORE HOSTS LEFT *****************************************************************************************************************************************************************************************************************************************************************************************************

PLAY RECAP *************************************************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0 
node1                      : ok=518  changed=26  unreachable=0    failed=1    skipped=611  rescued=0    ignored=0 
node2                      : ok=484  changed=26  unreachable=0    failed=0    skipped=528  rescued=0    ignored=0 
node3                      : ok=437  changed=22  unreachable=0    failed=0    skipped=506  rescued=0    ignored=0 

Monday 13 March 2023  07:19:17 +0000 (0:00:47.097)      0:08:28.625 **********
===============================================================================
etcd : Configure | Wait for etcd cluster to be healthy --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 47.10s
bootstrap-os : Install python-xml ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 44.31s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 34.17s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 21.84s
container-engine/containerd : download_file | Download item ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 17.07s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 15.86s
container-engine/crictl : download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 12.04s
etcd : Gen_certs | Write etcd member/admin and kube_control_plane client certs to other etcd nodes ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 11.50s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 11.48s
download : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.40s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 9.20s
container-engine/nerdctl : download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.90s
container-engine/runc : download_file | Download item ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.79s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 6.39s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 5.97s
etcd : Configure | Check if etcd cluster is healthy ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.51s
network_plugin/calico : Get current calico cluster version ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 4.08s
etcd : Gen_certs | Gather etcd member/admin and kube_control_plane client certs from first etcd node ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 3.59s
bootstrap-os : Install python3-cryptography --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.25s
container-engine/containerd : containerd | Unpack containerd archive -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.07s
qa@qa:~/kubespray$

Here is my host.yaml:

Code:

all:
  hosts:
    node1:
      ansible_host: 192.168.122.233
      ip: 192.168.122.233
      access_ip: 192.168.122.233
    node2:
      ansible_host: 192.168.122.120
      ip: 192.168.122.120
      access_ip: 192.168.122.120
    node3:
      ansible_host: 192.168.122.242
      ip: 192.168.122.242
      access_ip: 192.168.122.242
  children:
    kube_control_plane:
      hosts:
        node1:
        node2:
    kube_node:
      hosts:
        node1:
        node2:
        node3:
    etcd:
      hosts:
        node1:
        node2:
        node3:
    k8s_cluster:
      children:
        kube_control_plane:
        kube_node:
    calico_rr:
      hosts: {}

There is communication between the hosts.
Additional info :-
VMs - openSUSE
ssh & sudo - working fine
Kubespray version- v2.21.0-64
Client Version: v1.26.2
Kustomize Version: v4.5.7




Here's the solution (100% Works)
https://github.com/kubernetes-sigs/k...ay/issues/9884


All times are GMT -5. The time now is 08:29 AM.