From: "Bruce Ashfield" <bruce.ashfield@gmail.com>
To: Bruce Ashfield <bruce.ashfield@gmail.com>
Cc: Lance Yang <Lance.Yang@arm.com>,
Joakim Roubert <joakim.roubert@axis.com>,
"meta-virtualization@yoctoproject.org"
<meta-virtualization@yoctoproject.org>,
Michael Zhao <Michael.Zhao@arm.com>, Kaly Xin <Kaly.Xin@arm.com>
Subject: Re: [meta-virtualization][PATCH v5] Adding k3s recipe
Date: Tue, 10 Nov 2020 08:17:06 -0500 [thread overview]
Message-ID: <CADkTA4PtR2gjBdimC6-JDqVr4=B82YrbwWZrFfh2vzegsOhF7Q@mail.gmail.com> (raw)
In-Reply-To: <16462648E2B320A8.24110@lists.yoctoproject.org>
On Tue, Nov 10, 2020 at 7:46 AM Bruce Ashfield via
lists.yoctoproject.org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> On Tue, Nov 10, 2020 at 1:43 AM Lance Yang <Lance.Yang@arm.com> wrote:
> >
> > Hi Bruce and Joakim,
> >
> > Thanks for sharing this branch: k3s-wip. I have tested against my yocto build.
>
> The branch will be more functional shortly, I have quite a few changes
> to factor things for
> k8s and generally more usable :D
>
> modified: classes/cni_networking.bbclass
> modified: conf/layer.conf
> modified: recipes-containers/containerd/containerd-docker_git.bb
> modified:
> recipes-containers/containerd/containerd-opencontainers_git.bb
> modified: recipes-containers/k3s/README.md
> modified: recipes-containers/k3s/k3s_git.bb
> modified: recipes-kernel/linux/linux-yocto/kubernetes.cfg
> modified: recipes-networking/cni/cni_git.bb
> container-deploy.txt
> recipes-core/packagegroups/
>
> >
> > My Image: Linux qemuarm64 by yocto.
> >
> > The master node can be ready after I started the k3s server. However, the pods in kube-system (which are essential components for k3s) cannot turn to ready state on qemuarm64.
> >
>
> That's interesting, since in my configuration, the master never comes ready:
>
> root@qemux86-64:~# kubectl get nodes
> NAME STATUS ROLES AGE VERSION
> qemux86-64 NotReady master 15h v1.18.9-k3s1
>
Hah.
I finally got the node to show up as ready:
root@qemux86-64:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
qemux86-64 Ready master 112s v1.18.9-k3s1
I'm attempting to build an all-in-one node, and that is likely causing
me some issues.
I'm revisiting those potential conflicts now.
But if anyone else does have an all in one working and has some tips,
feel free to share :D
Bruce
> I've sorted out more of the dependencies, and have packagegroups to
> make them easier
> now.
>
> Hopefully, I can figure out what is now missing and keeping my master
> from moving into
> ready today.
>
> Bruce
>
> > After the master node itself turned to ready state, I check the pods with kubectl:
> >
> > kubectl get nodes
> > NAME STATUS ROLES AGE VERSION
> > qemuarm64 Ready master 11m v1.18.9-k3s1
> > root@qemuarm64:~# ls
> > root@qemuarm64:~# kubectl get pods -n kube-system
> > NAME READY STATUS RESTARTS AGE
> > local-path-provisioner-6d59f47c7-xxvbl 0/1 ContainerCreating 0 12m
> > coredns-7944c66d8d-tlrm9 0/1 ContainerCreating 0 12m
> > metrics-server-7566d596c8-svkff 0/1 ContainerCreating 0 12m
> > helm-install-traefik-s8p5g 0/1 ContainerCreating 0 12m
> >
> > Then I describe the pods with:
> >
> > Events:
> > Type Reason Age From Message
> > ---- ------ ---- ---- -------
> > Normal Scheduled 16m default-scheduler Successfully assigned kube-system/coredns-7944c66d8d-tlrm9 to qemuarm64
> > Warning FailedMount 5m23s (x3 over 14m) kubelet Unable to attach or mount volumes: unmounted volumes=[coredns-token-b7nlh], unattached volumes=[coredns-token-b7nlh config-volume]: timed out waiting for the condition
> > Warning FailedMount 50s (x4 over 12m) kubelet Unable to attach or mount volumes: unmounted volumes=[coredns-token-b7nlh], unattached volumes=[config-volume coredns-token-b7nlh]: timed out waiting for the condition
> > Warning FailedMount 11s (x16 over 16m) kubelet MountVolume.SetUp failed for volume "coredns-token-b7nlh" : mount failed: exec: "mount": executable file not found in $PATH
> >
> > I found the "mount" binary is not found in $PATH. However, I confirmed the $PATH and mount binary on my qemuarm64 image:
> >
> > root@qemuarm64:~# echo $PATH
> > /usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
> > root@qemuarm64:~# which mount
> > /bin/mount
> >
> > When I type mount command, it worked fine:
> >
> > /dev/root on / type ext4 (rw,relatime)
> > devtmpfs on /dev type devtmpfs (rw,relatime,size=2016212k,nr_inodes=504053,mode=755)
> > proc on /proc type proc (rw,relatime)
> > sysfs on /sys type sysfs (rw,relatime)
> > debugfs on /sys/kernel/debug type debugfs (rw,relatime)
> > tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
> > ...
> > ... (skipped the verbose output)
> >
> > I would like to know whether you have met this "mount" issue ever?
> >
> > Best Regards,
> > Lance
> >
> > > -----Original Message-----
> > > From: meta-virtualization@lists.yoctoproject.org <meta-virtualization@lists.yoctoproject.org>
> > > On Behalf Of Bruce Ashfield via lists.yoctoproject.org
> > > Sent: Monday, October 26, 2020 11:46 PM
> > > To: Joakim Roubert <joakim.roubert@axis.com>
> > > Cc: meta-virtualization@yoctoproject.org
> > > Subject: Re: [meta-virtualization][PATCH v5] Adding k3s recipe
> > >
> > > On Wed, Oct 21, 2020 at 2:00 AM Joakim Roubert <joakim.roubert@axis.com> wrote:
> > > >
> > > > On 2020-10-21 05:10, Bruce Ashfield wrote:
> > > > > Ha!!!!
> > > > >
> > > > > This applies.
> > > >
> > > > Wonderful, thank you! I guess this is what is called "five times lucky"...
> > > >
> > > > > I'm now testing and completing some of my networking factoring, as
> > > > > well as importing / forking some recipes to avoid extra layer
> > > > > depends.
> > > >
> > > > Excellent!
> > >
> > > I've pushed some of my WIP to:
> > > https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/log/?h=k3s-wip
> > >
> > > That includes the split of the networking, the import of some of the dependencies and some
> > > small tweaks I'm working on.
> > >
> > > I did have a couple of questions on the k3s packaging itself, I was getting the following
> > > error:
> > >
> > > ERROR: k3s-v1.18.9+k3s1-dirty-r0 do_package: QA Issue: k3s:
> > > Files/directories were installed but not shipped in any package:
> > > /usr/local/bin/k3s-clean
> > > /usr/local/bin/crictl
> > > /usr/local/bin/kubectl
> > > /usr/local/bin/k3s
> > >
> > > So I added them to the FILES of the k3s package itself (so both k3s-server and k3s-agent will get
> > > them), is that the split you were looking for ?
> > >
> > > Bruce
> > >
> > > >
> > > > BR,
> > > >
> > > > /Joakim
> > > > --
> > > > Joakim Roubert
> > > > Senior Engineer
> > > >
> > > > Axis Communications AB
> > > > Emdalavägen 14, SE-223 69 Lund, Sweden
> > > > Tel: +46 46 272 18 00, Tel (direct): +46 46 272 27 48
> > > > Fax: +46 46 13 61 30, www.axis.com
> > > >
> > > --
> > > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
> > > - "Use the force Harry" - Gandalf, Star Trek II
> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
next prev parent reply other threads:[~2020-11-10 13:17 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20200821205529.29901-1-erik.jansson@axis.com>
2020-09-21 8:38 ` [meta-virtualization][PATCH] Adding k3s recipe Joakim Roubert
2020-09-21 11:11 ` Bruce Ashfield
2020-09-21 13:15 ` Joakim Roubert
2020-09-24 14:02 ` Bruce Ashfield
2020-09-24 14:46 ` Joakim Roubert
2020-09-24 15:41 ` Bruce Ashfield
2020-09-25 6:20 ` Joakim Roubert
2020-09-25 13:12 ` Bruce Ashfield
2020-09-25 13:50 ` Joakim Roubert
[not found] ` <16380B0CA000AB98.28124@lists.yoctoproject.org>
2020-09-28 13:48 ` Joakim Roubert
2020-09-29 19:58 ` Bruce Ashfield
2020-09-30 8:12 ` Joakim Roubert
[not found] ` <1639818C3E50A226.8589@lists.yoctoproject.org>
2020-09-30 8:14 ` Joakim Roubert
2020-10-01 10:32 ` Joakim Roubert
[not found] ` <1639D7B9311FC65C.18704@lists.yoctoproject.org>
2020-10-01 10:32 ` Joakim Roubert
2020-10-14 16:38 ` Bruce Ashfield
2020-10-15 11:40 ` Joakim Roubert
2020-10-15 11:47 ` [meta-virtualization][PATCH v4] " Joakim Roubert
2020-10-15 15:02 ` Bruce Ashfield
2020-10-20 11:14 ` [meta-virtualization][PATCH v5] " Joakim Roubert
2020-10-21 3:10 ` Bruce Ashfield
2020-10-21 6:00 ` Joakim Roubert
2020-10-26 15:46 ` Bruce Ashfield
2020-10-28 8:32 ` Joakim Roubert
2020-11-06 21:20 ` Bruce Ashfield
2020-11-09 7:48 ` Joakim Roubert
2020-11-09 9:26 ` Lance.Yang
2020-11-09 13:45 ` Bruce Ashfield
2020-11-10 8:45 ` Lance Yang
2020-11-09 13:44 ` Bruce Ashfield
2020-11-10 6:43 ` Lance Yang
2020-11-10 12:46 ` Bruce Ashfield
[not found] ` <16462648E2B320A8.24110@lists.yoctoproject.org>
2020-11-10 13:17 ` Bruce Ashfield [this message]
2020-11-12 7:30 ` Lance Yang
2020-11-12 13:38 ` Bruce Ashfield
2020-11-12 14:26 ` [meta-virtualization][PATCH] k3s: Update README.md Joakim Roubert
2020-11-17 12:39 ` [meta-virtualization][PATCH] k3s: Bump to v1.19.3+k3s3 Joakim Roubert
2020-11-17 13:27 ` Bruce Ashfield
2020-11-17 13:31 ` Joakim Roubert
2020-11-17 13:40 ` Bruce Ashfield
2020-11-17 13:50 ` Joakim Roubert
2020-11-17 14:15 ` Bruce Ashfield
[not found] ` <16485135E3A12798.28066@lists.yoctoproject.org>
2020-11-17 14:19 ` Bruce Ashfield
2020-11-17 14:27 ` Joakim Roubert
2020-11-17 14:41 ` Bruce Ashfield
[not found] ` <1648529A6FD37D30.5807@lists.yoctoproject.org>
2020-11-17 19:39 ` Bruce Ashfield
2020-11-18 18:27 ` Joakim Roubert
2020-11-18 20:38 ` Bruce Ashfield
2020-12-11 6:31 ` Lance Yang
2020-12-11 13:43 ` Bruce Ashfield
2020-12-15 9:56 ` Lance Yang
2020-12-15 18:58 ` Bruce Ashfield
2020-12-18 14:23 ` Joakim Roubert
2020-12-22 16:15 ` Bruce Ashfield
2021-01-04 7:12 ` Joakim Roubert
2021-01-04 13:40 ` Bruce Ashfield
[not found] ` <16570B29E8680DE8.14857@lists.yoctoproject.org>
2021-01-05 13:58 ` Bruce Ashfield
[not found] ` <16484BFA14ED0B17.5807@lists.yoctoproject.org>
2020-11-17 13:05 ` Joakim Roubert
2020-11-12 13:43 ` [meta-virtualization][PATCH v5] Adding k3s recipe Joakim Roubert
2020-11-13 5:48 ` Lance Yang
2020-11-13 6:20 ` Bruce Ashfield
2020-11-12 13:40 ` Joakim Roubert
[not found] ` <164627F27D18DB55.10479@lists.yoctoproject.org>
2020-11-10 13:34 ` Bruce Ashfield
2020-11-11 10:06 ` Lance Yang
2020-11-11 13:40 ` Bruce Ashfield
2020-11-12 7:04 ` Lance Yang
2020-11-12 13:40 ` Bruce Ashfield
2020-11-12 14:07 ` Lance Yang
2020-11-17 14:13 ` Joakim Roubert
2021-03-13 19:30 ` Bruce Ashfield
2021-03-14 4:32 ` Yocto
2021-03-15 9:46 ` Joakim Roubert
2020-10-13 12:22 ` [meta-virtualization][PATCH] " Bruce Ashfield
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CADkTA4PtR2gjBdimC6-JDqVr4=B82YrbwWZrFfh2vzegsOhF7Q@mail.gmail.com' \
--to=bruce.ashfield@gmail.com \
--cc=Kaly.Xin@arm.com \
--cc=Lance.Yang@arm.com \
--cc=Michael.Zhao@arm.com \
--cc=joakim.roubert@axis.com \
--cc=meta-virtualization@yoctoproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.