* [PATCH] deploy docs to qemu-project.org from GitLab CI
@ 2021-01-19 13:26 Paolo Bonzini
2021-01-19 14:24 ` Daniel P. Berrangé
2021-01-19 14:56 ` Stefan Hajnoczi
0 siblings, 2 replies; 5+ messages in thread
From: Paolo Bonzini @ 2021-01-19 13:26 UTC (permalink / raw)
To: qemu-devel; +Cc: alex.bennee, stefanha
Currently, the website is rebuilt on qemu-project.org using
a separate container (https://github.com/stefanha/qemu-docs/)
cron job hook. We can instead reuse the GitLab's CI artifacts.
To do so, we use the same mechanism that is already in place for
qemu-web.git.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
.gitlab-ci.yml | 23 ++++++++++++++++++++++
tests/docker/dockerfiles/ubuntu2004.docker | 2 ++
2 files changed, 25 insertions(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 4532f1718a..729138064c 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -6,6 +6,7 @@ stages:
- containers-layer2
- build
- test
+ - update
include:
- local: '/.gitlab-ci.d/edk2.yml'
@@ -609,3 +610,25 @@ pages:
artifacts:
paths:
- public
+
+deploy:
+ image: $CI_REGISTRY_IMAGE/qemu/ubuntu2004:latest
+ stage: update
+ needs:
+ - job: pages
+ artifacts: true
+ before_script:
+ - eval $(ssh-agent -s)
+ - cat "$SSH_PRIVATE_KEY_FILE" | tr -d '\r' | ssh-add -
+ - mkdir -m700 -p ~/.ssh
+ - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config'
+ script:
+ - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && mkdir new-docs && rsync -az docs/ new-docs"
+ - rsync -avz --delete public/ $SSH_DEPLOY_DESTINATION:/var/www/qemu-project.org/new-docs
+ - ssh $SSH_DEPLOY_DESTINATION "cd /var/www/qemu-project.org && rm -rf old-docs && mv docs old-docs && mv new-docs docs"
+ only:
+ refs:
+ - master
+ variables:
+ - $SSH_PRIVATE_KEY_FILE
+ - $SSH_DEPLOY_DESTINATION
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index ae889d8482..2bb826c376 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -50,6 +50,7 @@ ENV PACKAGES flex bison \
make \
netcat-openbsd \
ninja-build \
+ openssh-client \
python3-numpy \
python3-opencv \
python3-pil \
@@ -58,6 +59,7 @@ ENV PACKAGES flex bison \
python3-venv \
python3-yaml \
rpm2cpio \
+ rsync \
sparse \
tesseract-ocr \
tesseract-ocr-eng \
--
2.29.2
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] deploy docs to qemu-project.org from GitLab CI
2021-01-19 13:26 [PATCH] deploy docs to qemu-project.org from GitLab CI Paolo Bonzini
@ 2021-01-19 14:24 ` Daniel P. Berrangé
2021-01-19 14:56 ` Stefan Hajnoczi
1 sibling, 0 replies; 5+ messages in thread
From: Daniel P. Berrangé @ 2021-01-19 14:24 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: alex.bennee, qemu-devel, stefanha
On Tue, Jan 19, 2021 at 02:26:19PM +0100, Paolo Bonzini wrote:
> Currently, the website is rebuilt on qemu-project.org using
> a separate container (https://github.com/stefanha/qemu-docs/)
> cron job hook. We can instead reuse the GitLab's CI artifacts.
>
> To do so, we use the same mechanism that is already in place for
> qemu-web.git.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> .gitlab-ci.yml | 23 ++++++++++++++++++++++
> tests/docker/dockerfiles/ubuntu2004.docker | 2 ++
> 2 files changed, 25 insertions(+)
> diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
> index ae889d8482..2bb826c376 100644
> --- a/tests/docker/dockerfiles/ubuntu2004.docker
> +++ b/tests/docker/dockerfiles/ubuntu2004.docker
> @@ -50,6 +50,7 @@ ENV PACKAGES flex bison \
> make \
> netcat-openbsd \
> ninja-build \
> + openssh-client \
> python3-numpy \
> python3-opencv \
> python3-pil \
> @@ -58,6 +59,7 @@ ENV PACKAGES flex bison \
> python3-venv \
> python3-yaml \
> rpm2cpio \
> + rsync \
Can we just stick to installing them in the deploy pre-script as for
the other job, as this dockerfile is going to be auto-generated with
just the real world QEMU dependancies present soon.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] deploy docs to qemu-project.org from GitLab CI
2021-01-19 13:26 [PATCH] deploy docs to qemu-project.org from GitLab CI Paolo Bonzini
2021-01-19 14:24 ` Daniel P. Berrangé
@ 2021-01-19 14:56 ` Stefan Hajnoczi
2021-01-19 15:00 ` Daniel P. Berrangé
2021-01-19 16:39 ` Paolo Bonzini
1 sibling, 2 replies; 5+ messages in thread
From: Stefan Hajnoczi @ 2021-01-19 14:56 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: alex.bennee, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1349 bytes --]
On Tue, Jan 19, 2021 at 02:26:19PM +0100, Paolo Bonzini wrote:
> Currently, the website is rebuilt on qemu-project.org using
> a separate container (https://github.com/stefanha/qemu-docs/)
> cron job hook. We can instead reuse the GitLab's CI artifacts.
>
> To do so, we use the same mechanism that is already in place for
> qemu-web.git.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> .gitlab-ci.yml | 23 ++++++++++++++++++++++
> tests/docker/dockerfiles/ubuntu2004.docker | 2 ++
> 2 files changed, 25 insertions(+)
Hmm...the UNIX account on qemu.org is locked down to some extent but I
don't feel comfortable with a GitLab CI job sshing into qemu.org.
ssh access aside, we are publishing HTML from a shared CI runner to
qemu.org. Effectively we are allowing an untrusted machine to publish
HTML/JS/CSS on qemu.org. It could steal HTTP Cookies or do other
malicious things. That is less of a problem when there is a dedicated
subdomain so that the Same Origin policy can provide isolation. Maybe
there are more recent web security mechanisms that allow us to define a
policy so browsers do not treat qemu.org/docs/* the same as other
qemu.org pages?
(This wasn't a problem before since the container was running on a
dedicated instance under our control.)
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] deploy docs to qemu-project.org from GitLab CI
2021-01-19 14:56 ` Stefan Hajnoczi
@ 2021-01-19 15:00 ` Daniel P. Berrangé
2021-01-19 16:39 ` Paolo Bonzini
1 sibling, 0 replies; 5+ messages in thread
From: Daniel P. Berrangé @ 2021-01-19 15:00 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: Paolo Bonzini, alex.bennee, qemu-devel
On Tue, Jan 19, 2021 at 02:56:22PM +0000, Stefan Hajnoczi wrote:
> On Tue, Jan 19, 2021 at 02:26:19PM +0100, Paolo Bonzini wrote:
> > Currently, the website is rebuilt on qemu-project.org using
> > a separate container (https://github.com/stefanha/qemu-docs/)
> > cron job hook. We can instead reuse the GitLab's CI artifacts.
> >
> > To do so, we use the same mechanism that is already in place for
> > qemu-web.git.
> >
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> > .gitlab-ci.yml | 23 ++++++++++++++++++++++
> > tests/docker/dockerfiles/ubuntu2004.docker | 2 ++
> > 2 files changed, 25 insertions(+)
>
> Hmm...the UNIX account on qemu.org is locked down to some extent but I
> don't feel comfortable with a GitLab CI job sshing into qemu.org.
>
> ssh access aside, we are publishing HTML from a shared CI runner to
> qemu.org. Effectively we are allowing an untrusted machine to publish
> HTML/JS/CSS on qemu.org. It could steal HTTP Cookies or do other
> malicious things. That is less of a problem when there is a dedicated
> subdomain so that the Same Origin policy can provide isolation. Maybe
> there are more recent web security mechanisms that allow us to define a
> policy so browsers do not treat qemu.org/docs/* the same as other
> qemu.org pages?
The "easy" option is to just stop using qemu.org/docs and instad hav
docs.qemu.org and make it a cname for qemu-project.gitlab.io. Then
gitlab can be serving the docs directly.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] deploy docs to qemu-project.org from GitLab CI
2021-01-19 14:56 ` Stefan Hajnoczi
2021-01-19 15:00 ` Daniel P. Berrangé
@ 2021-01-19 16:39 ` Paolo Bonzini
1 sibling, 0 replies; 5+ messages in thread
From: Paolo Bonzini @ 2021-01-19 16:39 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: alex.bennee, qemu-devel
On 19/01/21 15:56, Stefan Hajnoczi wrote:
> Hmm...the UNIX account on qemu.org is locked down to some extent but I
> don't feel comfortable with a GitLab CI job sshing into qemu.org.
As you say, the qemu-deploy account on qemu.org is limited to writing to
/var/www/qemu-project.org. Its own home directory is also limited with
"chattr +i".
The same CI runners are already using the qemu-deploy user to deploy the
website itself. (To state the obvious, you can only do this if you can
push to the qemu-project GitLab organization. Regular users can
configure their fork to deploy to a different server using a different
ssh private key, but their CI jobs won't touch qemu-project.org).
There are other ways to do defense in depth.
We could use https://www.hashicorp.com/cloud-platform for the ssh
private key. Right now the ssh private key (which of course only grants
access to the qemu-deploy user) is accessible to everyone with
administrator access to the QEMU GitLab project; a Vault instance could
have more limited access.
With respect to the ssh private key, however, a bigger risk factor is
that a botched (even if not malicious) patch can reach the QEMU or
qemu-web git repositories, causing the private key to appear in public
CI logs. To mitigate this we could set up a restricted bash for the
qemu-deploy user on qemu.org. It would require small changes to
gitlab-ci.yml to avoid the "cd" command, as well as configuring a
restricted PATH via ~/.ssh/environment, but overall it would be easy.
It would also protect against a malicious actor sneaking in a patch to
gitlab-ci.yml that makes it do bad things.
Neither of these has to be done now. The current way to do things is
more or less what GitLab recommends so, security-wise, it's not entirely
broken.
> ssh access aside, we are publishing HTML from a shared CI runner to
> qemu.org. Effectively we are allowing an untrusted machine to publish
> HTML/JS/CSS on qemu.org. It could steal HTTP Cookies or do other
> malicious things.
Note that we don't use cookies on www.qemu.org and don't have a CORS
policy either. Only wiki.qemu.org uses cookies.
Paolo
> That is less of a problem when there is a dedicated
> subdomain so that the Same Origin policy can provide isolation. Maybe
> there are more recent web security mechanisms that allow us to define a
> policy so browsers do not treat qemu.org/docs/* the same as other
> qemu.org pages?
>
> (This wasn't a problem before since the container was running on a
> dedicated instance under our control.)
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-01-19 18:44 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-19 13:26 [PATCH] deploy docs to qemu-project.org from GitLab CI Paolo Bonzini
2021-01-19 14:24 ` Daniel P. Berrangé
2021-01-19 14:56 ` Stefan Hajnoczi
2021-01-19 15:00 ` Daniel P. Berrangé
2021-01-19 16:39 ` Paolo Bonzini
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).