From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E312C4338F for ; Mon, 2 Aug 2021 13:00:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 12D2060E09 for ; Mon, 2 Aug 2021 13:00:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 12D2060E09 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:41922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mAXYp-0004d5-0x for qemu-devel@archiver.kernel.org; Mon, 02 Aug 2021 09:00:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60326) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mAXXn-0003P6-C4 for qemu-devel@nongnu.org; Mon, 02 Aug 2021 08:59:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28852) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mAXXk-0007Zp-L7 for qemu-devel@nongnu.org; Mon, 02 Aug 2021 08:59:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1627909183; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OcztPFBK2AhJgcTr3/SnEyRXZ2uYFidxiN1+BXeFM4s=; b=IuqiB4Z3CAkTT8Aa5Rf/cKxm1cjkdTl4Doik7hB/P7Lsjyuggj6OryOyle+95M7a34lxJU cBHkF7R2+680NJ6hq5uilZTxbue6MJzVtUtoQ/mo+AK6U3Ai1xTZ/qqb3+YygnqtUsOLxO k3hOM34ydeR9zz1RifIUi8K6EKZd51M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-14-Glrw6o70NXyK_Sq9HnZ1Rg-1; Mon, 02 Aug 2021 08:59:42 -0400 X-MC-Unique: Glrw6o70NXyK_Sq9HnZ1Rg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0705A10082F6; Mon, 2 Aug 2021 12:59:21 +0000 (UTC) Received: from redhat.com (unknown [10.39.194.210]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8985C19C44; Mon, 2 Aug 2021 12:59:19 +0000 (UTC) Date: Mon, 2 Aug 2021 13:59:16 +0100 From: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= To: Alex =?utf-8?Q?Benn=C3=A9e?= Subject: Re: "make check-acceptance" takes way too long Message-ID: References: <87bl6gggb2.fsf@linaro.org> MIME-Version: 1.0 In-Reply-To: <87bl6gggb2.fsf@linaro.org> User-Agent: Mutt/2.0.7 (2021-05-04) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.701, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= Cc: Peter Maydell , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= , QEMU Developers , Cleber Rosa Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Mon, Aug 02, 2021 at 01:47:37PM +0100, Alex Bennée wrote: > > Daniel P. Berrangé writes: > > > On Fri, Jul 30, 2021 at 04:12:27PM +0100, Peter Maydell wrote: > >> "make check-acceptance" takes way way too long. I just did a run > >> on an arm-and-aarch64-targets-only debug build and it took over > >> half an hour, and this despite it skipping or cancelling 26 out > >> of 58 tests! > >> > >> I think that ~10 minutes runtime is reasonable. 30 is not; > >> ideally no individual test would take more than a minute or so. > >> > >> Output saying where the time went. The first two tests take > >> more than 10 minutes *each*. I think a good start would be to find > >> a way of testing what they're testing that is less heavyweight. > > > > While there is certainly value in testing with a real world "full" guest > > OS, I think it is overkill as the default setup. I reckon we would get > > 80-90% of the value, by making our own test image repo, containing minimal > > kernel builds for each machine/target combo we need, together with a tiny > > initrd containing busybox. This could easily be made to boot in 1 second, > > which would make 'make check-acceptance' waaaaay faster, considering how > > many times we boot a guest. This would also solve our problem that we're > > pointing to URLs to download these giant images, and they're frequently > > break URLs. > > It's been discussed before but previously the worry has been the hassle > of maintaining such images along with such tediousness as ensuring GPL > compliance. We've outsourced that problem to the upstream. I don't recall discussing that directly - only discussions around hosting images / files from other distros on our own infra, that does indeed create a compliance burden. This is why I suggested /strictly/ nothing more than kernel+busybox built from source ourselves, probably using debian cross compilers. The busybox stuff would only need to be built once per architecture. The kernel would potentially need more builds to cope with machine board specific configs. We would not need to continually track new releases - we can fix on specific kernel + busybox versions for as long as they cope with the targets/archs we need. I'd expect it all to be done in a gitlab repo, with a CI job to publish the results, never any manual builds, so that we ensure license compliance. Of course the main problem is someone doing the leg work to get such a system up & running initially to prove the concept. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|