From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C501C433B4 for ; Wed, 19 May 2021 19:15:40 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 21B3461355 for ; Wed, 19 May 2021 19:15:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21B3461355 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56250 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljRfP-0003Ff-6l for qemu-devel@archiver.kernel.org; Wed, 19 May 2021 15:15:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55270) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR73-00061u-NY for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34283) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljR6w-0001z9-2w for qemu-devel@nongnu.org; Wed, 19 May 2021 14:40:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621449597; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cfw1pJJnQNieEEbqPWCtUB8RGpx680d1YC/REGKv2AE=; b=N1mzkaVNKcrmfWxm2K5T1LW4b3brkZTDy/rl4QEM3M0S1aT8gJNcJqD7Vos3MO7/Vptv1e PEp8O+XS1kJtkJHv1u2+zk1oMxwlt23LhY9ZqMYfu0irqxYGiVrNCMBRAe+7bVVZtsdlI1 qfHt3qXmokauTJuaYgB7XRxXbpVXg7Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-489-t7DRotyuPOykSZ8fmWvw4g-1; Wed, 19 May 2021 14:39:55 -0400 X-MC-Unique: t7DRotyuPOykSZ8fmWvw4g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 64C5C501F4; Wed, 19 May 2021 18:39:54 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id A7CD95944C; Wed, 19 May 2021 18:39:53 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 01/15] qapi/parser: Don't try to handle file errors Date: Wed, 19 May 2021 14:39:37 -0400 Message-Id: <20210519183951.3946870-2-jsnow@redhat.com> In-Reply-To: <20210519183951.3946870-1-jsnow@redhat.com> References: <20210519183951.3946870-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII" Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , Cleber Rosa , John Snow , Eduardo Habkost , Markus Armbruster Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Fixes: f5d4361cda Fixes: 52a474180a Fixes: 46f49468c6 Remove the try/except block that handles file-opening errors in QAPISchemaParser.__init__() and add one each to QAPISchemaParser._include() and QAPISchema.__init__() respectively. This simultaneously fixes the typing of info.fname (f5d4361cda), A static typing violation in test-qapi (46f49468c6), and a regression of an error message (52a474180a). The short-ish version of what motivates this patch is: - It's hard to write a good error message in the init method, because we need to determine the context of our caller to do so. It's easier to just let the caller write the message. - We don't want to allow QAPISourceInfo(None, None, None) to exist. The typing introduced by commit f5d4361cda types the 'fname' field as (non-optional) str, which was premature until the removal of this construct. - Errors made using such an object are currently incorrect (since 52a474180a) - It's not technically a semantic error if we cannot open the schema. - There are various typing constraints that make mixing these two cases undesirable for a single special case. - test-qapi's code handling an fname of 'None' is now dead, drop it. Additionally, Not all QAPIError objects have an 'info' field (since 46f49468), so deleting this stanza corrects a typing oversight in test-qapi introduced by that commit. Other considerations: - open() is moved to a 'with' block to ensure file pointers are cleaned up deterministically. - Python 3.3 deprecated IOError and made it a synonym for OSError. Avoid the misleading perception these exception handlers are narrower than they really are. The long version: The error message here is incorrect (since commit 52a474180a): > python3 qapi-gen.py 'fake.json' qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file or directory In pursuing it, we find that QAPISourceInfo has a special accommodation for when there's no filename. Meanwhile, the intent when QAPISourceInfo was typed (f5d4361cda) was non-optional 'str'. This usage was overlooked. To remove this, I'd want to avoid having a "fake" QAPISourceInfo object. I also don't want to explicitly begin accommodating QAPISourceInfo itself being None, because we actually want to eventually prove that this can never happen -- We don't want to confuse "The file isn't open yet" with "This error stems from a definition that wasn't defined in any file". (An earlier series tried to create a dummy info object, but it was tough to prove in review that it worked correctly without creating new regressions. This patch avoids that distraction. We would like to first prove that we never raise QAPISemError for any built-in object before we add "special" info objects. We aren't ready to do that yet.) So, which way out of the labyrinth? Here's one way: Don't try to handle errors at a level with "mixed" semantic contexts; i.e. don't mix inclusion errors (should report a source line where the include was triggered) and command line errors (where we specified a file we couldn't read). Remove the error handling from the initializer of the parser. Pythonic! Now it's the caller's job to figure out what to do about it. Handle the error in QAPISchemaParser._include() instead, where we can write a targeted error message where we are guaranteed to have an 'info' context to report with. The root level error can similarly move to QAPISchema.__init__(), where we know we'll never have an info context to report with, so we use a more abstract error type. Now the error looks sensible again: > python3 qapi-gen.py 'fake.json' qapi-gen.py: can't read schema file 'fake.json': No such file or directory With these error cases separated, QAPISourceInfo can be solidified as never having placeholder arguments that violate our desired types. Clean up test-qapi along similar lines. Signed-off-by: John Snow --- scripts/qapi/parser.py | 18 +++++++++--------- scripts/qapi/schema.py | 11 +++++++++-- scripts/qapi/source.py | 3 --- tests/qapi-schema/test-qapi.py | 3 --- 4 files changed, 18 insertions(+), 17 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index ca5e8e18e00..a53b735e7de 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -40,15 +40,9 @@ def __init__(self, fname, previously_included=None, incl_info=None): previously_included = previously_included or set() previously_included.add(os.path.abspath(fname)) - try: - fp = open(fname, 'r', encoding='utf-8') + # May raise OSError; allow the caller to handle it. + with open(fname, 'r', encoding='utf-8') as fp: self.src = fp.read() - except IOError as e: - raise QAPISemError(incl_info or QAPISourceInfo(None, None, None), - "can't read %s file '%s': %s" - % ("include" if incl_info else "schema", - fname, - e.strerror)) if self.src == '' or self.src[-1] != '\n': self.src += '\n' @@ -129,7 +123,13 @@ def _include(self, include, info, incl_fname, previously_included): if incl_abs_fname in previously_included: return None - return QAPISchemaParser(incl_fname, previously_included, info) + try: + return QAPISchemaParser(incl_fname, previously_included, info) + except OSError as err: + raise QAPISemError( + info, + f"can't read include file '{incl_fname}': {err.strerror}" + ) from err def _check_pragma_list_of_str(self, name, value, info): if (not isinstance(value, list) diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py index 3a4172fb749..d1d27ff7ee8 100644 --- a/scripts/qapi/schema.py +++ b/scripts/qapi/schema.py @@ -20,7 +20,7 @@ from typing import Optional from .common import POINTER_SUFFIX, c_name -from .error import QAPISemError, QAPISourceError +from .error import QAPIError, QAPISemError, QAPISourceError from .expr import check_exprs from .parser import QAPISchemaParser @@ -849,7 +849,14 @@ def visit(self, visitor): class QAPISchema: def __init__(self, fname): self.fname = fname - parser = QAPISchemaParser(fname) + + try: + parser = QAPISchemaParser(fname) + except OSError as err: + raise QAPIError( + f"can't read schema file '{fname}': {err.strerror}" + ) from err + exprs = check_exprs(parser.exprs) self.docs = parser.docs self._entity_list = [] diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py index 03b6ede0828..1ade864d7b9 100644 --- a/scripts/qapi/source.py +++ b/scripts/qapi/source.py @@ -10,7 +10,6 @@ # See the COPYING file in the top-level directory. import copy -import sys from typing import List, Optional, TypeVar @@ -53,8 +52,6 @@ def next_line(self: T) -> T: return info def loc(self) -> str: - if self.fname is None: - return sys.argv[0] ret = self.fname if self.line is not None: ret += ':%d' % self.line diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py index e8db9d09d91..f1c4deb9a51 100755 --- a/tests/qapi-schema/test-qapi.py +++ b/tests/qapi-schema/test-qapi.py @@ -128,9 +128,6 @@ def test_and_diff(test_name, dir_name, update): try: test_frontend(os.path.join(dir_name, test_name + '.json')) except QAPIError as err: - if err.info.fname is None: - print("%s" % err, file=sys.stderr) - return 2 errstr = str(err) + '\n' if dir_name: errstr = errstr.replace(dir_name + '/', '') -- 2.30.2