From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1m5x4w-0007Yk-Em for mharc-grub-devel@gnu.org; Tue, 20 Jul 2021 17:15:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46348) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5x4u-0007Vi-Gk for grub-devel@gnu.org; Tue, 20 Jul 2021 17:15:00 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:17420 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5x4s-0001tQ-I5 for grub-devel@gnu.org; Tue, 20 Jul 2021 17:15:00 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16KL4Etx039823; Tue, 20 Jul 2021 17:14:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=MjfDvV9NSM/vIXq9RWYrwXrvzfCsoP35QvE2Q1zbpxM=; b=Y8YrklIskbvy6l9phYAlPt/cW2HGSYVV4xAJDcFqGel8xXZq6tq/z1VOgNvUYORNBEBd JsjaFbfpLUKvPBcILp0VlI00ilGpP9GQmtr5ifteaq4R7bmOc3UZYDnEhnBDp+Q5Mqwh vrBv5XXT82RCXyOMHrD7NW2zNpJVBlKrv7bAUrwtQXBrY7ccCf0v/7Xgc/zqOmOgVEfP wpzdgx1n0Y8jYFNQNJQlnI1vp7yhBIbxnkTv2t9UFPUC2GA+UPT3fCNQRreukGD+YIbz LoO4C5Cs5QPsojYgsPJGV6tRSz2QQ9WvW6Rc7BAvKh6HCEpSotugr+e6nGZEfmcuDU9P ig== Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0b-001b2d01.pphosted.com with ESMTP id 39wwg6htyc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 20 Jul 2021 17:14:54 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 16KLCkTk016570; Tue, 20 Jul 2021 21:14:53 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma04dal.us.ibm.com with ESMTP id 39upuce4qx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 20 Jul 2021 21:14:53 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 16KLEqdp36569574 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Jul 2021 21:14:52 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F169FC605D; Tue, 20 Jul 2021 21:14:51 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB085C6059; Tue, 20 Jul 2021 21:14:51 +0000 (GMT) Received: from sbct-2.. (unknown [9.47.158.152]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Tue, 20 Jul 2021 21:14:51 +0000 (GMT) From: Stefan Berger To: grub-devel@gnu.org Cc: dkiper@net-space.pl, Daniel Axtens , Stefan Berger Subject: [PATCH v2 3/4] ieee1275: request memory with ibm, client-architecture-support Date: Tue, 20 Jul 2021 17:14:48 -0400 Message-Id: <20210720211449.572356-4-stefanb@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210720211449.572356-1-stefanb@linux.vnet.ibm.com> References: <20210720211449.572356-1-stefanb@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 6UNyRhmX47mWevLt4yorWuvZECISdqWM X-Proofpoint-ORIG-GUID: 6UNyRhmX47mWevLt4yorWuvZECISdqWM X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-07-20_13:2021-07-19, 2021-07-20 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 phishscore=0 impostorscore=0 suspectscore=0 mlxlogscore=999 malwarescore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 priorityscore=1501 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107200131 Received-SPF: none client-ip=148.163.158.5; envelope-from=stefanb@linux.vnet.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: 0 X-Spam_score: -0.1 X-Spam_bar: / X-Spam_report: (-0.1 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2021 21:15:00 -0000 From: Daniel Axtens On PowerVM, the first time we boot a Linux partition, we may only get 256MB of real memory area, even if the partition has more memory. This isn't really enough. Fortunately, the Power Architecture Platform Reference (PAPR) defines a method we can call to ask for more memory. This is part of the broad and powerful ibm,client-architecture-support (CAS) method. CAS can do an enormous amount of things on a PAPR platform: as well as asking for memory, you can set the supported processor level, the interrupt controller, hash vs radix mmu, and so on. We want to touch as little of this as possible because we don't want to step on the toes of the future OS. If: - we are running under what we think is PowerVM (compatible property of / begins with "IBM"), and - the full amount of RMA is less than 512MB (as determined by the reg property of /memory) then call CAS as follows: (refer to the Linux on Power Architecture Reference, LoPAR, which is public, at B.5.2.3): - Use the "any" PVR value and supply 2 option vectors. - Set option vector 1 (PowerPC Server Processor Architecture Level) to "ignore". - Set option vector 2 with default or Linux-like options, including a min-rma-size of 512MB. This will cause a CAS reboot and the partition will restart with 512MB of RMA. Grub will notice the 512MB and not call CAS again. (A partition can be configured with only 256MB of memory, which would mean this request couldn't be satisfied, but PFW refuses to load with only 256MB of memory, so it's a bit moot. SLOF will run fine with 256MB, but we will never call CAS under qemu/SLOF because /compatible won't begin with "IBM".) One of the first things Linux does while still running under OpenFirmware is to call CAS with a much fuller set of options (including asking for 512MB of memory). This includes a much more restrictive set of PVR values and processor support levels, and this will induce another reboot. On this reboot grub will again notice the higher RMA, and not call CAS. We will get to Linux, Linux will call CAS but because the values are now set for Linux this will not induce another CAS reboot and we will finally boot. On all subsequent boots, everything will be configured with 512MB of RMA and all the settings Linux likes, so there will be no further CAS reboots. (phyp is super sticky with the RMA size - it persists even on cold boots. So if you've ever booted Linux in a partition, you'll probably never have grub call CAS. It'll only ever fire the first time a partition loads grub, or if you deliberately lower the amount of memory your partition has below 512MB.) Signed-off-by: Daniel Axtens Tested-by: Stefan Berger --- grub-core/kern/ieee1275/cmain.c | 3 + grub-core/kern/ieee1275/init.c | 144 ++++++++++++++++++++++++++++++- include/grub/ieee1275/ieee1275.h | 6 ++ 3 files changed, 151 insertions(+), 2 deletions(-) diff --git a/grub-core/kern/ieee1275/cmain.c b/grub-core/kern/ieee1275/cmain.c index 20cbbd761..cc98811f4 100644 --- a/grub-core/kern/ieee1275/cmain.c +++ b/grub-core/kern/ieee1275/cmain.c @@ -124,6 +124,9 @@ grub_ieee1275_find_options (void) break; } } + + if (grub_strncmp (tmp, "IBM,", 4) == 0) + grub_ieee1275_set_flag (GRUB_IEEE1275_FLAG_CAN_TRY_CAS_FOR_MORE_MEMORY); } if (is_smartfirmware) diff --git a/grub-core/kern/ieee1275/init.c b/grub-core/kern/ieee1275/init.c index 4162b5949..4586bec93 100644 --- a/grub-core/kern/ieee1275/init.c +++ b/grub-core/kern/ieee1275/init.c @@ -240,6 +240,135 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, return 0; } +/* How much memory does OF believe it has? (regardless of whether + it's accessible or not) */ +static grub_err_t +grub_ieee1275_total_mem (grub_uint64_t *total) +{ + grub_ieee1275_phandle_t root; + grub_ieee1275_phandle_t memory; + grub_uint32_t reg[4]; + grub_ssize_t reg_size; + grub_uint32_t address_cells = 1; + grub_uint32_t size_cells = 1; + grub_uint64_t size; + + /* If we fail to get to the end, report 0. */ + *total = 0; + + /* Determine the format of each entry in `reg'. */ + grub_ieee1275_finddevice ("/", &root); + grub_ieee1275_get_integer_property (root, "#address-cells", &address_cells, + sizeof address_cells, 0); + grub_ieee1275_get_integer_property (root, "#size-cells", &size_cells, + sizeof size_cells, 0); + + if (size_cells > address_cells) + address_cells = size_cells; + + /* Load `/memory/reg'. */ + if (grub_ieee1275_finddevice ("/memory", &memory)) + return grub_error (GRUB_ERR_UNKNOWN_DEVICE, + "couldn't find /memory node"); + if (grub_ieee1275_get_integer_property (memory, "reg", reg, + sizeof reg, ®_size)) + return grub_error (GRUB_ERR_UNKNOWN_DEVICE, + "couldn't examine /memory/reg property"); + if (reg_size < 0 || (grub_size_t) reg_size > sizeof (reg)) + return grub_error (GRUB_ERR_UNKNOWN_DEVICE, + "/memory response buffer exceeded"); + + if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_BROKEN_ADDRESS_CELLS)) + { + address_cells = 1; + size_cells = 1; + } + + /* Decode only the size */ + size = reg[address_cells]; + if (size_cells == 2) + size = (size << 32) | reg[address_cells + 1]; + + *total = size; + + return grub_errno; +} + +/* Based on linux - arch/powerpc/kernel/prom_init.c */ +struct option_vector2 { + grub_uint8_t byte1; + grub_uint16_t reserved; + grub_uint32_t real_base; + grub_uint32_t real_size; + grub_uint32_t virt_base; + grub_uint32_t virt_size; + grub_uint32_t load_base; + grub_uint32_t min_rma; + grub_uint32_t min_load; + grub_uint8_t min_rma_percent; + grub_uint8_t max_pft_size; +} __attribute__((packed)); + +struct pvr_entry { + grub_uint32_t mask; + grub_uint32_t entry; +}; + +struct cas_vector { + struct { + struct pvr_entry terminal; + } pvr_list; + grub_uint8_t num_vecs; + grub_uint8_t vec1_size; + grub_uint8_t vec1; + grub_uint8_t vec2_size; + struct option_vector2 vec2; +} __attribute__((packed)); + +/* Call ibm,client-architecture-support to try to get more RMA. + We ask for 512MB which should be enough to verify a distro kernel. + We ignore most errors: if we don't succeed we'll proceed with whatever + memory we have. */ +static void +grub_ieee1275_ibm_cas (void) +{ + int rc; + grub_ieee1275_ihandle_t root; + struct cas_args { + struct grub_ieee1275_common_hdr common; + grub_ieee1275_cell_t method; + grub_ieee1275_ihandle_t ihandle; + grub_ieee1275_cell_t cas_addr; + grub_ieee1275_cell_t result; + } args; + struct cas_vector vector = { + .pvr_list = { { 0x00000000, 0xffffffff } }, /* any processor */ + .num_vecs = 2 - 1, + .vec1_size = 0, + .vec1 = 0x80, /* ignore */ + .vec2_size = 1 + sizeof(struct option_vector2) - 2, + .vec2 = { + 0, 0, -1, -1, -1, -1, -1, 512, -1, 0, 48 + }, + }; + + INIT_IEEE1275_COMMON (&args.common, "call-method", 3, 2); + args.method = (grub_ieee1275_cell_t)"ibm,client-architecture-support"; + rc = grub_ieee1275_open("/", &root); + if (rc) { + grub_error (GRUB_ERR_IO, "could not open root when trying to call CAS"); + return; + } + args.ihandle = root; + args.cas_addr = (grub_ieee1275_cell_t)&vector; + + grub_printf("Calling ibm,client-architecture-support..."); + IEEE1275_CALL_ENTRY_FN (&args); + grub_printf("done\n"); + + grub_ieee1275_close(root); +} + static void grub_claim_heap (void) { @@ -247,11 +376,22 @@ grub_claim_heap (void) if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_FORCE_CLAIM)) { - heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN, - 1, &total); + heap_init (GRUB_IEEE1275_STATIC_HEAP_START, + GRUB_IEEE1275_STATIC_HEAP_LEN, 1, &total); return; } + if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_CAN_TRY_CAS_FOR_MORE_MEMORY)) + { + grub_uint64_t rma_size; + grub_err_t err; + + err = grub_ieee1275_total_mem (&rma_size); + /* if we have an error, don't call CAS, just hope for the best */ + if (!err && rma_size < (512 * 1024 * 1024)) + grub_ieee1275_ibm_cas(); + } + grub_machine_mmap_iterate (heap_size, &total); total = total / 4; diff --git a/include/grub/ieee1275/ieee1275.h b/include/grub/ieee1275/ieee1275.h index 73e2f4644..18c479b66 100644 --- a/include/grub/ieee1275/ieee1275.h +++ b/include/grub/ieee1275/ieee1275.h @@ -148,6 +148,12 @@ enum grub_ieee1275_flag GRUB_IEEE1275_FLAG_CURSORONOFF_ANSI_BROKEN, GRUB_IEEE1275_FLAG_RAW_DEVNAMES, + + /* On PFW, the first time we boot a Linux partition, we may only get 256MB + of real memory area, even if the partition has more memory. Set this flag + if we think we're running under PFW. Then, if this flag is set, and the + RMA is only 256MB in size, try asking for more with CAS. */ + GRUB_IEEE1275_FLAG_CAN_TRY_CAS_FOR_MORE_MEMORY, }; extern int EXPORT_FUNC(grub_ieee1275_test_flag) (enum grub_ieee1275_flag flag); -- 2.25.1