From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C486C433E1 for ; Tue, 23 Jun 2020 19:11:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 47C3D207F9 for ; Tue, 23 Jun 2020 19:11:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1592939497; bh=rlQHqh/1EcPc9M4t4Y9qJ2pt9mUGEddxgyBOvlhwd3o=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=QMIvefDcPym6k+iQyVwQH8reBDt6viYiVJNYzw8ixyNKyomIlu1+UXkAsXG5T4MHb Dfv/r3xSDxWntr3WyChMSYEwJge+lQ6JnJ2P3BTJyZryTpU61elTP37kMjXeYvbK+R lPUxNA0gM2DHiK0tmdyBqJrFRSSatAjHBT5VzAFs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387479AbgFWTLg (ORCPT ); Tue, 23 Jun 2020 15:11:36 -0400 Received: from mail-ej1-f41.google.com ([209.85.218.41]:45691 "EHLO mail-ej1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733248AbgFWTLf (ORCPT ); Tue, 23 Jun 2020 15:11:35 -0400 Received: by mail-ej1-f41.google.com with SMTP id a1so8698389ejg.12; Tue, 23 Jun 2020 12:11:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=S6qzk+OKExXb2YpLZeuh1PmKG+c4jm0lpjuoeWwXFzk=; b=r9kTTD97qkKFw+q1SPYr0Wmmq3KWGMmYl1o9LsR0k5H/dgQxF1Fj/AYaVCjMl0Xfhx 952UvgI6M34fyLhH+gGlkFq+u/PMO6TWDqrp/3vpXe75uggWJLtNzU2smIEt1MrgPRsR 2LpwPNKlxLPZ1zQAgqJ/O031v3TGNxGQYDFywEUx1p4Axdgl8olSmu3knmuhr9dDI0o/ XfLw/tjQCoOrBNndsTlkMuVBK1NnExcM8UQxWElMLIF9UZH9+zIgsBYtzyWyiky7jBeU neE0PZ46Pqg0pkxQ5aM6CtRFI09dM5dNS9Mk4QCIRoNhymZ54j1MOxuf6kULC3Wwf0RY QKzQ== X-Gm-Message-State: AOAM533o31B/eNTpEmWZdnSOEhqe09EvV7r9zil3N8T3FHhrEPQ6FM96 290/5Q5+lgkg0SAxtxMMxm0= X-Google-Smtp-Source: ABdhPJzL8paryXZaLIV+aJhyo9RYBnt+N7daGvK76KL9xSHh8GkcyzPYHf1WG9BUOVfCwW6i7BNwCg== X-Received: by 2002:a17:906:4cd0:: with SMTP id q16mr13306634ejt.418.1592939492855; Tue, 23 Jun 2020 12:11:32 -0700 (PDT) Received: from kozik-lap ([194.230.155.235]) by smtp.googlemail.com with ESMTPSA id a2sm4479503ejg.76.2020.06.23.12.11.31 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 Jun 2020 12:11:31 -0700 (PDT) Date: Tue, 23 Jun 2020 21:11:29 +0200 From: Krzysztof Kozlowski To: Willy Wolff Cc: Chanwoo Choi , MyungJoo Ham , Kyungmin Park , Kukjin Kim , linux-pm@vger.kernel.org, "linux-samsung-soc@vger.kernel.org" , linux-arm-kernel@lists.infradead.org, "linux-kernel@vger.kernel.org" , Lukasz Luba Subject: Re: brocken devfreq simple_ondemand for Odroid XU3/4? Message-ID: <20200623191129.GA4171@kozik-lap> References: <20200623164733.qbhua7b6cg2umafj@macmini.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jun 23, 2020 at 09:02:38PM +0200, Krzysztof Kozlowski wrote: > On Tue, 23 Jun 2020 at 18:47, Willy Wolff wrote: > > > > Hi everybody, > > > > Is DVFS for memory bus really working on Odroid XU3/4 board? > > Using a simple microbenchmark that is doing only memory accesses, memory DVFS > > seems to not working properly: > > > > The microbenchmark is doing pointer chasing by following index in an array. > > Indices in the array are set to follow a random pattern (cutting prefetcher), > > and forcing RAM access. > > > > git clone https://github.com/wwilly/benchmark.git \ > > && cd benchmark \ > > && source env.sh \ > > && ./bench_build.sh \ > > && bash source/scripts/test_dvfs_mem.sh > > > > Python 3, cmake and sudo rights are required. > > > > Results: > > DVFS CPU with performance governor > > mem_gov = simple_ondemand at 165000000 Hz in idle, should be bumped when the > > benchmark is running. > > - on the LITTLE cluster it takes 4.74308 s to run (683.004 c per memory access), > > - on the big cluster it takes 4.76556 s to run (980.343 c per moemory access). > > > > While forcing DVFS memory bus to use performance governor, > > mem_gov = performance at 825000000 Hz in idle, > > - on the LITTLE cluster it takes 1.1451 s to run (164.894 c per memory access), > > - on the big cluster it takes 1.18448 s to run (243.664 c per memory access). > > > > The kernel used is the last 5.7.5 stable with default exynos_defconfig. > > Thanks for the report. Few thoughts: > 1. What trans_stat are saying? Except DMC driver you can also check > all other devfreq devices (e.g. wcore) - maybe the devfreq events > (nocp) are not properly assigned? > 2. Try running the measurement for ~1 minutes or longer. The counters > might have some delay (which would require probably fixing but the > point is to narrow the problem). > 3. What do you understand by "mem_gov"? Which device is it? +Cc Lukasz who was working on this. I just run memtester and more-or-less ondemand works (at least ramps up): Before: /sys/class/devfreq/10c20000.memory-controller$ cat trans_stat From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) * 165000000: 0 0 0 0 0 0 0 0 1795950 206000000: 1 0 0 0 0 0 0 0 4770 275000000: 0 1 0 0 0 0 0 0 15540 413000000: 0 0 1 0 0 0 0 0 20780 543000000: 0 0 0 1 0 0 0 1 10760 633000000: 0 0 0 0 2 0 0 0 10310 728000000: 0 0 0 0 0 0 0 0 0 825000000: 0 0 0 0 0 2 0 0 25920 Total transition : 9 $ sudo memtester 1G During memtester: /sys/class/devfreq/10c20000.memory-controller$ cat trans_stat From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) 165000000: 0 0 0 0 0 0 0 1 1801490 206000000: 1 0 0 0 0 0 0 0 4770 275000000: 0 1 0 0 0 0 0 0 15540 413000000: 0 0 1 0 0 0 0 0 20780 543000000: 0 0 0 1 0 0 0 2 11090 633000000: 0 0 0 0 3 0 0 0 17210 728000000: 0 0 0 0 0 0 0 0 0 * 825000000: 0 0 0 0 0 3 0 0 169020 Total transition : 13 However after killing memtester it stays at 633 MHz for very long time and does not slow down. This is indeed weird... Best regards, Krzysztof From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8AD7C433E0 for ; Tue, 23 Jun 2020 19:13:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 75A1E2084D for ; Tue, 23 Jun 2020 19:13:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="XBsF6h/Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75A1E2084D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JDnLsa58dJZGKUed+nU8swmjAfjoTF2ERHCACtEebHc=; b=XBsF6h/YksJ8Cmy5hkSmuqCxF fCZRmFJXVYXo+bH8/nkEPPjYA4iPYEpIMG5oZOJV2X3bVozy9FsNPBSqlYZ8ITioR+jVGO5+7l97z dQKVXuROw4CvsoOpAxw9oiaRTvwuk3N5X3UTPNXqMbKqv9OJPOmOz9FW04zjog2BDXqfBzYV70UJc Z4w4rXTMar5EsU3wFDeDAuG6/V8u4PpXmfYgFU5wbXQSsNV98PcPwlz5Nh18nDHjmGv2WkE+8V8bo pIM2Yrgw9iUjyb1hCeoF7Wyn1uyRf9nwoEnRO4AEIkbuCZhI7GwHbEWCxLvELOgy84vWrC3sSlm4j g56+kgQOw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jnoKZ-0007bk-W8; Tue, 23 Jun 2020 19:11:40 +0000 Received: from mail-ej1-f42.google.com ([209.85.218.42]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jnoKY-0007an-3y for linux-arm-kernel@lists.infradead.org; Tue, 23 Jun 2020 19:11:39 +0000 Received: by mail-ej1-f42.google.com with SMTP id p20so22671525ejd.13 for ; Tue, 23 Jun 2020 12:11:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=S6qzk+OKExXb2YpLZeuh1PmKG+c4jm0lpjuoeWwXFzk=; b=fmwlbaZOoZ8+aDPFLkTwFlP0/sRI5Bxdt/YomGphUjNTTrH09Cf1a2QGip2uZBfhzL 0h/2kI+zQ/zAIGiGw3OdZgCm+Ftw1FaMeXBJ1/igiP2QefzNpQHDYJtUpMifcYUQcn3D K3onmMq6s8Ba8rv1hwRlXiZpwb7WCuUnyAwn3/dzE0H05F0kyIKnYUte7+6zjGuWA9ik Z4+2rByh/XRwZ8elTIM8MIQpJsidkWliJvHkrIz6kYdwAl4ymXNaZgGuybXakRUoZu/u l7Z4AuiEKHldvmGdHOm5otYCUA+qKJ/KBKimYDxqQoCfFEk7SaPzoxQIP+nmL8SS0TC7 ErSw== X-Gm-Message-State: AOAM530MpPDE4K5q6j6YO0a/YbeMeLEz4aSyrqd1uX01eH+YtmabhAKx uEK79bgmSIIdfoJBVIPm7RI= X-Google-Smtp-Source: ABdhPJzL8paryXZaLIV+aJhyo9RYBnt+N7daGvK76KL9xSHh8GkcyzPYHf1WG9BUOVfCwW6i7BNwCg== X-Received: by 2002:a17:906:4cd0:: with SMTP id q16mr13306634ejt.418.1592939492855; Tue, 23 Jun 2020 12:11:32 -0700 (PDT) Received: from kozik-lap ([194.230.155.235]) by smtp.googlemail.com with ESMTPSA id a2sm4479503ejg.76.2020.06.23.12.11.31 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 Jun 2020 12:11:31 -0700 (PDT) Date: Tue, 23 Jun 2020 21:11:29 +0200 From: Krzysztof Kozlowski To: Willy Wolff Subject: Re: brocken devfreq simple_ondemand for Odroid XU3/4? Message-ID: <20200623191129.GA4171@kozik-lap> References: <20200623164733.qbhua7b6cg2umafj@macmini.local> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-samsung-soc@vger.kernel.org" , linux-pm@vger.kernel.org, "linux-kernel@vger.kernel.org" , Chanwoo Choi , Kyungmin Park , MyungJoo Ham , Kukjin Kim , Lukasz Luba , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Jun 23, 2020 at 09:02:38PM +0200, Krzysztof Kozlowski wrote: > On Tue, 23 Jun 2020 at 18:47, Willy Wolff wrote: > > > > Hi everybody, > > > > Is DVFS for memory bus really working on Odroid XU3/4 board? > > Using a simple microbenchmark that is doing only memory accesses, memory DVFS > > seems to not working properly: > > > > The microbenchmark is doing pointer chasing by following index in an array. > > Indices in the array are set to follow a random pattern (cutting prefetcher), > > and forcing RAM access. > > > > git clone https://github.com/wwilly/benchmark.git \ > > && cd benchmark \ > > && source env.sh \ > > && ./bench_build.sh \ > > && bash source/scripts/test_dvfs_mem.sh > > > > Python 3, cmake and sudo rights are required. > > > > Results: > > DVFS CPU with performance governor > > mem_gov = simple_ondemand at 165000000 Hz in idle, should be bumped when the > > benchmark is running. > > - on the LITTLE cluster it takes 4.74308 s to run (683.004 c per memory access), > > - on the big cluster it takes 4.76556 s to run (980.343 c per moemory access). > > > > While forcing DVFS memory bus to use performance governor, > > mem_gov = performance at 825000000 Hz in idle, > > - on the LITTLE cluster it takes 1.1451 s to run (164.894 c per memory access), > > - on the big cluster it takes 1.18448 s to run (243.664 c per memory access). > > > > The kernel used is the last 5.7.5 stable with default exynos_defconfig. > > Thanks for the report. Few thoughts: > 1. What trans_stat are saying? Except DMC driver you can also check > all other devfreq devices (e.g. wcore) - maybe the devfreq events > (nocp) are not properly assigned? > 2. Try running the measurement for ~1 minutes or longer. The counters > might have some delay (which would require probably fixing but the > point is to narrow the problem). > 3. What do you understand by "mem_gov"? Which device is it? +Cc Lukasz who was working on this. I just run memtester and more-or-less ondemand works (at least ramps up): Before: /sys/class/devfreq/10c20000.memory-controller$ cat trans_stat From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) * 165000000: 0 0 0 0 0 0 0 0 1795950 206000000: 1 0 0 0 0 0 0 0 4770 275000000: 0 1 0 0 0 0 0 0 15540 413000000: 0 0 1 0 0 0 0 0 20780 543000000: 0 0 0 1 0 0 0 1 10760 633000000: 0 0 0 0 2 0 0 0 10310 728000000: 0 0 0 0 0 0 0 0 0 825000000: 0 0 0 0 0 2 0 0 25920 Total transition : 9 $ sudo memtester 1G During memtester: /sys/class/devfreq/10c20000.memory-controller$ cat trans_stat From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) 165000000: 0 0 0 0 0 0 0 1 1801490 206000000: 1 0 0 0 0 0 0 0 4770 275000000: 0 1 0 0 0 0 0 0 15540 413000000: 0 0 1 0 0 0 0 0 20780 543000000: 0 0 0 1 0 0 0 2 11090 633000000: 0 0 0 0 3 0 0 0 17210 728000000: 0 0 0 0 0 0 0 0 0 * 825000000: 0 0 0 0 0 3 0 0 169020 Total transition : 13 However after killing memtester it stays at 633 MHz for very long time and does not slow down. This is indeed weird... Best regards, Krzysztof _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel