From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72466C433EF for ; Thu, 3 Mar 2022 15:23:19 +0000 (UTC) Received: from localhost ([::1] helo=shelob.surriel.com) by shelob.surriel.com with esmtp (Exim 4.94.2) (envelope-from ) id 1nPnIT-0003hh-TO for kernelnewbies@archiver.kernel.org; Thu, 03 Mar 2022 10:23:17 -0500 Received: from mail-lf1-x134.google.com ([2a00:1450:4864:20::134]) by shelob.surriel.com with esmtps (TLS1.2) tls TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1nPnGe-0001dx-Ob for kernelnewbies@kernelnewbies.org; Thu, 03 Mar 2022 10:21:24 -0500 Received: by mail-lf1-x134.google.com with SMTP id j15so9045699lfe.11 for ; Thu, 03 Mar 2022 07:21:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=fX4T2En0e5xP4RiO/aUCg+snuAO1oNvTpHTAx1fU6S0=; b=SJ8JVM+ciFXzaJSWwO/rEYqdeNRWCj4rU4PSkbEwBxtTnb9TlDYcUcdWTqzn2zjICI F6CKdreF53zDt4ZJVP1CnsTBdVzj9L1Li+RKB0yyHSd/E+fCxF/tABsBEvk2LE+DRBS8 EMGD0kK6vt/3s2ym/29l/3OfXL6kiZa60YIyH51F/U8lcaqSetSyppbq9zq2OynCcXdx Qdi899uckHFq31XqAsAb9PjqgHVpSXZmIayPDNYViFYRyVFPZiZWWi3O0XR6RIhne9pd 1ij8l0rlYnQ502AjVBIQcYQeGn/o8CRyWKzzZzOyJhn5AariPKU7/LGGUFlMMqz/kplA yBgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=fX4T2En0e5xP4RiO/aUCg+snuAO1oNvTpHTAx1fU6S0=; b=2BACLi4ROFG8NW+TIsQQlY26XcBdGpWkEpySzwTMkxnP5bL7D7lhb7tRv14pRmJfUS 3KNObzLyFOYfni++4zPthRImdNCdHsA1PXaeuaymfKkgRhUEB29sJ/dTMqHYvIIk2DJB 5U1EUr0bG2ycVsKaebl74GCcSv2TooPbDDhbivMJPPzWD1vtqEq1ZjhF3AGY+AgfwRhw 5pM0g/gNTNE/T79uVhriJhsQEpZP8cjIKxmskj7AwAUd9KJW8llOge7mHo0j4RcWNQ7A yYKoPwHtMcAuT2VKMGmYGhWF4XYAL3xHO6DU/2xmfLcAkd5DUaaZ79ZMl/OhYk+bIRye 2WAg== X-Gm-Message-State: AOAM530/4jCU6wn45Mf6t79ISHClbsvLMn6wfrk6zq1OjLRYLCBUx7ei NDAaGG15cVvQh2Ozrkn3yC+5vhvozfYVWTUqeXlUqbZ0 X-Google-Smtp-Source: ABdhPJxbTG6e6N/mJe2ZZffFo0NzyfMdkiP8uxBiKAO1ExS/GUC3q9BeFENUXxedjMiiIfEC0K/ysfEO9m+mQ/3s+0c= X-Received: by 2002:a05:6512:2804:b0:445:7592:3dc5 with SMTP id cf4-20020a056512280400b0044575923dc5mr20055333lfb.356.1646320882420; Thu, 03 Mar 2022 07:21:22 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Guddla Rupesh Date: Thu, 3 Mar 2022 20:51:09 +0530 Message-ID: Subject: Re: Exporting cpu instruction set to kernel .config file To: kernelnewbies@kernelnewbies.org X-BeenThere: kernelnewbies@kernelnewbies.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Learn about the Linux kernel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============7944287675949759938==" Errors-To: kernelnewbies-bounces+kernelnewbies=archiver.kernel.org@kernelnewbies.org --===============7944287675949759938== Content-Type: multipart/alternative; boundary="00000000000062107305d951f334" --00000000000062107305d951f334 Content-Type: text/plain; charset="UTF-8" Pentium MMX was launched about 30 years ago and at that time video processing was new and there was no proper hardware just to play any video file I mean if you play any video file video playback stops sometimes and resume after some time. At present there is lot of development in video hardware and at the same time new technologies in both hardware and software has been evolved. Coming into hardware CPUs with integrated graphics came and coming into software many video and audio codecs have been developed like h265, vpx, av1, aac, opus etc.,.Of these aac and opus are audio codecs which produces lowest disk size at reasonable quality and h265, av1 are video codecs which provides lowest disk size at reasonable quality. Most of the time I download videos from youtube and other sources which are of large size. When you record a video in android smartphone it takes around 600 mb for 3 minutes 50 seconds and it uses h265 codec. It is difficult to store such files in my android smartphone. Mostly I use Linux and ffmpeg tool to compress these video files. Of h265 and av1 aom encoders. av1 aom encoder provide lowest disk size at lowest video bitrate. H265 encoding and decoding are supported in all hardware including pc and smartphone but unfortunately av1 files are not supported in none but definitely I think that it will become defacto standard in future. If h265 encoding takes 1 minute then aom av1 encoding takes about 10 minutes. If video acceleration is present the encoding takes about 7 minutes. Video acceleration come into action only if GPU is detected properly and the necessary drivers are installed. If the system can detect opecl version of GPU then av1 encoding takes about 5 minutes through ffmpeg. As MMX was trending about 30 years ago I think that latest processors have vector instructions for video transcoding or encoding like avx2 or avx 512, simd etc.,. These new instructions are not present in 15 years back processors. As 64 bit processor came into action around 20 years back nowadays processors are 10 times are greater in performance than old. I think that all the kernels shipped by distros are not considering latest processors instruction set. I think that even windows is doing the same. If opencl is detected properly any application can have improvement in performance about 10 percent. I can't find any option to set opencl in the Linux kernel config file or in any of the kernel source files. Atleast I am lucky to select option core 2 in the main .config file of kernel. Finally my request is there any way to export instruct set of my cpu to the kernel configuration and so after compilation my video encoding takes less time than previous. If it's a difficult task leave it and if it's possible try to suggest how to do so. On Thu, 3 Mar 2022, 6:56 am FMDF, wrote: > On Thu, 3 Mar 2022, 02:05 FMDF, wrote: > >> On Wed, 2 Mar 2022, 17:57 Guddla Rupesh, wrote: >> >>> I have seen the config from /proc/config.gz and found that processor >>> family as generic x86 64. >>> >>> Actually my system consists of i3 10th gen comet lake 10100 which has >>> integrated graphics and pci express 4.0 and also lots of new instructions >>> like avx2, sse4 etc.,. >>> >> Therefore, in .config, you should enable x86-64. That's it. Simple. >> >>> The main reason of compiling source code of kernel is it detects all my >>> hardware and provide support for opencl, video acceleration etc.,. But I >>> can't find any option to select my processor especially comet lake or >>> atleast skylake and instead there is an option called core2 in the section >>> processor family. >>> >> There are no Cometlake or Skylake options for kernel compilation. Your >> recent CPU is already supported and the whole ISA is available. Compilers >> know your system and they know which instructions are better suited for >> your applications and kernel code. >> >>> Finally I have disabled secure boot mode in bios and compiled source >>> code successfully and now I am able to boot into the latest build kernel >>> but there is no difference in performance between old and new kernels. >>> >> Why have you disabled secure boot mode? If performance concerns you, >> don't worry about it: there is no loss of performance with secure boot. >> >>> I think that normal kernel shipped by the distro is generic and it >>> doesn't have support for latest instructions like avx2, sse4 etc.,. To get >>> support we must compile kernel with lots of tuning. >>> >> No, you're wrong :) >> >> Just select the highest available processor family (I can't recall now if >> it is "core2" or "generic x86_64", it's the last of the list in the >> "processor family" section if you use "make menuconfig") and the entire >> instruction set of Intel x86-64 will be enabled and available. >> >> There is neither a "CONFIG_GENERATION_11" nor similar options. >> >>> I have exported the modules running on system and disable others by >>> running the command >>> >>> make localmodconfig >>> >> Beware that using "localmodconfig" disables the build of lots of modules >> you may need in the future. It relies on the current system configuration. >> When you'll add new devices you'll have to enable their drivers and >> recompile the modules. >> >>> What I want is " is there any way to export my cpu instruction set to >>> kernel .config file and compile kernel and install the modules.". >>> >>> If I can compile kernel source code then my Linux system will run at >>> extreme speeds never before. >>> >> You might tune the optimization flags of (e.g., GCC's "-On", for n=0,1,2, >> and others) but I strongly advise you to not do this because (1) it is >> clear that you don't understand how the code is compiled and (2) you >> probably won't get the results you think you should get. >> >> Regards, >> >> Fabio M. De Francesco >> > > I was about to forget three more things: > > 1) Read Torin's answer because he is right. > 2) Don't touch configuration's options that you don't know. For instance, > do you know if you should enable "CONFIG_CORE_SCHED"? Just think that you > can inadvertently disable Intel Hyperthreading with that option. > 3) Just copy your distro's .config and re-compile :) > > --00000000000062107305d951f334 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Pentium MMX was launched about 30 years ago and at that t= ime video processing was new and there was no proper hardware just to play = any video file I mean if you play any video file video playback stops somet= imes and resume after some time.

At present there is lot of development in video hardware and at the same= time new technologies in both hardware and software has been evolved. Comi= ng into hardware CPUs with integrated graphics came and coming into softwar= e many video and audio codecs have been developed like h265, vpx, av1, aac,= opus etc.,.Of these aac and opus are audio codecs which produces lowest di= sk size at reasonable quality and h265, av1 are video codecs which provides= lowest disk size at reasonable quality.

<= div dir=3D"auto">Most of the time I download videos from youtube and other = sources which are of large size. When you record a video in android smartph= one it takes around 600 mb for 3 minutes 50 seconds and it uses h265 codec.= It is difficult to store such files in my android smartphone.

Mostly I use Linux and ffmpeg tool t= o compress these video files. Of h265 and av1 aom encoders. av1 aom encoder= provide lowest disk size at lowest video bitrate. H265 encoding and decodi= ng are supported in all hardware including pc and smartphone but unfortunat= ely av1 files are not supported in none but definitely I think that it will= become defacto standard in future.

If h265 encoding takes 1 minute then aom av1 encoding takes abo= ut 10 minutes. If video acceleration is present the encoding takes about 7 = minutes. Video acceleration come into action only if GPU is detected proper= ly and the necessary drivers are installed. If the system can detect opecl = version of GPU then av1 encoding takes about 5 minutes through ffmpeg.

As MMX was trending about 30= years ago I think that latest processors have vector instructions for vide= o transcoding or encoding like avx2 or avx 512, simd etc.,. These new instr= uctions are not present in 15 years back processors.

As 64 bit processor came into action around 20= years back nowadays processors are 10 times are greater in performance tha= n old.

I think that all = the kernels shipped by distros are not considering latest processors instru= ction set. I think that even windows is doing the same.=C2=A0

If opencl is detected properly any ap= plication can have improvement in performance about 10 percent.

I can't find any option to set = opencl in the Linux kernel config file or in any of the kernel source files= .

Atleast I am lucky to = select option core 2 in the main .config file of kernel.

Finally my request is there any way to exp= ort instruct set of my cpu to the kernel configuration and so after compila= tion my video encoding takes less time than previous.

If it's a difficult task leave it and if = it's possible try to suggest how to do so.

On Thu, 3 Mar 2022, 6:5= 6 am FMDF, <fmdefrancesco@gma= il.com> wrote:
= On Thu, 3 Mar 2022, 02:05 FMDF, <fmdefrancesco@gmail.com> wr= ote:
On Wed, 2 Mar 202= 2, 17:57 Guddla Rupesh, <rupeshforu3@gmail.com> wro= te:

I have see= n the config from /proc/config.gz and found that processor family as generi= c x86 64.

Actually my system consists of i3 10th gen comet lake 10100 which has integrated graphics and pci express 4.0 and also lots of new=20 instructions like avx2, sse4 etc.,.

Therefore, in .config, you should enable x86-64. That's it= . Simple.

The main reason of compiling source code of kernel is it detects all=20 my hardware and provide support for opencl, video acceleration etc.,.=20 But I can't find any option to select my processor especially comet lak= e or atleast skylake and instead there is an option called core2 in the=20 section processor family.

There are no Cometlake or Skylake options for kernel compilation. Your r= ecent CPU is already supported and the whole ISA is available. Compilers kn= ow your system and they know which instructions are better suited for your = applications and kernel code.=C2=A0

Finally I have disabled secure boot mode in bios and compiled source=20 code successfully and now I am able to boot into the latest build kernel but there is no difference in performance between old and new kernels.

=
Why have you disabled secu= re boot mode? If performance concerns you, don't worry about it: there = is no loss of performance with secure boot.

I think that normal kernel shipped by the distro is generic and it=20 doesn't have support for latest instructions like avx2, sse4 etc.,. To= =20 get support we must compile kernel with lots of tuning.

No, you're wrong :)

Just select the highest available processo= r family (I can't recall now if it is "core2" or "generi= c x86_64", it's the last of the list in the "processor family= " section if you use "make menuconfig") and the entire instr= uction set of Intel x86-64 will be enabled and available.=C2=A0

There is neither a "CONFIG_GEN= ERATION_11" nor similar options.

I have exported the modules running on system and disable others by runn= ing the command

make localmodconfig

= Beware that using "localmodconfig" disables the build of lots of = modules you may need in the future. It relies on the current system configu= ration. When you'll add new devices you'll have to enable their dri= vers and recompile the modules.

What I want is " is there any way to export my cpu instruction set = to kernel .config file and compile kernel and install the modules.".

If I can compile kernel source code then my Linux system will run at ext= reme speeds never before.

You might tune the optimization flags of (e.g., GCC's "-On"= ;, for n=3D0,1,2, and=C2=A0 others) but I strongly advise you to not do thi= s because (1) it is clear that you don't understand how the code is com= piled and (2) you probably won't get the results you think you should g= et.

Regards,

Fabio M. De Francesco

I was = about to forget three more things:

1) Read Torin's answer because he is right.
2) Don't touch configuration's options that you don't kn= ow. For instance, do you know if you should enable "CONFIG_CORE_SCHED&= quot;? Just think that you can inadvertently disable Intel Hyperthreading w= ith that option.=C2=A0
3) Just copy your distro'= s .config and re-compile :)

--00000000000062107305d951f334-- --===============7944287675949759938== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies --===============7944287675949759938==--