All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
       [not found]   ` <20130116131725.4e590c34@bluemoon>
@ 2013-01-16 20:37     ` Tom Gundersen
  2013-01-16 22:33       ` Rafael J. Wysocki
  0 siblings, 1 reply; 19+ messages in thread
From: Tom Gundersen @ 2013-01-16 20:37 UTC (permalink / raw)
  To: Matthew Garrett, Andre Przywara, Rafael J. Wysocki
  Cc: Leonid Isaev, cpufreq, linux-acpi

[dropped arch-general, added kernel folks]

On Wed, Jan 16, 2013 at 8:17 PM, Leonid Isaev <lisaev@umail.iu.edu> wrote:
> On Wed, 16 Jan 2013 11:10:19 +0100
> Tom Gundersen <teg@jklm.no> wrote:
>
>> On Wed, Jan 16, 2013 at 5:47 AM, Leonid Isaev <lisaev@umail.iu.edu> wrote:
>> >         I just installed linux 3.7.2 from [testing] on an AMD system and
>> > noticed that powernow-k8 is not loaded.
>>
>> Probably due to this:
>> <http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=e1f0b8e9b04a262834ed1>.

[...]

>> Hm, I thought acpi_cpufreq should have been loaded instead, no idea
>> why that does not happen for you. Could you try loading it manually?
>
> Neither acpi_cpufreq nor powernow_k8 is loaded automatically which of course
> leads to failure of all custom units configuring ondemand governor via
> sysfs. Manually modprob'ing acpi_cpufreq does work and indeed properly scales
> down the frequancy, but is accompanied by an info-level kernel message
> "acpi-cpufreq: overriding BIOS provided _PSD data".
>
> So, if the cpufreq driver works OK for other AMD users, I'll happily blame
> asus...

Matthew, Andre, Rafael,

Is what Leonid experinces above expected behavior or is this a bug? He
is using stock Arch kernel, which is almost vanilla 3.7.3 (see [0] for
the applied patches, nothing looks relevant).

Cheers,

Tom

[0]: <https://projects.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/linux>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-16 20:37     ` [arch-general] powernow-k8 fails to load with linux 3.7.2 Tom Gundersen
@ 2013-01-16 22:33       ` Rafael J. Wysocki
  2013-01-16 23:17         ` Borislav Petkov
  0 siblings, 1 reply; 19+ messages in thread
From: Rafael J. Wysocki @ 2013-01-16 22:33 UTC (permalink / raw)
  To: Tom Gundersen, Andre Przywara
  Cc: Matthew Garrett, Leonid Isaev, cpufreq, linux-acpi, Borislav Petkov

On Wednesday, January 16, 2013 09:37:58 PM Tom Gundersen wrote:
> [dropped arch-general, added kernel folks]
> 
> On Wed, Jan 16, 2013 at 8:17 PM, Leonid Isaev <lisaev@umail.iu.edu> wrote:
> > On Wed, 16 Jan 2013 11:10:19 +0100
> > Tom Gundersen <teg@jklm.no> wrote:
> >
> >> On Wed, Jan 16, 2013 at 5:47 AM, Leonid Isaev <lisaev@umail.iu.edu> wrote:
> >> >         I just installed linux 3.7.2 from [testing] on an AMD system and
> >> > noticed that powernow-k8 is not loaded.
> >>
> >> Probably due to this:
> >> <http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=e1f0b8e9b04a262834ed1>.
> 
> [...]
> 
> >> Hm, I thought acpi_cpufreq should have been loaded instead, no idea
> >> why that does not happen for you. Could you try loading it manually?
> >
> > Neither acpi_cpufreq nor powernow_k8 is loaded automatically which of course
> > leads to failure of all custom units configuring ondemand governor via
> > sysfs. Manually modprob'ing acpi_cpufreq does work and indeed properly scales
> > down the frequancy, but is accompanied by an info-level kernel message
> > "acpi-cpufreq: overriding BIOS provided _PSD data".
> >
> > So, if the cpufreq driver works OK for other AMD users, I'll happily blame
> > asus...
> 
> Matthew, Andre, Rafael,
> 
> Is what Leonid experinces above expected behavior or is this a bug? He
> is using stock Arch kernel, which is almost vanilla 3.7.3 (see [0] for
> the applied patches, nothing looks relevant).

Well, I was afraid that something like this would happen after the recent AMD
cpufreq changes.

Andre, do we have a clear way to address these problems?

Rafael


-- 
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-16 22:33       ` Rafael J. Wysocki
@ 2013-01-16 23:17         ` Borislav Petkov
       [not found]           ` <20130116181600.4c48ee3b@bluemoon>
  0 siblings, 1 reply; 19+ messages in thread
From: Borislav Petkov @ 2013-01-16 23:17 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Tom Gundersen, André Przywara, Matthew Garrett,
	Leonid Isaev, cpufreq, linux-acpi

[ fixup Andre's address. ]

On Wed, Jan 16, 2013 at 11:33:21PM +0100, Rafael J. Wysocki wrote:
> > down the frequancy, but is accompanied by an info-level kernel message
> > "acpi-cpufreq: overriding BIOS provided _PSD data".

That just says that BIOS has crappy data and we're using our own.

> Well, I was afraid that something like this would happen after the
> recent AMD cpufreq changes.
>
> Andre, do we have a clear way to address these problems?

I'm working on a patch (see thread at
http://marc.info/?l=linux-pm&m=135791578516584&w=2) that should take
care of really fixing handoff to acpi_cpufreq from powernow-k8, need to
sort out some stuff first though.

Btw, the handoff from powernow-k8 to acpi_cpufreq works on my systems,
but I haven't tested 3.7.2 so maybe we're missing some patches there.
Leonid, can you send your config pls?

Thanks.

-- 
Regards/Gruss,
Boris.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
       [not found]           ` <20130116181600.4c48ee3b@bluemoon>
@ 2013-01-17  9:46             ` Borislav Petkov
  2013-01-17 10:34               ` Tom Gundersen
  2013-01-18  1:40               ` Leonid Isaev
  0 siblings, 2 replies; 19+ messages in thread
From: Borislav Petkov @ 2013-01-17  9:46 UTC (permalink / raw)
  To: Leonid Isaev
  Cc: Rafael J. Wysocki, Tom Gundersen, André Przywara,
	Matthew Garrett, cpufreq, linux-acpi

On Wed, Jan 16, 2013 at 06:16:00PM -0600, Leonid Isaev wrote:
> > [ fixup Andre's address. ]
> > 
> > On Wed, Jan 16, 2013 at 11:33:21PM +0100, Rafael J. Wysocki wrote:
> > > > down the frequancy, but is accompanied by an info-level kernel message
> > > > "acpi-cpufreq: overriding BIOS provided _PSD data".
> > 
> > That just says that BIOS has crappy data and we're using our own.
> 
> OK. Yes, my BIOS is broken, so I thought it was the reason why autoloading does
> not work.

Hmm, come to think of it, we're issuing this message on SMP on *all*
AMDs with HW_PSTATE. Andre, remind me again why we're doing this? We're
basically saying that we're overriding ACPI data but we still read it
out from acpi_perf_data and use *that* data to prepare the frequencies
table. What am I missing?

> To be more precise: "modprobe powernow-k8" returns a "no such device" error,

That's correct - we say ENODEV on your CPU which supports hardware
P-states and hand off to acpi-cpufreq which has that functionality now.

> but in kernel logs I see that acpi-cpufreq is indeed loaded:
> -------
> Jan 15 22:42:19 metal-0 kernel: [  534.003995] acpi-cpufreq: overriding BIOS
> provided _PSD data
> Jan 15 22:42:32 metal-0 kernel: [  547.122695] powernow-k8: this CPU is not
> supported anymore, using acpi-cpufreq instead.
> Jan 15 22:42:45 metal-0 kernel: [  559.623068] powernow-k8: this CPU is not 
> supported anymore, using acpi-cpufreq instead.
> -------
> (the CPU in question is phenom II x4 955). However, this is only after I
> attempt to _manually_ load powernow-k8 (the timestamps in [...] are ~6 mins
> after boot is completed). So the above-mentioned handoff seems to work, but
> please find attached my kernel config.

Ok, this is maybe the issue. What used to load powernow-k8 on your
distro before? Because basically it's enough if some script did
'modprobe powernow-k8' for the handoff to just work.

But, we also have the CPU autoprobing deal:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246

so, *actually*, powernow-k8 should be loaded by default. I dunno, maybe
upgrade udev on your distro?

Btw, your config looks fine:

CONFIG_X86_POWERNOW_K8=m
CONFIG_X86_ACPI_CPUFREQ=m

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-17  9:46             ` Borislav Petkov
@ 2013-01-17 10:34               ` Tom Gundersen
  2013-01-18  1:27                 ` Leonid Isaev
  2013-01-18  1:40               ` Leonid Isaev
  1 sibling, 1 reply; 19+ messages in thread
From: Tom Gundersen @ 2013-01-17 10:34 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Leonid Isaev, Rafael J. Wysocki, André Przywara,
	Matthew Garrett, cpufreq, linux-acpi

On Thu, Jan 17, 2013 at 10:46 AM, Borislav Petkov <bp@alien8.de> wrote:
> On Wed, Jan 16, 2013 at 06:16:00PM -0600, Leonid Isaev wrote:
>> > [ fixup Andre's address. ]
>> >
>> > On Wed, Jan 16, 2013 at 11:33:21PM +0100, Rafael J. Wysocki wrote:
>> > > > down the frequancy, but is accompanied by an info-level kernel message
>> > > > "acpi-cpufreq: overriding BIOS provided _PSD data".
>> >
>> > That just says that BIOS has crappy data and we're using our own.
>>
>> OK. Yes, my BIOS is broken, so I thought it was the reason why autoloading does
>> not work.
>
> Hmm, come to think of it, we're issuing this message on SMP on *all*
> AMDs with HW_PSTATE. Andre, remind me again why we're doing this? We're
> basically saying that we're overriding ACPI data but we still read it
> out from acpi_perf_data and use *that* data to prepare the frequencies
> table. What am I missing?
>
>> To be more precise: "modprobe powernow-k8" returns a "no such device" error,
>
> That's correct - we say ENODEV on your CPU which supports hardware
> P-states and hand off to acpi-cpufreq which has that functionality now.
>
>> but in kernel logs I see that acpi-cpufreq is indeed loaded:
>> -------
>> Jan 15 22:42:19 metal-0 kernel: [  534.003995] acpi-cpufreq: overriding BIOS
>> provided _PSD data
>> Jan 15 22:42:32 metal-0 kernel: [  547.122695] powernow-k8: this CPU is not
>> supported anymore, using acpi-cpufreq instead.
>> Jan 15 22:42:45 metal-0 kernel: [  559.623068] powernow-k8: this CPU is not
>> supported anymore, using acpi-cpufreq instead.
>> -------
>> (the CPU in question is phenom II x4 955). However, this is only after I
>> attempt to _manually_ load powernow-k8 (the timestamps in [...] are ~6 mins
>> after boot is completed). So the above-mentioned handoff seems to work, but
>> please find attached my kernel config.
>
> Ok, this is maybe the issue. What used to load powernow-k8 on your
> distro before? Because basically it's enough if some script did
> 'modprobe powernow-k8' for the handoff to just work.
>
> But, we also have the CPU autoprobing deal:
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246

Arch relies entirely on the CPU autoprobing, there are no scripts
modprobing things at boot.

> so, *actually*, powernow-k8 should be loaded by default. I dunno, maybe
> upgrade udev on your distro?

Assuming Leonid's system is up-to-date, Arch ships the most recent
udev version (197).

Cheers,

Tom

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-17 10:34               ` Tom Gundersen
@ 2013-01-18  1:27                 ` Leonid Isaev
  0 siblings, 0 replies; 19+ messages in thread
From: Leonid Isaev @ 2013-01-18  1:27 UTC (permalink / raw)
  To: Tom Gundersen
  Cc: Borislav Petkov, Rafael J. Wysocki, André Przywara,
	Matthew Garrett, cpufreq, linux-acpi

[-- Attachment #1: Type: text/plain, Size: 3106 bytes --]

Sorry for a late reply...

On Thu, 17 Jan 2013 11:34:22 +0100
Tom Gundersen <teg@jklm.no> wrote:

> On Thu, Jan 17, 2013 at 10:46 AM, Borislav Petkov <bp@alien8.de> wrote:
> > On Wed, Jan 16, 2013 at 06:16:00PM -0600, Leonid Isaev wrote:
> >> > [ fixup Andre's address. ]
> >> >
> >> > On Wed, Jan 16, 2013 at 11:33:21PM +0100, Rafael J. Wysocki wrote:
> >> > > > down the frequancy, but is accompanied by an info-level kernel
> >> > > > message "acpi-cpufreq: overriding BIOS provided _PSD data".
> >> >
> >> > That just says that BIOS has crappy data and we're using our own.
> >>
> >> OK. Yes, my BIOS is broken, so I thought it was the reason why
> >> autoloading does not work.
> >
> > Hmm, come to think of it, we're issuing this message on SMP on *all*
> > AMDs with HW_PSTATE. Andre, remind me again why we're doing this? We're
> > basically saying that we're overriding ACPI data but we still read it
> > out from acpi_perf_data and use *that* data to prepare the frequencies
> > table. What am I missing?
> >
> >> To be more precise: "modprobe powernow-k8" returns a "no such device"
> >> error,
> >
> > That's correct - we say ENODEV on your CPU which supports hardware
> > P-states and hand off to acpi-cpufreq which has that functionality now.
> >
> >> but in kernel logs I see that acpi-cpufreq is indeed loaded:
> >> -------
> >> Jan 15 22:42:19 metal-0 kernel: [  534.003995] acpi-cpufreq: overriding
> >> BIOS provided _PSD data
> >> Jan 15 22:42:32 metal-0 kernel: [  547.122695] powernow-k8: this CPU is
> >> not supported anymore, using acpi-cpufreq instead.
> >> Jan 15 22:42:45 metal-0 kernel: [  559.623068] powernow-k8: this CPU is
> >> not supported anymore, using acpi-cpufreq instead.
> >> -------
> >> (the CPU in question is phenom II x4 955). However, this is only after I
> >> attempt to _manually_ load powernow-k8 (the timestamps in [...] are ~6
> >> mins after boot is completed). So the above-mentioned handoff seems to
> >> work, but please find attached my kernel config.
> >
> > Ok, this is maybe the issue. What used to load powernow-k8 on your
> > distro before? Because basically it's enough if some script did
> > 'modprobe powernow-k8' for the handoff to just work.
> >
> > But, we also have the CPU autoprobing deal:
> >
> > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246
> 
> Arch relies entirely on the CPU autoprobing, there are no scripts
> modprobing things at boot.

That's right, processor drivers used to be automatically loaded alongside
with kvm and kvm_amd. Now, only the KVM stuff is loaded.

> 
> > so, *actually*, powernow-k8 should be loaded by default. I dunno, maybe
> > upgrade udev on your distro?
> 
> Assuming Leonid's system is up-to-date, Arch ships the most recent
> udev version (197).

Yes, the system is fully updated running systemd/udev 197.

> 
> Cheers,
> 
> Tom

Thank you,
L.

-- 
Leonid Isaev
GnuPG key: 0x164B5A6D
Fingerprint: C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-17  9:46             ` Borislav Petkov
  2013-01-17 10:34               ` Tom Gundersen
@ 2013-01-18  1:40               ` Leonid Isaev
  2013-01-18 11:37                 ` Borislav Petkov
  1 sibling, 1 reply; 19+ messages in thread
From: Leonid Isaev @ 2013-01-18  1:40 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Rafael J. Wysocki, Tom Gundersen, André Przywara,
	Matthew Garrett, cpufreq, linux-acpi

[-- Attachment #1: Type: text/plain, Size: 3015 bytes --]

On Thu, 17 Jan 2013 10:46:48 +0100
Borislav Petkov <bp@alien8.de> wrote:

> On Wed, Jan 16, 2013 at 06:16:00PM -0600, Leonid Isaev wrote:
> > > [ fixup Andre's address. ]
> > > 
> > > On Wed, Jan 16, 2013 at 11:33:21PM +0100, Rafael J. Wysocki wrote:
> > > > > down the frequancy, but is accompanied by an info-level kernel
> > > > > message "acpi-cpufreq: overriding BIOS provided _PSD data".
> > > 
> > > That just says that BIOS has crappy data and we're using our own.
> > 
> > OK. Yes, my BIOS is broken, so I thought it was the reason why autoloading
> > does not work.
> 
> Hmm, come to think of it, we're issuing this message on SMP on *all*
> AMDs with HW_PSTATE. Andre, remind me again why we're doing this? We're
> basically saying that we're overriding ACPI data but we still read it
> out from acpi_perf_data and use *that* data to prepare the frequencies
> table. What am I missing?
> 
> > To be more precise: "modprobe powernow-k8" returns a "no such device"
> > error,
> 
> That's correct - we say ENODEV on your CPU which supports hardware
> P-states and hand off to acpi-cpufreq which has that functionality now.
> 
> > but in kernel logs I see that acpi-cpufreq is indeed loaded:
> > -------
> > Jan 15 22:42:19 metal-0 kernel: [  534.003995] acpi-cpufreq: overriding
> > BIOS provided _PSD data
> > Jan 15 22:42:32 metal-0 kernel: [  547.122695] powernow-k8: this CPU is not
> > supported anymore, using acpi-cpufreq instead.
> > Jan 15 22:42:45 metal-0 kernel: [  559.623068] powernow-k8: this CPU is
> > not supported anymore, using acpi-cpufreq instead.
> > -------
> > (the CPU in question is phenom II x4 955). However, this is only after I
> > attempt to _manually_ load powernow-k8 (the timestamps in [...] are ~6 mins
> > after boot is completed). So the above-mentioned handoff seems to work, but
> > please find attached my kernel config.
> 
> Ok, this is maybe the issue. What used to load powernow-k8 on your
> distro before? Because basically it's enough if some script did
> 'modprobe powernow-k8' for the handoff to just work.

Arrh, modprobe still does not exit cleanly with this...

> 
> But, we also have the CPU autoprobing deal:
> 
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246
> 
> so, *actually*, powernow-k8 should be loaded by default. I dunno, maybe
> upgrade udev on your distro?

Udev 196 and 197 + linux <= 3.6.11 yield correct CPU autoprobing. However,
udev 197 + linux 3.7.2 do not. I haven't tested udev 196 with 3.7.2 though,
but it does not look like a udev issue.

I don't know if it matters, but I also have AMD microcode "0x10000c8", albeit
it didn't cause problems with 3.6.11...

> 
> Btw, your config looks fine:
> 
> CONFIG_X86_POWERNOW_K8=m
> CONFIG_X86_ACPI_CPUFREQ=m
> 
> Thanks.
>  

Sincerely,
L.
-- 
Leonid Isaev
GnuPG key: 0x164B5A6D
Fingerprint: C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18  1:40               ` Leonid Isaev
@ 2013-01-18 11:37                 ` Borislav Petkov
  2013-01-18 12:32                   ` André Przywara
  2013-01-18 16:59                   ` Leonid Isaev
  0 siblings, 2 replies; 19+ messages in thread
From: Borislav Petkov @ 2013-01-18 11:37 UTC (permalink / raw)
  To: Leonid Isaev
  Cc: Rafael J. Wysocki, Tom Gundersen, André Przywara,
	Matthew Garrett, cpufreq, linux-acpi

On Thu, Jan 17, 2013 at 07:40:44PM -0600, Leonid Isaev wrote:
> Arrh, modprobe still does not exit cleanly with this...

As I said already above:

>> That's correct - we say ENODEV on your CPU which supports hardware
>> P-states and hand off to acpi-cpufreq which has that functionality
>> now.

It is supposed to work that way: we return an error from the powernow-k8
init function so that it doesn't load but hand off to acpi-cpufreq so
that it gets loaded instead.

> > But, we also have the CPU autoprobing deal:
> > 
> > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246
> > 
> > so, *actually*, powernow-k8 should be loaded by default. I dunno, maybe
> > upgrade udev on your distro?
> 
> Udev 196 and 197 + linux <= 3.6.11 yield correct CPU autoprobing. However,
> udev 197 + linux 3.7.2 do not. I haven't tested udev 196 with 3.7.2 though,
> but it does not look like a udev issue.

Hmm, I'll bet this has something to do with the fact that HW_PSTATE is
in another CPUID function on AMD than on Intel and udev doesn't see that
bit to load acpi-cpufreq automatically.

Before I go and install archlinux here, where can I get the udev sources
which are in your archlinux installation to stare at them a little? :)
Especially the cpu autoprobing part which supposedly uses the cpuid
kernel driver.

> I don't know if it matters, but I also have AMD microcode "0x10000c8",
> albeit it didn't cause problems with 3.6.11...

Nah, microcode doesn't have anything to do with it.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 11:37                 ` Borislav Petkov
@ 2013-01-18 12:32                   ` André Przywara
  2013-01-18 15:17                     ` Matthew Garrett
  2013-01-18 17:13                     ` Leonid Isaev
  2013-01-18 16:59                   ` Leonid Isaev
  1 sibling, 2 replies; 19+ messages in thread
From: André Przywara @ 2013-01-18 12:32 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Leonid Isaev, Rafael J. Wysocki, Tom Gundersen, Matthew Garrett,
	cpufreq, linux-acpi

On Fri, 18 Jan 2013 12:37:17 +0100
Borislav Petkov <bp@alien8.de> wrote:

(updating Matthew's email as per MAINTAINERS)

> On Thu, Jan 17, 2013 at 07:40:44PM -0600, Leonid Isaev wrote:
> > Arrh, modprobe still does not exit cleanly with this...
> 
> As I said already above:
> 
> >> That's correct - we say ENODEV on your CPU which supports hardware
> >> P-states and hand off to acpi-cpufreq which has that functionality
> >> now.
> 
> It is supposed to work that way: we return an error from the
> powernow-k8 init function so that it doesn't load but hand off to
> acpi-cpufreq so that it gets loaded instead.
> 
> > > But, we also have the CPU autoprobing deal:
> > > 
> > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246
> > > 
> > > so, *actually*, powernow-k8 should be loaded by default. I dunno,
> > > maybe upgrade udev on your distro?
> > 
> > Udev 196 and 197 + linux <= 3.6.11 yield correct CPU autoprobing.
> > However, udev 197 + linux 3.7.2 do not. I haven't tested udev 196
> > with 3.7.2 though, but it does not look like a udev issue.
> 
> Hmm, I'll bet this has something to do with the fact that HW_PSTATE is
> in another CPUID function on AMD than on Intel and udev doesn't see
> that bit to load acpi-cpufreq automatically.

But the acpi-cpufreq does not export it's dependency on some CPUID
flags, since it is an _ACPI_ module. Shouldn't the load be triggered
while parsing the ACPI tree instead?

$ /sbin/modinfo drivers/cpufreq/acpi-cpufreq.ko | grep alias
alias:          acpi

$ /sbin/modinfo drivers/cpufreq/powernow-k8.ko | grep alias
alias:          x86cpu:vendor:0002:family:000F:model:*:feature:*

How does ArchLinux loads acpi-cpufreq on Intel CPUs? It should now be
the same mechanism on AMD. Or does it have acpi-cpufreq somehow
hard-coded in the scripts for GenuineIntel?

Regards,
Andre.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 12:32                   ` André Przywara
@ 2013-01-18 15:17                     ` Matthew Garrett
  2013-01-18 15:26                       ` Borislav Petkov
  2013-01-19 12:08                       ` Borislav Petkov
  2013-01-18 17:13                     ` Leonid Isaev
  1 sibling, 2 replies; 19+ messages in thread
From: Matthew Garrett @ 2013-01-18 15:17 UTC (permalink / raw)
  To: André Przywara
  Cc: Borislav Petkov, Leonid Isaev, Rafael J. Wysocki, Tom Gundersen,
	cpufreq, linux-acpi

Can you try this (entirely untested) patch?

diff --git a/drivers/cpufreq/acpi-cpufreq.c
b/drivers/cpufreq/acpi-cpufreq.c
index 0d048f6..7b0d49d 100644
--- a/drivers/cpufreq/acpi-cpufreq.c
+++ b/drivers/cpufreq/acpi-cpufreq.c
@@ -1030,4 +1030,11 @@ MODULE_PARM_DESC(acpi_pstate_strict,
 late_initcall(acpi_cpufreq_init);
 module_exit(acpi_cpufreq_exit);
 
+static const struct x86_cpu_id acpi_cpufreq_ids[] = {
+	X86_FEATURE_MATCH(X86_FEATURE_ACPI),
+	X86_FEATURE_MATCH(X86_FEATURE_HW_PSTATE),
+	{}
+};
+MODULE_DEVICE_TABLE(x86cpu, acpi_cpufreq_ids);
+
 MODULE_ALIAS("acpi");


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 15:17                     ` Matthew Garrett
@ 2013-01-18 15:26                       ` Borislav Petkov
  2013-01-18 15:32                         ` Matthew Garrett
  2013-01-19 12:08                       ` Borislav Petkov
  1 sibling, 1 reply; 19+ messages in thread
From: Borislav Petkov @ 2013-01-18 15:26 UTC (permalink / raw)
  To: Matthew Garrett
  Cc: André Przywara, Leonid Isaev, Rafael J. Wysocki,
	Tom Gundersen, cpufreq, linux-acpi

On Fri, Jan 18, 2013 at 03:17:04PM +0000, Matthew Garrett wrote:
> Can you try this (entirely untested) patch?
> 
> diff --git a/drivers/cpufreq/acpi-cpufreq.c
> b/drivers/cpufreq/acpi-cpufreq.c
> index 0d048f6..7b0d49d 100644
> --- a/drivers/cpufreq/acpi-cpufreq.c
> +++ b/drivers/cpufreq/acpi-cpufreq.c
> @@ -1030,4 +1030,11 @@ MODULE_PARM_DESC(acpi_pstate_strict,
>  late_initcall(acpi_cpufreq_init);
>  module_exit(acpi_cpufreq_exit);
>  
> +static const struct x86_cpu_id acpi_cpufreq_ids[] = {
> +	X86_FEATURE_MATCH(X86_FEATURE_ACPI),
> +	X86_FEATURE_MATCH(X86_FEATURE_HW_PSTATE),
> +	{}

Yep, that should be one way to fix it.

One other fix IMHO would be if udev is looking at CPUID bits, to teach
it to check the proper P-States feature bits on Intel and AMD:

On Intel: CPUID_0x00000001[ECX] bit 7
On AMD  : CPUID_0x80000007[EDX] bit 7

AFAICT.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 15:26                       ` Borislav Petkov
@ 2013-01-18 15:32                         ` Matthew Garrett
  2013-01-18 17:10                           ` Borislav Petkov
  0 siblings, 1 reply; 19+ messages in thread
From: Matthew Garrett @ 2013-01-18 15:32 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: André Przywara, Leonid Isaev, Rafael J. Wysocki,
	Tom Gundersen, cpufreq, linux-acpi

On Fri, 2013-01-18 at 16:26 +0100, Borislav Petkov wrote:
> Yep, that should be one way to fix it.
> 
> One other fix IMHO would be if udev is looking at CPUID bits, to teach
> it to check the proper P-States feature bits on Intel and AMD:

I think it makes more sense to use the existing CPU modaliases than to
special-case it in udev.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 11:37                 ` Borislav Petkov
  2013-01-18 12:32                   ` André Przywara
@ 2013-01-18 16:59                   ` Leonid Isaev
  1 sibling, 0 replies; 19+ messages in thread
From: Leonid Isaev @ 2013-01-18 16:59 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Rafael J. Wysocki, Tom Gundersen, André Przywara,
	Matthew Garrett, cpufreq, linux-acpi

[-- Attachment #1: Type: text/plain, Size: 2096 bytes --]

On Fri, 18 Jan 2013 12:37:17 +0100
Borislav Petkov <bp@alien8.de> wrote:

> On Thu, Jan 17, 2013 at 07:40:44PM -0600, Leonid Isaev wrote:
> > Arrh, modprobe still does not exit cleanly with this...
> 
> As I said already above:
> 
> >> That's correct - we say ENODEV on your CPU which supports hardware
> >> P-states and hand off to acpi-cpufreq which has that functionality
> >> now.
> 
> It is supposed to work that way: we return an error from the powernow-k8
> init function so that it doesn't load but hand off to acpi-cpufreq so
> that it gets loaded instead.
> 
> > > But, we also have the CPU autoprobing deal:
> > > 
> > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246
> > > 
> > > so, *actually*, powernow-k8 should be loaded by default. I dunno, maybe
> > > upgrade udev on your distro?
> > 
> > Udev 196 and 197 + linux <= 3.6.11 yield correct CPU autoprobing. However,
> > udev 197 + linux 3.7.2 do not. I haven't tested udev 196 with 3.7.2 though,
> > but it does not look like a udev issue.
> 
> Hmm, I'll bet this has something to do with the fact that HW_PSTATE is
> in another CPUID function on AMD than on Intel and udev doesn't see that
> bit to load acpi-cpufreq automatically.
> 
> Before I go and install archlinux here, where can I get the udev sources
> which are in your archlinux installation to stare at them a little? :)
> Especially the cpu autoprobing part which supposedly uses the cpuid
> kernel driver.

They are vanilla systemd-197
(http://www.freedesktop.org/software/systemd/systemd-197.tar.xz) with only 1
dbus-related patch
(https://projects.archlinux.org/svntogit/packages.git/tree/trunk?h=packages/systemd).

> 
> > I don't know if it matters, but I also have AMD microcode "0x10000c8",
> > albeit it didn't cause problems with 3.6.11...
> 
> Nah, microcode doesn't have anything to do with it.
> 
> Thanks.
> 

Thanks,
L.

-- 
Leonid Isaev
GnuPG key: 0x164B5A6D
Fingerprint: C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 15:32                         ` Matthew Garrett
@ 2013-01-18 17:10                           ` Borislav Petkov
  2013-01-18 17:17                             ` Matthew Garrett
  0 siblings, 1 reply; 19+ messages in thread
From: Borislav Petkov @ 2013-01-18 17:10 UTC (permalink / raw)
  To: Matthew Garrett
  Cc: André Przywara, Leonid Isaev, Rafael J. Wysocki,
	Tom Gundersen, cpufreq, linux-acpi

On Fri, Jan 18, 2013 at 03:32:45PM +0000, Matthew Garrett wrote:
> On Fri, 2013-01-18 at 16:26 +0100, Borislav Petkov wrote:
> > Yep, that should be one way to fix it.
> > 
> > One other fix IMHO would be if udev is looking at CPUID bits, to teach
> > it to check the proper P-States feature bits on Intel and AMD:
> 
> I think it makes more sense to use the existing CPU modaliases than to
> special-case it in udev.

Well, actually, those CPUID bits are there for exactly that reason: to
query them and enable software features. And this is one of the reasons
CPUID is a ring 3 instruction: so that even userspace can use it.

So, in a perfect world, udev should simply run CPUID, check the
respective bits, and, if they're set, load the respective driver. That
is, if it doesn't do it already.

No need for additional code glue in the kernel or anywhere else.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 12:32                   ` André Przywara
  2013-01-18 15:17                     ` Matthew Garrett
@ 2013-01-18 17:13                     ` Leonid Isaev
  1 sibling, 0 replies; 19+ messages in thread
From: Leonid Isaev @ 2013-01-18 17:13 UTC (permalink / raw)
  To: André Przywara
  Cc: Borislav Petkov, Rafael J. Wysocki, Tom Gundersen,
	Matthew Garrett, cpufreq, linux-acpi

[-- Attachment #1: Type: text/plain, Size: 2624 bytes --]

On Fri, 18 Jan 2013 13:32:33 +0100
André Przywara <andre@andrep.de> wrote:

> On Fri, 18 Jan 2013 12:37:17 +0100
> Borislav Petkov <bp@alien8.de> wrote:
> 
> (updating Matthew's email as per MAINTAINERS)
> 
> > On Thu, Jan 17, 2013 at 07:40:44PM -0600, Leonid Isaev wrote:
> > > Arrh, modprobe still does not exit cleanly with this...
> > 
> > As I said already above:
> > 
> > >> That's correct - we say ENODEV on your CPU which supports hardware
> > >> P-states and hand off to acpi-cpufreq which has that functionality
> > >> now.
> > 
> > It is supposed to work that way: we return an error from the
> > powernow-k8 init function so that it doesn't load but hand off to
> > acpi-cpufreq so that it gets loaded instead.
> > 
> > > > But, we also have the CPU autoprobing deal:
> > > > 
> > > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=644e9cbbe3fc032cc92d0936057e166a994dc246
> > > > 
> > > > so, *actually*, powernow-k8 should be loaded by default. I dunno,
> > > > maybe upgrade udev on your distro?
> > > 
> > > Udev 196 and 197 + linux <= 3.6.11 yield correct CPU autoprobing.
> > > However, udev 197 + linux 3.7.2 do not. I haven't tested udev 196
> > > with 3.7.2 though, but it does not look like a udev issue.
> > 
> > Hmm, I'll bet this has something to do with the fact that HW_PSTATE is
> > in another CPUID function on AMD than on Intel and udev doesn't see
> > that bit to load acpi-cpufreq automatically.
> 
> But the acpi-cpufreq does not export it's dependency on some CPUID
> flags, since it is an _ACPI_ module. Shouldn't the load be triggered
> while parsing the ACPI tree instead?
> 
> $ /sbin/modinfo drivers/cpufreq/acpi-cpufreq.ko | grep alias
> alias:          acpi
> 
> $ /sbin/modinfo drivers/cpufreq/powernow-k8.ko | grep alias
> alias:          x86cpu:vendor:0002:family:000F:model:*:feature:*
> 
> How does ArchLinux loads acpi-cpufreq on Intel CPUs? It should now be
> the same mechanism on AMD. Or does it have acpi-cpufreq somehow
> hard-coded in the scripts for GenuineIntel?

In both cases Intel and AMD we rely on the kernel/udev autoprobing to insert
correct CPU and KVM drivers, i.e. there are no custom scripts or manual loading
of modules via systemd. However, in Intel's case acpi_cpufreq is loaded
correctly (and always has been). Indeed, on an Intel core2duo:
 $ dmesg | grep -i cpufreq
[    9.005851] ACPI: Requesting acpi_cpufreq

> 
> Regards,
> Andre.

Sincerely,
L.

-- 
Leonid Isaev
GnuPG key: 0x164B5A6D
Fingerprint: C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 17:10                           ` Borislav Petkov
@ 2013-01-18 17:17                             ` Matthew Garrett
  0 siblings, 0 replies; 19+ messages in thread
From: Matthew Garrett @ 2013-01-18 17:17 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: André Przywara, Leonid Isaev, Rafael J. Wysocki,
	Tom Gundersen, cpufreq, linux-acpi

On Fri, 2013-01-18 at 18:10 +0100, Borislav Petkov wrote:

> Well, actually, those CPUID bits are there for exactly that reason: to
> query them and enable software features. And this is one of the reasons
> CPUID is a ring 3 instruction: so that even userspace can use it.
> 
> So, in a perfect world, udev should simply run CPUID, check the
> respective bits, and, if they're set, load the respective driver. That
> is, if it doesn't do it already.
> 
> No need for additional code glue in the kernel or anywhere else.

We already have the additional code in the kernel - it means we can add
modaliases when new functionality gets added to the kernel, instead of
having to update udev as well. We /could/ have udev use outb and inb to
identify PCI devices as well, but it seems more reasonable to use the
information the kernel already has and exposes :)


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-18 15:17                     ` Matthew Garrett
  2013-01-18 15:26                       ` Borislav Petkov
@ 2013-01-19 12:08                       ` Borislav Petkov
  2013-01-21  5:14                         ` Leonid Isaev
  1 sibling, 1 reply; 19+ messages in thread
From: Borislav Petkov @ 2013-01-19 12:08 UTC (permalink / raw)
  To: Matthew Garrett, Leonid Isaev
  Cc: André Przywara, Rafael J. Wysocki, Tom Gundersen, cpufreq,
	linux-acpi

On Fri, Jan 18, 2013 at 03:17:04PM +0000, Matthew Garrett wrote:
> Can you try this (entirely untested) patch?
> 
> diff --git a/drivers/cpufreq/acpi-cpufreq.c
> b/drivers/cpufreq/acpi-cpufreq.c
> index 0d048f6..7b0d49d 100644
> --- a/drivers/cpufreq/acpi-cpufreq.c
> +++ b/drivers/cpufreq/acpi-cpufreq.c
> @@ -1030,4 +1030,11 @@ MODULE_PARM_DESC(acpi_pstate_strict,
>  late_initcall(acpi_cpufreq_init);
>  module_exit(acpi_cpufreq_exit);
>  
> +static const struct x86_cpu_id acpi_cpufreq_ids[] = {
> +	X86_FEATURE_MATCH(X86_FEATURE_ACPI),
> +	X86_FEATURE_MATCH(X86_FEATURE_HW_PSTATE),
> +	{}
> +};
> +MODULE_DEVICE_TABLE(x86cpu, acpi_cpufreq_ids);
> +
>  MODULE_ALIAS("acpi");

Leonid, can you try Matthew's patch?

This is adding the additional cpufreatures aliases:

alias:          x86cpu:vendor:*:family:*:model:*:feature:*00E8*
alias:          x86cpu:vendor:*:family:*:model:*:feature:*0016*

and since powernow-k8 used to load on 3.6-stable, it probably happened
because the 0xe8 feature and since 3.7 moved it to acpi-cpufreq, this
patch should actually fix it.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-19 12:08                       ` Borislav Petkov
@ 2013-01-21  5:14                         ` Leonid Isaev
  2013-01-21 10:54                           ` Borislav Petkov
  0 siblings, 1 reply; 19+ messages in thread
From: Leonid Isaev @ 2013-01-21  5:14 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Matthew Garrett, André Przywara, Rafael J. Wysocki,
	Tom Gundersen, cpufreq, linux-acpi

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

On Sat, 19 Jan 2013 13:08:14 +0100
Borislav Petkov <bp@alien8.de> wrote:

> On Fri, Jan 18, 2013 at 03:17:04PM +0000, Matthew Garrett wrote:
> > Can you try this (entirely untested) patch?
> > 
> > diff --git a/drivers/cpufreq/acpi-cpufreq.c
> > b/drivers/cpufreq/acpi-cpufreq.c
> > index 0d048f6..7b0d49d 100644
> > --- a/drivers/cpufreq/acpi-cpufreq.c
> > +++ b/drivers/cpufreq/acpi-cpufreq.c
> > @@ -1030,4 +1030,11 @@ MODULE_PARM_DESC(acpi_pstate_strict,
> >  late_initcall(acpi_cpufreq_init);
> >  module_exit(acpi_cpufreq_exit);
> >  
> > +static const struct x86_cpu_id acpi_cpufreq_ids[] = {
> > +	X86_FEATURE_MATCH(X86_FEATURE_ACPI),
> > +	X86_FEATURE_MATCH(X86_FEATURE_HW_PSTATE),
> > +	{}
> > +};
> > +MODULE_DEVICE_TABLE(x86cpu, acpi_cpufreq_ids);
> > +
> >  MODULE_ALIAS("acpi");
> 
> Leonid, can you try Matthew's patch?
> 
> This is adding the additional cpufreatures aliases:
> 
> alias:          x86cpu:vendor:*:family:*:model:*:feature:*00E8*
> alias:          x86cpu:vendor:*:family:*:model:*:feature:*0016*
> 
> and since powernow-k8 used to load on 3.6-stable, it probably happened
> because the 0xe8 feature and since 3.7 moved it to acpi-cpufreq, this
> patch should actually fix it.
> 
> Thanks.
> 

Matthew and Borislav, thanks a lot for the patch and explanation!

I applied the above patch on top of linux 3.7.3 -- it does generate the
aliases and fixes the autoprobing issue, at least on my AMD machine. Should we
expect it in 3.7.4 or 3.8? Otherwise, I'll forward it to archlinux devel ML to
be possibly included in current -stable kernel. Also, Borislav, this patch is
not in your patch series, is it?

Thanks again,
L.

-- 
Leonid Isaev
GnuPG key: 0x164B5A6D
Fingerprint: C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [arch-general] powernow-k8 fails to load with linux 3.7.2
  2013-01-21  5:14                         ` Leonid Isaev
@ 2013-01-21 10:54                           ` Borislav Petkov
  0 siblings, 0 replies; 19+ messages in thread
From: Borislav Petkov @ 2013-01-21 10:54 UTC (permalink / raw)
  To: Leonid Isaev
  Cc: Matthew Garrett, André Przywara, Rafael J. Wysocki,
	Tom Gundersen, cpufreq, linux-acpi

On Sun, Jan 20, 2013 at 11:14:32PM -0600, Leonid Isaev wrote:
> Matthew and Borislav, thanks a lot for the patch and explanation!
>
> I applied the above patch on top of linux 3.7.3 -- it does generate
> the aliases and fixes the autoprobing issue, at least on my AMD
> machine. Should we expect it in 3.7.4 or 3.8? Otherwise, I'll forward
> it to archlinux devel ML to be possibly included in current -stable
> kernel. Also, Borislav, this patch is not in your patch series, is it?

Yeah, it's not.

Matthew, care to write a proper commit message with your SOB and
Leonid's Tested-by pls?

Then Rafael could take it for 3.8 and tag it for 3.7-stable, methinks.

Leonid, thanks for testing.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2013-01-21 10:54 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20130115224755.085aa6b7@bluemoon>
     [not found] ` <CAG-2HqWPOYKVCj-Z=z_1e=mE6b6z9hMYFJzHf2GYAmTwQovN0A@mail.gmail.com>
     [not found]   ` <20130116131725.4e590c34@bluemoon>
2013-01-16 20:37     ` [arch-general] powernow-k8 fails to load with linux 3.7.2 Tom Gundersen
2013-01-16 22:33       ` Rafael J. Wysocki
2013-01-16 23:17         ` Borislav Petkov
     [not found]           ` <20130116181600.4c48ee3b@bluemoon>
2013-01-17  9:46             ` Borislav Petkov
2013-01-17 10:34               ` Tom Gundersen
2013-01-18  1:27                 ` Leonid Isaev
2013-01-18  1:40               ` Leonid Isaev
2013-01-18 11:37                 ` Borislav Petkov
2013-01-18 12:32                   ` André Przywara
2013-01-18 15:17                     ` Matthew Garrett
2013-01-18 15:26                       ` Borislav Petkov
2013-01-18 15:32                         ` Matthew Garrett
2013-01-18 17:10                           ` Borislav Petkov
2013-01-18 17:17                             ` Matthew Garrett
2013-01-19 12:08                       ` Borislav Petkov
2013-01-21  5:14                         ` Leonid Isaev
2013-01-21 10:54                           ` Borislav Petkov
2013-01-18 17:13                     ` Leonid Isaev
2013-01-18 16:59                   ` Leonid Isaev

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.