[PATCH v2 0/3] Cache id

[PATCH v2 0/3] Cache id

[Fenghua Yu]

From: Fenghua Yu <fenghua.yu@intel.com>

This patch set introduces cache id to identify a cache in platform. It can
be useful in such areas as Cach Allocation Technology (CAT) where user
needs to specify how much cache is allocated on which cache. Cache id
provides a concise way to identify the cache. CAT patches will be released
separately.

Changes:
v2: Split one patch into three patches and add ABI documentation.

Fenghua Yu (3):
  cacheinfo: Introduce cache id
  Documentation, ABI: Add a document entry for cache id
  x86, intel_cacheinfo: Enable cache id in x86

 Documentation/ABI/testing/sysfs-devices-system-cpu | 13 +++++++++++++
 arch/x86/kernel/cpu/intel_cacheinfo.c              | 20 ++++++++++++++++++++
 drivers/base/cacheinfo.c                           |  5 +++++
 include/linux/cacheinfo.h                          |  3 +++
 4 files changed, 41 insertions(+)

-- 
2.5.0

[PATCH v2 1/3] cacheinfo: Introduce cache id

[Fenghua Yu]

From: Fenghua Yu <fenghua.yu@intel.com>

Each cache is described by cacheinfo and is unique in the same index
across the platform. But there is no id for a cache. We introduce cache
ID to identify a cache.

Intel Cache Allocation Technology (CAT) allows some control on the
allocation policy within each cache that it controls. We need a unique
cache ID for each cache level to allow the user to specify which
controls are applied to which cache. Cache id is a concise way to specify
a cache.

Cache id is first enabled on x86. It can be enabled on other platforms
as well. The cache id is not necessary contiguous.

Add an "id" entry to /sys/devices/system/cpu/cpu*/cache/index*/

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 drivers/base/cacheinfo.c  | 5 +++++
 include/linux/cacheinfo.h | 3 +++
 2 files changed, 8 insertions(+)

diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c
index e9fd32e..2a21c15 100644
--- a/drivers/base/cacheinfo.c
+++ b/drivers/base/cacheinfo.c
@@ -233,6 +233,7 @@ static ssize_t file_name##_show(struct device *dev,		\
 	return sprintf(buf, "%u\n", this_leaf->object);		\
 }
 
+show_one(id, id);
 show_one(level, level);
 show_one(coherency_line_size, coherency_line_size);
 show_one(number_of_sets, number_of_sets);
@@ -314,6 +315,7 @@ static ssize_t write_policy_show(struct device *dev,
 	return n;
 }
 
+static DEVICE_ATTR_RO(id);
 static DEVICE_ATTR_RO(level);
 static DEVICE_ATTR_RO(type);
 static DEVICE_ATTR_RO(coherency_line_size);
@@ -327,6 +329,7 @@ static DEVICE_ATTR_RO(shared_cpu_list);
 static DEVICE_ATTR_RO(physical_line_partition);
 
 static struct attribute *cache_default_attrs[] = {
+	&dev_attr_id.attr,
 	&dev_attr_type.attr,
 	&dev_attr_level.attr,
 	&dev_attr_shared_cpu_map.attr,
@@ -350,6 +353,8 @@ cache_default_attrs_is_visible(struct kobject *kobj,
 	const struct cpumask *mask = &this_leaf->shared_cpu_map;
 	umode_t mode = attr->mode;
 
+	if ((attr == &dev_attr_id.attr) && this_leaf->attributes & CACHE_ID)
+		return mode;
 	if ((attr == &dev_attr_type.attr) && this_leaf->type)
 		return mode;
 	if ((attr == &dev_attr_level.attr) && this_leaf->level)
diff --git a/include/linux/cacheinfo.h b/include/linux/cacheinfo.h
index 2189935..cf6984d 100644
--- a/include/linux/cacheinfo.h
+++ b/include/linux/cacheinfo.h
@@ -18,6 +18,7 @@ enum cache_type {
 
 /**
  * struct cacheinfo - represent a cache leaf node
+ * @id: This cache's id. ID is unique in the same index on the platform.
  * @type: type of the cache - data, inst or unified
  * @level: represents the hierarchy in the multi-level cache
  * @coherency_line_size: size of each cache line usually representing
@@ -44,6 +45,7 @@ enum cache_type {
  * keeping, the remaining members form the core properties of the cache
  */
 struct cacheinfo {
+	unsigned int id;
 	enum cache_type type;
 	unsigned int level;
 	unsigned int coherency_line_size;
@@ -61,6 +63,7 @@ struct cacheinfo {
 #define CACHE_WRITE_ALLOCATE	BIT(3)
 #define CACHE_ALLOCATE_POLICY_MASK	\
 	(CACHE_READ_ALLOCATE | CACHE_WRITE_ALLOCATE)
+#define CACHE_ID		BIT(4)
 
 	struct device_node *of_node;
 	bool disable_sysfs;
-- 
2.5.0

[PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Fenghua Yu]

From: Fenghua Yu <fenghua.yu@intel.com>

Add an ABI document entry for /sys/devices/system/cpu/cpu*/cache/index*/id.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 Documentation/ABI/testing/sysfs-devices-system-cpu | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index 1650133..cc62034 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -272,6 +272,19 @@ Description:	Parameters for the CPU cache attributes
 				     the modified cache line is written to main
 				     memory only when it is replaced
 
+
+What:		/sys/devices/system/cpu/cpu*/cache/index*/id
+Date:		July 2016
+Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>
+Description:	Cache id
+
+		The id identifies a cache in the platform. In same index, the id
+		is unique across the platform.
+
+		Currently id is implemented on x86. On other platforms, id is
+		not enabled yet.
+
+
 What:		/sys/devices/system/cpu/cpuX/cpufreq/throttle_stats
 		/sys/devices/system/cpu/cpuX/cpufreq/throttle_stats/turbo_stat
 		/sys/devices/system/cpu/cpuX/cpufreq/throttle_stats/sub_turbo_stat
-- 
2.5.0

[PATCH v2 3/3] x86, intel_cacheinfo: Enable cache id in x86

[Fenghua Yu]

From: Fenghua Yu <fenghua.yu@intel.com>

Enable cache id in x86. Cache id comes from APIC ID and CPUID4.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_cacheinfo.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_cacheinfo.c b/arch/x86/kernel/cpu/intel_cacheinfo.c
index de6626c..8dc5720 100644
--- a/arch/x86/kernel/cpu/intel_cacheinfo.c
+++ b/arch/x86/kernel/cpu/intel_cacheinfo.c
@@ -153,6 +153,7 @@ struct _cpuid4_info_regs {
 	union _cpuid4_leaf_eax eax;
 	union _cpuid4_leaf_ebx ebx;
 	union _cpuid4_leaf_ecx ecx;
+	unsigned int id;
 	unsigned long size;
 	struct amd_northbridge *nb;
 };
@@ -894,6 +895,8 @@ static void __cache_cpumap_setup(unsigned int cpu, int index,
 static void ci_leaf_init(struct cacheinfo *this_leaf,
 			 struct _cpuid4_info_regs *base)
 {
+	this_leaf->id = base->id;
+	this_leaf->attributes = CACHE_ID;
 	this_leaf->level = base->eax.split.level;
 	this_leaf->type = cache_type_map[base->eax.split.type];
 	this_leaf->coherency_line_size =
@@ -920,6 +923,22 @@ static int __init_cache_level(unsigned int cpu)
 	return 0;
 }
 
+/*
+ * The max shared threads number comes from CPUID.4:EAX[25-14] with input
+ * ECX as cache index. Then right shift apicid by the number's order to get
+ * cache id for this cache node.
+ */
+static void get_cache_id(int cpu, struct _cpuid4_info_regs *id4_regs)
+{
+	struct cpuinfo_x86 *c = &cpu_data(cpu);
+	unsigned long num_threads_sharing;
+	int index_msb;
+
+	num_threads_sharing = 1 + id4_regs->eax.split.num_threads_sharing;
+	index_msb = get_count_order(num_threads_sharing);
+	id4_regs->id = c->apicid >> index_msb;
+}
+
 static int __populate_cache_leaves(unsigned int cpu)
 {
 	unsigned int idx, ret;
@@ -931,6 +950,7 @@ static int __populate_cache_leaves(unsigned int cpu)
 		ret = cpuid4_cache_lookup_regs(idx, &id4_regs);
 		if (ret)
 			return ret;
+		get_cache_id(cpu, &id4_regs);
 		ci_leaf_init(this_leaf++, &id4_regs);
 		__cache_cpumap_setup(cpu, idx, &id4_regs);
 	}
-- 
2.5.0

Re: [PATCH v2 0/3] Cache id

[Borislav Petkov]

On Wed, Jul 06, 2016 at 03:07:15PM -0700, Fenghua Yu wrote:
> From: Fenghua Yu <fenghua.yu@intel.com>
> 
> This patch set introduces cache id to identify a cache in platform. It can
> be useful in such areas as Cach Allocation Technology (CAT) where user
> needs to specify how much cache is allocated on which cache. Cache id
> provides a concise way to identify the cache. CAT patches will be released
> separately.
> 
> Changes:
> v2: Split one patch into three patches and add ABI documentation.
> 
> Fenghua Yu (3):
>   cacheinfo: Introduce cache id
>   Documentation, ABI: Add a document entry for cache id
>   x86, intel_cacheinfo: Enable cache id in x86
> 
>  Documentation/ABI/testing/sysfs-devices-system-cpu | 13 +++++++++++++
>  arch/x86/kernel/cpu/intel_cacheinfo.c              | 20 ++++++++++++++++++++
>  drivers/base/cacheinfo.c                           |  5 +++++
>  include/linux/cacheinfo.h                          |  3 +++
>  4 files changed, 41 insertions(+)

All 3:

Acked-by: Borislav Petkov <bp@suse.de>

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

RE: [PATCH v2 0/3] Cache id

[Yu, Fenghua]

> From: Borislav Petkov [mailto:bp@suse.de]
> Sent: Thursday, July 07, 2016 9:21 AM
> On Wed, Jul 06, 2016 at 03:07:15PM -0700, Fenghua Yu wrote:
> > From: Fenghua Yu <fenghua.yu@intel.com>
> >
> > This patch set introduces cache id to identify a cache in platform. It
> > can be useful in such areas as Cach Allocation Technology (CAT) where
> > user needs to specify how much cache is allocated on which cache.
> > Cache id provides a concise way to identify the cache. CAT patches
> > will be released separately.
> >
> > Changes:
> > v2: Split one patch into three patches and add ABI documentation.
> >
> > Fenghua Yu (3):
> >   cacheinfo: Introduce cache id
> >   Documentation, ABI: Add a document entry for cache id
> >   x86, intel_cacheinfo: Enable cache id in x86
> >
> >  Documentation/ABI/testing/sysfs-devices-system-cpu | 13
> +++++++++++++
> >  arch/x86/kernel/cpu/intel_cacheinfo.c              | 20
> ++++++++++++++++++++
> >  drivers/base/cacheinfo.c                           |  5 +++++
> >  include/linux/cacheinfo.h                          |  3 +++
> >  4 files changed, 41 insertions(+)
> 
> All 3:
> 
> Acked-by: Borislav Petkov <bp@suse.de>

That's great!

Is it possible to merge the patches to 4.8? Then I don't need to carry these patches with upcoming CAT enabling patches:)

Thanks.

-Fenghua

Re: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Ingo Molnar]

* Fenghua Yu <fenghua.yu@intel.com> wrote:

> From: Fenghua Yu <fenghua.yu@intel.com>
> 
> Add an ABI document entry for /sys/devices/system/cpu/cpu*/cache/index*/id.
> 
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> ---
>  Documentation/ABI/testing/sysfs-devices-system-cpu | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
> index 1650133..cc62034 100644
> --- a/Documentation/ABI/testing/sysfs-devices-system-cpu
> +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
> @@ -272,6 +272,19 @@ Description:	Parameters for the CPU cache attributes
>  				     the modified cache line is written to main
>  				     memory only when it is replaced
>  
> +
> +What:		/sys/devices/system/cpu/cpu*/cache/index*/id
> +Date:		July 2016
> +Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>
> +Description:	Cache id
> +
> +		The id identifies a cache in the platform. In same index, the id
> +		is unique across the platform.

What does 'In same index' mean?

Thanks,

	Ingo

RE: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Yu, Fenghua]

> From: Ingo Molnar [mailto:mingo.kernel.org@gmail.com] On Behalf Of Ingo
> Molnar
> Sent: Friday, July 08, 2016 1:42 AM
> * Fenghua Yu <fenghua.yu@intel.com> wrote:
> 
> > From: Fenghua Yu <fenghua.yu@intel.com>
> >
> > Add an ABI document entry for
> /sys/devices/system/cpu/cpu*/cache/index*/id.
> >
> > Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> > ---
> >  Documentation/ABI/testing/sysfs-devices-system-cpu | 13
> +++++++++++++
> >  1 file changed, 13 insertions(+)
> >
> > diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu
> b/Documentation/ABI/testing/sysfs-devices-system-cpu
> > index 1650133..cc62034 100644
> > --- a/Documentation/ABI/testing/sysfs-devices-system-cpu
> > +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
> > @@ -272,6 +272,19 @@ Description:	Parameters for the CPU cache
> attributes
> >  				     the modified cache line is written to main
> >  				     memory only when it is replaced
> >
> > +
> > +What:		/sys/devices/system/cpu/cpu*/cache/index*/id
> > +Date:		July 2016
> > +Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>
> > +Description:	Cache id
> > +
> > +		The id identifies a cache in the platform. In same index, the id
> > +		is unique across the platform.
> 
> What does 'In same index' mean?

It means one cache's id is unique in all caches with same cache index number. For example, in all caches with index3 (i.e. level3), cache id 0 is unique to identify a L3 cache. But in caches with index 0 (i.e. Level0), there is also a cache id 0. So cache id is unique in one index. But not unique in two different index.

Does that make sense? I hope I express that correctly.

Thanks.

-Fenghua

RE: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Luck, Tony]

> It means one cache's id is unique in all caches with same cache index number.
> For example, in all caches with index3 (i.e. level3), cache id 0 is unique to identify
> a L3 cache. But in caches with index 0 (i.e. Level0), there is also a cache id 0.
> So cache id is unique in one index. But not unique in two different index.

> Does that make sense? I hope I express that correctly.

We use "index" rather than "level" because that is the terminology used
in /sys/devices/system/cpu/cpu*/cache/index*

E.g. on most Intel cpus you'll typically find "index0" is the L1-data cache,
"index1" is the L1-instruction cache, "index3" is the L2-unified cache and
"index4" is the L3-unified cache.

-Tony

Re: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Ingo Molnar]

* Yu, Fenghua <fenghua.yu@intel.com> wrote:

> > From: Ingo Molnar [mailto:mingo.kernel.org@gmail.com] On Behalf Of Ingo
> > Molnar
> > Sent: Friday, July 08, 2016 1:42 AM
> > * Fenghua Yu <fenghua.yu@intel.com> wrote:
> > 
> > > From: Fenghua Yu <fenghua.yu@intel.com>
> > >
> > > Add an ABI document entry for
> > /sys/devices/system/cpu/cpu*/cache/index*/id.
> > >
> > > Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> > > ---
> > >  Documentation/ABI/testing/sysfs-devices-system-cpu | 13
> > +++++++++++++
> > >  1 file changed, 13 insertions(+)
> > >
> > > diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu
> > b/Documentation/ABI/testing/sysfs-devices-system-cpu
> > > index 1650133..cc62034 100644
> > > --- a/Documentation/ABI/testing/sysfs-devices-system-cpu
> > > +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
> > > @@ -272,6 +272,19 @@ Description:	Parameters for the CPU cache
> > attributes
> > >  				     the modified cache line is written to main
> > >  				     memory only when it is replaced
> > >
> > > +
> > > +What:		/sys/devices/system/cpu/cpu*/cache/index*/id
> > > +Date:		July 2016
> > > +Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>
> > > +Description:	Cache id
> > > +
> > > +		The id identifies a cache in the platform. In same index, the id
> > > +		is unique across the platform.
> > 
> > What does 'In same index' mean?
> 
> It means one cache's id is unique in all caches with same cache index number. 
> For example, in all caches with index3 (i.e. level3), cache id 0 is unique to 
> identify a L3 cache. But in caches with index 0 (i.e. Level0), there is also a 
> cache id 0. So cache id is unique in one index. But not unique in two different 
> index.
> 
> Does that make sense? I hope I express that correctly.

Yeah, makes sense!

I'd express it like that:

	The ID identifies a specific hardware cache of the machine
	the kernel is running on. The (depth_index, ID) pair uniquely
	identifies a cache.

And wherever you mention 'index' I'd change it over to 'depth index' - because 
'index' alone is way too opaque - the 'id' is an index too.

Also note how I avoided the 'platform' word - that is really ambigious as well: it 
might mean something generic like 'x86', not just the actual system we are on.

BTW., are there any ID enumeration continuity guarantees, or can there be holes in 
the depth index or the ID? (In the normal case, not considering CPU hotplug 
creating holes.) Is it the kernel that does the enumeration or do we trust what 
the hardware (CPUID) is telling us?

Thanks,

	Ingo

Re: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Ingo Molnar]

* Luck, Tony <tony.luck@intel.com> wrote:

> > It means one cache's id is unique in all caches with same cache index number.
> > For example, in all caches with index3 (i.e. level3), cache id 0 is unique to identify
> > a L3 cache. But in caches with index 0 (i.e. Level0), there is also a cache id 0.
> > So cache id is unique in one index. But not unique in two different index.
> 
> > Does that make sense? I hope I express that correctly.
> 
> We use "index" rather than "level" because that is the terminology used
> in /sys/devices/system/cpu/cpu*/cache/index*

Who can we ... thank for that nonsensical naming? :-/

> E.g. on most Intel cpus you'll typically find "index0" is the L1-data cache, 
> "index1" is the L1-instruction cache, "index3" is the L2-unified cache and 
> "index4" is the L3-unified cache.

Crazy. What was wrong with using 'level' or 'depth'?

Thanks,

	Ingo

Re: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Borislav Petkov]

On Fri, Jul 08, 2016 at 08:07:46PM +0200, Ingo Molnar wrote:
> Who can we ... thank for that nonsensical naming? :-/
> 
> > E.g. on most Intel cpus you'll typically find "index0" is the L1-data cache, 
> > "index1" is the L1-instruction cache, "index3" is the L2-unified cache and 
> > "index4" is the L3-unified cache.
> 
> Crazy. What was wrong with using 'level' or 'depth'?

It is all there:

$ grep . /sys/devices/system/cpu/cpu0/cache/index?/level
/sys/devices/system/cpu/cpu0/cache/index0/level:1
/sys/devices/system/cpu/cpu0/cache/index1/level:1
/sys/devices/system/cpu/cpu0/cache/index2/level:2
/sys/devices/system/cpu/cpu0/cache/index3/level:3
$ grep . /sys/devices/system/cpu/cpu0/cache/index?/type
/sys/devices/system/cpu/cpu0/cache/index0/type:Data
/sys/devices/system/cpu/cpu0/cache/index1/type:Instruction
/sys/devices/system/cpu/cpu0/cache/index2/type:Unified
/sys/devices/system/cpu/cpu0/cache/index3/type:Unified



for i in /sys/devices/system/cpu/cpu0/cache/index?;
do
	l=$(cat $i/level)
	t=$(cat $i/type)

	printf "L%d-%s\n" $l $t
done

->

L1-Data
L1-Instruction
L2-Unified
L3-Unified

I believe the index naming is simply enumerating the caches...

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

RE: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Luck, Tony]

> > > "index4" is the L3-unified cache.
> > 
> > Crazy. What was wrong with using 'level' or 'depth'?
>
> It is all there:
>
> $ grep . /sys/devices/system/cpu/cpu0/cache/index?/level
> /sys/devices/system/cpu/cpu0/cache/index0/level:1

The term "index" came from the Intel Software developer manual, volume
2, description of CPUID instruction which talks about the index into leaf
and sub leaf.  I think Ingo might have been making a small dig at Intel
documentation :-)

-Tony
 

Re: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Borislav Petkov]

On Fri, Jul 08, 2016 at 06:47:33PM +0000, Luck, Tony wrote:
> The term "index" came from the Intel Software developer manual, volume
> 2, description of CPUID instruction which talks about the index into leaf
> and sub leaf.

Ah, CPUID(4) - the caches description loop :-)

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

Re: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Ingo Molnar]

Yeah, so if the 'cache index' naming has already been settled, then I suspect 
we'll have to live with it - but at least let's name it 'cache-index' when we 
refer to it, not just 'index' which is super confusing (to me!).

I.e. instead:

  The id identifies a cache in the platform. In same index, the id
  is unique across the platform.

Something like this is more readable I think:

  The cache-id identifies a hardware cache of the system within a given
  cache-index category of caches. The (cache-index,cache-id) pair is
  unique for the whole system.

agreed?

Thanks,

	Ingo

Re: [PATCH v2 2/3] Documentation, ABI: Add a document entry for cache id

[Borislav Petkov]

On Fri, Jul 08, 2016 at 09:34:57PM +0200, Ingo Molnar wrote:
>   The cache-id identifies a hardware cache of the system within a given
>   cache-index category of caches. The (cache-index,cache-id) pair is

I'd say something like:

   "... within a given cache index in a set of cache indices. The
   "index" name is simply a nomenclature from CPUID's leaf 4 which
   enumerates all caches on the system by referring to each one as a
   cache index. ... "

Because this way we hint at from where the "index" thing comes and
people can go and read the SDM for more info.

And also we're explaining what it is because, IMHO, whatever formulation
or spelling we do with "cache index", there's simply not a good way of
explaining what it means without *actually* explaining it and where it
comes from.

:-)

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
--