All of lore.kernel.org
 help / color / mirror / Atom feed
From: Thomas Gleixner <tglx@linutronix.de>
To: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>, x86@kernel.org
Cc: Tony Luck <tony.luck@intel.com>, Andi Kleen <ak@linux.intel.com>,
	Stephane Eranian <eranian@google.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,
	Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>,
	David Woodhouse <dwmw2@infradead.org>,
	Lu Baolu <baolu.lu@linux.intel.com>,
	Nicholas Piggin <npiggin@gmail.com>,
	"Ravi V. Shankar" <ravi.v.shankar@intel.com>,
	Ricardo Neri <ricardo.neri@intel.com>,
	iommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org,
	linux-kernel@vger.kernel.org,
	Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Subject: Re: [PATCH v6 03/29] x86/apic/msi: Set the delivery mode individually for each IRQ
Date: Fri, 06 May 2022 22:05:34 +0200	[thread overview]
Message-ID: <8735hmh1f5.ffs@tglx> (raw)
In-Reply-To: <20220506000008.30892-4-ricardo.neri-calderon@linux.intel.com>

On Thu, May 05 2022 at 16:59, Ricardo Neri wrote:
> There are no restrictions in hardware to set  MSI messages with its
> own delivery mode.

"messages with its own" ? Plural/singular confusion.

> Use the mode specified in the provided IRQ hardware
> configuration data. Since most of the IRQs are configured to use the
> delivery mode of the APIC driver in use (set in all of them to
> APIC_DELIVERY_MODE_FIXED), the only functional changes are where
> IRQs are configured to use a specific delivery mode.

This does not parse. There are no functional changes due to this patch
and there is no point talking about functional changes in subsequent
patches here.

> Changing the utility function __irq_msi_compose_msg() takes care of
> implementing the change in the in the local APIC, PCI-MSI, and DMAR-MSI

in the in the

> irq_chips.
>
> The IO-APIC irq_chip configures the entries in the interrupt redirection
> table using the delivery mode specified in the corresponding MSI message.
> Since the MSI message is composed by a higher irq_chip in the hierarchy,
> it does not need to be updated.

The point is that updating __irq_msi_compose_msg() covers _all_ MSI
consumers including IO-APIC.

I had to read that changelog 3 times to make sense of it. Something like
this perhaps:

  "x86/apic/msi: Use the delivery mode from irq_cfg for message composition

   irq_cfg provides a delivery mode for each interrupt. Use it instead
   of the hardcoded APIC_DELIVERY_MODE_FIXED. This allows to compose
   messages for NMI delivery mode which is required to implement a HPET
   based NMI watchdog.

   No functional change as the default delivery mode is set to
   APIC_DELIVERY_MODE_FIXED."

Thanks,

        tglx

WARNING: multiple messages have this Message-ID (diff)
From: Thomas Gleixner <tglx@linutronix.de>
To: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>, x86@kernel.org
Cc: "Ravi V. Shankar" <ravi.v.shankar@intel.com>,
	Andi Kleen <ak@linux.intel.com>,
	linuxppc-dev@lists.ozlabs.org,
	Ricardo Neri <ricardo.neri@intel.com>,
	Stephane Eranian <eranian@google.com>,
	linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org,
	Tony Luck <tony.luck@intel.com>,
	Nicholas Piggin <npiggin@gmail.com>,
	Ricardo Neri <ricardo.neri-calderon@linux.intel.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	David Woodhouse <dwmw2@infradead.org>
Subject: Re: [PATCH v6 03/29] x86/apic/msi: Set the delivery mode individually for each IRQ
Date: Fri, 06 May 2022 22:05:34 +0200	[thread overview]
Message-ID: <8735hmh1f5.ffs@tglx> (raw)
In-Reply-To: <20220506000008.30892-4-ricardo.neri-calderon@linux.intel.com>

On Thu, May 05 2022 at 16:59, Ricardo Neri wrote:
> There are no restrictions in hardware to set  MSI messages with its
> own delivery mode.

"messages with its own" ? Plural/singular confusion.

> Use the mode specified in the provided IRQ hardware
> configuration data. Since most of the IRQs are configured to use the
> delivery mode of the APIC driver in use (set in all of them to
> APIC_DELIVERY_MODE_FIXED), the only functional changes are where
> IRQs are configured to use a specific delivery mode.

This does not parse. There are no functional changes due to this patch
and there is no point talking about functional changes in subsequent
patches here.

> Changing the utility function __irq_msi_compose_msg() takes care of
> implementing the change in the in the local APIC, PCI-MSI, and DMAR-MSI

in the in the

> irq_chips.
>
> The IO-APIC irq_chip configures the entries in the interrupt redirection
> table using the delivery mode specified in the corresponding MSI message.
> Since the MSI message is composed by a higher irq_chip in the hierarchy,
> it does not need to be updated.

The point is that updating __irq_msi_compose_msg() covers _all_ MSI
consumers including IO-APIC.

I had to read that changelog 3 times to make sense of it. Something like
this perhaps:

  "x86/apic/msi: Use the delivery mode from irq_cfg for message composition

   irq_cfg provides a delivery mode for each interrupt. Use it instead
   of the hardcoded APIC_DELIVERY_MODE_FIXED. This allows to compose
   messages for NMI delivery mode which is required to implement a HPET
   based NMI watchdog.

   No functional change as the default delivery mode is set to
   APIC_DELIVERY_MODE_FIXED."

Thanks,

        tglx
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

WARNING: multiple messages have this Message-ID (diff)
From: Thomas Gleixner <tglx@linutronix.de>
To: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>, x86@kernel.org
Cc: "Ravi V. Shankar" <ravi.v.shankar@intel.com>,
	Andi Kleen <ak@linux.intel.com>,
	linuxppc-dev@lists.ozlabs.org, Joerg Roedel <joro@8bytes.org>,
	Ricardo Neri <ricardo.neri@intel.com>,
	Stephane Eranian <eranian@google.com>,
	linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org,
	Tony Luck <tony.luck@intel.com>,
	Nicholas Piggin <npiggin@gmail.com>,
	Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>,
	Ricardo Neri <ricardo.neri-calderon@linux.intel.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	David Woodhouse <dwmw2@infradead.org>,
	Lu Baolu <baolu.lu@linux.intel.com>
Subject: Re: [PATCH v6 03/29] x86/apic/msi: Set the delivery mode individually for each IRQ
Date: Fri, 06 May 2022 22:05:34 +0200	[thread overview]
Message-ID: <8735hmh1f5.ffs@tglx> (raw)
In-Reply-To: <20220506000008.30892-4-ricardo.neri-calderon@linux.intel.com>

On Thu, May 05 2022 at 16:59, Ricardo Neri wrote:
> There are no restrictions in hardware to set  MSI messages with its
> own delivery mode.

"messages with its own" ? Plural/singular confusion.

> Use the mode specified in the provided IRQ hardware
> configuration data. Since most of the IRQs are configured to use the
> delivery mode of the APIC driver in use (set in all of them to
> APIC_DELIVERY_MODE_FIXED), the only functional changes are where
> IRQs are configured to use a specific delivery mode.

This does not parse. There are no functional changes due to this patch
and there is no point talking about functional changes in subsequent
patches here.

> Changing the utility function __irq_msi_compose_msg() takes care of
> implementing the change in the in the local APIC, PCI-MSI, and DMAR-MSI

in the in the

> irq_chips.
>
> The IO-APIC irq_chip configures the entries in the interrupt redirection
> table using the delivery mode specified in the corresponding MSI message.
> Since the MSI message is composed by a higher irq_chip in the hierarchy,
> it does not need to be updated.

The point is that updating __irq_msi_compose_msg() covers _all_ MSI
consumers including IO-APIC.

I had to read that changelog 3 times to make sense of it. Something like
this perhaps:

  "x86/apic/msi: Use the delivery mode from irq_cfg for message composition

   irq_cfg provides a delivery mode for each interrupt. Use it instead
   of the hardcoded APIC_DELIVERY_MODE_FIXED. This allows to compose
   messages for NMI delivery mode which is required to implement a HPET
   based NMI watchdog.

   No functional change as the default delivery mode is set to
   APIC_DELIVERY_MODE_FIXED."

Thanks,

        tglx

  reply	other threads:[~2022-05-06 20:05 UTC|newest]

Thread overview: 207+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-05 23:59 [PATCH v6 00/29] x86: Implement an HPET-based hardlockup detector Ricardo Neri
2022-05-05 23:59 ` Ricardo Neri
2022-05-05 23:59 ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 01/29] irq/matrix: Expose functions to allocate the best CPU for new vectors Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 19:48   ` Thomas Gleixner
2022-05-06 19:48     ` Thomas Gleixner
2022-05-06 19:48     ` Thomas Gleixner
2022-05-12  0:09     ` Ricardo Neri
2022-05-12  0:09       ` Ricardo Neri
2022-05-12  0:09       ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 02/29] x86/apic: Add irq_cfg::delivery_mode Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 19:53   ` Thomas Gleixner
2022-05-06 19:53     ` Thomas Gleixner
2022-05-06 19:53     ` Thomas Gleixner
2022-05-12  0:26     ` Ricardo Neri
2022-05-12  0:26       ` Ricardo Neri
2022-05-12  0:26       ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 03/29] x86/apic/msi: Set the delivery mode individually for each IRQ Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 20:05   ` Thomas Gleixner [this message]
2022-05-06 20:05     ` Thomas Gleixner
2022-05-06 20:05     ` Thomas Gleixner
2022-05-12  0:38     ` Ricardo Neri
2022-05-12  0:38       ` Ricardo Neri
2022-05-12  0:38       ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 04/29] x86/apic: Add the X86_IRQ_ALLOC_AS_NMI irq allocation flag Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 05/29] x86/apic/vector: Do not allocate vectors for NMIs Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 21:12   ` Thomas Gleixner
2022-05-06 21:12     ` Thomas Gleixner
2022-05-06 21:12     ` Thomas Gleixner
2022-05-13 18:03     ` Ricardo Neri
2022-05-13 18:03       ` Ricardo Neri
2022-05-13 18:03       ` Ricardo Neri
2022-05-13 20:50       ` Thomas Gleixner
2022-05-13 20:50         ` Thomas Gleixner
2022-05-13 20:50         ` Thomas Gleixner
2022-05-13 23:45         ` Ricardo Neri
2022-05-13 23:45           ` Ricardo Neri
2022-05-13 23:45           ` Ricardo Neri
2022-05-14  8:15           ` Thomas Gleixner
2022-05-14  8:15             ` Thomas Gleixner
2022-05-14  8:15             ` Thomas Gleixner
2022-05-05 23:59 ` [PATCH v6 06/29] x86/apic/vector: Implement support for NMI delivery mode Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 07/29] iommu/vt-d: Clear the redirection hint when the destination mode is physical Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 08/29] iommu/vt-d: Rework prepare_irte() to support per-IRQ delivery mode Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 09/29] iommu/vt-d: Set the IRTE delivery mode individually for each IRQ Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 10/29] iommu/vt-d: Implement minor tweaks for NMI irqs Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 21:23   ` Thomas Gleixner
2022-05-06 21:23     ` Thomas Gleixner
2022-05-06 21:23     ` Thomas Gleixner
2022-05-13 18:07     ` Ricardo Neri
2022-05-13 18:07       ` Ricardo Neri
2022-05-13 18:07       ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 11/29] iommu/amd: Expose [set|get]_dev_entry_bit() Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 12/29] iommu/amd: Enable NMIPass when allocating an NMI irq Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 21:26   ` Thomas Gleixner
2022-05-06 21:26     ` Thomas Gleixner
2022-05-06 21:26     ` Thomas Gleixner
2022-05-13 19:01     ` Ricardo Neri
2022-05-13 19:01       ` Ricardo Neri
2022-05-13 19:01       ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 13/29] iommu/amd: Compose MSI messages for NMI irqs in non-IR format Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 21:31   ` Thomas Gleixner
2022-05-06 21:31     ` Thomas Gleixner
2022-05-06 21:31     ` Thomas Gleixner
2022-05-13 19:03     ` Ricardo Neri
2022-05-13 19:03       ` Ricardo Neri
2022-05-13 19:03       ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 14/29] x86/hpet: Expose hpet_writel() in header Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 15/29] x86/hpet: Add helper function hpet_set_comparator_periodic() Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-06 21:41   ` Thomas Gleixner
2022-05-06 21:41     ` Thomas Gleixner
2022-05-06 21:41     ` Thomas Gleixner
2022-05-06 21:51     ` Thomas Gleixner
2022-05-06 21:51       ` Thomas Gleixner
2022-05-06 21:51       ` Thomas Gleixner
2022-05-13 21:29       ` Ricardo Neri
2022-05-13 21:29         ` Ricardo Neri
2022-05-13 21:29         ` Ricardo Neri
2022-05-13 21:19     ` Ricardo Neri
2022-05-13 21:19       ` Ricardo Neri
2022-05-13 21:19       ` Ricardo Neri
2022-05-14  8:17       ` Thomas Gleixner
2022-05-14  8:17         ` Thomas Gleixner
2022-05-14  8:17         ` Thomas Gleixner
2022-05-17 22:54         ` Ricardo Neri
2022-05-17 22:54           ` Ricardo Neri
2022-05-17 22:54           ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 16/29] x86/hpet: Prepare IRQ assignments to use the X86_ALLOC_AS_NMI flag Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 17/29] x86/hpet: Reserve an HPET channel for the hardlockup detector Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 18/29] watchdog/hardlockup: Define a generic function to detect hardlockups Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 19/29] watchdog/hardlockup: Decouple the hardlockup detector from perf Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59 ` [PATCH v6 20/29] init/main: Delay initialization of the lockup detector after smp_init() Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-05 23:59   ` Ricardo Neri
2022-05-10 10:38   ` Nicholas Piggin
2022-05-10 10:38     ` Nicholas Piggin
2022-05-10 10:38     ` Nicholas Piggin
2022-05-13 23:16     ` Ricardo Neri
2022-05-13 23:16       ` Ricardo Neri
2022-05-13 23:16       ` Ricardo Neri
2022-05-20  0:25       ` Nicholas Piggin
2022-05-20  0:25         ` Nicholas Piggin
2022-05-20  0:25         ` Nicholas Piggin
2022-05-06  0:00 ` [PATCH v6 21/29] x86/nmi: Add an NMI_WATCHDOG NMI handler category Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-09 13:59   ` Thomas Gleixner
2022-05-09 13:59     ` Thomas Gleixner
2022-05-09 13:59     ` Thomas Gleixner
2022-05-17 18:41     ` Ricardo Neri
2022-05-17 18:41       ` Ricardo Neri
2022-05-17 18:41       ` Ricardo Neri
2022-05-06  0:00 ` [PATCH v6 22/29] x86/watchdog/hardlockup: Add an HPET-based hardlockup detector Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-09 14:03   ` Thomas Gleixner
2022-05-09 14:03     ` Thomas Gleixner
2022-05-09 14:03     ` Thomas Gleixner
2022-05-13 22:16     ` Ricardo Neri
2022-05-13 22:16       ` Ricardo Neri
2022-05-13 22:16       ` Ricardo Neri
2022-05-14 14:04       ` Thomas Gleixner
2022-05-14 14:04         ` Thomas Gleixner
2022-05-14 14:04         ` Thomas Gleixner
2022-05-06  0:00 ` [PATCH v6 23/29] x86/watchdog/hardlockup/hpet: Determine if HPET timer caused NMI Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00 ` [PATCH v6 24/29] watchdog/hardlockup: Use parse_option_str() to handle "nmi_watchdog" Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-10 10:46   ` Nicholas Piggin
2022-05-10 10:46     ` Nicholas Piggin
2022-05-10 10:46     ` Nicholas Piggin
2022-05-13 23:17     ` Ricardo Neri
2022-05-13 23:17       ` Ricardo Neri
2022-05-13 23:17       ` Ricardo Neri
2022-05-06  0:00 ` [PATCH v6 25/29] watchdog/hardlockup/hpet: Only enable the HPET watchdog via a boot parameter Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00 ` [PATCH v6 26/29] x86/watchdog: Add a shim hardlockup detector Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00 ` [PATCH v6 27/29] watchdog: Expose lockup_detector_reconfigure() Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00 ` [PATCH v6 28/29] x86/tsc: Restart NMI watchdog after refining tsc_khz Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-10 11:16   ` Nicholas Piggin
2022-05-10 11:16     ` Nicholas Piggin
2022-05-10 11:16     ` Nicholas Piggin
2022-05-10 11:44     ` Thomas Gleixner
2022-05-10 11:44       ` Thomas Gleixner
2022-05-10 11:44       ` Thomas Gleixner
2022-05-17 22:53       ` Ricardo Neri
2022-05-17 22:53         ` Ricardo Neri
2022-05-17 22:53         ` Ricardo Neri
2022-05-17 22:08     ` Ricardo Neri
2022-05-17 22:08       ` Ricardo Neri
2022-05-17 22:08       ` Ricardo Neri
2022-05-06  0:00 ` [PATCH v6 29/29] x86/tsc: Switch to perf-based hardlockup detector if TSC become unstable Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-06  0:00   ` Ricardo Neri
2022-05-10 12:14   ` Nicholas Piggin
2022-05-10 12:14     ` Nicholas Piggin
2022-05-10 12:14     ` Nicholas Piggin
2022-05-17  3:09     ` Ricardo Neri
2022-05-17  3:09       ` Ricardo Neri
2022-05-17  3:09       ` Ricardo Neri

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8735hmh1f5.ffs@tglx \
    --to=tglx@linutronix.de \
    --cc=Suravee.Suthikulpanit@amd.com \
    --cc=ak@linux.intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=baolu.lu@linux.intel.com \
    --cc=dwmw2@infradead.org \
    --cc=eranian@google.com \
    --cc=iommu@lists.linux-foundation.org \
    --cc=joro@8bytes.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=npiggin@gmail.com \
    --cc=ravi.v.shankar@intel.com \
    --cc=ricardo.neri-calderon@linux.intel.com \
    --cc=ricardo.neri@intel.com \
    --cc=tony.luck@intel.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.