linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [git patches] libata updates, GPG signed (but see admin notes)
@ 2011-10-26 20:22 Jeff Garzik
  2011-10-29 19:13 ` Valdis.Kletnieks
  2011-10-30 10:05 ` James Bottomley
  0 siblings, 2 replies; 101+ messages in thread
From: Jeff Garzik @ 2011-10-26 20:22 UTC (permalink / raw)
  To: Andrew Morton, Linus Torvalds; +Cc: linux-ide, LKML


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


changes, nothing major:

1) many minor PATA fixes and cleanups from Bart Z

2) pata_legacy has supported QDI PATA for a while, so we may
   eliminate pata_qdi driver as duplicate. (Bart Z, acked by Alan)

3) add power management support to several PATA drivers (Bart Z)

4) other minor PATA, SATA driver fixes and cleanups (various)

5) fix ATAPI device + Sandybridge issue (Intel, Canonical)

6) make pata_of_platform work again on !PPC -- though note we hope to replace
   this with pata_platform + DT (patch by Grant L exists)

7) core bug fix from Tejun: garbage data could be transferred to an
   ATAPI device as padding,  Presumably, as this is padding, the bytes are
   irrelevant -- but at least one nutty device wants padding == zeroes.

administrivia notes:

1) This entire pull request is GPG-signed by a new GPG key 01782B2F
   (jgarzik@pobox.com, AF8B E07C 7049 F3A2 6B23  9D53 25B3 0832 0178 2B2F),
   created for new kernel.org interactions.  However, not being at KS, I've
   only gotten Andre Hedrick to sign this new key so far.  In addition to
   pgp.mit.edu keyserver, my pubkey is published on my personal website at
   http://yyz.us/publickey.txt

2) The GPG wrapping causes all '^-' lines to become '^- -'.  This does NOT
   reflect the true 'git diff' output, of course.

   For this reason, I am tempted to modify my new, GPG-signed pull request
   template as follows, for subsequent pull requests:

	---<begin GPG signed text>---

   	pull req description

	"Please pull from $branch\n$url $branch"

	"Top of tree is $sha1_commit"

	---<end GPG signed text>---

	diffstat
	diff

   That ensures that the critical part -- sha1 commit for top of tree --
   is GPG signed, while the diff will be outside the signed area and therefore
   not mangled by wrapping.


Please pull from 'upstream-linus' branch of
git://github.com/jgarzik/libata-dev.git upstream-linus

ToT commit a0da19149f589e3e6fd5bab527b3326368ab92bc

to receive the following updates:

 drivers/ata/Kconfig            |    3 +-
 drivers/ata/Makefile           |    1 -
 drivers/ata/ahci_platform.c    |   44 ++++-
 drivers/ata/ata_piix.c         |  153 ++++++++---------
 drivers/ata/libata-core.c      |    2 +-
 drivers/ata/libata-eh.c        |    5 +-
 drivers/ata/libata-sff.c       |  117 ++++++--------
 drivers/ata/pata_ali.c         |    2 +-
 drivers/ata/pata_arasan_cf.c   |    6 +-
 drivers/ata/pata_artop.c       |  107 +++++++------
 drivers/ata/pata_at91.c        |    5 +-
 drivers/ata/pata_atiixp.c      |   35 +++-
 drivers/ata/pata_cmd64x.c      |   44 +++---
 drivers/ata/pata_cs5535.c      |   16 +--
 drivers/ata/pata_efar.c        |   22 ++--
 drivers/ata/pata_hpt366.c      |   35 +++-
 drivers/ata/pata_it8213.c      |   24 ++--
 drivers/ata/pata_legacy.c      |  129 +++++----------
 drivers/ata/pata_mpc52xx.c     |    2 +-
 drivers/ata/pata_of_platform.c |    6 +-
 drivers/ata/pata_pdc2027x.c    |   30 ++++
 drivers/ata/pata_qdi.c         |  366 ----------------------------------------
 drivers/ata/pata_rdc.c         |   16 ++
 drivers/ata/pata_sc1200.c      |   14 +-
 drivers/ata/pata_scc.c         |   14 +--
 drivers/ata/pata_serverworks.c |  127 +++++---------
 drivers/ata/pata_sil680.c      |   94 ++++++-----
 drivers/ata/pata_sis.c         |  154 +++++++++++------
 drivers/ata/pata_sl82c105.c    |   37 ++++-
 drivers/ata/pata_via.c         |   49 +++---
 drivers/ata/sata_mv.c          |   16 +-
 drivers/ata/sata_sil24.c       |    4 +-
 32 files changed, 692 insertions(+), 987 deletions(-)
 delete mode 100644 drivers/ata/pata_qdi.c

Bartlomiej Zolnierkiewicz (34):
      pata_sil680: documentation fixes
      pata_sil680: constify tables
      pata_sil680: minor CodingStyle fixups
      pata_sc1200: fix DRV_NAME
      pata_cs5535: fix DRV_NAME
      libata: reduce ata_pci_[sff,bmdma]_init_one() size
      pata_sc1200: do not use c99 style comments
      pata_artop: unify ->prereset methods
      pata_hpt366: add hpt36x_find_mode() helper
      pata_cmd64x: add cmd64x_fixup()
      pata_cs5535: no need to program PIO0 timings during device init
      pata_ali: fix "Satelite" typo
      pata_cmd64x: documentation fix
      pata_serverworks: use standard cable detection methods
      pata_serverworks: cleanup cable detection
      pata_serverworks: add serverworks_fixup()
      pata_atiixp: add proper ->prereset method
      pata_scc: add proper cable detection method
      libata: ata_timing_compute() fixup
      pata_via: add via_fixup()
      pata_rdc: parallel scanning needs an extra locking
      pata_it8213: add UDMA100 and UDMA133 support
      pata_it8213: fix register naming used in it8213_set_piomode()
      libata: remove no longer needed pata_qdi driver
      pata_legacy: unify QDI ->set_piomode methods
      pata_legacy: use PIO mask defines
      pata_pdc2027x: add Power Management support
      pata_sl82c105: add Power Management support
      pata_artop: add Power Management support
      libata: make ata_sff_data_xfer_noirq() work with 32-bit PIO
      ata_piix: SITRE handling fix
      ata_piix: unify code for programming PIO and MWDMA timings
      pata_rdc: add Power Management support
      pata_efar: fix register naming used in efar_set_piomode()

Dan McGee (5):
      pata_sis: extract a sis_port_base() method
      pata_sis: add mode_filter method for certain sis5513 chipsets
      pata_sis: enable MWDMA for UDMA 133 chipset
      pata_sis: mark most const data static as well
      pata_sis: code style cleanups for consistency

Joe Perches (1):
      sata_sil24: Use const

Ming Lei (1):
      ata_piix: make DVD Drive recognisable on systems with Intel Sandybridge chipsets(v2)

Pawel Moll (1):
      ata: Make pata_of_platform.c compile again and work on non-PPC platforms

Richard Zhu (1):
      [libata] ahci_platform: Add the board_ids and pi refer to different features

Sergei Shtylyov (5):
      sata_mv: release clock on ata_host_activate() failure
      sata_mv: use {platform|pci}_get_drvdata()
      libata-eh: ata_eh_followup_srst_needed() does not need 'classes' parameter
      pata_at91: call clk_put() on ata_host_activate() failure
      pata_arasan_cf: remove bogus to_platform_device() calls

Tejun Heo (1):
      libata: clear PIO pad area

Yong Zhang (1):
      ata: irq: Remove IRQF_DISABLED

diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig
index 5987e0b..6bdedd7 100644
- --- a/drivers/ata/Kconfig
+++ b/drivers/ata/Kconfig
@@ -820,7 +820,7 @@ config PATA_PLATFORM
 
 config PATA_OF_PLATFORM
 	tristate "OpenFirmware platform device PATA support"
- -	depends on PATA_PLATFORM && PPC_OF
+	depends on PATA_PLATFORM && OF
 	help
 	  This option enables support for generic directly connected ATA
 	  devices commonly found on embedded systems with OpenFirmware
@@ -831,6 +831,7 @@ config PATA_OF_PLATFORM
 config PATA_QDI
 	tristate "QDI VLB PATA support"
 	depends on ISA
+	select PATA_LEGACY
 	help
 	  Support for QDI 6500 and 6580 PATA controllers on VESA local bus.
 
diff --git a/drivers/ata/Makefile b/drivers/ata/Makefile
index 9550d69..6ece5b7 100644
- --- a/drivers/ata/Makefile
+++ b/drivers/ata/Makefile
@@ -88,7 +88,6 @@ obj-$(CONFIG_PATA_PCMCIA)	+= pata_pcmcia.o
 obj-$(CONFIG_PATA_PALMLD)	+= pata_palmld.o
 obj-$(CONFIG_PATA_PLATFORM)	+= pata_platform.o
 obj-$(CONFIG_PATA_OF_PLATFORM)	+= pata_of_platform.o
- -obj-$(CONFIG_PATA_QDI)		+= pata_qdi.o
 obj-$(CONFIG_PATA_RB532)	+= pata_rb532_cf.o
 obj-$(CONFIG_PATA_RZ1000)	+= pata_rz1000.o
 obj-$(CONFIG_PATA_SAMSUNG_CF)	+= pata_samsung_cf.o
diff --git a/drivers/ata/ahci_platform.c b/drivers/ata/ahci_platform.c
index 6fef1fa..c03277d 100644
- --- a/drivers/ata/ahci_platform.c
+++ b/drivers/ata/ahci_platform.c
@@ -23,6 +23,41 @@
 #include <linux/ahci_platform.h>
 #include "ahci.h"
 
+enum ahci_type {
+	AHCI,		/* standard platform ahci */
+	IMX53_AHCI,	/* ahci on i.mx53 */
+};
+
+static struct platform_device_id ahci_devtype[] = {
+	{
+		.name = "ahci",
+		.driver_data = AHCI,
+	}, {
+		.name = "imx53-ahci",
+		.driver_data = IMX53_AHCI,
+	}, {
+		/* sentinel */
+	}
+};
+MODULE_DEVICE_TABLE(platform, ahci_devtype);
+
+
+static const struct ata_port_info ahci_port_info[] = {
+	/* by features */
+	[AHCI] = {
+		.flags		= AHCI_FLAG_COMMON,
+		.pio_mask	= ATA_PIO4,
+		.udma_mask	= ATA_UDMA6,
+		.port_ops	= &ahci_ops,
+	},
+	[IMX53_AHCI] = {
+		.flags		= AHCI_FLAG_COMMON,
+		.pio_mask	= ATA_PIO4,
+		.udma_mask	= ATA_UDMA6,
+		.port_ops	= &ahci_pmp_retry_srst_ops,
+	},
+};
+
 static struct scsi_host_template ahci_platform_sht = {
 	AHCI_SHT("ahci_platform"),
 };
@@ -31,12 +66,8 @@ static int __init ahci_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
 	struct ahci_platform_data *pdata = dev->platform_data;
- -	struct ata_port_info pi = {
- -		.flags		= AHCI_FLAG_COMMON,
- -		.pio_mask	= ATA_PIO4,
- -		.udma_mask	= ATA_UDMA6,
- -		.port_ops	= &ahci_ops,
- -	};
+	const struct platform_device_id *id = platform_get_device_id(pdev);
+	struct ata_port_info pi = ahci_port_info[id->driver_data];
 	const struct ata_port_info *ppi[] = { &pi, NULL };
 	struct ahci_host_priv *hpriv;
 	struct ata_host *host;
@@ -177,6 +208,7 @@ static struct platform_driver ahci_driver = {
 		.name = "ahci",
 		.owner = THIS_MODULE,
 	},
+	.id_table	= ahci_devtype,
 };
 
 static int __init ahci_init(void)
diff --git a/drivers/ata/ata_piix.c b/drivers/ata/ata_piix.c
index 43107e9..69ac373 100644
- --- a/drivers/ata/ata_piix.c
+++ b/drivers/ata/ata_piix.c
@@ -113,6 +113,8 @@ enum {
 	PIIX_PATA_FLAGS		= ATA_FLAG_SLAVE_POSS,
 	PIIX_SATA_FLAGS		= ATA_FLAG_SATA | PIIX_FLAG_CHECKINTR,
 
+	PIIX_FLAG_PIO16		= (1 << 30), /*support 16bit PIO only*/
+
 	PIIX_80C_PRI		= (1 << 5) | (1 << 4),
 	PIIX_80C_SEC		= (1 << 7) | (1 << 6),
 
@@ -147,6 +149,7 @@ enum piix_controller_ids {
 	ich8m_apple_sata,	/* locks up on second port enable */
 	tolapai_sata,
 	piix_pata_vmw,			/* PIIX4 for VMware, spurious DMA_ERR */
+	ich8_sata_snb,
 };
 
 struct piix_map_db {
@@ -177,6 +180,7 @@ static int piix_sidpr_scr_write(struct ata_link *link,
 static int piix_sidpr_set_lpm(struct ata_link *link, enum ata_lpm_policy policy,
 			      unsigned hints);
 static bool piix_irq_check(struct ata_port *ap);
+static int piix_port_start(struct ata_port *ap);
 #ifdef CONFIG_PM
 static int piix_pci_device_suspend(struct pci_dev *pdev, pm_message_t mesg);
 static int piix_pci_device_resume(struct pci_dev *pdev);
@@ -298,21 +302,21 @@ static const struct pci_device_id piix_pci_tbl[] = {
 	/* SATA Controller IDE (PCH) */
 	{ 0x8086, 0x3b2e, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata },
 	/* SATA Controller IDE (CPT) */
- -	{ 0x8086, 0x1c00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata },
+	{ 0x8086, 0x1c00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata_snb },
 	/* SATA Controller IDE (CPT) */
- -	{ 0x8086, 0x1c01, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata },
+	{ 0x8086, 0x1c01, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata_snb },
 	/* SATA Controller IDE (CPT) */
 	{ 0x8086, 0x1c08, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
 	/* SATA Controller IDE (CPT) */
 	{ 0x8086, 0x1c09, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
 	/* SATA Controller IDE (PBG) */
- -	{ 0x8086, 0x1d00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata },
+	{ 0x8086, 0x1d00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata_snb },
 	/* SATA Controller IDE (PBG) */
 	{ 0x8086, 0x1d08, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
 	/* SATA Controller IDE (Panther Point) */
- -	{ 0x8086, 0x1e00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata },
+	{ 0x8086, 0x1e00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata_snb },
 	/* SATA Controller IDE (Panther Point) */
- -	{ 0x8086, 0x1e01, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata },
+	{ 0x8086, 0x1e01, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata_snb },
 	/* SATA Controller IDE (Panther Point) */
 	{ 0x8086, 0x1e08, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
 	/* SATA Controller IDE (Panther Point) */
@@ -338,6 +342,7 @@ static struct scsi_host_template piix_sht = {
 static struct ata_port_operations piix_sata_ops = {
 	.inherits		= &ata_bmdma32_port_ops,
 	.sff_irq_check		= piix_irq_check,
+	.port_start		= piix_port_start,
 };
 
 static struct ata_port_operations piix_pata_ops = {
@@ -478,6 +483,7 @@ static const struct piix_map_db *piix_map_db_table[] = {
 	[ich8_2port_sata]	= &ich8_2port_map_db,
 	[ich8m_apple_sata]	= &ich8m_apple_map_db,
 	[tolapai_sata]		= &tolapai_map_db,
+	[ich8_sata_snb]		= &ich8_map_db,
 };
 
 static struct ata_port_info piix_port_info[] = {
@@ -606,6 +612,19 @@ static struct ata_port_info piix_port_info[] = {
 		.port_ops	= &piix_vmw_ops,
 	},
 
+	/*
+	 * some Sandybridge chipsets have broken 32 mode up to now,
+	 * see https://bugzilla.kernel.org/show_bug.cgi?id=40592
+	 */
+	[ich8_sata_snb] =
+	{
+		.flags		= PIIX_SATA_FLAGS | PIIX_FLAG_SIDPR | PIIX_FLAG_PIO16,
+		.pio_mask	= ATA_PIO4,
+		.mwdma_mask	= ATA_MWDMA2,
+		.udma_mask	= ATA_UDMA6,
+		.port_ops	= &piix_sata_ops,
+	},
+
 };
 
 static struct pci_bits piix_enable_bits[] = {
@@ -649,6 +668,14 @@ static const struct ich_laptop ich_laptop[] = {
 	{ 0, }
 };
 
+static int piix_port_start(struct ata_port *ap)
+{
+	if (!(ap->flags & PIIX_FLAG_PIO16))
+		ap->pflags |= ATA_PFLAG_PIO32 | ATA_PFLAG_PIO32CHANGE;
+
+	return ata_bmdma_port_start(ap);
+}
+
 /**
  *	ich_pata_cable_detect - Probe host controller cable detect info
  *	@ap: Port for which cable detect info is desired
@@ -704,22 +731,11 @@ static int piix_pata_prereset(struct ata_link *link, unsigned long deadline)
 
 static DEFINE_SPINLOCK(piix_lock);
 
- -/**
- - *	piix_set_piomode - Initialize host controller PATA PIO timings
- - *	@ap: Port whose timings we are configuring
- - *	@adev: um
- - *
- - *	Set PIO mode for device, in host controller PCI config space.
- - *
- - *	LOCKING:
- - *	None (inherited from caller).
- - */
- -
- -static void piix_set_piomode(struct ata_port *ap, struct ata_device *adev)
+static void piix_set_timings(struct ata_port *ap, struct ata_device *adev,
+			     u8 pio)
 {
 	struct pci_dev *dev	= to_pci_dev(ap->host->dev);
 	unsigned long flags;
- -	unsigned int pio	= adev->pio_mode - XFER_PIO_0;
 	unsigned int is_slave	= (adev->devno != 0);
 	unsigned int master_port= ap->port_no ? 0x42 : 0x40;
 	unsigned int slave_port	= 0x44;
@@ -744,10 +760,16 @@ static void piix_set_piomode(struct ata_port *ap, struct ata_device *adev)
 		control |= 1;	/* TIME1 enable */
 	if (ata_pio_need_iordy(adev))
 		control |= 2;	/* IE enable */
- -
 	/* Intel specifies that the PPE functionality is for disk only */
 	if (adev->class == ATA_DEV_ATA)
 		control |= 4;	/* PPE enable */
+	/*
+	 * If the drive MWDMA is faster than it can do PIO then
+	 * we must force PIO into PIO0
+	 */
+	if (adev->pio_mode < XFER_PIO_0 + pio)
+		/* Enable DMA timing only */
+		control |= 8;	/* PIO cycles in PIO0 */
 
 	spin_lock_irqsave(&piix_lock, flags);
 
@@ -759,8 +781,6 @@ static void piix_set_piomode(struct ata_port *ap, struct ata_device *adev)
 	if (is_slave) {
 		/* clear TIME1|IE1|PPE1|DTE1 */
 		master_data &= 0xff0f;
- -		/* Enable SITRE (separate slave timing register) */
- -		master_data |= 0x4000;
 		/* enable PPE1, IE1 and TIME1 as needed */
 		master_data |= (control << 4);
 		pci_read_config_byte(dev, slave_port, &slave_data);
@@ -778,6 +798,9 @@ static void piix_set_piomode(struct ata_port *ap, struct ata_device *adev)
 			(timings[pio][0] << 12) |
 			(timings[pio][1] << 8);
 	}
+
+	/* Enable SITRE (separate slave timing register) */
+	master_data |= 0x4000;
 	pci_write_config_word(dev, master_port, master_data);
 	if (is_slave)
 		pci_write_config_byte(dev, slave_port, slave_data);
@@ -795,6 +818,22 @@ static void piix_set_piomode(struct ata_port *ap, struct ata_device *adev)
 }
 
 /**
+ *	piix_set_piomode - Initialize host controller PATA PIO timings
+ *	@ap: Port whose timings we are configuring
+ *	@adev: Drive in question
+ *
+ *	Set PIO mode for device, in host controller PCI config space.
+ *
+ *	LOCKING:
+ *	None (inherited from caller).
+ */
+
+static void piix_set_piomode(struct ata_port *ap, struct ata_device *adev)
+{
+	piix_set_timings(ap, adev, adev->pio_mode - XFER_PIO_0);
+}
+
+/**
  *	do_pata_set_dmamode - Initialize host controller PATA PIO timings
  *	@ap: Port whose timings we are configuring
  *	@adev: Drive in question
@@ -810,31 +849,20 @@ static void do_pata_set_dmamode(struct ata_port *ap, struct ata_device *adev, in
 {
 	struct pci_dev *dev	= to_pci_dev(ap->host->dev);
 	unsigned long flags;
- -	u8 master_port		= ap->port_no ? 0x42 : 0x40;
- -	u16 master_data;
 	u8 speed		= adev->dma_mode;
 	int devid		= adev->devno + 2 * ap->port_no;
 	u8 udma_enable		= 0;
 
- -	static const	 /* ISP  RTC */
- -	u8 timings[][2]	= { { 0, 0 },
- -			    { 0, 0 },
- -			    { 1, 0 },
- -			    { 2, 1 },
- -			    { 2, 3 }, };
- -
- -	spin_lock_irqsave(&piix_lock, flags);
- -
- -	pci_read_config_word(dev, master_port, &master_data);
- -	if (ap->udma_mask)
- -		pci_read_config_byte(dev, 0x48, &udma_enable);
- -
 	if (speed >= XFER_UDMA_0) {
- -		unsigned int udma = adev->dma_mode - XFER_UDMA_0;
+		unsigned int udma = speed - XFER_UDMA_0;
 		u16 udma_timing;
 		u16 ideconf;
 		int u_clock, u_speed;
 
+		spin_lock_irqsave(&piix_lock, flags);
+
+		pci_read_config_byte(dev, 0x48, &udma_enable);
+
 		/*
 		 * UDMA is handled by a combination of clock switching and
 		 * selection of dividers
@@ -867,56 +895,21 @@ static void do_pata_set_dmamode(struct ata_port *ap, struct ata_device *adev, in
 			   performance (WR_PingPong_En) */
 			pci_write_config_word(dev, 0x54, ideconf);
 		}
+
+		pci_write_config_byte(dev, 0x48, udma_enable);
+
+		spin_unlock_irqrestore(&piix_lock, flags);
 	} else {
- -		/*
- -		 * MWDMA is driven by the PIO timings. We must also enable
- -		 * IORDY unconditionally along with TIME1. PPE has already
- -		 * been set when the PIO timing was set.
- -		 */
- -		unsigned int mwdma	= adev->dma_mode - XFER_MW_DMA_0;
- -		unsigned int control;
- -		u8 slave_data;
+		/* MWDMA is driven by the PIO timings. */
+		unsigned int mwdma = speed - XFER_MW_DMA_0;
 		const unsigned int needed_pio[3] = {
 			XFER_PIO_0, XFER_PIO_3, XFER_PIO_4
 		};
 		int pio = needed_pio[mwdma] - XFER_PIO_0;
 
- -		control = 3;	/* IORDY|TIME1 */
- -
- -		/* If the drive MWDMA is faster than it can do PIO then
- -		   we must force PIO into PIO0 */
- -
- -		if (adev->pio_mode < needed_pio[mwdma])
- -			/* Enable DMA timing only */
- -			control |= 8;	/* PIO cycles in PIO0 */
- -
- -		if (adev->devno) {	/* Slave */
- -			master_data &= 0xFF4F;  /* Mask out IORDY|TIME1|DMAONLY */
- -			master_data |= control << 4;
- -			pci_read_config_byte(dev, 0x44, &slave_data);
- -			slave_data &= (ap->port_no ? 0x0f : 0xf0);
- -			/* Load the matching timing */
- -			slave_data |= ((timings[pio][0] << 2) | timings[pio][1]) << (ap->port_no ? 4 : 0);
- -			pci_write_config_byte(dev, 0x44, slave_data);
- -		} else { 	/* Master */
- -			master_data &= 0xCCF4;	/* Mask out IORDY|TIME1|DMAONLY
- -						   and master timing bits */
- -			master_data |= control;
- -			master_data |=
- -				(timings[pio][0] << 12) |
- -				(timings[pio][1] << 8);
- -		}
- -
- -		if (ap->udma_mask)
- -			udma_enable &= ~(1 << devid);
- -
- -		pci_write_config_word(dev, master_port, master_data);
+		/* XFER_PIO_0 is never used currently */
+		piix_set_timings(ap, adev, pio);
 	}
- -	/* Don't scribble on 0x48 if the controller does not support UDMA */
- -	if (ap->udma_mask)
- -		pci_write_config_byte(dev, 0x48, udma_enable);
- -
- -	spin_unlock_irqrestore(&piix_lock, flags);
 }
 
 /**
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 4a3a5ae..32fc41c 100644
- --- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -2938,7 +2938,7 @@ int ata_timing_compute(struct ata_device *adev, unsigned short speed,
 	if (id[ATA_ID_FIELD_VALID] & 2) {	/* EIDE drive */
 		memset(&p, 0, sizeof(p));
 
- -		if (speed >= XFER_PIO_0 && speed <= XFER_SW_DMA_0) {
+		if (speed >= XFER_PIO_0 && speed < XFER_SW_DMA_0) {
 			if (speed <= XFER_PIO_2)
 				p.cycle = p.cyc8b = id[ATA_ID_EIDE_PIO];
 			else if ((speed <= XFER_PIO_4) ||
diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c
index ed16fbe..c021186 100644
- --- a/drivers/ata/libata-eh.c
+++ b/drivers/ata/libata-eh.c
@@ -2532,8 +2532,7 @@ static int ata_do_reset(struct ata_link *link, ata_reset_fn_t reset,
 	return reset(link, classes, deadline);
 }
 
- -static int ata_eh_followup_srst_needed(struct ata_link *link,
- -				       int rc, const unsigned int *classes)
+static int ata_eh_followup_srst_needed(struct ata_link *link, int rc)
 {
 	if ((link->flags & ATA_LFLAG_NO_SRST) || ata_link_offline(link))
 		return 0;
@@ -2726,7 +2725,7 @@ int ata_eh_reset(struct ata_link *link, int classify,
 
 		/* perform follow-up SRST if necessary */
 		if (reset == hardreset &&
- -		    ata_eh_followup_srst_needed(link, rc, classes)) {
+		    ata_eh_followup_srst_needed(link, rc)) {
 			reset = softreset;
 
 			if (!reset) {
diff --git a/drivers/ata/libata-sff.c b/drivers/ata/libata-sff.c
index c24127d..239bfa6 100644
- --- a/drivers/ata/libata-sff.c
+++ b/drivers/ata/libata-sff.c
@@ -569,7 +569,7 @@ unsigned int ata_sff_data_xfer(struct ata_device *dev, unsigned char *buf,
 
 	/* Transfer trailing byte, if any. */
 	if (unlikely(buflen & 0x01)) {
- -		unsigned char pad[2];
+		unsigned char pad[2] = { };
 
 		/* Point buf to the tail of buffer */
 		buf += buflen - 1;
@@ -628,7 +628,7 @@ unsigned int ata_sff_data_xfer32(struct ata_device *dev, unsigned char *buf,
 
 	/* Transfer trailing bytes, if any */
 	if (unlikely(slop)) {
- -		unsigned char pad[4];
+		unsigned char pad[4] = { };
 
 		/* Point buf to the tail of buffer */
 		buf += buflen - slop;
@@ -678,7 +678,7 @@ unsigned int ata_sff_data_xfer_noirq(struct ata_device *dev, unsigned char *buf,
 	unsigned int consumed;
 
 	local_irq_save(flags);
- -	consumed = ata_sff_data_xfer(dev, buf, buflen, rw);
+	consumed = ata_sff_data_xfer32(dev, buf, buflen, rw);
 	local_irq_restore(flags);
 
 	return consumed;
@@ -2507,31 +2507,10 @@ static const struct ata_port_info *ata_sff_find_valid_pi(
 	return NULL;
 }
 
- -/**
- - *	ata_pci_sff_init_one - Initialize/register PIO-only PCI IDE controller
- - *	@pdev: Controller to be initialized
- - *	@ppi: array of port_info, must be enough for two ports
- - *	@sht: scsi_host_template to use when registering the host
- - *	@host_priv: host private_data
- - *	@hflag: host flags
- - *
- - *	This is a helper function which can be called from a driver's
- - *	xxx_init_one() probe function if the hardware uses traditional
- - *	IDE taskfile registers and is PIO only.
- - *
- - *	ASSUMPTION:
- - *	Nobody makes a single channel controller that appears solely as
- - *	the secondary legacy port on PCI.
- - *
- - *	LOCKING:
- - *	Inherited from PCI layer (may sleep).
- - *
- - *	RETURNS:
- - *	Zero on success, negative on errno-based value on error.
- - */
- -int ata_pci_sff_init_one(struct pci_dev *pdev,
- -		 const struct ata_port_info * const *ppi,
- -		 struct scsi_host_template *sht, void *host_priv, int hflag)
+static int ata_pci_init_one(struct pci_dev *pdev,
+		const struct ata_port_info * const *ppi,
+		struct scsi_host_template *sht, void *host_priv,
+		int hflags, bool bmdma)
 {
 	struct device *dev = &pdev->dev;
 	const struct ata_port_info *pi;
@@ -2553,14 +2532,22 @@ int ata_pci_sff_init_one(struct pci_dev *pdev,
 	if (rc)
 		goto out;
 
- -	/* prepare and activate SFF host */
- -	rc = ata_pci_sff_prepare_host(pdev, ppi, &host);
+	if (bmdma)
+		/* prepare and activate BMDMA host */
+		rc = ata_pci_bmdma_prepare_host(pdev, ppi, &host);
+	else
+		/* prepare and activate SFF host */
+		rc = ata_pci_sff_prepare_host(pdev, ppi, &host);
 	if (rc)
 		goto out;
 	host->private_data = host_priv;
- -	host->flags |= hflag;
+	host->flags |= hflags;
 
- -	rc = ata_pci_sff_activate_host(host, ata_sff_interrupt, sht);
+	if (bmdma) {
+		pci_set_master(pdev);
+		rc = ata_pci_sff_activate_host(host, ata_bmdma_interrupt, sht);
+	} else
+		rc = ata_pci_sff_activate_host(host, ata_sff_interrupt, sht);
 out:
 	if (rc == 0)
 		devres_remove_group(&pdev->dev, NULL);
@@ -2569,6 +2556,35 @@ out:
 
 	return rc;
 }
+
+/**
+ *	ata_pci_sff_init_one - Initialize/register PIO-only PCI IDE controller
+ *	@pdev: Controller to be initialized
+ *	@ppi: array of port_info, must be enough for two ports
+ *	@sht: scsi_host_template to use when registering the host
+ *	@host_priv: host private_data
+ *	@hflag: host flags
+ *
+ *	This is a helper function which can be called from a driver's
+ *	xxx_init_one() probe function if the hardware uses traditional
+ *	IDE taskfile registers and is PIO only.
+ *
+ *	ASSUMPTION:
+ *	Nobody makes a single channel controller that appears solely as
+ *	the secondary legacy port on PCI.
+ *
+ *	LOCKING:
+ *	Inherited from PCI layer (may sleep).
+ *
+ *	RETURNS:
+ *	Zero on success, negative on errno-based value on error.
+ */
+int ata_pci_sff_init_one(struct pci_dev *pdev,
+		 const struct ata_port_info * const *ppi,
+		 struct scsi_host_template *sht, void *host_priv, int hflag)
+{
+	return ata_pci_init_one(pdev, ppi, sht, host_priv, hflag, 0);
+}
 EXPORT_SYMBOL_GPL(ata_pci_sff_init_one);
 
 #endif /* CONFIG_PCI */
@@ -3286,42 +3302,7 @@ int ata_pci_bmdma_init_one(struct pci_dev *pdev,
 			   struct scsi_host_template *sht, void *host_priv,
 			   int hflags)
 {
- -	struct device *dev = &pdev->dev;
- -	const struct ata_port_info *pi;
- -	struct ata_host *host = NULL;
- -	int rc;
- -
- -	DPRINTK("ENTER\n");
- -
- -	pi = ata_sff_find_valid_pi(ppi);
- -	if (!pi) {
- -		dev_err(&pdev->dev, "no valid port_info specified\n");
- -		return -EINVAL;
- -	}
- -
- -	if (!devres_open_group(dev, NULL, GFP_KERNEL))
- -		return -ENOMEM;
- -
- -	rc = pcim_enable_device(pdev);
- -	if (rc)
- -		goto out;
- -
- -	/* prepare and activate BMDMA host */
- -	rc = ata_pci_bmdma_prepare_host(pdev, ppi, &host);
- -	if (rc)
- -		goto out;
- -	host->private_data = host_priv;
- -	host->flags |= hflags;
- -
- -	pci_set_master(pdev);
- -	rc = ata_pci_sff_activate_host(host, ata_bmdma_interrupt, sht);
- - out:
- -	if (rc == 0)
- -		devres_remove_group(&pdev->dev, NULL);
- -	else
- -		devres_release_group(&pdev->dev, NULL);
- -
- -	return rc;
+	return ata_pci_init_one(pdev, ppi, sht, host_priv, hflags, 1);
 }
 EXPORT_SYMBOL_GPL(ata_pci_bmdma_init_one);
 
diff --git a/drivers/ata/pata_ali.c b/drivers/ata/pata_ali.c
index cadd679..61da069 100644
- --- a/drivers/ata/pata_ali.c
+++ b/drivers/ata/pata_ali.c
@@ -56,7 +56,7 @@ static const struct dmi_system_id cable_dmi_table[] = {
 		},
 	},
 	{
- -		.ident = "Toshiba Satelite S1800-814",
+		.ident = "Toshiba Satellite S1800-814",
 		.matches = {
 			DMI_MATCH(DMI_SYS_VENDOR, "TOSHIBA"),
 			DMI_MATCH(DMI_PRODUCT_NAME, "S1800-814"),
diff --git a/drivers/ata/pata_arasan_cf.c b/drivers/ata/pata_arasan_cf.c
index 719bb73..e8574bb 100644
- --- a/drivers/ata/pata_arasan_cf.c
+++ b/drivers/ata/pata_arasan_cf.c
@@ -922,8 +922,7 @@ static int __devexit arasan_cf_remove(struct platform_device *pdev)
 #ifdef CONFIG_PM
 static int arasan_cf_suspend(struct device *dev)
 {
- -	struct platform_device *pdev = to_platform_device(dev);
- -	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	struct ata_host *host = dev_get_drvdata(dev);
 	struct arasan_cf_dev *acdev = host->ports[0]->private_data;
 
 	if (acdev->dma_chan) {
@@ -937,8 +936,7 @@ static int arasan_cf_suspend(struct device *dev)
 
 static int arasan_cf_resume(struct device *dev)
 {
- -	struct platform_device *pdev = to_platform_device(dev);
- -	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	struct ata_host *host = dev_get_drvdata(dev);
 	struct arasan_cf_dev *acdev = host->ports[0]->private_data;
 
 	cf_init(acdev);
diff --git a/drivers/ata/pata_artop.c b/drivers/ata/pata_artop.c
index 78a93b6..4b8b22e 100644
- --- a/drivers/ata/pata_artop.c
+++ b/drivers/ata/pata_artop.c
@@ -2,7 +2,7 @@
  *    pata_artop.c - ARTOP ATA controller driver
  *
  *	(C) 2006 Red Hat
- - *	(C) 2007 Bartlomiej Zolnierkiewicz
+ *	(C) 2007,2011 Bartlomiej Zolnierkiewicz
  *
  *    Based in part on drivers/ide/pci/aec62xx.c
  *	Copyright (C) 1999-2002	Andre Hedrick <andre@linux-ide.org>
@@ -28,7 +28,7 @@
 #include <linux/ata.h>
 
 #define DRV_NAME	"pata_artop"
- -#define DRV_VERSION	"0.4.5"
+#define DRV_VERSION	"0.4.6"
 
 /*
  *	The ARTOP has 33 Mhz and "over clocked" timing tables. Until we
@@ -39,31 +39,15 @@
 
 static int clock = 0;
 
- -static int artop6210_pre_reset(struct ata_link *link, unsigned long deadline)
- -{
- -	struct ata_port *ap = link->ap;
- -	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
- -	const struct pci_bits artop_enable_bits[] = {
- -		{ 0x4AU, 1U, 0x02UL, 0x02UL },	/* port 0 */
- -		{ 0x4AU, 1U, 0x04UL, 0x04UL },	/* port 1 */
- -	};
- -
- -	if (!pci_test_config_bits(pdev, &artop_enable_bits[ap->port_no]))
- -		return -ENOENT;
- -
- -	return ata_sff_prereset(link, deadline);
- -}
- -
 /**
- - *	artop6260_pre_reset	-	check for 40/80 pin
+ *	artop62x0_pre_reset	-	probe begin
  *	@link: link
  *	@deadline: deadline jiffies for the operation
  *
- - *	The ARTOP hardware reports the cable detect bits in register 0x49.
  *	Nothing complicated needed here.
  */
 
- -static int artop6260_pre_reset(struct ata_link *link, unsigned long deadline)
+static int artop62x0_pre_reset(struct ata_link *link, unsigned long deadline)
 {
 	static const struct pci_bits artop_enable_bits[] = {
 		{ 0x4AU, 1U, 0x02UL, 0x02UL },	/* port 0 */
@@ -73,7 +57,7 @@ static int artop6260_pre_reset(struct ata_link *link, unsigned long deadline)
 	struct ata_port *ap = link->ap;
 	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 
- -	/* Odd numbered device ids are the units with enable bits (the -R cards) */
+	/* Odd numbered device ids are the units with enable bits. */
 	if ((pdev->device & 1) &&
 	    !pci_test_config_bits(pdev, &artop_enable_bits[ap->port_no]))
 		return -ENOENT;
@@ -317,7 +301,7 @@ static struct ata_port_operations artop6210_ops = {
 	.cable_detect		= ata_cable_40wire,
 	.set_piomode		= artop6210_set_piomode,
 	.set_dmamode		= artop6210_set_dmamode,
- -	.prereset		= artop6210_pre_reset,
+	.prereset		= artop62x0_pre_reset,
 	.qc_defer		= artop6210_qc_defer,
 };
 
@@ -326,9 +310,36 @@ static struct ata_port_operations artop6260_ops = {
 	.cable_detect		= artop6260_cable_detect,
 	.set_piomode		= artop6260_set_piomode,
 	.set_dmamode		= artop6260_set_dmamode,
- -	.prereset		= artop6260_pre_reset,
+	.prereset		= artop62x0_pre_reset,
 };
 
+static void atp8xx_fixup(struct pci_dev *pdev)
+{
+	if (pdev->device == 0x0005)
+		/* BIOS may have left us in UDMA, clear it before libata probe */
+		pci_write_config_byte(pdev, 0x54, 0);
+	else if (pdev->device == 0x0008 || pdev->device == 0x0009) {
+		u8 reg;
+
+		/* Mac systems come up with some registers not set as we
+		   will need them */
+
+		/* Clear reset & test bits */
+		pci_read_config_byte(pdev, 0x49, &reg);
+		pci_write_config_byte(pdev, 0x49, reg & ~0x30);
+
+		/* PCI latency must be > 0x80 for burst mode, tweak it
+		 * if required.
+		 */
+		pci_read_config_byte(pdev, PCI_LATENCY_TIMER, &reg);
+		if (reg <= 0x80)
+			pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 0x90);
+
+		/* Enable IRQ output and burst mode */
+		pci_read_config_byte(pdev, 0x4a, &reg);
+		pci_write_config_byte(pdev, 0x4a, (reg & ~0x01) | 0x80);
+	}
+}
 
 /**
  *	artop_init_one - Register ARTOP ATA PCI device with kernel services
@@ -383,42 +394,22 @@ static int artop_init_one (struct pci_dev *pdev, const struct pci_device_id *id)
 	if (rc)
 		return rc;
 
- -	if (id->driver_data == 0) {	/* 6210 variant */
+	if (id->driver_data == 0)	/* 6210 variant */
 		ppi[0] = &info_6210;
- -		/* BIOS may have left us in UDMA, clear it before libata probe */
- -		pci_write_config_byte(pdev, 0x54, 0);
- -	}
 	else if (id->driver_data == 1)	/* 6260 */
 		ppi[0] = &info_626x;
 	else if (id->driver_data == 2)	{ /* 6280 or 6280 + fast */
 		unsigned long io = pci_resource_start(pdev, 4);
- -		u8 reg;
 
 		ppi[0] = &info_628x;
 		if (inb(io) & 0x10)
 			ppi[0] = &info_628x_fast;
- -		/* Mac systems come up with some registers not set as we
- -		   will need them */
- -
- -		/* Clear reset & test bits */
- -		pci_read_config_byte(pdev, 0x49, &reg);
- -		pci_write_config_byte(pdev, 0x49, reg & ~ 0x30);
- -
- -		/* PCI latency must be > 0x80 for burst mode, tweak it
- -		 * if required.
- -		 */
- -		pci_read_config_byte(pdev, PCI_LATENCY_TIMER, &reg);
- -		if (reg <= 0x80)
- -			pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 0x90);
- -
- -		/* Enable IRQ output and burst mode */
- -		pci_read_config_byte(pdev, 0x4a, &reg);
- -		pci_write_config_byte(pdev, 0x4a, (reg & ~0x01) | 0x80);
- -
 	}
 
 	BUG_ON(ppi[0] == NULL);
 
+	atp8xx_fixup(pdev);
+
 	return ata_pci_bmdma_init_one(pdev, ppi, &artop_sht, NULL, 0);
 }
 
@@ -432,11 +423,32 @@ static const struct pci_device_id artop_pci_tbl[] = {
 	{ }	/* terminate list */
 };
 
+#ifdef CONFIG_PM
+static int atp8xx_reinit_one(struct pci_dev *pdev)
+{
+	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	int rc;
+
+	rc = ata_pci_device_do_resume(pdev);
+	if (rc)
+		return rc;
+
+	atp8xx_fixup(pdev);
+
+	ata_host_resume(host);
+	return 0;
+}
+#endif
+
 static struct pci_driver artop_pci_driver = {
 	.name			= DRV_NAME,
 	.id_table		= artop_pci_tbl,
 	.probe			= artop_init_one,
 	.remove			= ata_pci_remove_one,
+#ifdef CONFIG_PM
+	.suspend		= ata_pci_device_suspend,
+	.resume			= atp8xx_reinit_one,
+#endif
 };
 
 static int __init artop_init(void)
@@ -452,9 +464,8 @@ static void __exit artop_exit(void)
 module_init(artop_init);
 module_exit(artop_exit);
 
- -MODULE_AUTHOR("Alan Cox");
+MODULE_AUTHOR("Alan Cox, Bartlomiej Zolnierkiewicz");
 MODULE_DESCRIPTION("SCSI low-level driver for ARTOP PATA");
 MODULE_LICENSE("GPL");
 MODULE_DEVICE_TABLE(pci, artop_pci_tbl);
 MODULE_VERSION(DRV_VERSION);
- -
diff --git a/drivers/ata/pata_at91.c b/drivers/ata/pata_at91.c
index 960c725..c6d1409 100644
- --- a/drivers/ata/pata_at91.c
+++ b/drivers/ata/pata_at91.c
@@ -414,10 +414,13 @@ static int __devinit pata_at91_probe(struct platform_device *pdev)
 
 	host->private_data = info;
 
- -	return ata_host_activate(host, irq ? gpio_to_irq(irq) : 0,
+	ret = ata_host_activate(host, irq ? gpio_to_irq(irq) : 0,
 			irq ? ata_sff_interrupt : NULL,
 			irq_flags, &pata_at91_sht);
 
+	if (!ret)
+		return 0;
+
 err_put:
 	clk_put(info->mck);
 	return ret;
diff --git a/drivers/ata/pata_atiixp.c b/drivers/ata/pata_atiixp.c
index 4375561..be1aa14 100644
- --- a/drivers/ata/pata_atiixp.c
+++ b/drivers/ata/pata_atiixp.c
@@ -49,6 +49,31 @@ static int atiixp_cable_detect(struct ata_port *ap)
 static DEFINE_SPINLOCK(atiixp_lock);
 
 /**
+ *	atiixp_prereset	-	perform reset handling
+ *	@link: ATA link
+ *	@deadline: deadline jiffies for the operation
+ *
+ *	Reset sequence checking enable bits to see which ports are
+ *	active.
+ */
+
+static int atiixp_prereset(struct ata_link *link, unsigned long deadline)
+{
+	static const struct pci_bits atiixp_enable_bits[] = {
+		{ 0x48, 1, 0x01, 0x00 },
+		{ 0x48, 1, 0x08, 0x00 }
+	};
+
+	struct ata_port *ap = link->ap;
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
+
+	if (!pci_test_config_bits(pdev, &atiixp_enable_bits[ap->port_no]))
+		return -ENOENT;
+
+	return ata_sff_prereset(link, deadline);
+}
+
+/**
  *	atiixp_set_pio_timing	-	set initial PIO mode data
  *	@ap: ATA interface
  *	@adev: ATA device
@@ -221,6 +246,7 @@ static struct ata_port_operations atiixp_port_ops = {
 	.bmdma_start 	= atiixp_bmdma_start,
 	.bmdma_stop	= atiixp_bmdma_stop,
 
+	.prereset	= atiixp_prereset,
 	.cable_detect	= atiixp_cable_detect,
 	.set_piomode	= atiixp_set_piomode,
 	.set_dmamode	= atiixp_set_dmamode,
@@ -235,16 +261,7 @@ static int atiixp_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 		.udma_mask = ATA_UDMA5,
 		.port_ops = &atiixp_port_ops
 	};
- -	static const struct pci_bits atiixp_enable_bits[] = {
- -		{ 0x48, 1, 0x01, 0x00 },
- -		{ 0x48, 1, 0x08, 0x00 }
- -	};
 	const struct ata_port_info *ppi[] = { &info, &info };
- -	int i;
- -
- -	for (i = 0; i < 2; i++)
- -		if (!pci_test_config_bits(pdev, &atiixp_enable_bits[i]))
- -			ppi[i] = &ata_dummy_port_info;
 
 	return ata_pci_bmdma_init_one(pdev, ppi, &atiixp_sht, NULL,
 				      ATA_HOST_PARALLEL_SCAN);
diff --git a/drivers/ata/pata_cmd64x.c b/drivers/ata/pata_cmd64x.c
index 7bafc16..e1fb39a 100644
- --- a/drivers/ata/pata_cmd64x.c
+++ b/drivers/ata/pata_cmd64x.c
@@ -82,7 +82,7 @@ static int cmd648_cable_detect(struct ata_port *ap)
 }
 
 /**
- - *	cmd64x_set_piomode	-	set PIO and MWDMA timing
+ *	cmd64x_set_timing	-	set PIO and MWDMA timing
  *	@ap: ATA interface
  *	@adev: ATA device
  *	@mode: mode
@@ -288,6 +288,22 @@ static struct ata_port_operations cmd648_port_ops = {
 	.cable_detect	= cmd648_cable_detect,
 };
 
+static void cmd64x_fixup(struct pci_dev *pdev)
+{
+	u8 mrdmode;
+
+	pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 64);
+	pci_read_config_byte(pdev, MRDMODE, &mrdmode);
+	mrdmode &= ~0x30;	/* IRQ set up */
+	mrdmode |= 0x02;	/* Memory read line enable */
+	pci_write_config_byte(pdev, MRDMODE, mrdmode);
+
+	/* PPC specific fixup copied from old driver */
+#ifdef CONFIG_PPC
+	pci_write_config_byte(pdev, UDIDETCR0, 0xF0);
+#endif
+}
+
 static int cmd64x_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 {
 	static const struct ata_port_info cmd_info[6] = {
@@ -336,7 +352,7 @@ static int cmd64x_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 		&cmd_info[id->driver_data],
 		NULL
 	};
- -	u8 mrdmode, reg;
+	u8 reg;
 	int rc;
 	struct pci_dev *bridge = pdev->bus->self;
 	/* mobility split bridges don't report enabled ports correctly */
@@ -368,11 +384,7 @@ static int cmd64x_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 			cntrl_ch0_ok = 0;
 	}
 
- -	pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 64);
- -	pci_read_config_byte(pdev, MRDMODE, &mrdmode);
- -	mrdmode &= ~ 0x30;	/* IRQ set up */
- -	mrdmode |= 0x02;	/* Memory read line enable */
- -	pci_write_config_byte(pdev, MRDMODE, mrdmode);
+	cmd64x_fixup(pdev);
 
 	/* check for enabled ports */
 	pci_read_config_byte(pdev, CNTRL, &reg);
@@ -388,13 +400,6 @@ static int cmd64x_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 		ppi[1] = &ata_dummy_port_info;
 	}
 
- -	/* Force PIO 0 here.. */
- -
- -	/* PPC specific fixup copied from old driver */
- -#ifdef CONFIG_PPC
- -	pci_write_config_byte(pdev, UDIDETCR0, 0xF0);
- -#endif
- -
 	return ata_pci_bmdma_init_one(pdev, ppi, &cmd64x_sht, NULL, 0);
 }
 
@@ -402,21 +407,14 @@ static int cmd64x_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 static int cmd64x_reinit_one(struct pci_dev *pdev)
 {
 	struct ata_host *host = dev_get_drvdata(&pdev->dev);
- -	u8 mrdmode;
 	int rc;
 
 	rc = ata_pci_device_do_resume(pdev);
 	if (rc)
 		return rc;
 
- -	pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 64);
- -	pci_read_config_byte(pdev, MRDMODE, &mrdmode);
- -	mrdmode &= ~ 0x30;	/* IRQ set up */
- -	mrdmode |= 0x02;	/* Memory read line enable */
- -	pci_write_config_byte(pdev, MRDMODE, mrdmode);
- -#ifdef CONFIG_PPC
- -	pci_write_config_byte(pdev, UDIDETCR0, 0xF0);
- -#endif
+	cmd64x_fixup(pdev);
+
 	ata_host_resume(host);
 	return 0;
 }
diff --git a/drivers/ata/pata_cs5535.c b/drivers/ata/pata_cs5535.c
index 03a9318..a0b4640 100644
- --- a/drivers/ata/pata_cs5535.c
+++ b/drivers/ata/pata_cs5535.c
@@ -38,7 +38,7 @@
 #include <linux/libata.h>
 #include <asm/msr.h>
 
- -#define DRV_NAME	"cs5535"
+#define DRV_NAME	"pata_cs5535"
 #define DRV_VERSION	"0.2.12"
 
 /*
@@ -67,8 +67,6 @@
 
 #define CS5535_CABLE_DETECT    0x48
 
- -#define CS5535_BAD_PIO(timings) ( (timings&~0x80000000UL)==0x00009172 )
- -
 /**
  *	cs5535_cable_detect	-	detect cable type
  *	@ap: Port to detect on
@@ -188,16 +186,6 @@ static int cs5535_init_one(struct pci_dev *dev, const struct pci_device_id *id)
 	};
 	const struct ata_port_info *ppi[] = { &info, &ata_dummy_port_info };
 
- -	u32 timings, dummy;
- -
- -	/* Check the BIOS set the initial timing clock. If not set the
- -	   timings for PIO0 */
- -	rdmsr(ATAC_CH0D0_PIO, timings, dummy);
- -	if (CS5535_BAD_PIO(timings))
- -		wrmsr(ATAC_CH0D0_PIO, 0xF7F4F7F4UL, 0);
- -	rdmsr(ATAC_CH0D1_PIO, timings, dummy);
- -	if (CS5535_BAD_PIO(timings))
- -		wrmsr(ATAC_CH0D1_PIO, 0xF7F4F7F4UL, 0);
 	return ata_pci_bmdma_init_one(dev, ppi, &cs5535_sht, NULL, 0);
 }
 
@@ -230,7 +218,7 @@ static void __exit cs5535_exit(void)
 }
 
 MODULE_AUTHOR("Alan Cox, Jens Altmann, Wolfgan Zuleger, Alexander Kiausch");
- -MODULE_DESCRIPTION("low-level driver for the NS/AMD 5530");
+MODULE_DESCRIPTION("low-level driver for the NS/AMD 5535");
 MODULE_LICENSE("GPL");
 MODULE_DEVICE_TABLE(pci, cs5535);
 MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/ata/pata_efar.c b/drivers/ata/pata_efar.c
index aca47e4..f0243ed 100644
- --- a/drivers/ata/pata_efar.c
+++ b/drivers/ata/pata_efar.c
@@ -73,7 +73,7 @@ static DEFINE_SPINLOCK(efar_lock);
 /**
  *	efar_set_piomode - Initialize host controller PATA PIO timings
  *	@ap: Port whose timings we are configuring
- - *	@adev: um
+ *	@adev: Device to program
  *
  *	Set PIO mode for device, in host controller PCI config space.
  *
@@ -85,9 +85,9 @@ static void efar_set_piomode (struct ata_port *ap, struct ata_device *adev)
 {
 	unsigned int pio	= adev->pio_mode - XFER_PIO_0;
 	struct pci_dev *dev	= to_pci_dev(ap->host->dev);
- -	unsigned int idetm_port= ap->port_no ? 0x42 : 0x40;
+	unsigned int master_port = ap->port_no ? 0x42 : 0x40;
 	unsigned long flags;
- -	u16 idetm_data;
+	u16 master_data;
 	u8 udma_enable;
 	int control = 0;
 
@@ -113,20 +113,20 @@ static void efar_set_piomode (struct ata_port *ap, struct ata_device *adev)
 
 	spin_lock_irqsave(&efar_lock, flags);
 
- -	pci_read_config_word(dev, idetm_port, &idetm_data);
+	pci_read_config_word(dev, master_port, &master_data);
 
 	/* Set PPE, IE, and TIME as appropriate */
 	if (adev->devno == 0) {
- -		idetm_data &= 0xCCF0;
- -		idetm_data |= control;
- -		idetm_data |= (timings[pio][0] << 12) |
+		master_data &= 0xCCF0;
+		master_data |= control;
+		master_data |= (timings[pio][0] << 12) |
 			(timings[pio][1] << 8);
 	} else {
 		int shift = 4 * ap->port_no;
 		u8 slave_data;
 
- -		idetm_data &= 0xFF0F;
- -		idetm_data |= (control << 4);
+		master_data &= 0xFF0F;
+		master_data |= (control << 4);
 
 		/* Slave timing in separate register */
 		pci_read_config_byte(dev, 0x44, &slave_data);
@@ -135,8 +135,8 @@ static void efar_set_piomode (struct ata_port *ap, struct ata_device *adev)
 		pci_write_config_byte(dev, 0x44, slave_data);
 	}
 
- -	idetm_data |= 0x4000;	/* Ensure SITRE is set */
- -	pci_write_config_word(dev, idetm_port, idetm_data);
+	master_data |= 0x4000;	/* Ensure SITRE is set */
+	pci_write_config_word(dev, master_port, master_data);
 
 	pci_read_config_byte(dev, 0x48, &udma_enable);
 	udma_enable &= ~(1 << (2 * ap->port_no + adev->devno));
diff --git a/drivers/ata/pata_hpt366.c b/drivers/ata/pata_hpt366.c
index 6c77d68..42cffd3 100644
- --- a/drivers/ata/pata_hpt366.c
+++ b/drivers/ata/pata_hpt366.c
@@ -111,6 +111,28 @@ static const struct hpt_clock hpt366_25[] = {
 	{	0,		0x01208585	}
 };
 
+/**
+ *	hpt36x_find_mode	-	find the hpt36x timing
+ *	@ap: ATA port
+ *	@speed: transfer mode
+ *
+ *	Return the 32bit register programming information for this channel
+ *	that matches the speed provided.
+ */
+
+static u32 hpt36x_find_mode(struct ata_port *ap, int speed)
+{
+	struct hpt_clock *clocks = ap->host->private_data;
+
+	while (clocks->xfer_mode) {
+		if (clocks->xfer_mode == speed)
+			return clocks->timing;
+		clocks++;
+	}
+	BUG();
+	return 0xffffffffU;	/* silence compiler warning */
+}
+
 static const char * const bad_ata33[] = {
 	"Maxtor 92720U8", "Maxtor 92040U6", "Maxtor 91360U4", "Maxtor 91020U3",
 	"Maxtor 90845U3", "Maxtor 90650U2",
@@ -210,10 +232,9 @@ static int hpt36x_cable_detect(struct ata_port *ap)
 static void hpt366_set_mode(struct ata_port *ap, struct ata_device *adev,
 			    u8 mode)
 {
- -	struct hpt_clock *clocks = ap->host->private_data;
 	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	u32 addr = 0x40 + 4 * adev->devno;
- -	u32 mask, reg;
+	u32 mask, reg, t;
 
 	/* determine timing mask and find matching clock entry */
 	if (mode < XFER_MW_DMA_0)
@@ -223,13 +244,7 @@ static void hpt366_set_mode(struct ata_port *ap, struct ata_device *adev,
 	else
 		mask = 0x30070000;
 
- -	while (clocks->xfer_mode) {
- -		if (clocks->xfer_mode == mode)
- -			break;
- -		clocks++;
- -	}
- -	if (!clocks->xfer_mode)
- -		BUG();
+	t = hpt36x_find_mode(ap, mode);
 
 	/*
 	 * Combine new mode bits with old config bits and disable
@@ -237,7 +252,7 @@ static void hpt366_set_mode(struct ata_port *ap, struct ata_device *adev,
 	 * problems handling I/O errors later.
 	 */
 	pci_read_config_dword(pdev, addr, &reg);
- -	reg = ((reg & ~mask) | (clocks->timing & mask)) & ~0xc0000000;
+	reg = ((reg & ~mask) | (t & mask)) & ~0xc0000000;
 	pci_write_config_dword(pdev, addr, reg);
 }
 
diff --git a/drivers/ata/pata_it8213.c b/drivers/ata/pata_it8213.c
index 998af0e..cf9164d 100644
- --- a/drivers/ata/pata_it8213.c
+++ b/drivers/ata/pata_it8213.c
@@ -76,8 +76,8 @@ static void it8213_set_piomode (struct ata_port *ap, struct ata_device *adev)
 {
 	unsigned int pio	= adev->pio_mode - XFER_PIO_0;
 	struct pci_dev *dev	= to_pci_dev(ap->host->dev);
- -	unsigned int idetm_port= ap->port_no ? 0x42 : 0x40;
- -	u16 idetm_data;
+	unsigned int master_port = ap->port_no ? 0x42 : 0x40;
+	u16 master_data;
 	int control = 0;
 
 	/*
@@ -100,19 +100,19 @@ static void it8213_set_piomode (struct ata_port *ap, struct ata_device *adev)
 	if (adev->class != ATA_DEV_ATA)
 		control |= 4;	/* PPE */
 
- -	pci_read_config_word(dev, idetm_port, &idetm_data);
+	pci_read_config_word(dev, master_port, &master_data);
 
 	/* Set PPE, IE, and TIME as appropriate */
 	if (adev->devno == 0) {
- -		idetm_data &= 0xCCF0;
- -		idetm_data |= control;
- -		idetm_data |= (timings[pio][0] << 12) |
+		master_data &= 0xCCF0;
+		master_data |= control;
+		master_data |= (timings[pio][0] << 12) |
 			(timings[pio][1] << 8);
 	} else {
 		u8 slave_data;
 
- -		idetm_data &= 0xFF0F;
- -		idetm_data |= (control << 4);
+		master_data &= 0xFF0F;
+		master_data |= (control << 4);
 
 		/* Slave timing in separate register */
 		pci_read_config_byte(dev, 0x44, &slave_data);
@@ -121,8 +121,8 @@ static void it8213_set_piomode (struct ata_port *ap, struct ata_device *adev)
 		pci_write_config_byte(dev, 0x44, slave_data);
 	}
 
- -	idetm_data |= 0x4000;	/* Ensure SITRE is set */
- -	pci_write_config_word(dev, idetm_port, idetm_data);
+	master_data |= 0x4000;	/* Ensure SITRE is set */
+	pci_write_config_word(dev, master_port, master_data);
 }
 
 /**
@@ -163,7 +163,7 @@ static void it8213_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 
 		/* Clocks follow the PIIX style */
 		u_speed = min(2 - (udma & 1), udma);
- -		if (udma == 5)
+		if (udma > 4)
 			u_clock = 0x1000;	/* 100Mhz */
 		else if (udma > 2)
 			u_clock = 1;		/* 66Mhz */
@@ -262,7 +262,7 @@ static int it8213_init_one (struct pci_dev *pdev, const struct pci_device_id *en
 		.flags		= ATA_FLAG_SLAVE_POSS,
 		.pio_mask	= ATA_PIO4,
 		.mwdma_mask	= ATA_MWDMA12_ONLY,
- -		.udma_mask 	= ATA_UDMA4, /* FIXME: want UDMA 100? */
+		.udma_mask	= ATA_UDMA6,
 		.port_ops	= &it8213_ops,
 	};
 	/* Current IT8213 stuff is single port */
diff --git a/drivers/ata/pata_legacy.c b/drivers/ata/pata_legacy.c
index d960f8e..35aca7d 100644
- --- a/drivers/ata/pata_legacy.c
+++ b/drivers/ata/pata_legacy.c
@@ -79,15 +79,6 @@ static int all;
 module_param(all, int, 0444);
 MODULE_PARM_DESC(all, "Grab all legacy port devices, even if PCI(0=off, 1=on)");
 
- -struct legacy_data {
- -	unsigned long timing;
- -	u8 clock[2];
- -	u8 last;
- -	int fast;
- -	struct platform_device *platform_dev;
- -
- -};
- -
 enum controller {
 	BIOS = 0,
 	SNOOP = 1,
@@ -104,6 +95,14 @@ enum controller {
 	UNKNOWN = -1
 };
 
+struct legacy_data {
+	unsigned long timing;
+	u8 clock[2];
+	u8 last;
+	int fast;
+	enum controller type;
+	struct platform_device *platform_dev;
+};
 
 struct legacy_probe {
 	unsigned char *name;
@@ -137,11 +136,17 @@ static int ht6560a;		/* HT 6560A on primary 1, second 2, both 3 */
 static int ht6560b;		/* HT 6560A on primary 1, second 2, both 3 */
 static int opti82c611a;		/* Opti82c611A on primary 1, sec 2, both 3 */
 static int opti82c46x;		/* Opti 82c465MV present(pri/sec autodetect) */
- -static int qdi;			/* Set to probe QDI controllers */
 static int autospeed;		/* Chip present which snoops speed changes */
 static int pio_mask = ATA_PIO4;	/* PIO range for autospeed devices */
 static int iordy_mask = 0xFFFFFFFF;	/* Use iordy if available */
 
+/* Set to probe QDI controllers */
+#ifdef CONFIG_PATA_QDI_MODULE
+static int qdi = 1;
+#else
+static int qdi;
+#endif
+
 #ifdef CONFIG_PATA_WINBOND_VLB_MODULE
 static int winbond = 1;		/* Set to probe Winbond controllers,
 					give I/O port if non standard */
@@ -631,40 +636,20 @@ static struct ata_port_operations opti82c46x_port_ops = {
 	.qc_issue	= opti82c46x_qc_issue,
 };
 
- -static void qdi6500_set_piomode(struct ata_port *ap, struct ata_device *adev)
- -{
- -	struct ata_timing t;
- -	struct legacy_data *ld_qdi = ap->host->private_data;
- -	int active, recovery;
- -	u8 timing;
- -
- -	/* Get the timing data in cycles */
- -	ata_timing_compute(adev, adev->pio_mode, &t, 30303, 1000);
- -
- -	if (ld_qdi->fast) {
- -		active = 8 - clamp_val(t.active, 1, 8);
- -		recovery = 18 - clamp_val(t.recover, 3, 18);
- -	} else {
- -		active = 9 - clamp_val(t.active, 2, 9);
- -		recovery = 15 - clamp_val(t.recover, 0, 15);
- -	}
- -	timing = (recovery << 4) | active | 0x08;
- -
- -	ld_qdi->clock[adev->devno] = timing;
- -
- -	outb(timing, ld_qdi->timing);
- -}
- -
 /**
- - *	qdi6580dp_set_piomode		-	PIO setup for dual channel
+ *	qdi65x0_set_piomode		-	PIO setup for QDI65x0
  *	@ap: Port
  *	@adev: Device
  *
+ *	In single channel mode the 6580 has one clock per device and we can
+ *	avoid the requirement to clock switch. We also have to load the timing
+ *	into the right clock according to whether we are master or slave.
+ *
  *	In dual channel mode the 6580 has one clock per channel and we have
  *	to software clockswitch in qc_issue.
  */
 
- -static void qdi6580dp_set_piomode(struct ata_port *ap, struct ata_device *adev)
+static void qdi65x0_set_piomode(struct ata_port *ap, struct ata_device *adev)
 {
 	struct ata_timing t;
 	struct legacy_data *ld_qdi = ap->host->private_data;
@@ -682,47 +667,15 @@ static void qdi6580dp_set_piomode(struct ata_port *ap, struct ata_device *adev)
 		recovery = 15 - clamp_val(t.recover, 0, 15);
 	}
 	timing = (recovery << 4) | active | 0x08;
- -
 	ld_qdi->clock[adev->devno] = timing;
 
- -	outb(timing, ld_qdi->timing + 2 * ap->port_no);
- -	/* Clear the FIFO */
- -	if (adev->class != ATA_DEV_ATA)
- -		outb(0x5F, (ld_qdi->timing & 0xFFF0) + 3);
- -}
- -
- -/**
- - *	qdi6580_set_piomode		-	PIO setup for single channel
- - *	@ap: Port
- - *	@adev: Device
- - *
- - *	In single channel mode the 6580 has one clock per device and we can
- - *	avoid the requirement to clock switch. We also have to load the timing
- - *	into the right clock according to whether we are master or slave.
- - */
- -
- -static void qdi6580_set_piomode(struct ata_port *ap, struct ata_device *adev)
- -{
- -	struct ata_timing t;
- -	struct legacy_data *ld_qdi = ap->host->private_data;
- -	int active, recovery;
- -	u8 timing;
- -
- -	/* Get the timing data in cycles */
- -	ata_timing_compute(adev, adev->pio_mode, &t, 30303, 1000);
+	if (ld_qdi->type == QDI6580)
+		outb(timing, ld_qdi->timing + 2 * adev->devno);
+	else
+		outb(timing, ld_qdi->timing + 2 * ap->port_no);
 
- -	if (ld_qdi->fast) {
- -		active = 8 - clamp_val(t.active, 1, 8);
- -		recovery = 18 - clamp_val(t.recover, 3, 18);
- -	} else {
- -		active = 9 - clamp_val(t.active, 2, 9);
- -		recovery = 15 - clamp_val(t.recover, 0, 15);
- -	}
- -	timing = (recovery << 4) | active | 0x08;
- -	ld_qdi->clock[adev->devno] = timing;
- -	outb(timing, ld_qdi->timing + 2 * adev->devno);
 	/* Clear the FIFO */
- -	if (adev->class != ATA_DEV_ATA)
+	if (ld_qdi->type != QDI6500 && adev->class != ATA_DEV_ATA)
 		outb(0x5F, (ld_qdi->timing & 0xFFF0) + 3);
 }
 
@@ -789,20 +742,20 @@ static int qdi_port(struct platform_device *dev,
 
 static struct ata_port_operations qdi6500_port_ops = {
 	.inherits	= &legacy_base_port_ops,
- -	.set_piomode	= qdi6500_set_piomode,
+	.set_piomode	= qdi65x0_set_piomode,
 	.qc_issue	= qdi_qc_issue,
 	.sff_data_xfer	= vlb32_data_xfer,
 };
 
 static struct ata_port_operations qdi6580_port_ops = {
 	.inherits	= &legacy_base_port_ops,
- -	.set_piomode	= qdi6580_set_piomode,
+	.set_piomode	= qdi65x0_set_piomode,
 	.sff_data_xfer	= vlb32_data_xfer,
 };
 
 static struct ata_port_operations qdi6580dp_port_ops = {
 	.inherits	= &legacy_base_port_ops,
- -	.set_piomode	= qdi6580dp_set_piomode,
+	.set_piomode	= qdi65x0_set_piomode,
 	.qc_issue	= qdi_qc_issue,
 	.sff_data_xfer	= vlb32_data_xfer,
 };
@@ -879,29 +832,29 @@ static struct ata_port_operations winbond_port_ops = {
 };
 
 static struct legacy_controller controllers[] = {
- -	{"BIOS",	&legacy_port_ops, 	0x1F,
+	{"BIOS",	&legacy_port_ops, 	ATA_PIO4,
 			ATA_FLAG_NO_IORDY,	0,			NULL },
- -	{"Snooping", 	&simple_port_ops, 	0x1F,
+	{"Snooping", 	&simple_port_ops, 	ATA_PIO4,
 			0,			0,			NULL },
- -	{"PDC20230",	&pdc20230_port_ops,	0x7,
+	{"PDC20230",	&pdc20230_port_ops,	ATA_PIO2,
 			ATA_FLAG_NO_IORDY,
 			ATA_PFLAG_PIO32 | ATA_PFLAG_PIO32CHANGE,	NULL },
- -	{"HT6560A",	&ht6560a_port_ops,	0x07,
+	{"HT6560A",	&ht6560a_port_ops,	ATA_PIO2,
 			ATA_FLAG_NO_IORDY,	0,			NULL },
- -	{"HT6560B",	&ht6560b_port_ops,	0x1F,
+	{"HT6560B",	&ht6560b_port_ops,	ATA_PIO4,
 			ATA_FLAG_NO_IORDY,	0,			NULL },
- -	{"OPTI82C611A",	&opti82c611a_port_ops,	0x0F,
+	{"OPTI82C611A",	&opti82c611a_port_ops,	ATA_PIO3,
 			0,			0,			NULL },
- -	{"OPTI82C46X",	&opti82c46x_port_ops,	0x0F,
+	{"OPTI82C46X",	&opti82c46x_port_ops,	ATA_PIO3,
 			0,			0,			NULL },
- -	{"QDI6500",	&qdi6500_port_ops,	0x07,
+	{"QDI6500",	&qdi6500_port_ops,	ATA_PIO2,
 			ATA_FLAG_NO_IORDY,
 			ATA_PFLAG_PIO32 | ATA_PFLAG_PIO32CHANGE,    qdi_port },
- -	{"QDI6580",	&qdi6580_port_ops,	0x1F,
+	{"QDI6580",	&qdi6580_port_ops,	ATA_PIO4,
 			0, ATA_PFLAG_PIO32 | ATA_PFLAG_PIO32CHANGE, qdi_port },
- -	{"QDI6580DP",	&qdi6580dp_port_ops,	0x1F,
+	{"QDI6580DP",	&qdi6580dp_port_ops,	ATA_PIO4,
 			0, ATA_PFLAG_PIO32 | ATA_PFLAG_PIO32CHANGE, qdi_port },
- -	{"W83759A",	&winbond_port_ops,	0x1F,
+	{"W83759A",	&winbond_port_ops,	ATA_PIO4,
 			0, ATA_PFLAG_PIO32 | ATA_PFLAG_PIO32CHANGE,
 								winbond_port }
 };
@@ -1022,6 +975,7 @@ static __init int legacy_init_one(struct legacy_probe *probe)
 	ctrl_addr = devm_ioport_map(&pdev->dev, io + 0x0206, 1);
 	if (!io_addr || !ctrl_addr)
 		goto fail;
+	ld->type = probe->type;
 	if (controller->setup)
 		if (controller->setup(pdev, probe, ld) < 0)
 			goto fail;
@@ -1306,6 +1260,7 @@ MODULE_AUTHOR("Alan Cox");
 MODULE_DESCRIPTION("low-level driver for legacy ATA");
 MODULE_LICENSE("GPL");
 MODULE_VERSION(DRV_VERSION);
+MODULE_ALIAS("pata_qdi");
 MODULE_ALIAS("pata_winbond");
 
 module_param(probe_all, int, 0);
diff --git a/drivers/ata/pata_mpc52xx.c b/drivers/ata/pata_mpc52xx.c
index 2fcac51..3e17463 100644
- --- a/drivers/ata/pata_mpc52xx.c
+++ b/drivers/ata/pata_mpc52xx.c
@@ -780,7 +780,7 @@ mpc52xx_ata_probe(struct platform_device *op)
 	}
 
 	task_irq = bcom_get_task_irq(dmatsk);
- -	ret = request_irq(task_irq, &mpc52xx_ata_task_irq, IRQF_DISABLED,
+	ret = request_irq(task_irq, &mpc52xx_ata_task_irq, 0,
 				"ATA task", priv);
 	if (ret) {
 		dev_err(&op->dev, "error requesting DMA IRQ\n");
diff --git a/drivers/ata/pata_of_platform.c b/drivers/ata/pata_of_platform.c
index f305400..a72ab0d 100644
- --- a/drivers/ata/pata_of_platform.c
+++ b/drivers/ata/pata_of_platform.c
@@ -11,6 +11,8 @@
 
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
 #include <linux/of_platform.h>
 #include <linux/ata_platform.h>
 
@@ -57,11 +59,11 @@ static int __devinit pata_of_platform_probe(struct platform_device *ofdev)
 
 	prop = of_get_property(dn, "reg-shift", NULL);
 	if (prop)
- -		reg_shift = *prop;
+		reg_shift = be32_to_cpup(prop);
 
 	prop = of_get_property(dn, "pio-mode", NULL);
 	if (prop) {
- -		pio_mode = *prop;
+		pio_mode = be32_to_cpup(prop);
 		if (pio_mode > 6) {
 			dev_err(&ofdev->dev, "invalid pio-mode\n");
 			return -EINVAL;
diff --git a/drivers/ata/pata_pdc2027x.c b/drivers/ata/pata_pdc2027x.c
index b1511f3..7d63f24 100644
- --- a/drivers/ata/pata_pdc2027x.c
+++ b/drivers/ata/pata_pdc2027x.c
@@ -63,6 +63,7 @@ enum {
 };
 
 static int pdc2027x_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
+static int pdc2027x_reinit_one(struct pci_dev *pdev);
 static int pdc2027x_prereset(struct ata_link *link, unsigned long deadline);
 static void pdc2027x_set_piomode(struct ata_port *ap, struct ata_device *adev);
 static void pdc2027x_set_dmamode(struct ata_port *ap, struct ata_device *adev);
@@ -126,6 +127,10 @@ static struct pci_driver pdc2027x_pci_driver = {
 	.id_table		= pdc2027x_pci_tbl,
 	.probe			= pdc2027x_init_one,
 	.remove			= ata_pci_remove_one,
+#ifdef CONFIG_PM
+	.suspend		= ata_pci_device_suspend,
+	.resume			= pdc2027x_reinit_one,
+#endif
 };
 
 static struct scsi_host_template pdc2027x_sht = {
@@ -754,6 +759,31 @@ static int __devinit pdc2027x_init_one(struct pci_dev *pdev, const struct pci_de
 				 IRQF_SHARED, &pdc2027x_sht);
 }
 
+#ifdef CONFIG_PM
+static int pdc2027x_reinit_one(struct pci_dev *pdev)
+{
+	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	unsigned int board_idx;
+	int rc;
+
+	rc = ata_pci_device_do_resume(pdev);
+	if (rc)
+		return rc;
+
+	if (pdev->device == PCI_DEVICE_ID_PROMISE_20268 ||
+	    pdev->device == PCI_DEVICE_ID_PROMISE_20270)
+		board_idx = PDC_UDMA_100;
+	else
+		board_idx = PDC_UDMA_133;
+
+	if (pdc_hardware_init(host, board_idx))
+		return -EIO;
+
+	ata_host_resume(host);
+	return 0;
+}
+#endif
+
 /**
  * pdc2027x_init - Called after this module is loaded into the kernel.
  */
diff --git a/drivers/ata/pata_qdi.c b/drivers/ata/pata_qdi.c
deleted file mode 100644
index 45879dc..0000000
- --- a/drivers/ata/pata_qdi.c
+++ /dev/null
@@ -1,366 +0,0 @@
- -/*
- - *    pata_qdi.c - QDI VLB ATA controllers
- - *	(C) 2006 Red Hat
- - *
- - * This driver mostly exists as a proof of concept for non PCI devices under
- - * libata. While the QDI6580 was 'neat' in 1993 it is no longer terribly
- - * useful.
- - *
- - * Tuning code written from the documentation at
- - * http://www.ryston.cz/petr/vlb/qd6500.html
- - * http://www.ryston.cz/petr/vlb/qd6580.html
- - *
- - * Probe code based on drivers/ide/legacy/qd65xx.c
- - * Rewritten from the work of Colten Edwards <pje120@cs.usask.ca> by
- - * Samuel Thibault <samuel.thibault@ens-lyon.org>
- - */
- -
- -#include <linux/kernel.h>
- -#include <linux/module.h>
- -#include <linux/pci.h>
- -#include <linux/init.h>
- -#include <linux/blkdev.h>
- -#include <linux/delay.h>
- -#include <scsi/scsi_host.h>
- -#include <linux/libata.h>
- -#include <linux/platform_device.h>
- -
- -#define DRV_NAME "pata_qdi"
- -#define DRV_VERSION "0.3.1"
- -
- -#define NR_HOST 4	/* Two 6580s */
- -
- -struct qdi_data {
- -	unsigned long timing;
- -	u8 clock[2];
- -	u8 last;
- -	int fast;
- -	struct platform_device *platform_dev;
- -
- -};
- -
- -static struct ata_host *qdi_host[NR_HOST];
- -static struct qdi_data qdi_data[NR_HOST];
- -static int nr_qdi_host;
- -
- -#ifdef MODULE
- -static int probe_qdi = 1;
- -#else
- -static int probe_qdi;
- -#endif
- -
- -static void qdi6500_set_piomode(struct ata_port *ap, struct ata_device *adev)
- -{
- -	struct ata_timing t;
- -	struct qdi_data *qdi = ap->host->private_data;
- -	int active, recovery;
- -	u8 timing;
- -
- -	/* Get the timing data in cycles */
- -	ata_timing_compute(adev, adev->pio_mode, &t, 30303, 1000);
- -
- -	if (qdi->fast) {
- -		active = 8 - clamp_val(t.active, 1, 8);
- -		recovery = 18 - clamp_val(t.recover, 3, 18);
- -	} else {
- -		active = 9 - clamp_val(t.active, 2, 9);
- -		recovery = 15 - clamp_val(t.recover, 0, 15);
- -	}
- -	timing = (recovery << 4) | active | 0x08;
- -
- -	qdi->clock[adev->devno] = timing;
- -
- -	outb(timing, qdi->timing);
- -}
- -
- -static void qdi6580_set_piomode(struct ata_port *ap, struct ata_device *adev)
- -{
- -	struct ata_timing t;
- -	struct qdi_data *qdi = ap->host->private_data;
- -	int active, recovery;
- -	u8 timing;
- -
- -	/* Get the timing data in cycles */
- -	ata_timing_compute(adev, adev->pio_mode, &t, 30303, 1000);
- -
- -	if (qdi->fast) {
- -		active = 8 - clamp_val(t.active, 1, 8);
- -		recovery = 18 - clamp_val(t.recover, 3, 18);
- -	} else {
- -		active = 9 - clamp_val(t.active, 2, 9);
- -		recovery = 15 - clamp_val(t.recover, 0, 15);
- -	}
- -	timing = (recovery << 4) | active | 0x08;
- -
- -	qdi->clock[adev->devno] = timing;
- -
- -	outb(timing, qdi->timing);
- -
- -	/* Clear the FIFO */
- -	if (adev->class != ATA_DEV_ATA)
- -		outb(0x5F, (qdi->timing & 0xFFF0) + 3);
- -}
- -
- -/**
- - *	qdi_qc_issue		-	command issue
- - *	@qc: command pending
- - *
- - *	Called when the libata layer is about to issue a command. We wrap
- - *	this interface so that we can load the correct ATA timings.
- - */
- -
- -static unsigned int qdi_qc_issue(struct ata_queued_cmd *qc)
- -{
- -	struct ata_port *ap = qc->ap;
- -	struct ata_device *adev = qc->dev;
- -	struct qdi_data *qdi = ap->host->private_data;
- -
- -	if (qdi->clock[adev->devno] != qdi->last) {
- -		if (adev->pio_mode) {
- -			qdi->last = qdi->clock[adev->devno];
- -			outb(qdi->clock[adev->devno], qdi->timing);
- -		}
- -	}
- -	return ata_sff_qc_issue(qc);
- -}
- -
- -static unsigned int qdi_data_xfer(struct ata_device *dev, unsigned char *buf,
- -				  unsigned int buflen, int rw)
- -{
- -	if (ata_id_has_dword_io(dev->id)) {
- -		struct ata_port *ap = dev->link->ap;
- -		int slop = buflen & 3;
- -
- -		if (rw == READ)
- -			ioread32_rep(ap->ioaddr.data_addr, buf, buflen >> 2);
- -		else
- -			iowrite32_rep(ap->ioaddr.data_addr, buf, buflen >> 2);
- -
- -		if (unlikely(slop)) {
- -			__le32 pad;
- -			if (rw == READ) {
- -				pad = cpu_to_le32(ioread32(ap->ioaddr.data_addr));
- -				memcpy(buf + buflen - slop, &pad, slop);
- -			} else {
- -				memcpy(&pad, buf + buflen - slop, slop);
- -				iowrite32(le32_to_cpu(pad), ap->ioaddr.data_addr);
- -			}
- -			buflen += 4 - slop;
- -		}
- -	} else
- -		buflen = ata_sff_data_xfer(dev, buf, buflen, rw);
- -
- -	return buflen;
- -}
- -
- -static struct scsi_host_template qdi_sht = {
- -	ATA_PIO_SHT(DRV_NAME),
- -};
- -
- -static struct ata_port_operations qdi6500_port_ops = {
- -	.inherits	= &ata_sff_port_ops,
- -	.qc_issue	= qdi_qc_issue,
- -	.sff_data_xfer	= qdi_data_xfer,
- -	.cable_detect	= ata_cable_40wire,
- -	.set_piomode	= qdi6500_set_piomode,
- -};
- -
- -static struct ata_port_operations qdi6580_port_ops = {
- -	.inherits	= &qdi6500_port_ops,
- -	.set_piomode	= qdi6580_set_piomode,
- -};
- -
- -/**
- - *	qdi_init_one		-	attach a qdi interface
- - *	@type: Type to display
- - *	@io: I/O port start
- - *	@irq: interrupt line
- - *	@fast: True if on a > 33Mhz VLB
- - *
- - *	Register an ISA bus IDE interface. Such interfaces are PIO and we
- - *	assume do not support IRQ sharing.
- - */
- -
- -static __init int qdi_init_one(unsigned long port, int type, unsigned long io, int irq, int fast)
- -{
- -	unsigned long ctl = io + 0x206;
- -	struct platform_device *pdev;
- -	struct ata_host *host;
- -	struct ata_port *ap;
- -	void __iomem *io_addr, *ctl_addr;
- -	int ret;
- -
- -	/*
- -	 *	Fill in a probe structure first of all
- -	 */
- -
- -	pdev = platform_device_register_simple(DRV_NAME, nr_qdi_host, NULL, 0);
- -	if (IS_ERR(pdev))
- -		return PTR_ERR(pdev);
- -
- -	ret = -ENOMEM;
- -	io_addr = devm_ioport_map(&pdev->dev, io, 8);
- -	ctl_addr = devm_ioport_map(&pdev->dev, ctl, 1);
- -	if (!io_addr || !ctl_addr)
- -		goto fail;
- -
- -	ret = -ENOMEM;
- -	host = ata_host_alloc(&pdev->dev, 1);
- -	if (!host)
- -		goto fail;
- -	ap = host->ports[0];
- -
- -	if (type == 6580) {
- -		ap->ops = &qdi6580_port_ops;
- -		ap->pio_mask = ATA_PIO4;
- -		ap->flags |= ATA_FLAG_SLAVE_POSS;
- -	} else {
- -		ap->ops = &qdi6500_port_ops;
- -		ap->pio_mask = ATA_PIO2; /* Actually PIO3 !IORDY is possible */
- -		ap->flags = ATA_FLAG_SLAVE_POSS | ATA_FLAG_NO_IORDY;
- -	}
- -
- -	ap->ioaddr.cmd_addr = io_addr;
- -	ap->ioaddr.altstatus_addr = ctl_addr;
- -	ap->ioaddr.ctl_addr = ctl_addr;
- -	ata_sff_std_ports(&ap->ioaddr);
- -
- -	ata_port_desc(ap, "cmd %lx ctl %lx", io, ctl);
- -
- -	/*
- -	 *	Hook in a private data structure per channel
- -	 */
- -	ap->private_data = &qdi_data[nr_qdi_host];
- -
- -	qdi_data[nr_qdi_host].timing = port;
- -	qdi_data[nr_qdi_host].fast = fast;
- -	qdi_data[nr_qdi_host].platform_dev = pdev;
- -
- -	printk(KERN_INFO DRV_NAME": qd%d at 0x%lx.\n", type, io);
- -
- -	/* activate */
- -	ret = ata_host_activate(host, irq, ata_sff_interrupt, 0, &qdi_sht);
- -	if (ret)
- -		goto fail;
- -
- -	qdi_host[nr_qdi_host++] = dev_get_drvdata(&pdev->dev);
- -	return 0;
- -
- - fail:
- -	platform_device_unregister(pdev);
- -	return ret;
- -}
- -
- -/**
- - *	qdi_init		-	attach qdi interfaces
- - *
- - *	Attach qdi IDE interfaces by scanning the ports it may occupy.
- - */
- -
- -static __init int qdi_init(void)
- -{
- -	unsigned long flags;
- -	static const unsigned long qd_port[2] = { 0x30, 0xB0 };
- -	static const unsigned long ide_port[2] = { 0x170, 0x1F0 };
- -	static const int ide_irq[2] = { 14, 15 };
- -
- -	int ct = 0;
- -	int i;
- -
- -	if (probe_qdi == 0)
- -		return -ENODEV;
- -
- -	/*
- - 	 *	Check each possible QD65xx base address
- -	 */
- -
- -	for (i = 0; i < 2; i++) {
- -		unsigned long port = qd_port[i];
- -		u8 r, res;
- -
- -
- -		if (request_region(port, 2, "pata_qdi")) {
- -			/* Check for a card */
- -			local_irq_save(flags);
- -			r = inb_p(port);
- -			outb_p(0x19, port);
- -			res = inb_p(port);
- -			outb_p(r, port);
- -			local_irq_restore(flags);
- -
- -			/* Fail */
- -			if (res == 0x19)
- -			{
- -				release_region(port, 2);
- -				continue;
- -			}
- -
- -			/* Passes the presence test */
- -			r = inb_p(port + 1);	/* Check port agrees with port set */
- -			if ((r & 2) >> 1 != i) {
- -				release_region(port, 2);
- -				continue;
- -			}
- -
- -			/* Check card type */
- -			if ((r & 0xF0) == 0xC0) {
- -				/* QD6500: single channel */
- -				if (r & 8) {
- -					/* Disabled ? */
- -					release_region(port, 2);
- -					continue;
- -				}
- -				if (qdi_init_one(port, 6500, ide_port[r & 0x01], ide_irq[r & 0x01], r & 0x04) == 0)
- -					ct++;
- -			}
- -			if (((r & 0xF0) == 0xA0) || (r & 0xF0) == 0x50) {
- -				/* QD6580: dual channel */
- -				if (!request_region(port + 2 , 2, "pata_qdi"))
- -				{
- -					release_region(port, 2);
- -					continue;
- -				}
- -				res = inb(port + 3);
- -				if (res & 1) {
- -					/* Single channel mode */
- -					if (qdi_init_one(port, 6580, ide_port[r & 0x01], ide_irq[r & 0x01], r & 0x04) == 0)
- -						ct++;
- -				} else {
- -					/* Dual channel mode */
- -					if (qdi_init_one(port, 6580, 0x1F0, 14, r & 0x04) == 0)
- -						ct++;
- -					if (qdi_init_one(port + 2, 6580, 0x170, 15, r & 0x04) == 0)
- -						ct++;
- -				}
- -			}
- -		}
- -	}
- -	if (ct != 0)
- -		return 0;
- -	return -ENODEV;
- -}
- -
- -static __exit void qdi_exit(void)
- -{
- -	int i;
- -
- -	for (i = 0; i < nr_qdi_host; i++) {
- -		ata_host_detach(qdi_host[i]);
- -		/* Free the control resource. The 6580 dual channel has the resources
- -		 * claimed as a pair of 2 byte resources so we need no special cases...
- -		 */
- -		release_region(qdi_data[i].timing, 2);
- -		platform_device_unregister(qdi_data[i].platform_dev);
- -	}
- -}
- -
- -MODULE_AUTHOR("Alan Cox");
- -MODULE_DESCRIPTION("low-level driver for qdi ATA");
- -MODULE_LICENSE("GPL");
- -MODULE_VERSION(DRV_VERSION);
- -
- -module_init(qdi_init);
- -module_exit(qdi_exit);
- -
- -module_param(probe_qdi, int, 0);
- -
diff --git a/drivers/ata/pata_rdc.c b/drivers/ata/pata_rdc.c
index 4d318f8..e6a2dd7 100644
- --- a/drivers/ata/pata_rdc.c
+++ b/drivers/ata/pata_rdc.c
@@ -86,6 +86,8 @@ static int rdc_pata_prereset(struct ata_link *link, unsigned long deadline)
 	return ata_sff_prereset(link, deadline);
 }
 
+static DEFINE_SPINLOCK(rdc_lock);
+
 /**
  *	rdc_set_piomode - Initialize host controller PATA PIO timings
  *	@ap: Port whose timings we are configuring
@@ -101,6 +103,7 @@ static void rdc_set_piomode(struct ata_port *ap, struct ata_device *adev)
 {
 	unsigned int pio	= adev->pio_mode - XFER_PIO_0;
 	struct pci_dev *dev	= to_pci_dev(ap->host->dev);
+	unsigned long flags;
 	unsigned int is_slave	= (adev->devno != 0);
 	unsigned int master_port= ap->port_no ? 0x42 : 0x40;
 	unsigned int slave_port	= 0x44;
@@ -124,6 +127,8 @@ static void rdc_set_piomode(struct ata_port *ap, struct ata_device *adev)
 	if (adev->class == ATA_DEV_ATA)
 		control |= 4;	/* PPE enable */
 
+	spin_lock_irqsave(&rdc_lock, flags);
+
 	/* PIO configuration clears DTE unconditionally.  It will be
 	 * programmed in set_dmamode which is guaranteed to be called
 	 * after set_piomode if any DMA mode is available.
@@ -161,6 +166,8 @@ static void rdc_set_piomode(struct ata_port *ap, struct ata_device *adev)
 	pci_read_config_byte(dev, 0x48, &udma_enable);
 	udma_enable &= ~(1 << (2 * ap->port_no + adev->devno));
 	pci_write_config_byte(dev, 0x48, udma_enable);
+
+	spin_unlock_irqrestore(&rdc_lock, flags);
 }
 
 /**
@@ -177,6 +184,7 @@ static void rdc_set_piomode(struct ata_port *ap, struct ata_device *adev)
 static void rdc_set_dmamode(struct ata_port *ap, struct ata_device *adev)
 {
 	struct pci_dev *dev	= to_pci_dev(ap->host->dev);
+	unsigned long flags;
 	u8 master_port		= ap->port_no ? 0x42 : 0x40;
 	u16 master_data;
 	u8 speed		= adev->dma_mode;
@@ -190,6 +198,8 @@ static void rdc_set_dmamode(struct ata_port *ap, struct ata_device *adev)
 			    { 2, 1 },
 			    { 2, 3 }, };
 
+	spin_lock_irqsave(&rdc_lock, flags);
+
 	pci_read_config_word(dev, master_port, &master_data);
 	pci_read_config_byte(dev, 0x48, &udma_enable);
 
@@ -271,6 +281,8 @@ static void rdc_set_dmamode(struct ata_port *ap, struct ata_device *adev)
 		pci_write_config_word(dev, master_port, master_data);
 	}
 	pci_write_config_byte(dev, 0x48, udma_enable);
+
+	spin_unlock_irqrestore(&rdc_lock, flags);
 }
 
 static struct ata_port_operations rdc_pata_ops = {
@@ -375,6 +387,10 @@ static struct pci_driver rdc_pci_driver = {
 	.id_table		= rdc_pci_tbl,
 	.probe			= rdc_init_one,
 	.remove			= rdc_remove_one,
+#ifdef CONFIG_PM
+	.suspend		= ata_pci_device_suspend,
+	.resume			= ata_pci_device_resume,
+#endif
 };
 
 
diff --git a/drivers/ata/pata_sc1200.c b/drivers/ata/pata_sc1200.c
index e2c1825..c0e603a 100644
- --- a/drivers/ata/pata_sc1200.c
+++ b/drivers/ata/pata_sc1200.c
@@ -38,7 +38,7 @@
 #include <scsi/scsi_host.h>
 #include <linux/libata.h>
 
- -#define DRV_NAME	"sc1200"
+#define DRV_NAME	"pata_sc1200"
 #define DRV_VERSION	"0.2.6"
 
 #define SC1200_REV_A	0x00
@@ -86,10 +86,14 @@ static int sc1200_clock(void)
 static void sc1200_set_piomode(struct ata_port *ap, struct ata_device *adev)
 {
 	static const u32 pio_timings[4][5] = {
- -		{0x00009172, 0x00012171, 0x00020080, 0x00032010, 0x00040010},	// format0  33Mhz
- -		{0xd1329172, 0x71212171, 0x30200080, 0x20102010, 0x00100010},	// format1, 33Mhz
- -		{0xfaa3f4f3, 0xc23232b2, 0x513101c1, 0x31213121, 0x10211021},	// format1, 48Mhz
- -		{0xfff4fff4, 0xf35353d3, 0x814102f1, 0x42314231, 0x11311131}	// format1, 66Mhz
+		/* format0, 33Mhz */
+		{ 0x00009172, 0x00012171, 0x00020080, 0x00032010, 0x00040010 },
+		/* format1, 33Mhz */
+		{ 0xd1329172, 0x71212171, 0x30200080, 0x20102010, 0x00100010 },
+		/* format1, 48Mhz */
+		{ 0xfaa3f4f3, 0xc23232b2, 0x513101c1, 0x31213121, 0x10211021 },
+		/* format1, 66Mhz */
+		{ 0xfff4fff4, 0xf35353d3, 0x814102f1, 0x42314231, 0x11311131 }
 	};
 
 	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
diff --git a/drivers/ata/pata_scc.c b/drivers/ata/pata_scc.c
index eb748e3..e265f83 100644
- --- a/drivers/ata/pata_scc.c
+++ b/drivers/ata/pata_scc.c
@@ -826,18 +826,6 @@ static unsigned int scc_data_xfer (struct ata_device *dev, unsigned char *buf,
 }
 
 /**
- - *	scc_pata_prereset - prepare for reset
- - *	@ap: ATA port to be reset
- - *	@deadline: deadline jiffies for the operation
- - */
- -
- -static int scc_pata_prereset(struct ata_link *link, unsigned long deadline)
- -{
- -	link->ap->cbl = ATA_CBL_PATA80;
- -	return ata_sff_prereset(link, deadline);
- -}
- -
- -/**
  *	scc_postreset - standard postreset callback
  *	@ap: the target ata_port
  *	@classes: classes of attached devices
@@ -946,7 +934,7 @@ static struct ata_port_operations scc_pata_ops = {
 	.bmdma_status		= scc_bmdma_status,
 	.sff_data_xfer		= scc_data_xfer,
 
- -	.prereset		= scc_pata_prereset,
+	.cable_detect		= ata_cable_80wire,
 	.softreset		= scc_softreset,
 	.postreset		= scc_postreset,
 
diff --git a/drivers/ata/pata_serverworks.c b/drivers/ata/pata_serverworks.c
index 86dd714..71eaf38 100644
- --- a/drivers/ata/pata_serverworks.c
+++ b/drivers/ata/pata_serverworks.c
@@ -58,31 +58,15 @@ static const char *csb_bad_ata100[] = {
 };
 
 /**
- - *	dell_cable	-	Dell serverworks cable detection
+ *	oem_cable	-	Dell/Sun serverworks cable detection
  *	@ap: ATA port to do cable detect
  *
- - *	Dell hide the 40/80 pin select for their interfaces in the top two
- - *	bits of the subsystem ID.
+ *	Dell PowerEdge and Sun Cobalt 'Alpine' hide the 40/80 pin select
+ *	for their interfaces in the top two bits of the subsystem ID.
  */
 
- -static int dell_cable(struct ata_port *ap) {
- -	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
- -
- -	if (pdev->subsystem_device & (1 << (ap->port_no + 14)))
- -		return ATA_CBL_PATA80;
- -	return ATA_CBL_PATA40;
- -}
- -
- -/**
- - *	sun_cable	-	Sun Cobalt 'Alpine' cable detection
- - *	@ap: ATA port to do cable select
- - *
- - *	Cobalt CSB5 IDE hides the 40/80pin in the top two bits of the
- - *	subsystem ID the same as dell. We could use one function but we may
- - *	need to extend the Dell one in future
- - */
- -
- -static int sun_cable(struct ata_port *ap) {
+static int oem_cable(struct ata_port *ap)
+{
 	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 
 	if (pdev->subsystem_device & (1 << (ap->port_no + 14)))
@@ -90,49 +74,21 @@ static int sun_cable(struct ata_port *ap) {
 	return ATA_CBL_PATA40;
 }
 
- -/**
- - *	osb4_cable	-	OSB4 cable detect
- - *	@ap: ATA port to check
- - *
- - *	The OSB4 isn't UDMA66 capable so this is easy
- - */
- -
- -static int osb4_cable(struct ata_port *ap) {
- -	return ATA_CBL_PATA40;
- -}
- -
- -/**
- - *	csb_cable	-	CSB5/6 cable detect
- - *	@ap: ATA port to check
- - *
- - *	Serverworks default arrangement is to use the drive side detection
- - *	only.
- - */
- -
- -static int csb_cable(struct ata_port *ap) {
- -	return ATA_CBL_PATA_UNK;
- -}
- -
 struct sv_cable_table {
 	int device;
 	int subvendor;
 	int (*cable_detect)(struct ata_port *ap);
 };
 
- -/*
- - *	Note that we don't copy the old serverworks code because the old
- - *	code contains obvious mistakes
- - */
- -
 static struct sv_cable_table cable_detect[] = {
- -	{ PCI_DEVICE_ID_SERVERWORKS_CSB5IDE, PCI_VENDOR_ID_DELL, dell_cable },
- -	{ PCI_DEVICE_ID_SERVERWORKS_CSB6IDE, PCI_VENDOR_ID_DELL, dell_cable },
- -	{ PCI_DEVICE_ID_SERVERWORKS_CSB5IDE, PCI_VENDOR_ID_SUN,  sun_cable },
- -	{ PCI_DEVICE_ID_SERVERWORKS_OSB4IDE, PCI_ANY_ID, osb4_cable },
- -	{ PCI_DEVICE_ID_SERVERWORKS_CSB5IDE, PCI_ANY_ID, csb_cable },
- -	{ PCI_DEVICE_ID_SERVERWORKS_CSB6IDE, PCI_ANY_ID, csb_cable },
- -	{ PCI_DEVICE_ID_SERVERWORKS_CSB6IDE2, PCI_ANY_ID, csb_cable },
- -	{ PCI_DEVICE_ID_SERVERWORKS_HT1000IDE, PCI_ANY_ID, csb_cable },
+	{ PCI_DEVICE_ID_SERVERWORKS_CSB5IDE,   PCI_VENDOR_ID_DELL, oem_cable },
+	{ PCI_DEVICE_ID_SERVERWORKS_CSB6IDE,   PCI_VENDOR_ID_DELL, oem_cable },
+	{ PCI_DEVICE_ID_SERVERWORKS_CSB5IDE,   PCI_VENDOR_ID_SUN,  oem_cable },
+	{ PCI_DEVICE_ID_SERVERWORKS_OSB4IDE,   PCI_ANY_ID, ata_cable_40wire  },
+	{ PCI_DEVICE_ID_SERVERWORKS_CSB5IDE,   PCI_ANY_ID, ata_cable_unknown },
+	{ PCI_DEVICE_ID_SERVERWORKS_CSB6IDE,   PCI_ANY_ID, ata_cable_unknown },
+	{ PCI_DEVICE_ID_SERVERWORKS_CSB6IDE2,  PCI_ANY_ID, ata_cable_unknown },
+	{ PCI_DEVICE_ID_SERVERWORKS_HT1000IDE, PCI_ANY_ID, ata_cable_unknown },
 	{ }
 };
 
@@ -393,6 +349,31 @@ static void serverworks_fixup_ht1000(struct pci_dev *pdev)
 	pci_write_config_byte(pdev, 0x5A, btr);
 }
 
+static int serverworks_fixup(struct pci_dev *pdev)
+{
+	int rc = 0;
+
+	/* Force master latency timer to 64 PCI clocks */
+	pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 0x40);
+
+	switch (pdev->device) {
+	case PCI_DEVICE_ID_SERVERWORKS_OSB4IDE:
+		rc = serverworks_fixup_osb4(pdev);
+		break;
+	case PCI_DEVICE_ID_SERVERWORKS_CSB5IDE:
+		ata_pci_bmdma_clear_simplex(pdev);
+		/* fall through */
+	case PCI_DEVICE_ID_SERVERWORKS_CSB6IDE:
+	case PCI_DEVICE_ID_SERVERWORKS_CSB6IDE2:
+		rc = serverworks_fixup_csb(pdev);
+		break;
+	case PCI_DEVICE_ID_SERVERWORKS_HT1000IDE:
+		serverworks_fixup_ht1000(pdev);
+		break;
+	}
+
+	return rc;
+}
 
 static int serverworks_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 {
@@ -430,13 +411,12 @@ static int serverworks_init_one(struct pci_dev *pdev, const struct pci_device_id
 	if (rc)
 		return rc;
 
- -	/* Force master latency timer to 64 PCI clocks */
- -	pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 0x40);
+	rc = serverworks_fixup(pdev);
 
 	/* OSB4 : South Bridge and IDE */
 	if (pdev->device == PCI_DEVICE_ID_SERVERWORKS_OSB4IDE) {
 		/* Select non UDMA capable OSB4 if we can't do fixups */
- -		if ( serverworks_fixup_osb4(pdev) < 0)
+		if (rc < 0)
 			ppi[0] = &info[1];
 	}
 	/* setup CSB5/CSB6 : South Bridge and IDE option RAID */
@@ -446,19 +426,13 @@ static int serverworks_init_one(struct pci_dev *pdev, const struct pci_device_id
 
 		 /* If the returned btr is the newer revision then
 		    select the right info block */
- -		 if (serverworks_fixup_csb(pdev) == 3)
+		 if (rc == 3)
 		 	ppi[0] = &info[3];
 
 		/* Is this the 3rd channel CSB6 IDE ? */
 		if (pdev->device == PCI_DEVICE_ID_SERVERWORKS_CSB6IDE2)
 			ppi[1] = &ata_dummy_port_info;
 	}
- -	/* setup HT1000E */
- -	else if (pdev->device == PCI_DEVICE_ID_SERVERWORKS_HT1000IDE)
- -		serverworks_fixup_ht1000(pdev);
- -
- -	if (pdev->device == PCI_DEVICE_ID_SERVERWORKS_CSB5IDE)
- -		ata_pci_bmdma_clear_simplex(pdev);
 
 	return ata_pci_bmdma_init_one(pdev, ppi, &serverworks_sht, NULL, 0);
 }
@@ -473,24 +447,7 @@ static int serverworks_reinit_one(struct pci_dev *pdev)
 	if (rc)
 		return rc;
 
- -	/* Force master latency timer to 64 PCI clocks */
- -	pci_write_config_byte(pdev, PCI_LATENCY_TIMER, 0x40);
- -
- -	switch (pdev->device) {
- -		case PCI_DEVICE_ID_SERVERWORKS_OSB4IDE:
- -			serverworks_fixup_osb4(pdev);
- -			break;
- -		case PCI_DEVICE_ID_SERVERWORKS_CSB5IDE:
- -			ata_pci_bmdma_clear_simplex(pdev);
- -			/* fall through */
- -		case PCI_DEVICE_ID_SERVERWORKS_CSB6IDE:
- -		case PCI_DEVICE_ID_SERVERWORKS_CSB6IDE2:
- -			serverworks_fixup_csb(pdev);
- -			break;
- -		case PCI_DEVICE_ID_SERVERWORKS_HT1000IDE:
- -			serverworks_fixup_ht1000(pdev);
- -			break;
- -	}
+	(void)serverworks_fixup(pdev);
 
 	ata_host_resume(host);
 	return 0;
diff --git a/drivers/ata/pata_sil680.c b/drivers/ata/pata_sil680.c
index 31f759b..b92eacf 100644
- --- a/drivers/ata/pata_sil680.c
+++ b/drivers/ata/pata_sil680.c
@@ -38,11 +38,12 @@
 
 /**
  *	sil680_selreg		-	return register base
- - *	@hwif: interface
+ *	@ap: ATA interface
  *	@r: config offset
  *
- - *	Turn a config register offset into the right address in either
- - *	PCI space or MMIO space to access the control register in question
+ *	Turn a config register offset into the right address in PCI space
+ *	to access the control register in question.
+ *
  *	Thankfully this is a configuration operation so isn't performance
  *	criticial.
  */
@@ -56,12 +57,12 @@ static unsigned long sil680_selreg(struct ata_port *ap, int r)
 
 /**
  *	sil680_seldev		-	return register base
- - *	@hwif: interface
+ *	@ap: ATA interface
  *	@r: config offset
  *
- - *	Turn a config register offset into the right address in either
- - *	PCI space or MMIO space to access the control register in question
- - *	including accounting for the unit shift.
+ *	Turn a config register offset into the right address in PCI space
+ *	to access the control register in question including accounting for
+ *	the unit shift.
  */
 
 static unsigned long sil680_seldev(struct ata_port *ap, struct ata_device *adev, int r)
@@ -81,7 +82,8 @@ static unsigned long sil680_seldev(struct ata_port *ap, struct ata_device *adev,
  *	space for us.
  */
 
- -static int sil680_cable_detect(struct ata_port *ap) {
+static int sil680_cable_detect(struct ata_port *ap)
+{
 	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	unsigned long addr = sil680_selreg(ap, 0);
 	u8 ata66;
@@ -93,7 +95,7 @@ static int sil680_cable_detect(struct ata_port *ap) {
 }
 
 /**
- - *	sil680_set_piomode	-	set initial PIO mode data
+ *	sil680_set_piomode	-	set PIO mode data
  *	@ap: ATA interface
  *	@adev: ATA device
  *
@@ -104,8 +106,12 @@ static int sil680_cable_detect(struct ata_port *ap) {
 
 static void sil680_set_piomode(struct ata_port *ap, struct ata_device *adev)
 {
- -	static u16 speed_p[5] = { 0x328A, 0x2283, 0x1104, 0x10C3, 0x10C1 };
- -	static u16 speed_t[5] = { 0x328A, 0x2283, 0x1281, 0x10C3, 0x10C1 };
+	static const u16 speed_p[5] = {
+		0x328A, 0x2283, 0x1104, 0x10C3, 0x10C1
+	};
+	static const u16 speed_t[5] = {
+		0x328A, 0x2283, 0x1281, 0x10C3, 0x10C1
+	};
 
 	unsigned long tfaddr = sil680_selreg(ap, 0x02);
 	unsigned long addr = sil680_seldev(ap, adev, 0x04);
@@ -140,22 +146,23 @@ static void sil680_set_piomode(struct ata_port *ap, struct ata_device *adev)
 }
 
 /**
- - *	sil680_set_dmamode	-	set initial DMA mode data
+ *	sil680_set_dmamode	-	set DMA mode data
  *	@ap: ATA interface
  *	@adev: ATA device
  *
- - *	Program the MWDMA/UDMA modes for the sil680 k
- - *	chipset. The MWDMA mode values are pulled from a lookup table
+ *	Program the MWDMA/UDMA modes for the sil680 chipset.
+ *
+ *	The MWDMA mode values are pulled from a lookup table
  *	while the chipset uses mode number for UDMA.
  */
 
 static void sil680_set_dmamode(struct ata_port *ap, struct ata_device *adev)
 {
- -	static u8 ultra_table[2][7] = {
+	static const u8 ultra_table[2][7] = {
 		{ 0x0C, 0x07, 0x05, 0x04, 0x02, 0x01, 0xFF },	/* 100MHz */
 		{ 0x0F, 0x0B, 0x07, 0x05, 0x03, 0x02, 0x01 },	/* 133Mhz */
 	};
- -	static u16 dma_table[3] = { 0x2208, 0x10C2, 0x10C1 };
+	static const u16 dma_table[3] = { 0x2208, 0x10C2, 0x10C1 };
 
 	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	unsigned long ma = sil680_seldev(ap, adev, 0x08);
@@ -175,7 +182,7 @@ static void sil680_set_dmamode(struct ata_port *ap, struct ata_device *adev)
 	mode &= ~(0x03 << port_shift);
 
 	/* Extract scsc */
- -	scsc = (scsc & 0x30) ? 1: 0;
+	scsc = (scsc & 0x30) ? 1 : 0;
 
 	if (adev->dma_mode >= XFER_UDMA_0) {
 		multi = 0x10C1;
@@ -248,7 +255,7 @@ static u8 sil680_init_chip(struct pci_dev *pdev, int *try_mmio)
 {
 	u8 tmpbyte	= 0;
 
- -        /* FIXME: double check */
+	/* FIXME: double check */
 	pci_write_config_byte(pdev, PCI_CACHE_LINE_SIZE,
 			      pdev->revision ? 1 : 255);
 
@@ -266,22 +273,22 @@ static u8 sil680_init_chip(struct pci_dev *pdev, int *try_mmio)
 		*try_mmio = (tmpbyte & 1) || pci_resource_start(pdev, 5);
 #endif
 
- -	switch(tmpbyte & 0x30) {
- -		case 0x00:
- -			/* 133 clock attempt to force it on */
- -			pci_write_config_byte(pdev, 0x8A, tmpbyte|0x10);
- -			break;
- -		case 0x30:
- -			/* if clocking is disabled */
- -			/* 133 clock attempt to force it on */
- -			pci_write_config_byte(pdev, 0x8A, tmpbyte & ~0x20);
- -			break;
- -		case 0x10:
- -			/* 133 already */
- -			break;
- -		case 0x20:
- -			/* BIOS set PCI x2 clocking */
- -			break;
+	switch (tmpbyte & 0x30) {
+	case 0x00:
+		/* 133 clock attempt to force it on */
+		pci_write_config_byte(pdev, 0x8A, tmpbyte|0x10);
+		break;
+	case 0x30:
+		/* if clocking is disabled */
+		/* 133 clock attempt to force it on */
+		pci_write_config_byte(pdev, 0x8A, tmpbyte & ~0x20);
+		break;
+	case 0x10:
+		/* 133 already */
+		break;
+	case 0x20:
+		/* BIOS set PCI x2 clocking */
+		break;
 	}
 
 	pci_read_config_byte(pdev,   0x8A, &tmpbyte);
@@ -299,12 +306,19 @@ static u8 sil680_init_chip(struct pci_dev *pdev, int *try_mmio)
 	pci_write_config_dword(pdev, 0xB8, 0x43924392);
 	pci_write_config_dword(pdev, 0xBC, 0x40094009);
 
- -	switch(tmpbyte & 0x30) {
- -		case 0x00: printk(KERN_INFO "sil680: 100MHz clock.\n");break;
- -		case 0x10: printk(KERN_INFO "sil680: 133MHz clock.\n");break;
- -		case 0x20: printk(KERN_INFO "sil680: Using PCI clock.\n");break;
- -		/* This last case is _NOT_ ok */
- -		case 0x30: printk(KERN_ERR "sil680: Clock disabled ?\n");
+	switch (tmpbyte & 0x30) {
+	case 0x00:
+		printk(KERN_INFO "sil680: 100MHz clock.\n");
+		break;
+	case 0x10:
+		printk(KERN_INFO "sil680: 133MHz clock.\n");
+		break;
+	case 0x20:
+		printk(KERN_INFO "sil680: Using PCI clock.\n");
+		break;
+	/* This last case is _NOT_ ok */
+	case 0x30:
+		printk(KERN_ERR "sil680: Clock disabled ?\n");
 	}
 	return tmpbyte & 0x30;
 }
diff --git a/drivers/ata/pata_sis.c b/drivers/ata/pata_sis.c
index 533f2ae..b0edc7d 100644
- --- a/drivers/ata/pata_sis.c
+++ b/drivers/ata/pata_sis.c
@@ -55,7 +55,7 @@ static const struct sis_laptop sis_laptop[] = {
 	/* devid, subvendor, subdev */
 	{ 0x5513, 0x1043, 0x1107 },	/* ASUS A6K */
 	{ 0x5513, 0x1734, 0x105F },	/* FSC Amilo A1630 */
- -	{ 0x5513, 0x1071, 0x8640 },     /* EasyNote K5305 */
+	{ 0x5513, 0x1071, 0x8640 },	/* EasyNote K5305 */
 	/* end marker */
 	{ 0, }
 };
@@ -76,7 +76,7 @@ static int sis_short_ata40(struct pci_dev *dev)
 }
 
 /**
- - *	sis_old_port_base		-	return PCI configuration base for dev
+ *	sis_old_port_base - return PCI configuration base for dev
  *	@adev: device
  *
  *	Returns the base of the PCI configuration registers for this port
@@ -85,11 +85,34 @@ static int sis_short_ata40(struct pci_dev *dev)
 
 static int sis_old_port_base(struct ata_device *adev)
 {
- -	return  0x40 + (4 * adev->link->ap->port_no) +  (2 * adev->devno);
+	return 0x40 + (4 * adev->link->ap->port_no) + (2 * adev->devno);
 }
 
 /**
- - *	sis_133_cable_detect	-	check for 40/80 pin
+ *	sis_port_base - return PCI configuration base for dev
+ *	@adev: device
+ *
+ *	Returns the base of the PCI configuration registers for this port
+ *	number.
+ */
+
+static int sis_port_base(struct ata_device *adev)
+{
+	struct ata_port *ap = adev->link->ap;
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
+	int port = 0x40;
+	u32 reg54;
+
+	/* If bit 30 is set then the registers are mapped at 0x70 not 0x40 */
+	pci_read_config_dword(pdev, 0x54, &reg54);
+	if (reg54 & 0x40000000)
+		port = 0x70;
+
+	return port + (8 * ap->port_no) + (4 * adev->devno);
+}
+
+/**
+ *	sis_133_cable_detect - check for 40/80 pin
  *	@ap: Port
  *	@deadline: deadline jiffies for the operation
  *
@@ -110,7 +133,7 @@ static int sis_133_cable_detect(struct ata_port *ap)
 }
 
 /**
- - *	sis_66_cable_detect	-	check for 40/80 pin
+ *	sis_66_cable_detect - check for 40/80 pin
  *	@ap: Port
  *
  *	Perform cable detection on the UDMA66, UDMA100 and early UDMA133
@@ -132,7 +155,7 @@ static int sis_66_cable_detect(struct ata_port *ap)
 
 
 /**
- - *	sis_pre_reset		-	probe begin
+ *	sis_pre_reset - probe begin
  *	@link: ATA link
  *	@deadline: deadline jiffies for the operation
  *
@@ -160,7 +183,7 @@ static int sis_pre_reset(struct ata_link *link, unsigned long deadline)
 
 
 /**
- - *	sis_set_fifo	-	Set RWP fifo bits for this device
+ *	sis_set_fifo - Set RWP fifo bits for this device
  *	@ap: Port
  *	@adev: Device
  *
@@ -203,13 +226,13 @@ static void sis_set_fifo(struct ata_port *ap, struct ata_device *adev)
 
 static void sis_old_set_piomode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	int port = sis_old_port_base(adev);
 	u8 t1, t2;
 	int speed = adev->pio_mode - XFER_PIO_0;
 
- -	const u8 active[]   = { 0x00, 0x07, 0x04, 0x03, 0x01 };
- -	const u8 recovery[] = { 0x00, 0x06, 0x04, 0x03, 0x03 };
+	static const u8 active[]   = { 0x00, 0x07, 0x04, 0x03, 0x01 };
+	static const u8 recovery[] = { 0x00, 0x06, 0x04, 0x03, 0x03 };
 
 	sis_set_fifo(ap, adev);
 
@@ -240,11 +263,11 @@ static void sis_old_set_piomode (struct ata_port *ap, struct ata_device *adev)
 
 static void sis_100_set_piomode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	int port = sis_old_port_base(adev);
 	int speed = adev->pio_mode - XFER_PIO_0;
 
- -	const u8 actrec[] = { 0x00, 0x67, 0x44, 0x33, 0x31 };
+	static const u8 actrec[] = { 0x00, 0x67, 0x44, 0x33, 0x31 };
 
 	sis_set_fifo(ap, adev);
 
@@ -265,20 +288,19 @@ static void sis_100_set_piomode (struct ata_port *ap, struct ata_device *adev)
 
 static void sis_133_set_piomode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
- -	int port = 0x40;
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
+	int port;
 	u32 t1;
- -	u32 reg54;
 	int speed = adev->pio_mode - XFER_PIO_0;
 
- -	const u32 timing133[] = {
+	static const u32 timing133[] = {
 		0x28269000,	/* Recovery << 24 | Act << 16 | Ini << 12 */
 		0x0C266000,
 		0x04263000,
 		0x0C0A3000,
 		0x05093000
 	};
- -	const u32 timing100[] = {
+	static const u32 timing100[] = {
 		0x1E1C6000,	/* Recovery << 24 | Act << 16 | Ini << 12 */
 		0x091C4000,
 		0x031C2000,
@@ -288,12 +310,7 @@ static void sis_133_set_piomode (struct ata_port *ap, struct ata_device *adev)
 
 	sis_set_fifo(ap, adev);
 
- -	/* If bit 14 is set then the registers are mapped at 0x70 not 0x40 */
- -	pci_read_config_dword(pdev, 0x54, &reg54);
- -	if (reg54 & 0x40000000)
- -		port = 0x70;
- -	port += 8 * ap->port_no +  4 * adev->devno;
- -
+	port = sis_port_base(adev);
 	pci_read_config_dword(pdev, port, &t1);
 	t1 &= 0xC0C00FFF;	/* Mask out timing */
 
@@ -319,13 +336,13 @@ static void sis_133_set_piomode (struct ata_port *ap, struct ata_device *adev)
 
 static void sis_old_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	int speed = adev->dma_mode - XFER_MW_DMA_0;
 	int drive_pci = sis_old_port_base(adev);
 	u16 timing;
 
- -	const u16 mwdma_bits[] = { 0x008, 0x302, 0x301 };
- -	const u16 udma_bits[]  = { 0xE000, 0xC000, 0xA000 };
+	static const u16 mwdma_bits[] = { 0x008, 0x302, 0x301 };
+	static const u16 udma_bits[]  = { 0xE000, 0xC000, 0xA000 };
 
 	pci_read_config_word(pdev, drive_pci, &timing);
 
@@ -358,14 +375,14 @@ static void sis_old_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 
 static void sis_66_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	int speed = adev->dma_mode - XFER_MW_DMA_0;
 	int drive_pci = sis_old_port_base(adev);
 	u16 timing;
 
 	/* MWDMA 0-2 and UDMA 0-5 */
- -	const u16 mwdma_bits[] = { 0x008, 0x302, 0x301 };
- -	const u16 udma_bits[]  = { 0xF000, 0xD000, 0xB000, 0xA000, 0x9000, 0x8000 };
+	static const u16 mwdma_bits[] = { 0x008, 0x302, 0x301 };
+	static const u16 udma_bits[]  = { 0xF000, 0xD000, 0xB000, 0xA000, 0x9000, 0x8000 };
 
 	pci_read_config_word(pdev, drive_pci, &timing);
 
@@ -397,12 +414,12 @@ static void sis_66_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 
 static void sis_100_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	int speed = adev->dma_mode - XFER_MW_DMA_0;
 	int drive_pci = sis_old_port_base(adev);
 	u8 timing;
 
- -	const u8 udma_bits[]  = { 0x8B, 0x87, 0x85, 0x83, 0x82, 0x81};
+	static const u8 udma_bits[]  = { 0x8B, 0x87, 0x85, 0x83, 0x82, 0x81};
 
 	pci_read_config_byte(pdev, drive_pci + 1, &timing);
 
@@ -431,7 +448,7 @@ static void sis_100_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 
 static void sis_133_early_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
 	int speed = adev->dma_mode - XFER_MW_DMA_0;
 	int drive_pci = sis_old_port_base(adev);
 	u8 timing;
@@ -464,32 +481,34 @@ static void sis_133_early_set_dmamode (struct ata_port *ap, struct ata_device *a
 
 static void sis_133_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 {
- -	struct pci_dev *pdev	= to_pci_dev(ap->host->dev);
- -	int speed = adev->dma_mode - XFER_MW_DMA_0;
- -	int port = 0x40;
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
+	int port;
 	u32 t1;
- -	u32 reg54;
- -
- -	/* bits 4- cycle time 8 - cvs time */
- -	static const u32 timing_u100[] = { 0x6B0, 0x470, 0x350, 0x140, 0x120, 0x110, 0x000 };
- -	static const u32 timing_u133[] = { 0x9F0, 0x6A0, 0x470, 0x250, 0x230, 0x220, 0x210 };
- -
- -	/* If bit 14 is set then the registers are mapped at 0x70 not 0x40 */
- -	pci_read_config_dword(pdev, 0x54, &reg54);
- -	if (reg54 & 0x40000000)
- -		port = 0x70;
- -	port += (8 * ap->port_no) +  (4 * adev->devno);
 
+	port = sis_port_base(adev);
 	pci_read_config_dword(pdev, port, &t1);
 
 	if (adev->dma_mode < XFER_UDMA_0) {
+		/* Recovery << 24 | Act << 16 | Ini << 12, like PIO modes */
+		static const u32 timing_u100[] = { 0x19154000, 0x06072000, 0x04062000 };
+		static const u32 timing_u133[] = { 0x221C6000, 0x0C0A3000, 0x05093000 };
+		int speed = adev->dma_mode - XFER_MW_DMA_0;
+
+		t1 &= 0xC0C00FFF;
+		/* disable UDMA */
 		t1 &= ~0x00000004;
- -		/* FIXME: need data sheet to add MWDMA here. Also lacking on
- -		   ide/pci driver */
+		if (t1 & 0x08)
+			t1 |= timing_u133[speed];
+		else
+			t1 |= timing_u100[speed];
 	} else {
- -		speed = adev->dma_mode - XFER_UDMA_0;
- -		/* if & 8 no UDMA133 - need info for ... */
+		/* bits 4- cycle time 8 - cvs time */
+		static const u32 timing_u100[] = { 0x6B0, 0x470, 0x350, 0x140, 0x120, 0x110, 0x000 };
+		static const u32 timing_u133[] = { 0x9F0, 0x6A0, 0x470, 0x250, 0x230, 0x220, 0x210 };
+		int speed = adev->dma_mode - XFER_UDMA_0;
+
 		t1 &= ~0x00000FF0;
+		/* enable UDMA */
 		t1 |= 0x00000004;
 		if (t1 & 0x08)
 			t1 |= timing_u133[speed];
@@ -499,6 +518,27 @@ static void sis_133_set_dmamode (struct ata_port *ap, struct ata_device *adev)
 	pci_write_config_dword(pdev, port, t1);
 }
 
+/**
+ *	sis_133_mode_filter - mode selection filter
+ *	@adev: ATA device
+ *
+ *	Block UDMA6 on devices that do not support it.
+ */
+
+static unsigned long sis_133_mode_filter(struct ata_device *adev, unsigned long mask)
+{
+	struct ata_port *ap = adev->link->ap;
+	struct pci_dev *pdev = to_pci_dev(ap->host->dev);
+	int port = sis_port_base(adev);
+	u32 t1;
+
+	pci_read_config_dword(pdev, port, &t1);
+	/* if ATA133 is disabled, mask it out */
+	if (!(t1 & 0x08))
+		mask &= ~(0xC0 << ATA_SHIFT_UDMA);
+	return mask;
+}
+
 static struct scsi_host_template sis_sht = {
 	ATA_BMDMA_SHT(DRV_NAME),
 };
@@ -520,6 +560,7 @@ static struct ata_port_operations sis_133_ops = {
 	.set_piomode		= sis_133_set_piomode,
 	.set_dmamode		= sis_133_set_dmamode,
 	.cable_detect		= sis_133_cable_detect,
+	.mode_filter		= sis_133_mode_filter,
 };
 
 static struct ata_port_operations sis_133_early_ops = {
@@ -588,7 +629,7 @@ static const struct ata_port_info sis_info100_early = {
 static const struct ata_port_info sis_info133 = {
 	.flags		= ATA_FLAG_SLAVE_POSS,
 	.pio_mask	= ATA_PIO4,
- -	/* No MWDMA */
+	.mwdma_mask	= ATA_MWDMA2,
 	.udma_mask	= ATA_UDMA6,
 	.port_ops	= &sis_133_ops,
 };
@@ -669,7 +710,7 @@ static void sis_fixup(struct pci_dev *pdev, struct sis_chipset *sis)
  *	@pdev: PCI device to register
  *	@ent: Entry in sis_pci_tbl matching with @pdev
  *
- - *	Called from kernel PCI layer.  We probe for combined mode (sigh),
+ *	Called from kernel PCI layer. We probe for combined mode (sigh),
  *	and then hand over control to libata, for it to do the rest.
  *
  *	LOCKING:
@@ -769,17 +810,20 @@ static int sis_init_one (struct pci_dev *pdev, const struct pci_device_id *ent)
 
 		switch(trueid) {
 		case 0x5518:	/* SIS 962/963 */
+			dev_info(&pdev->dev,
+				 "SiS 962/963 MuTIOL IDE UDMA133 controller\n");
 			chipset = &sis133;
 			if ((idemisc & 0x40000000) == 0) {
 				pci_write_config_dword(pdev, 0x54, idemisc | 0x40000000);
- -				printk(KERN_INFO "SIS5513: Switching to 5513 register mapping\n");
+				dev_info(&pdev->dev,
+					 "Switching to 5513 register mapping\n");
 			}
 			break;
 		case 0x0180:	/* SIS 965/965L */
- -			chipset =  &sis133;
+			chipset = &sis133;
 			break;
 		case 0x1180:	/* SIS 966/966L */
- -			chipset =  &sis133;
+			chipset = &sis133;
 			break;
 		}
 	}
diff --git a/drivers/ata/pata_sl82c105.c b/drivers/ata/pata_sl82c105.c
index c06ce8c..24cf200 100644
- --- a/drivers/ata/pata_sl82c105.c
+++ b/drivers/ata/pata_sl82c105.c
@@ -1,6 +1,7 @@
 /*
  * pata_sl82c105.c 	- SL82C105 PATA for new ATA layer
  *			  (C) 2005 Red Hat Inc
+ *			  (C) 2011 Bartlomiej Zolnierkiewicz
  *
  * Based in part on linux/drivers/ide/pci/sl82c105.c
  * 		SL82C105/Winbond 553 IDE driver
@@ -289,6 +290,14 @@ static int sl82c105_bridge_revision(struct pci_dev *pdev)
 	return bridge->revision;
 }
 
+static void sl82c105_fixup(struct pci_dev *pdev)
+{
+	u32 val;
+
+	pci_read_config_dword(pdev, 0x40, &val);
+	val |= CTRL_P0EN | CTRL_P0F16 | CTRL_P1F16;
+	pci_write_config_dword(pdev, 0x40, val);
+}
 
 static int sl82c105_init_one(struct pci_dev *dev, const struct pci_device_id *id)
 {
@@ -306,7 +315,6 @@ static int sl82c105_init_one(struct pci_dev *dev, const struct pci_device_id *id
 	/* for now use only the first port */
 	const struct ata_port_info *ppi[] = { &info_early,
 					       NULL };
- -	u32 val;
 	int rev;
 	int rc;
 
@@ -325,13 +333,28 @@ static int sl82c105_init_one(struct pci_dev *dev, const struct pci_device_id *id
 	else
 		ppi[0] = &info_dma;
 
- -	pci_read_config_dword(dev, 0x40, &val);
- -	val |= CTRL_P0EN | CTRL_P0F16 | CTRL_P1F16;
- -	pci_write_config_dword(dev, 0x40, val);
+	sl82c105_fixup(dev);
 
 	return ata_pci_bmdma_init_one(dev, ppi, &sl82c105_sht, NULL, 0);
 }
 
+#ifdef CONFIG_PM
+static int sl82c105_reinit_one(struct pci_dev *pdev)
+{
+	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	int rc;
+
+	rc = ata_pci_device_do_resume(pdev);
+	if (rc)
+		return rc;
+
+	sl82c105_fixup(pdev);
+
+	ata_host_resume(host);
+	return 0;
+}
+#endif
+
 static const struct pci_device_id sl82c105[] = {
 	{ PCI_VDEVICE(WINBOND, PCI_DEVICE_ID_WINBOND_82C105), },
 
@@ -342,7 +365,11 @@ static struct pci_driver sl82c105_pci_driver = {
 	.name 		= DRV_NAME,
 	.id_table	= sl82c105,
 	.probe 		= sl82c105_init_one,
- -	.remove		= ata_pci_remove_one
+	.remove		= ata_pci_remove_one,
+#ifdef CONFIG_PM
+	.suspend	= ata_pci_device_suspend,
+	.resume		= sl82c105_reinit_one,
+#endif
 };
 
 static int __init sl82c105_init(void)
diff --git a/drivers/ata/pata_via.c b/drivers/ata/pata_via.c
index 8e9f504..255f336 100644
- --- a/drivers/ata/pata_via.c
+++ b/drivers/ata/pata_via.c
@@ -509,6 +509,27 @@ static void via_config_fifo(struct pci_dev *pdev, unsigned int flags)
 	}
 }
 
+static void via_fixup(struct pci_dev *pdev, const struct via_isa_bridge *config)
+{
+	u32 timing;
+
+	/* Initialise the FIFO for the enabled channels. */
+	via_config_fifo(pdev, config->flags);
+
+	if (config->udma_mask == ATA_UDMA4) {
+		/* The 66 MHz devices require we enable the clock */
+		pci_read_config_dword(pdev, 0x50, &timing);
+		timing |= 0x80008;
+		pci_write_config_dword(pdev, 0x50, timing);
+	}
+	if (config->flags & VIA_BAD_CLK66) {
+		/* Disable the 66MHz clock on problem devices */
+		pci_read_config_dword(pdev, 0x50, &timing);
+		timing &= ~0x80008;
+		pci_write_config_dword(pdev, 0x50, timing);
+	}
+}
+
 /**
  *	via_init_one		-	discovery callback
  *	@pdev: PCI device
@@ -570,7 +591,6 @@ static int via_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 	struct pci_dev *isa;
 	const struct via_isa_bridge *config;
 	u8 enable;
- -	u32 timing;
 	unsigned long flags = id->driver_data;
 	int rc;
 
@@ -609,9 +629,6 @@ static int via_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 			return -ENODEV;
 	}
 
- -	/* Initialise the FIFO for the enabled channels. */
- -	via_config_fifo(pdev, config->flags);
- -
 	/* Clock set up */
 	switch (config->udma_mask) {
 	case 0x00:
@@ -637,12 +654,7 @@ static int via_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 		return -ENODEV;
  	}
 
- -	if (config->flags & VIA_BAD_CLK66) {
- -		/* Disable the 66MHz clock on problem devices */
- -		pci_read_config_dword(pdev, 0x50, &timing);
- -		timing &= ~0x80008;
- -		pci_write_config_dword(pdev, 0x50, timing);
- -	}
+	via_fixup(pdev, config);
 
 	/* We have established the device type, now fire it up */
 	return ata_pci_bmdma_init_one(pdev, ppi, &via_sht, (void *)config, 0);
@@ -661,29 +673,14 @@ static int via_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
 
 static int via_reinit_one(struct pci_dev *pdev)
 {
- -	u32 timing;
 	struct ata_host *host = dev_get_drvdata(&pdev->dev);
- -	const struct via_isa_bridge *config = host->private_data;
 	int rc;
 
 	rc = ata_pci_device_do_resume(pdev);
 	if (rc)
 		return rc;
 
- -	via_config_fifo(pdev, config->flags);
- -
- -	if (config->udma_mask == ATA_UDMA4) {
- -		/* The 66 MHz devices require we enable the clock */
- -		pci_read_config_dword(pdev, 0x50, &timing);
- -		timing |= 0x80008;
- -		pci_write_config_dword(pdev, 0x50, timing);
- -	}
- -	if (config->flags & VIA_BAD_CLK66) {
- -		/* Disable the 66MHz clock on problem devices */
- -		pci_read_config_dword(pdev, 0x50, &timing);
- -		timing &= ~0x80008;
- -		pci_write_config_dword(pdev, 0x50, timing);
- -	}
+	via_fixup(pdev, host->private_data);
 
 	ata_host_resume(host);
 	return 0;
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 4b6b209..0b8b8b4 100644
- --- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -4087,8 +4087,11 @@ static int mv_platform_probe(struct platform_device *pdev)
 	dev_info(&pdev->dev, "slots %u ports %d\n",
 		 (unsigned)MV_MAX_Q_DEPTH, host->n_ports);
 
- -	return ata_host_activate(host, platform_get_irq(pdev, 0), mv_interrupt,
- -				 IRQF_SHARED, &mv6_sht);
+	rc = ata_host_activate(host, platform_get_irq(pdev, 0), mv_interrupt,
+			       IRQF_SHARED, &mv6_sht);
+	if (!rc)
+		return 0;
+
 err:
 #if defined(CONFIG_HAVE_CLK)
 	if (!IS_ERR(hpriv->clk)) {
@@ -4110,8 +4113,7 @@ err:
  */
 static int __devexit mv_platform_remove(struct platform_device *pdev)
 {
- -	struct device *dev = &pdev->dev;
- -	struct ata_host *host = dev_get_drvdata(dev);
+	struct ata_host *host = platform_get_drvdata(pdev);
 #if defined(CONFIG_HAVE_CLK)
 	struct mv_host_priv *hpriv = host->private_data;
 #endif
@@ -4129,7 +4131,7 @@ static int __devexit mv_platform_remove(struct platform_device *pdev)
 #ifdef CONFIG_PM
 static int mv_platform_suspend(struct platform_device *pdev, pm_message_t state)
 {
- -	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	struct ata_host *host = platform_get_drvdata(pdev);
 	if (host)
 		return ata_host_suspend(host, state);
 	else
@@ -4138,7 +4140,7 @@ static int mv_platform_suspend(struct platform_device *pdev, pm_message_t state)
 
 static int mv_platform_resume(struct platform_device *pdev)
 {
- -	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	struct ata_host *host = platform_get_drvdata(pdev);
 	int ret;
 
 	if (host) {
@@ -4353,7 +4355,7 @@ static int mv_pci_init_one(struct pci_dev *pdev,
 #ifdef CONFIG_PM
 static int mv_pci_device_resume(struct pci_dev *pdev)
 {
- -	struct ata_host *host = dev_get_drvdata(&pdev->dev);
+	struct ata_host *host = pci_get_drvdata(pdev);
 	int rc;
 
 	rc = ata_pci_device_do_resume(pdev);
diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
index 55470f3..1e91406 100644
- --- a/drivers/ata/sata_sil24.c
+++ b/drivers/ata/sata_sil24.c
@@ -268,7 +268,7 @@ union sil24_cmd_block {
 	struct sil24_atapi_block atapi;
 };
 
- -static struct sil24_cerr_info {
+static const struct sil24_cerr_info {
 	unsigned int err_mask, action;
 	const char *desc;
 } sil24_cerr_db[] = {
@@ -1019,7 +1019,7 @@ static void sil24_error_intr(struct ata_port *ap)
 
 	/* deal with command error */
 	if (irq_stat & PORT_IRQ_ERROR) {
- -		struct sil24_cerr_info *ci = NULL;
+		const struct sil24_cerr_info *ci = NULL;
 		unsigned int err_mask = 0, action = 0;
 		u32 context, cerr;
 		int pmp;
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIVAwUBTqhrjCWzCDIBeCsvAQJ1Ww/+P9ZBEwQNcucxdt1MkF3h4Nd9jy+jf3Ha
8Pj0l2mvzaP4veGfM8kq51ptKI5rTpeuiRNvZw7S/uTR3WqNXcUpJrVex7anvv/s
tLU17JxfQbsB9Sbo3929X0xbeYXizR+wJRPYr/4sy8ukkGsEBbSEqmiSAiL+vZ68
e/RRBSi8t1InVRzlQ6riCDRSC0unYfBNKN8BjY6TiFziBFeZjsXpTo0oSVq4bq12
eAz134scTZBDmnbXYxq8FvOip2Oe2rUPF8HFkCkAQgnlXAfUZXacyNqG3lcOytib
AAwnPgbiemVjEwDhG6Lf2B8b3Q3EMFU79x+h9BXmibSkKo0lgyOSRrvyyWF2KJ1c
ZWZAZE3f4JR6jMtrMZjyaawLnvTZqrRLFJkONEQLFF7d02oDnIPq8Ie+5CpZ8HOh
KwtWVRFr+7L2iY/dUWo/WG9MtLrgxglA08PvojZPPkEXyX8A2W6EzQRc5g8aBJca
jLrjXEJ8l9C5BtQl+WQQwsLOLJumKBC7WN3YDyTapddVQEY4CKXIG/nSQlOz84y7
yKXpye4T+/IR7NmxllFcYI9mPpf5jvX642ZsZJXOfKE9UV1p8OqYq775CDzguPU4
Njx8LGb4PlK6wubDgMdX8svJH5rWa77sEyFkNK6tKdrxsAvYgXohnZr+AI18tJtu
lPmrRSR6ESA=
=a7fj
-----END PGP SIGNATURE-----

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-26 20:22 [git patches] libata updates, GPG signed (but see admin notes) Jeff Garzik
@ 2011-10-29 19:13 ` Valdis.Kletnieks
  2011-10-29 19:27   ` Jeff Garzik
  2011-10-30 10:05 ` James Bottomley
  1 sibling, 1 reply; 101+ messages in thread
From: Valdis.Kletnieks @ 2011-10-29 19:13 UTC (permalink / raw)
  To: Jeff Garzik; +Cc: Andrew Morton, Linus Torvalds, linux-ide, LKML

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

On Wed, 26 Oct 2011 16:22:35 EDT, Jeff Garzik said:

>    For this reason, I am tempted to modify my new, GPG-signed pull request
>    template as follows, for subsequent pull requests:
> 
> 	---<begin GPG signed text>---
> 
>    	pull req description
> 
> 	"Please pull from $branch\n$url $branch"
> 
> 	"Top of tree is $sha1_commit"
> 
> 	---<end GPG signed text>---
> 
> 	diffstat
> 	diff

Note that this can be problematic as well - there's probably a lot of tools out there
that will be less than gracious when presented with a GPG-signed text with non-signed
trailing text.  Maybe GPG-sign from "pull request" to "diffstat" and make the diff an
attachment? (That would also allow attaching a gzip'ed diff, which would save vger's
bandwidth :)

[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-29 19:13 ` Valdis.Kletnieks
@ 2011-10-29 19:27   ` Jeff Garzik
  0 siblings, 0 replies; 101+ messages in thread
From: Jeff Garzik @ 2011-10-29 19:27 UTC (permalink / raw)
  To: Valdis.Kletnieks; +Cc: Andrew Morton, Linus Torvalds, linux-ide, LKML

On 10/29/2011 03:13 PM, Valdis.Kletnieks@vt.edu wrote:
> On Wed, 26 Oct 2011 16:22:35 EDT, Jeff Garzik said:
>
>>     For this reason, I am tempted to modify my new, GPG-signed pull request
>>     template as follows, for subsequent pull requests:
>>
>> 	---<begin GPG signed text>---
>>
>>     	pull req description
>>
>> 	"Please pull from $branch\n$url $branch"
>>
>> 	"Top of tree is $sha1_commit"
>>
>> 	---<end GPG signed text>---
>>
>> 	diffstat
>> 	diff
>
> Note that this can be problematic as well - there's probably a lot of tools out there
> that will be less than gracious when presented with a GPG-signed text with non-signed
> trailing text.  Maybe GPG-sign from "pull request" to "diffstat" and make the diff an
> attachment? (That would also allow attaching a gzip'ed diff, which would save vger's
> bandwidth :)

I'm not sure I buy that.  A GPG-signed email, something that's been 
around for more than decade, transits the system with unsigned email 
headers.  GPG content is commonly delivered buried inside a larger 
container.

	Jeff




^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-26 20:22 [git patches] libata updates, GPG signed (but see admin notes) Jeff Garzik
  2011-10-29 19:13 ` Valdis.Kletnieks
@ 2011-10-30 10:05 ` James Bottomley
  2011-10-30 17:12   ` Linus Torvalds
  1 sibling, 1 reply; 101+ messages in thread
From: James Bottomley @ 2011-10-30 10:05 UTC (permalink / raw)
  To: Jeff Garzik; +Cc: Andrew Morton, Linus Torvalds, linux-ide, LKML

On Wed, 2011-10-26 at 16:22 -0400, Jeff Garzik wrote:
> 2) The GPG wrapping causes all '^-' lines to become '^- -'.  This does NOT
>    reflect the true 'git diff' output, of course.
> 
>    For this reason, I am tempted to modify my new, GPG-signed pull request
>    template as follows, for subsequent pull requests:
> 
> 	---<begin GPG signed text>---
> 
>    	pull req description
> 
> 	"Please pull from $branch\n$url $branch"
> 
> 	"Top of tree is $sha1_commit"
> 
> 	---<end GPG signed text>---
> 
> 	diffstat
> 	diff
> 
>    That ensures that the critical part -- sha1 commit for top of tree --
>    is GPG signed, while the diff will be outside the signed area and therefore
>    not mangled by wrapping.

You can fix this by using mime and detached signatures as well but I
wouldn't worry too much about it.  What emerged at KS is that Linus uses
gmail and gmail has no integration with pgp, thus pgp signing of pull
requests is superfluous since Linus won't add the steps of saving the
message to a text file and manually running pgp over it to verify
because of the huge elongation in workflow this causes especially during
a merge window.

Ted Ts'o recommended using signed tags, but they're also a manual check.
I think the ultimate consensus will be that we'll have to wait until git
itself has pgp key handling built in (which the git people are looking
at) before we move to using the web of trust in pull requests.

James



^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-30 10:05 ` James Bottomley
@ 2011-10-30 17:12   ` Linus Torvalds
  2011-10-31  8:19     ` James Bottomley
                       ` (3 more replies)
  0 siblings, 4 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-10-30 17:12 UTC (permalink / raw)
  To: James Bottomley; +Cc: Jeff Garzik, Andrew Morton, linux-ide, LKML

On Sun, Oct 30, 2011 at 3:05 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> You can fix this by using mime and detached signatures as well but I
> wouldn't worry too much about it.  What emerged at KS is that Linus uses
> gmail and gmail has no integration with pgp, thus pgp signing of pull
> requests is superfluous since Linus won't add the steps of saving the
> message to a text file and manually running pgp over it to verify
> because of the huge elongation in workflow this causes especially during
> a merge window.

Actually, I have been running "gpg --verify" on the email that use the
standard PGP encapsulation (ie the kind that Jeff used, that has
"-----BEGIN PGP SIGNED MESSAGE-----" in it.

It's the rfc3516-type email (aka protocol="application/pgp-encrypted")
type that I can't even verify, because that's not something that gpg
knows inherently how to check: you have to decode the message the
right way and know what the rules are, and I have no intention of
trying to figure it out. I have yet to find any usable tool that is
able to check it, and I'm certainly not going to waste time trying to
parse the rfc and write my own.

That said, even the "BEGIN PGP SIGNED MESSAGE" things are a massive
pain in the butt. We need to automate this some sane way, both for the
sender and for the recipient.

                                      Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-30 17:12   ` Linus Torvalds
@ 2011-10-31  8:19     ` James Bottomley
  2011-10-31 15:53       ` Linus Torvalds
  2011-10-31  8:40     ` Ingo Molnar
                       ` (2 subsequent siblings)
  3 siblings, 1 reply; 101+ messages in thread
From: James Bottomley @ 2011-10-31  8:19 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Jeff Garzik, Andrew Morton, linux-ide, LKML

On Sun, 2011-10-30 at 10:12 -0700, Linus Torvalds wrote:
> On Sun, Oct 30, 2011 at 3:05 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> >
> > You can fix this by using mime and detached signatures as well but I
> > wouldn't worry too much about it.  What emerged at KS is that Linus uses
> > gmail and gmail has no integration with pgp, thus pgp signing of pull
> > requests is superfluous since Linus won't add the steps of saving the
> > message to a text file and manually running pgp over it to verify
> > because of the huge elongation in workflow this causes especially during
> > a merge window.
> 
> Actually, I have been running "gpg --verify" on the email that use the
> standard PGP encapsulation (ie the kind that Jeff used, that has
> "-----BEGIN PGP SIGNED MESSAGE-----" in it.
> 
> It's the rfc3516-type email (aka protocol="application/pgp-encrypted")
> type that I can't even verify, because that's not something that gpg
> knows inherently how to check: you have to decode the message the
> right way and know what the rules are, and I have no intention of
> trying to figure it out. I have yet to find any usable tool that is
> able to check it, and I'm certainly not going to waste time trying to
> parse the rfc and write my own.

OK, so I get clearly no partial PGP messages and no using rfc3516
formatted pgp mail.

> That said, even the "BEGIN PGP SIGNED MESSAGE" things are a massive
> pain in the butt. We need to automate this some sane way, both for the
> sender and for the recipient.

But this doesn't help with what practise you want us to follow.  Do you
want us to send full signed email using pgp encapsulation for pull
requests in spite of the mangling it does to attached patches and the
amount of extra pain it causes you?  Or is relying on looking at the
received headers and always using public email lists to detect spoofing
by revocation OK until we find a mechanism for integrating pgp into git?

James



^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-30 17:12   ` Linus Torvalds
  2011-10-31  8:19     ` James Bottomley
@ 2011-10-31  8:40     ` Ingo Molnar
  2011-10-31 21:46     ` H. Peter Anvin
       [not found]     ` <20111031084048.GA11807__21610.4542407722$1320051469$gmane$org@elte.hu>
  3 siblings, 0 replies; 101+ messages in thread
From: Ingo Molnar @ 2011-10-31  8:40 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML


* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> That said, even the "BEGIN PGP SIGNED MESSAGE" things are a massive 
> pain in the butt. We need to automate this some sane way, both for 
> the sender and for the recipient.

The most practical form would be if Git supported such oneliner pull 
requests:

 git pull git://foo.com bar.branch                           \
  --pull-sha1 0acf00014bcfd71090c3b0d43c98e970108064e4       \
  --gpg-by: "Ingo Molnar <mingo@kernel.org>"                 \
  --gpg-sig: 8a6f134afd1d212fe21345

maintainers could just paste them into a shell and it would abort if 
it's not trusted. The maintainer verifies the visible, 'Ingo Molnar' 
bit. The 8a6f134afd1d212fe21345 is a signed-by-Ingo-Molnar version of 
this content:

    git://foo.com bar.branch 0acf00014bcfd71090c3b0d43c98e970108064e4

And Git would verify that what ends up being pulled is indeed 
0acf00014bcfd and also verifies that it was signed by me.

[ If we are extra diligent/paranoid then beyond the sha1 we might 
  even GPG sign the shortlog, or even the full raw log of all commits 
  leading to the sha1: this introduces some Git shortlog and patch 
  formatting version dependency though.

  Git could also double check foo.com's DNS coherency, or check it 
  against a known-trusted whitelist of domain names specified in the 
  maintainer's .gitconfig, as an extra layer. ]

Doing it in this form would remove all the mail formatting madness - 
one could paste such a pull request into a shell straight away, from 
HTML email, from text email, from MIME email, etc.

In fact i would trust such a Git based solution far more than any 
opaque, invisible tool that claims to have checked a signature with 
cooperation of my mail client (ha!).

The only somewhat non-obvious bit is that Git should be *very* 
careful about its key ID and signature parsing strategy, to protect 
against social engineering attacks.

For example neither this:

  --gpg-by: "Ingo Molnar <mingo@kernal.org>"

nor this:

  --pgp-by: "Ingo Molnar <mingo@kernel.org>"

malicious pull request should slip through in any fashion:

 - Git should only use keys that are in your ring of trust - not pull 
   keys from the public keyring automatically and just check 
   coherency of the pull request or such. [I'm sure people will be 
   tempted to have such a feature - but that temptation should be 
   resisted.]

 - Git should abort the moment it sees an unknown option

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31  8:19     ` James Bottomley
@ 2011-10-31 15:53       ` Linus Torvalds
  2011-10-31 18:23         ` Junio C Hamano
  0 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 15:53 UTC (permalink / raw)
  To: James Bottomley, Junio C Hamano
  Cc: Jeff Garzik, Andrew Morton, linux-ide, LKML

[-- Attachment #1: Type: text/plain, Size: 2874 bytes --]

On Mon, Oct 31, 2011 at 1:19 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
>> That said, even the "BEGIN PGP SIGNED MESSAGE" things are a massive
>> pain in the butt. We need to automate this some sane way, both for the
>> sender and for the recipient.
>
> But this doesn't help with what practise you want us to follow.  Do you
> want us to send full signed email using pgp encapsulation for pull
> requests in spite of the mangling it does to attached patches and the
> amount of extra pain it causes you?

No. I don't want the *whole* email signed, because that is quite
inconvenient: it means that I can't just cut-and-paste some signature,
I have to save the email and verify it etc etc.

So my preferred thing would literally be to make the signed part as
small as possible with no odd characters or whitespace (top commit and
probably repository name), so that I can cut-and-paste it and just
have a terminal window open with "gpg --verify + paste + ^D" and I'm
done.

For the people who use "git request-pull", I'm attaching a trivial
patch to make it add this kind of signature if you give it the "-s"
flag. It basically just adds a hunk like the appended crazy example to
the pull request, and it's small enough and simple enough that it
makes verification simple too with just the above kind of trivial
cut-and-paste thing.

(Junio cc'd, I think he had something more complicated in mind)

Now, admittedly it would be *even nicer* if this gpg-signed block was
instead uploaded as a signed tag automatically, and "git pull" would
notice such a signed tag (tagname the same as the branch name + date
or something) and would download and verify the tag as I pull. Then I
wouldn't even need to actually do the cut-and-paste at all. But this
is the *really* simple approach that gets up 95% of the way there.

And the attached patch is so trivial that if you aren't actually using
"git request-pull" but instead have some home-cooked script to do the
same, then you can just look at this patch and trivially change your
script to do something very similar.

                 Linus

[ Example gpg-signed small block that the attached patch adds to the
pull request: ]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Commit be3fa9125e708348c7baf04ebe9507a72a9d1800
from git.kernel.org/pub/git
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iQEcBAEBAgAGBQJOrsILAAoJEHm+PkMAQRiGxZcH/31e0RrBitXUPKxHJajD58yh
SIEe/7i6E2RUSFva3KybEuFslcR8p8DYzDQTPLejStvnkO8v0lXu9s9R53tvjLMF
aaQXLOgrOC2RqvzP4F27O972h32YpLBkwIdWQGAhYcUOdKYDZ9RfgEgtdJwSYuL+
oJ7TjLrtkcILaFmr9nYZC+0Fh7z+84R8kR53v0iBHJQOFfssuMjUWCoj9aEY12t+
pywXuVk2FsuYvhniCAcyU6Y1K9aXaf6w5iOY2hx/ysXtUBnv92F7lcathxQkvgjO
fA7/TXEcummOv5KQFc9vckd5Z1gN2ync5jhfnmlT2uiobE6mNdCbOVlCOpsKQkU=
=l5PG
-----END PGP SIGNATURE-----

[-- Attachment #2: patch.diff --]
[-- Type: text/x-patch, Size: 756 bytes --]

 git-request-pull.sh |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/git-request-pull.sh b/git-request-pull.sh
index fc080cc5e45d..22b51930959f 100755
--- a/git-request-pull.sh
+++ b/git-request-pull.sh
@@ -20,11 +20,14 @@ GIT_PAGER=
 export GIT_PAGER
 
 patch=
+sign=
 while	case "$#" in 0) break ;; esac
 do
 	case "$1" in
 	-p)
 		patch=-p ;;
+	-s)
+		sign=-s ;;
 	--)
 		shift; break ;;
 	-*)
@@ -73,6 +76,12 @@ are available in the git repository at:' $baserev &&
 echo "  $url $branch" &&
 echo &&
 
+if test -n "$sign"
+then
+	printf "Commit $headrev\nfrom $url\n" | gpg --clearsign
+	echo
+fi &&
+
 git shortlog ^$baserev $headrev &&
 git diff -M --stat --summary $patch $merge_base..$headrev || exit
 exit $status

^ permalink raw reply related	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 15:53       ` Linus Torvalds
@ 2011-10-31 18:23         ` Junio C Hamano
  2011-10-31 22:18           ` Linus Torvalds
  0 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-10-31 18:23 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> For the people who use "git request-pull", I'm attaching a trivial
> patch to make it add this kind of signature if you give it the "-s"
> flag. It basically just adds a hunk like the appended crazy example to
> the pull request, and it's small enough and simple enough that it
> makes verification simple too with just the above kind of trivial
> cut-and-paste thing.
>
> (Junio cc'd, I think he had something more complicated in mind)

You have misread me this time.

I think the minimalistic "paste this line to your 'git pull' command line
and expect to get history leading to this commit" like you did in your
patch would be the solution that is the least painful and still useful,
which is an important criterion for wide adoption.

> Now, admittedly it would be *even nicer* if this gpg-signed block was
> instead uploaded as a signed tag automatically, and "git pull" would
> notice such a signed tag (tagname the same as the branch name + date
> or something) and would download and verify the tag as I pull. Then I
> wouldn't even need to actually do the cut-and-paste at all. But this
> is the *really* simple approach that gets up 95% of the way there.

I however have a small trouble with "lieutenants use signed tags in order
to prove who they are to Linus", depending on the details.

It certainly lets you run "git tag --verify" after you pulled and will
give you assurance that you pulled the right thing from the right person,
but what do you plan to do to the tag from your lieutenants after you
fetched and verified?  I count 379 merges by you between 3.0 (2011-07-21)
and 3.1 (2011-10-24), which would mean you would see 4-5 tags per day on
average.  Will these tags be pushed out to your public history?

On one hand, we (not just you but the consumers of "Linus kernel") can
consider these tags are of ephemeral nature. Once they are used for _you_
to verify the authenticity, they are not needed anymore. The consumers of
"Linus kernel" by definition trusts what you publish, so as long as they
have a way to verify the tip commit you push out, they _should_ be happy.
If you take this stance, you would not push these tags out so that you do
not have to contaminate the tags namespace with them, and you might even
choose to discard them once you pulled and verified the lieutenants' tips
to avoid contamination of your own refs namespace.

On the other hand, the consumers of "Linus kernel" may want to say that
they trust your tree and your tags because they can verify them with your
GPG signature, but also they can independently verify the lieutenants'
trees you pulled from are genuine. Keeping signed tags and publishing them
is one way to make it possible, but 400 extra tags in 3 months feels like
an approach with too much downside (i.e. noise) for that benefit.

On Git mailing list, we have been toying with a couple of ideas. The
simplest one (cooking in next) is to allow committers to add gpg signature
in an additional header of the commit objects. "git show" and friends are
taught how to verify these signatures when asked.

This might have a potential downside on the lieutenants' workflow; after
integrating the work by their sub-lieutenants and by themselves, they
would test and review the result to convince themselves that it is worth
asking you to pull, and then they have to either

    (1) "commit --amend --gpg-signature" the tip; or

    (2) "commit --allow-empty --gpg-signature" to add an empty commit
        whose sole purpose is to hold the signature (and avoid amending
        the tip)

before pushing it out, asking you to pull.

An alternative we have discussed was to store gpg signature for the commit
("push certificate") somewhere in notes tree and push that out, certifying
that the commit indeed came from the pusher, but that would:

 (1) require upstreams to fetch (and possibly suffer from merge conflicts
     in notes tree) push certificate whenever they pull from their
     lieutenants; and

 (2) require downstreams to also fetch the notes tree for "push
     certificates" (especially when the central repository is shared among
     multiple people) before adding their own signature and then push it
     back (and possiblly suffer from "non-fast-forward" in notes tree).

both of which are downsides coming from "notes" being not a very good
match for what these signatures are trying to achieve.

Namely, the current "notes" mechanism is designed to keep track of history
of changes made to notes attached to commits, but for the signature
application, we do not care about the order that signatures came to two
separate commits. "Non-fast-forward" conflicts while pushing, or having to
fetch and merge before adding one's own signature, are unwanted burden
imposed only by choosing to use "notes" for storing and conveying the
signature.

Also the "notes" approach would end up mixing "push certificates" for
different branches (this won't be an issue in your repository where there
is only one branch) into a single "notes" tree. We would want to use
something that behaves more like the "auto-following" semantics of tag
objects. You would want to fetch only signatures that are attached to the
commits you are fetching. Use of signed tags, or commit objects that can
be signed in-place, have this property, but storing signature in notes
tree does not give it to us.

I think further discussions on this should continue on the git mailing
list.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-30 17:12   ` Linus Torvalds
  2011-10-31  8:19     ` James Bottomley
  2011-10-31  8:40     ` Ingo Molnar
@ 2011-10-31 21:46     ` H. Peter Anvin
  2011-10-31 22:21       ` Linus Torvalds
       [not found]     ` <20111031084048.GA11807__21610.4542407722$1320051469$gmane$org@elte.hu>
  3 siblings, 1 reply; 101+ messages in thread
From: H. Peter Anvin @ 2011-10-31 21:46 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On 10/30/2011 10:12 AM, Linus Torvalds wrote:
> 
> Actually, I have been running "gpg --verify" on the email that use the
> standard PGP encapsulation (ie the kind that Jeff used, that has
> "-----BEGIN PGP SIGNED MESSAGE-----" in it.
> 
> It's the rfc3516-type email (aka protocol="application/pgp-encrypted")
> type that I can't even verify, because that's not something that gpg
> knows inherently how to check: you have to decode the message the
> right way and know what the rules are, and I have no intention of
> trying to figure it out. I have yet to find any usable tool that is
> able to check it, and I'm certainly not going to waste time trying to
> parse the rfc and write my own.
> 

What client are you using (or are you using the gmail web interface?)

	-hpa


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
       [not found]     ` <20111031084048.GA11807__21610.4542407722$1320051469$gmane$org@elte.hu>
@ 2011-10-31 22:03       ` Junio C Hamano
  0 siblings, 0 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-10-31 22:03 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: git, Linus Torvalds, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Ingo Molnar <mingo@elte.hu> writes:

> * Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
>> That said, even the "BEGIN PGP SIGNED MESSAGE" things are a massive 
>> pain in the butt. We need to automate this some sane way, both for 
>> the sender and for the recipient.
>
> The most practical form would be if Git supported such oneliner pull 
> requests:
>
>  git pull git://foo.com bar.branch                           \
>   --pull-sha1 0acf00014bcfd71090c3b0d43c98e970108064e4       \
>   --gpg-by: "Ingo Molnar <mingo@kernel.org>"                 \
>   --gpg-sig: 8a6f134afd1d212fe21345
>
> maintainers could just paste them into a shell and it would abort if 
> it's not trusted. The maintainer verifies the visible, 'Ingo Molnar' 
> bit. The 8a6f134afd1d212fe21345 is a signed-by-Ingo-Molnar version of 
> this content:
>
>     git://foo.com bar.branch 0acf00014bcfd71090c3b0d43c98e970108064e4

As a command line syntax, I think the new "--flag"s should all come
before non flag options to the "pull" subcommand, i.e.

    git pull --sha1 0acf00014bcfd71090c3b0d43c98e970108064e4 \
    	     --gpg-by "Ingo Molnar <mingo@kernel.org>" \
             git://foo.com bar.branch

I do not understand what you meant by that "8a6f13...". When I run

    $ echo "git://foo.com bar.branch 0acf00014bcfd71090c3b0d43c98e970108064e4" |
      gpg -sa

I would get about 20 lines of solid gibberish, nothing close to that
clean and concise 20-or-so character sequence.

In any case, I do not think that "this site, that branch" information is
essential for the purpose of validation. I think I saw Linus responding to
a pull request saying "Your pull request says master but I found nothing
there; I assume you meant for-linus branch" or something similar, and as
long as that matches the expectation of the contributor, especially if you
specify "I want you to get _this_ commit" in your request-pull message, it
should not matter how/where Linus gets the history leading to that commit.

As "git-pull" is still a scripted Porcelain, interested people should be
able to experiment this idea by doing something like this:

 1. The requestor signs the tip commit to be fetched with the version of
    git from the "next" branch, i.e. "git commit -S", and pushes it to his
    publishing location;

 2. Around line 207, "git pull" spawns "git fetch", stops if dry-run. At
    that point, you can:

    - parse FETCH_HEAD and verify the SHA-1 matches what you got from the
      command line;

    - run "git show -s --show-signature FETCH_HEAD" (again, use the
      version of git from the "next" branch) to let GPG parse the
      signature.

    and stop if either test fails.


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 18:23         ` Junio C Hamano
@ 2011-10-31 22:18           ` Linus Torvalds
  2011-10-31 22:20             ` H. Peter Anvin
  2011-11-01 19:47             ` Junio C Hamano
  0 siblings, 2 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 22:18 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Mon, Oct 31, 2011 at 11:23 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> It certainly lets you run "git tag --verify" after you pulled and will
> give you assurance that you pulled the right thing from the right person,
> but what do you plan to do to the tag from your lieutenants after you
> fetched and verified?  I count 379 merges by you between 3.0 (2011-07-21)
> and 3.1 (2011-10-24), which would mean you would see 4-5 tags per day on
> average.  Will these tags be pushed out to your public history?

No, you misunderstand.

I can do that kind of "crazy manual check of a tag" today. And it's
too painful to be useful in the long run (or even the short run - I'd
much prefer the pgp signature in the email which is easier to check
and more visible anyway). Fetching a tag by name and saving it as a
tag is indeed pointless.

But what would be nice is that "git pull" would fetch the tag (based
on name) *automatically*, and not actually create a tag in my
repository at all. Instead, if would use the tag to check the
signature, and - if we do this right - also use the tag contents to
populate the merge commit message.

In other words, no actual tag would ever be left around as a turd, it
would simply be used as an automatic communication channel between the
"git push -s" of the submitter and my subsequent "git pull". Neither
side would have to do anything special, and the tag would never show
up in any relevant tree (it could even be in a totally separate
namespace like "refs/pullmarker/<branchname>" or something).

                                 Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:18           ` Linus Torvalds
@ 2011-10-31 22:20             ` H. Peter Anvin
  2011-10-31 22:30               ` Linus Torvalds
  2011-10-31 22:33               ` Jiri Kosina
  2011-11-01 19:47             ` Junio C Hamano
  1 sibling, 2 replies; 101+ messages in thread
From: H. Peter Anvin @ 2011-10-31 22:20 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On 10/31/2011 03:18 PM, Linus Torvalds wrote:
> On Mon, Oct 31, 2011 at 11:23 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> It certainly lets you run "git tag --verify" after you pulled and will
>> give you assurance that you pulled the right thing from the right person,
>> but what do you plan to do to the tag from your lieutenants after you
>> fetched and verified?  I count 379 merges by you between 3.0 (2011-07-21)
>> and 3.1 (2011-10-24), which would mean you would see 4-5 tags per day on
>> average.  Will these tags be pushed out to your public history?
> 
> No, you misunderstand.
> 
> I can do that kind of "crazy manual check of a tag" today. And it's
> too painful to be useful in the long run (or even the short run - I'd
> much prefer the pgp signature in the email which is easier to check
> and more visible anyway). Fetching a tag by name and saving it as a
> tag is indeed pointless.
> 
> But what would be nice is that "git pull" would fetch the tag (based
> on name) *automatically*, and not actually create a tag in my
> repository at all. Instead, if would use the tag to check the
> signature, and - if we do this right - also use the tag contents to
> populate the merge commit message.
> 
> In other words, no actual tag would ever be left around as a turd, it
> would simply be used as an automatic communication channel between the
> "git push -s" of the submitter and my subsequent "git pull". Neither
> side would have to do anything special, and the tag would never show
> up in any relevant tree (it could even be in a totally separate
> namespace like "refs/pullmarker/<branchname>" or something).
> 

Perhaps we should introduce the notion of a "private tag" or something
along those lines?  (I guess that would still have to be possible to
push it, but not pull it by default...)

	-hpa


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 21:46     ` H. Peter Anvin
@ 2011-10-31 22:21       ` Linus Torvalds
  2011-10-31 22:23         ` H. Peter Anvin
  0 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 22:21 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Mon, Oct 31, 2011 at 2:46 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>
> What client are you using (or are you using the gmail web interface?)

I'm using the gmail web interface, but I can save the email
individually for gpg checking. It's just that when I do that, no sane
tool will actually check it (ie there is no equivalent of "gpg
--verify" for that mbox) that I've found.

And it's kind of sad when the gpg inative nterfaces are better than
some other tool. gpg is famous for having a *horrible* user interface.

                        Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:21       ` Linus Torvalds
@ 2011-10-31 22:23         ` H. Peter Anvin
  2011-10-31 22:34           ` Linus Torvalds
  0 siblings, 1 reply; 101+ messages in thread
From: H. Peter Anvin @ 2011-10-31 22:23 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On 10/31/2011 03:21 PM, Linus Torvalds wrote:
> On Mon, Oct 31, 2011 at 2:46 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>>
>> What client are you using (or are you using the gmail web interface?)
> 
> I'm using the gmail web interface, but I can save the email
> individually for gpg checking. It's just that when I do that, no sane
> tool will actually check it (ie there is no equivalent of "gpg
> --verify" for that mbox) that I've found.
> 
> And it's kind of sad when the gpg inative nterfaces are better than
> some other tool. gpg is famous for having a *horrible* user interface.
> 

I use enigmail which is a Thunderbird plugin; it works quite well, but,
well, it requires Thunderbird.  I think many GUI MUAs have similar
plugins, but text MTAs might be worse, I don't know.

	-hpa


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:20             ` H. Peter Anvin
@ 2011-10-31 22:30               ` Linus Torvalds
  2011-10-31 22:33                 ` H. Peter Anvin
  2011-10-31 22:33               ` Jiri Kosina
  1 sibling, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 22:30 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, Oct 31, 2011 at 3:20 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>
> Perhaps we should introduce the notion of a "private tag" or something
> along those lines?  (I guess that would still have to be possible to
> push it, but not pull it by default...)

All tags are private by default.

We actually *only* fetch tags if somebody explicitly asks for them
(--tags), or when fetching from a named remote (and even then it will
only fetch tags that point to objects you fetched by default iirc -
you have to mark the remote specially to get *all* tags).

But if you do the normal "git pull git://git.kernel.org/name/of/repo"
- which is how things happen as a result of a pull request - you won't
get tags at all - you have to ask for them by name or use "--tags" to
get them all.

                   Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:20             ` H. Peter Anvin
  2011-10-31 22:30               ` Linus Torvalds
@ 2011-10-31 22:33               ` Jiri Kosina
  1 sibling, 0 replies; 101+ messages in thread
From: Jiri Kosina @ 2011-10-31 22:33 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Linus Torvalds, Junio C Hamano, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Mon, 31 Oct 2011, H. Peter Anvin wrote:

> Perhaps we should introduce the notion of a "private tag" or something
> along those lines?  (I guess that would still have to be possible to
> push it, but not pull it by default...)

That's exactly what git does now, right? (unless you pull from a very 
specific remote).

-- 
Jiri Kosina
SUSE Labs


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:30               ` Linus Torvalds
@ 2011-10-31 22:33                 ` H. Peter Anvin
  2011-10-31 22:38                   ` Linus Torvalds
  2011-10-31 22:44                   ` Junio C Hamano
  0 siblings, 2 replies; 101+ messages in thread
From: H. Peter Anvin @ 2011-10-31 22:33 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On 10/31/2011 03:30 PM, Linus Torvalds wrote:
> 
> But if you do the normal "git pull git://git.kernel.org/name/of/repo"
> - which is how things happen as a result of a pull request - you won't
> get tags at all - you have to ask for them by name or use "--tags" to
> get them all.
> 

Didn't realize that... I guess I'm too used to named remotes.

If so, just using a tag should be fine, no?

	-hpa


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:23         ` H. Peter Anvin
@ 2011-10-31 22:34           ` Linus Torvalds
  2011-11-01  2:17             ` david
  0 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 22:34 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Mon, Oct 31, 2011 at 3:23 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>
> I use enigmail which is a Thunderbird plugin; it works quite well, but,
> well, it requires Thunderbird.  I think many GUI MUAs have similar
> plugins, but text MTAs might be worse, I don't know.

I have never met a GUI MUA I like before google webmail. And the
advantage of google webmail isn't so much the GUI MUA, as the fact
that it doesn't suck like IMAP, and does the automatic folders and
good handling of archives right.

The reason I switched away from pine was that pine is fine (apart from
the archival thing) if you have all your mail local. But "local mail"
is painful when traveling, or when trying to interact with mail using
a mobile phone etc.

And IMAP mail in turn is totally useless both when traveling *and*
when at home.

So no, thunderbird isn't an option. A command line interface that just
verifies the gpg signature is what I'd like to see.

                    Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:33                 ` H. Peter Anvin
@ 2011-10-31 22:38                   ` Linus Torvalds
  2011-10-31 22:51                     ` Junio C Hamano
  2011-10-31 22:44                   ` Junio C Hamano
  1 sibling, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 22:38 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, Oct 31, 2011 at 3:33 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>
> Didn't realize that... I guess I'm too used to named remotes.
>
> If so, just using a tag should be fine, no?

Yes, that's what I think. But the argument for using a separate
namespace is that
 (a) you never get confused
 (b) it would make it easier to make the 1:1 relationship between
branch names and these "pull request signature tags" without limiting
the naming of *normal* tags in any way
 (c) they do have separate lifetimes from "real" tags.

But seriously, I don't care about the *implementation* all that much.
If people want to use the crazy git "notes" capability, you can do
that too, although quite frankly, I don't see the point. What actually
matters is that "git push" and "git pull" would JustWork(tm), and
check the signature if one exists, without having to cut-and-paste
data that simply shouldn't be visible to the user.

I abhor the interface Ingo suggested, for example. Why would we have
stupid command line options that we should cut-and-paste? Automation
is for computers, not for people.

                          Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:33                 ` H. Peter Anvin
  2011-10-31 22:38                   ` Linus Torvalds
@ 2011-10-31 22:44                   ` Junio C Hamano
  2011-10-31 22:47                     ` H. Peter Anvin
                                       ` (3 more replies)
  1 sibling, 4 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-10-31 22:44 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Linus Torvalds, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

"H. Peter Anvin" <hpa@zytor.com> writes:

> On 10/31/2011 03:30 PM, Linus Torvalds wrote:
>> 
>> But if you do the normal "git pull git://git.kernel.org/name/of/repo"
>> - which is how things happen as a result of a pull request - you won't
>> get tags at all - you have to ask for them by name or use "--tags" to
>> get them all.
>> 
>
> Didn't realize that... I guess I'm too used to named remotes.
>
> If so, just using a tag should be fine, no?

So nobody is worried about this (quoting from my earlier message)?

   On the other hand, the consumers of "Linus kernel" may want to say that
   they trust your tree and your tags because they can verify them with your
   GPG signature, but also they can independently verify the lieutenants'
   trees you pulled from are genuine.

A signed emphemeral tag is usable as means to verify authenticity in a
hop-by-hop fashion, but that does not leave a permanent trail that can be
used for auditing.


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:44                   ` Junio C Hamano
@ 2011-10-31 22:47                     ` H. Peter Anvin
  2011-10-31 22:49                     ` Ted Ts'o
                                       ` (2 subsequent siblings)
  3 siblings, 0 replies; 101+ messages in thread
From: H. Peter Anvin @ 2011-10-31 22:47 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Linus Torvalds, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On 10/31/2011 03:44 PM, Junio C Hamano wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
> 
>> On 10/31/2011 03:30 PM, Linus Torvalds wrote:
>>>
>>> But if you do the normal "git pull git://git.kernel.org/name/of/repo"
>>> - which is how things happen as a result of a pull request - you won't
>>> get tags at all - you have to ask for them by name or use "--tags" to
>>> get them all.
>>>
>>
>> Didn't realize that... I guess I'm too used to named remotes.
>>
>> If so, just using a tag should be fine, no?
> 
> So nobody is worried about this (quoting from my earlier message)?
> 
>    On the other hand, the consumers of "Linus kernel" may want to say that
>    they trust your tree and your tags because they can verify them with your
>    GPG signature, but also they can independently verify the lieutenants'
>    trees you pulled from are genuine.
> 
> A signed emphemeral tag is usable as means to verify authenticity in a
> hop-by-hop fashion, but that does not leave a permanent trail that can be
> used for auditing.
> 

Well, the permanent trail is in the maintainer's tree, but that might
still be suboptimal.  The problem with Linus pulling those tags I assume
that it makes the tree too noisy?

	-hpa


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:44                   ` Junio C Hamano
  2011-10-31 22:47                     ` H. Peter Anvin
@ 2011-10-31 22:49                     ` Ted Ts'o
  2011-10-31 22:51                       ` H. Peter Anvin
  2011-10-31 22:52                     ` Linus Torvalds
  2011-10-31 23:55                     ` Jeff Garzik
  3 siblings, 1 reply; 101+ messages in thread
From: Ted Ts'o @ 2011-10-31 22:49 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: H. Peter Anvin, Linus Torvalds, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Mon, Oct 31, 2011 at 03:44:25PM -0700, Junio C Hamano wrote:
> So nobody is worried about this (quoting from my earlier message)?
> 
>    On the other hand, the consumers of "Linus kernel" may want to say that
>    they trust your tree and your tags because they can verify them with your
>    GPG signature, but also they can independently verify the lieutenants'
>    trees you pulled from are genuine.
> 
> A signed emphemeral tag is usable as means to verify authenticity in a
> hop-by-hop fashion, but that does not leave a permanent trail that can be
> used for auditing.

Oh, there are definitely people who worry about this.  They tend to be
security poeple, though, so the goal is how do we leave the permanent
trail in a way that doesn't generate too much noise or otherwise makes
life difficult for developers who don't care.

							- Ted

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:38                   ` Linus Torvalds
@ 2011-10-31 22:51                     ` Junio C Hamano
  2011-10-31 22:56                       ` Linus Torvalds
  2011-10-31 23:09                       ` Junio C Hamano
  0 siblings, 2 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-10-31 22:51 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: H. Peter Anvin, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> But seriously, I don't care about the *implementation* all that much.
> If people want to use the crazy git "notes" capability, you can do
> that too, although quite frankly, I don't see the point.

As I already said, I do not think notes is a good match as a tool to do
this.

> matters is that "git push" and "git pull" would JustWork(tm), and
> check the signature if one exists, without having to cut-and-paste
> data that simply shouldn't be visible to the user.
>
> I abhor the interface Ingo suggested, for example....

Some cut-and-paste (or piping the e-mail to a command) would be necessary
evil, though, as you would have GPG keys from more than one trusted person
in your keyring, and when you are responding to a pull-request from person
A, finding a valid commit signed by person B should not be a success, but
at least should raise a warning.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:49                     ` Ted Ts'o
@ 2011-10-31 22:51                       ` H. Peter Anvin
  0 siblings, 0 replies; 101+ messages in thread
From: H. Peter Anvin @ 2011-10-31 22:51 UTC (permalink / raw)
  To: Ted Ts'o, Junio C Hamano, Linus Torvalds, git,
	James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On 10/31/2011 03:49 PM, Ted Ts'o wrote:
> On Mon, Oct 31, 2011 at 03:44:25PM -0700, Junio C Hamano wrote:
>> So nobody is worried about this (quoting from my earlier message)?
>>
>>    On the other hand, the consumers of "Linus kernel" may want to say that
>>    they trust your tree and your tags because they can verify them with your
>>    GPG signature, but also they can independently verify the lieutenants'
>>    trees you pulled from are genuine.
>>
>> A signed emphemeral tag is usable as means to verify authenticity in a
>> hop-by-hop fashion, but that does not leave a permanent trail that can be
>> used for auditing.
> 
> Oh, there are definitely people who worry about this.  They tend to be
> security poeple, though, so the goal is how do we leave the permanent
> trail in a way that doesn't generate too much noise or otherwise makes
> life difficult for developers who don't care.
> 

Could we introduce a tag namespace that doesn't show up in gitweb by
default, and perhaps doesn't resolve in abbreviated form?

This is basically what Linus suggested, as far as I understand:
something like refs/pulls/hpa/tip-123-456 which is otherwise a normal
tag object?

	-hpa



^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:44                   ` Junio C Hamano
  2011-10-31 22:47                     ` H. Peter Anvin
  2011-10-31 22:49                     ` Ted Ts'o
@ 2011-10-31 22:52                     ` Linus Torvalds
  2011-10-31 22:54                       ` H. Peter Anvin
  2011-11-01  5:39                       ` James Bottomley
  2011-10-31 23:55                     ` Jeff Garzik
  3 siblings, 2 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 22:52 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: H. Peter Anvin, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, Oct 31, 2011 at 3:44 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> So nobody is worried about this (quoting from my earlier message)?

No, because you haven't been reading what we write.

The tag is useless.

The information *in* the tag is not. But it shouldn't be saved in the
tag (or note, or whatever). Because that's just an annoying place for
it to be, with no upside.

Save it in the commit we generate. BAM! Useful, readable, permanent,
and independently verifiable.

And the advantage is that we can make that same mechanism add
"maintainer notes" to the merge message too. Right now some
maintainers write good notes about what the merge will bring in, but
they are basically lost, because git is so good at merging and doesn't
even stop to ask people to edit the merge message.

                    Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:52                     ` Linus Torvalds
@ 2011-10-31 22:54                       ` H. Peter Anvin
  2011-10-31 23:03                         ` Linus Torvalds
  2011-11-01  5:39                       ` James Bottomley
  1 sibling, 1 reply; 101+ messages in thread
From: H. Peter Anvin @ 2011-10-31 22:54 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On 10/31/2011 03:52 PM, Linus Torvalds wrote:
> 
> Save it in the commit we generate. BAM! Useful, readable, permanent,
> and independently verifiable.
> 

Note: this means creating a commit even for a fast-forward merge.  Not
that there is any technical problem with that, of course.

	-hpa


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:51                     ` Junio C Hamano
@ 2011-10-31 22:56                       ` Linus Torvalds
  2011-11-02  9:11                         ` Ingo Molnar
  2011-10-31 23:09                       ` Junio C Hamano
  1 sibling, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 22:56 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: H. Peter Anvin, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, Oct 31, 2011 at 3:51 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Some cut-and-paste (or piping the e-mail to a command) would be necessary
> evil, though, as you would have GPG keys from more than one trusted person
> in your keyring, and when you are responding to a pull-request from person
> A, finding a valid commit signed by person B should not be a success, but
> at least should raise a warning.

Why?

The signer of the message needs to be printed out *anyway*. I can
match that up with the pull request, the same way I already match up
diffstat information.

So any extra cut-and-paste is (a) stupid, (b) unnecessary and (c) annoying.

It's also "bad user interface". The whole point is that we should make
the user interface *good*. Which means that the pushing side should
only need to add a "-s" to ask for signing, have to type his
passphrase (and even that would go away when using gpg-agent or
something), and perhaps a message (which would not be about the
signing, but about something that could be added to the merge commit.

And the receiving side would just do the "git pull" and automatically
just get notified that "Yes, this push has been signed by key Xyz
Abcdef"

                     Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:54                       ` H. Peter Anvin
@ 2011-10-31 23:03                         ` Linus Torvalds
  0 siblings, 0 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-10-31 23:03 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, Oct 31, 2011 at 3:54 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 10/31/2011 03:52 PM, Linus Torvalds wrote:
>>
>> Save it in the commit we generate. BAM! Useful, readable, permanent,
>> and independently verifiable.
>>
>
> Note: this means creating a commit even for a fast-forward merge.  Not
> that there is any technical problem with that, of course.

Well, only for the signed case, but yes. And for that case it's likely
a good thing.

In fact, even without signing, some projects always use --no-ff,
because they want the merge messages with the nice summary in them.
I've played around with it too, but haven't generally found it to be
worth it, and tend to think that it aggrandizes the merger too much.

It generates nice merge summaries, and it can look nice, but if the
*only* upside is the merge summary I think it's borderline worth it.
But with a signature, it would suddenly actually contain real
information, and I think that changes the equation.

                           Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:51                     ` Junio C Hamano
  2011-10-31 22:56                       ` Linus Torvalds
@ 2011-10-31 23:09                       ` Junio C Hamano
  1 sibling, 0 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-10-31 23:09 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: H. Peter Anvin, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Junio C Hamano <gitster@pobox.com> writes:

> Linus Torvalds <torvalds@linux-foundation.org> writes:
> ...
> As I already said, I do not think notes is a good match as a tool to do
> this.
>
>> matters is that "git push" and "git pull" would JustWork(tm), and
>> check the signature if one exists, without having to cut-and-paste
>> data that simply shouldn't be visible to the user.
>>
>> I abhor the interface Ingo suggested, for example....
>
> Some cut-and-paste (or piping the e-mail to a command) would be necessary
> evil, though, as you would have GPG keys from more than one trusted person
> in your keyring, and when you are responding to a pull-request from person
> A, finding a valid commit signed by person B should not be a success, but
> at least should raise a warning.

So here is a quick hack that does not involve cut-and-paste (it depends on
the signed-commit topic in 'next').

 $ git pull --require-signature

would trigger signature verification and stops you after fetching but
before merging.

 git-pull.sh |   25 ++++++++++++++++++++++++-
 1 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/git-pull.sh b/git-pull.sh
index 9868a0b..f3b4c93 100755
--- a/git-pull.sh
+++ b/git-pull.sh
@@ -39,7 +39,7 @@ test -z "$(git ls-files -u)" || die_conflict
 test -f "$GIT_DIR/MERGE_HEAD" && die_merge
 
 strategy_args= diffstat= no_commit= squash= no_ff= ff_only=
-log_arg= verbosity= progress= recurse_submodules=
+log_arg= verbosity= progress= recurse_submodules= must_be_signed=
 merge_args=
 curr_branch=$(git symbolic-ref -q HEAD)
 curr_branch_short="${curr_branch#refs/heads/}"
@@ -60,6 +60,8 @@ do
 		diffstat=--no-stat ;;
 	--stat|--summary)
 		diffstat=--stat ;;
+	--require-signature)
+		must_be_signed=yes ;;
 	--log|--no-log)
 		log_arg=$1 ;;
 	--no-c|--no-co|--no-com|--no-comm|--no-commi|--no-commit)
@@ -208,6 +210,27 @@ orig_head=$(git rev-parse -q --verify HEAD)
 git fetch $verbosity $progress $dry_run $recurse_submodules --update-head-ok "$@" || exit 1
 test -z "$dry_run" || exit 0
 
+if test -n "$must_be_signed"
+then
+	signature=$(git show -s --format='%G?' FETCH_HEAD)
+	case "$signature" in
+	G)
+		case "$verbosity" in
+		*' '-v*)
+			git show -s --show-signature FETCH_HEAD ;;
+		esac
+		;;
+	B)
+		echo >&2 "Bad signature on the tip commit"
+		exit 1
+		;;
+	*)
+		echo >&2 "Tip commit must be signed"
+		exit 1
+		;;
+	fi
+fi
+
 curr_head=$(git rev-parse -q --verify HEAD)
 if test -n "$orig_head" && test "$curr_head" != "$orig_head"
 then

^ permalink raw reply related	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:44                   ` Junio C Hamano
                                       ` (2 preceding siblings ...)
  2011-10-31 22:52                     ` Linus Torvalds
@ 2011-10-31 23:55                     ` Jeff Garzik
  2011-11-01  0:42                       ` H. Peter Anvin
  3 siblings, 1 reply; 101+ messages in thread
From: Jeff Garzik @ 2011-10-31 23:55 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: H. Peter Anvin, Linus Torvalds, git, James Bottomley,
	Andrew Morton, linux-ide, LKML

On 10/31/2011 06:44 PM, Junio C Hamano wrote:
> "H. Peter Anvin"<hpa@zytor.com>  writes:
>
>> On 10/31/2011 03:30 PM, Linus Torvalds wrote:
>>>
>>> But if you do the normal "git pull git://git.kernel.org/name/of/repo"
>>> - which is how things happen as a result of a pull request - you won't
>>> get tags at all - you have to ask for them by name or use "--tags" to
>>> get them all.
>>>
>>
>> Didn't realize that... I guess I'm too used to named remotes.
>>
>> If so, just using a tag should be fine, no?
>
> So nobody is worried about this (quoting from my earlier message)?
>
>     On the other hand, the consumers of "Linus kernel" may want to say that
>     they trust your tree and your tags because they can verify them with your
>     GPG signature, but also they can independently verify the lieutenants'
>     trees you pulled from are genuine.
>
> A signed emphemeral tag is usable as means to verify authenticity in a
> hop-by-hop fashion, but that does not leave a permanent trail that can be
> used for auditing.

The main worry is Linus ($human_who_pulls) gets 
cryptographically-verified data at the time he pulls.  Once Linus 
republishes his tree (git push), there will be few, if any, wanting to 
verify Jeff Garzik's signature.

So no, I don't see that as a _driving_ need in the kernel's case.

And IMO the kernel will be a mix of signed and unsigned content for a 
while, possibly forever.


And Linus wrote:
> [ Example gpg-signed small block that the attached patch adds to the
> pull request: ]
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Commit be3fa9125e708348c7baf04ebe9507a72a9d1800
> from git.kernel.org/pub/git
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.18 (GNU/Linux)
>
> iQEcBAEBAgAGBQJOrsILAAoJEHm+PkMAQRiGxZcH/31e0RrBitXUPKxHJajD58yh
> SIEe/7i6E2RUSFva3KybEuFslcR8p8DYzDQTPLejStvnkO8v0lXu9s9R53tvjLMF
> aaQXLOgrOC2RqvzP4F27O972h32YpLBkwIdWQGAhYcUOdKYDZ9RfgEgtdJwSYuL+
> oJ7TjLrtkcILaFmr9nYZC+0Fh7z+84R8kR53v0iBHJQOFfssuMjUWCoj9aEY12t+
> pywXuVk2FsuYvhniCAcyU6Y1K9aXaf6w5iOY2hx/ysXtUBnv92F7lcathxQkvgjO
> fA7/TXEcummOv5KQFc9vckd5Z1gN2ync5jhfnmlT2uiobE6mNdCbOVlCOpsKQkU=
> =l5PG
> -----END PGP SIGNATURE-----


This is my preference for kernel pull requests at the moment.  That has 
one advantage over Junio's "git pull --require-signature" and signed 
commits, notably, the URL is signed.

But in general signed commits would be nice, too.  pull-generated merge 
requests would need to be signed, potentially introducing an additional 
interactive step (GPG passphrase request) into an automated process.

	Jeff



^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 23:55                     ` Jeff Garzik
@ 2011-11-01  0:42                       ` H. Peter Anvin
  0 siblings, 0 replies; 101+ messages in thread
From: H. Peter Anvin @ 2011-11-01  0:42 UTC (permalink / raw)
  To: Jeff Garzik
  Cc: Junio C Hamano, Linus Torvalds, git, James Bottomley,
	Andrew Morton, linux-ide, LKML

> 
> The main worry is Linus ($human_who_pulls) gets
> cryptographically-verified data at the time he pulls.  Once Linus
> republishes his tree (git push), there will be few, if any, wanting to
> verify Jeff Garzik's signature.
> 
> So no, I don't see that as a _driving_ need in the kernel's case.
> 
> And IMO the kernel will be a mix of signed and unsigned content for a
> while, possibly forever.
> 

I think the desire is to be able to deconstruct things if things were to
go wrong.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:34           ` Linus Torvalds
@ 2011-11-01  2:17             ` david
  2011-11-01  3:25               ` H. Peter Anvin
  2011-11-01  3:31               ` Linus Torvalds
  0 siblings, 2 replies; 101+ messages in thread
From: david @ 2011-11-01  2:17 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: H. Peter Anvin, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1299 bytes --]

On Mon, 31 Oct 2011, Linus Torvalds wrote:

> On Mon, Oct 31, 2011 at 3:23 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>>
>> I use enigmail which is a Thunderbird plugin; it works quite well, but,
>> well, it requires Thunderbird.  I think many GUI MUAs have similar
>> plugins, but text MTAs might be worse, I don't know.
>
> I have never met a GUI MUA I like before google webmail. And the
> advantage of google webmail isn't so much the GUI MUA, as the fact
> that it doesn't suck like IMAP, and does the automatic folders and
> good handling of archives right.
>
> The reason I switched away from pine was that pine is fine (apart from
> the archival thing) if you have all your mail local. But "local mail"
> is painful when traveling, or when trying to interact with mail using
> a mobile phone etc.
>
> And IMAP mail in turn is totally useless both when traveling *and*
> when at home.

what IMAP servers have you used? I find that with a good IMAP server I can 
get away with _very_ little processing power in the client and get good 
performance (I still am using pine, but with a cyrus IMAP server)

there are a bunch of IMAP servers out there that are horrible (and even 
more IMAP clients that are really doing POP and storing everything 
locally, just using the IMAP protocol)

David Lang

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  2:17             ` david
@ 2011-11-01  3:25               ` H. Peter Anvin
  2011-11-01  3:42                 ` Linus Torvalds
  2011-11-01  5:03                 ` david
  2011-11-01  3:31               ` Linus Torvalds
  1 sibling, 2 replies; 101+ messages in thread
From: H. Peter Anvin @ 2011-11-01  3:25 UTC (permalink / raw)
  To: david
  Cc: Linus Torvalds, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On 10/31/2011 07:17 PM, david@lang.hm wrote:
> 
> what IMAP servers have you used? I find that with a good IMAP server I
> can get away with _very_ little processing power in the client and get
> good performance (I still am using pine, but with a cyrus IMAP server)
> 
> there are a bunch of IMAP servers out there that are horrible (and even
> more IMAP clients that are really doing POP and storing everything
> locally, just using the IMAP protocol)
> 

Last I checked if you used pine against an IMAP server it did absolutely
no caching, in which case you're of course screwed.

I use IMAP on my phone, and with Thunderpants... I mean Thunderbird...
and it works quite nicely.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  2:17             ` david
  2011-11-01  3:25               ` H. Peter Anvin
@ 2011-11-01  3:31               ` Linus Torvalds
  2011-11-01  4:58                 ` david
  1 sibling, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-01  3:31 UTC (permalink / raw)
  To: david
  Cc: H. Peter Anvin, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, Oct 31, 2011 at 7:17 PM,  <david@lang.hm> wrote:
>
> what IMAP servers have you used? I find that with a good IMAP server I can
> get away with _very_ little processing power in the client and get good
> performance (I still am using pine, but with a cyrus IMAP server)

I've used pine (and then alpine) to cyrus servers too. That was what
LF used to have on the server side. It absolutely sucked.

That combination is supposed to do server-side searching etc, but you
couldn't tell from the performance. It was disgusting.

I'm sure it works much better with a fast local network, but quite
frankly, that obviates the need for IMAP in the first place. If you
have your mail locally, there are better models than IMAP for handling
it.

So the only situation I've found IMAP reasonable has been at corporate
settings where you're not talking DSL or cable modem speeds, but use
IMAP as a way to avoid NFS-mounting the mail spool, which is even
worse. But actually working over slowish internet connections? No
thank you.

                         Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  3:25               ` H. Peter Anvin
@ 2011-11-01  3:42                 ` Linus Torvalds
  2011-11-01  4:25                   ` hpanvin@gmail.com
  2011-11-01 17:06                   ` Tony Luck
  2011-11-01  5:03                 ` david
  1 sibling, 2 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-01  3:42 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: david, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Mon, Oct 31, 2011 at 8:25 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>
> I use IMAP on my phone, and with Thunderpants... I mean Thunderbird...
> and it works quite nicely.

Including things like searching lkml etc?

I've been successful in using IMAP for just reading my inbox (even
over a cell network). Then IMAP is reasonably fine, at least if you do
client-side caching.

But not for actual real *work*, where I actually search for specific
email authors and words in the body? Not so much.

Maybe it's because I'm used to having all my mail locally, but
searching all my emails is one of my most common operations. The gmail
web interface does that fine. IMAP has never worked for me, despite
some client/server combinations allegedly supporting server-side
searches. Maybe I never hit the right combination, but I tested things
that were *supposed* to do it.

Actually, most of my attempts at using IMAP have been unacceptably
slow even when I don't do searches. I used to do huge inboxes, which
brought just about anything to a standstill. I only got rid of my
habit of big inboxes thanks to another gmail feature - you don't have
to save your non-inbox email in a folder, you can just "archive" it
and it goes away but is still easily searchable.

                         Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  3:42                 ` Linus Torvalds
@ 2011-11-01  4:25                   ` hpanvin@gmail.com
  2011-11-01  5:19                     ` James Bottomley
  2011-11-01 17:06                   ` Tony Luck
  1 sibling, 1 reply; 101+ messages in thread
From: hpanvin@gmail.com @ 2011-11-01  4:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: david, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

Body searches are still slow, at least with dovecot on the server.  No idea about gmail as the server.

Linus Torvalds <torvalds@linux-foundation.org> wrote:

>On Mon, Oct 31, 2011 at 8:25 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>>
>> I use IMAP on my phone, and with Thunderpants... I mean
>Thunderbird...
>> and it works quite nicely.
>
>Including things like searching lkml etc?
>
>I've been successful in using IMAP for just reading my inbox (even
>over a cell network). Then IMAP is reasonably fine, at least if you do
>client-side caching.
>
>But not for actual real *work*, where I actually search for specific
>email authors and words in the body? Not so much.
>
>Maybe it's because I'm used to having all my mail locally, but
>searching all my emails is one of my most common operations. The gmail
>web interface does that fine. IMAP has never worked for me, despite
>some client/server combinations allegedly supporting server-side
>searches. Maybe I never hit the right combination, but I tested things
>that were *supposed* to do it.
>
>Actually, most of my attempts at using IMAP have been unacceptably
>slow even when I don't do searches. I used to do huge inboxes, which
>brought just about anything to a standstill. I only got rid of my
>habit of big inboxes thanks to another gmail feature - you don't have
>to save your non-inbox email in a folder, you can just "archive" it
>and it goes away but is still easily searchable.
>
>                         Linus

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  3:31               ` Linus Torvalds
@ 2011-11-01  4:58                 ` david
  0 siblings, 0 replies; 101+ messages in thread
From: david @ 2011-11-01  4:58 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: H. Peter Anvin, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, 31 Oct 2011, Linus Torvalds wrote:

> On Mon, Oct 31, 2011 at 7:17 PM,  <david@lang.hm> wrote:
>>
>> what IMAP servers have you used? I find that with a good IMAP server I can
>> get away with _very_ little processing power in the client and get good
>> performance (I still am using pine, but with a cyrus IMAP server)
>
> I've used pine (and then alpine) to cyrus servers too. That was what
> LF used to have on the server side. It absolutely sucked.
>
> That combination is supposed to do server-side searching etc, but you
> couldn't tell from the performance. It was disgusting.

interesting, that's the setup I have and the server side searching works 
well for me (searching for random text works much better if I have the 
server update it's squatter indexes frequently). I did find that different 
filesystems made a HUGE difference in system performance (with ext2/3 
being the worst). I use XFS, but I see a lot of people reporting good 
results with ext4, btrfs, and ZFS.

the biggest folder I have has >250K messages in it and takes >20G. random 
text searches on that take 5-10 seconds (much better than I could do on 
my local drives)

> I'm sure it works much better with a fast local network, but quite
> frankly, that obviates the need for IMAP in the first place. If you
> have your mail locally, there are better models than IMAP for handling
> it.

I agree that it's not strictly needed if you store everything local, but I 
don't want to have all that mail on my non-mirrored, small laptop drive 
(even if it is a SSD). I've suffered too many disk failures over the years 
to not have my mail store on redundant drives :-)

David Lang

> So the only situation I've found IMAP reasonable has been at corporate
> settings where you're not talking DSL or cable modem speeds, but use
> IMAP as a way to avoid NFS-mounting the mail spool, which is even
> worse. But actually working over slowish internet connections? No
> thank you.
>
>                         Linus
>

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  3:25               ` H. Peter Anvin
  2011-11-01  3:42                 ` Linus Torvalds
@ 2011-11-01  5:03                 ` david
  2011-11-01 13:32                   ` Theodore Tso
  1 sibling, 1 reply; 101+ messages in thread
From: david @ 2011-11-01  5:03 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Linus Torvalds, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, 31 Oct 2011, H. Peter Anvin wrote:

> On 10/31/2011 07:17 PM, david@lang.hm wrote:
>>
>> what IMAP servers have you used? I find that with a good IMAP server I
>> can get away with _very_ little processing power in the client and get
>> good performance (I still am using pine, but with a cyrus IMAP server)
>>
>> there are a bunch of IMAP servers out there that are horrible (and even
>> more IMAP clients that are really doing POP and storing everything
>> locally, just using the IMAP protocol)
>>
>
> Last I checked if you used pine against an IMAP server it did absolutely
> no caching, in which case you're of course screwed.

my biggest gripe with most IMAP clients is that they try so hard to cache 
everything locally that you may as well just copy the mail locally to 
begin with...

David Lang

> I use IMAP on my phone, and with Thunderpants... I mean Thunderbird...
> and it works quite nicely.
>
> 	-hpa
>
>

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  4:25                   ` hpanvin@gmail.com
@ 2011-11-01  5:19                     ` James Bottomley
  2011-11-01 13:13                       ` Henrique de Moraes Holschuh
  0 siblings, 1 reply; 101+ messages in thread
From: James Bottomley @ 2011-11-01  5:19 UTC (permalink / raw)
  To: hpanvin@gmail.com
  Cc: Linus Torvalds, david, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Mon, 2011-10-31 at 21:25 -0700, hpanvin@gmail.com wrote:
> Body searches are still slow, at least with dovecot on the server.  No
> idea about gmail as the server.

Actually, my experience is different.  I do use body searches a lot (not
TSA approved ones, obviously) and I found they're much faster after
switching to dovecot (from wu-imapd).  However, they also burn massive
amounts of CPU time on my server, so it probably helps that I only have
a couple of mail users.

James



^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:52                     ` Linus Torvalds
  2011-10-31 22:54                       ` H. Peter Anvin
@ 2011-11-01  5:39                       ` James Bottomley
  1 sibling, 0 replies; 101+ messages in thread
From: James Bottomley @ 2011-11-01  5:39 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, H. Peter Anvin, git, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Mon, 2011-10-31 at 15:52 -0700, Linus Torvalds wrote:
> On Mon, Oct 31, 2011 at 3:44 PM, Junio C Hamano <gitster@pobox.com> wrote:
> >
> > So nobody is worried about this (quoting from my earlier message)?
> 
> No, because you haven't been reading what we write.
> 
> The tag is useless.

It's not useless to people who want to verify the tree after it's been
released by you (say for forensics or something).  As Peter said, we can
put it in a normally invisible namespace, but having a flag to make it
visible allows tools like git describe --contains to tell me which
signed tag was used to send a particular commit.

> The information *in* the tag is not. But it shouldn't be saved in the
> tag (or note, or whatever). Because that's just an annoying place for
> it to be, with no upside.
> 
> Save it in the commit we generate. BAM! Useful, readable, permanent,
> and independently verifiable.
> 
> And the advantage is that we can make that same mechanism add
> "maintainer notes" to the merge message too. Right now some
> maintainers write good notes about what the merge will bring in, but
> they are basically lost, because git is so good at merging and doesn't
> even stop to ask people to edit the merge message.

A signed empty commit containing the merge message as a comment also
looks fine to me.  We'd need extra tooling to say which signed merge
corresponds to this patch, but I'd say its workable.  The only slightly
counter intuitive thing is that for a non-trivial merge, my signed merge
description will have to be the next commit below rather than in the
actual merge you do (because we can't alter a cryptographically signed
commit).

James



^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  5:19                     ` James Bottomley
@ 2011-11-01 13:13                       ` Henrique de Moraes Holschuh
  0 siblings, 0 replies; 101+ messages in thread
From: Henrique de Moraes Holschuh @ 2011-11-01 13:13 UTC (permalink / raw)
  To: James Bottomley
  Cc: hpanvin@gmail.com, Linus Torvalds, david, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Tue, 01 Nov 2011, James Bottomley wrote:
> On Mon, 2011-10-31 at 21:25 -0700, hpanvin@gmail.com wrote:
> > Body searches are still slow, at least with dovecot on the server.  No
> > idea about gmail as the server.
> 
> Actually, my experience is different.  I do use body searches a lot (not
> TSA approved ones, obviously) and I found they're much faster after
> switching to dovecot (from wu-imapd).  However, they also burn massive
> amounts of CPU time on my server, so it probably helps that I only have
> a couple of mail users.

Well, Cyrus IMAP provides fast server-side body searches using indexes, but
you'd have to configure it properly to use that feature.  It is lightweight
on the server side after the initial indexing (yes, it can do incremental
and staggered indexing).  Maybe Dovecot also needs some configuration to
index mailboxes for fast searching?

wu-imapd is not something one should bring up on polite conversation.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  5:03                 ` david
@ 2011-11-01 13:32                   ` Theodore Tso
  0 siblings, 0 replies; 101+ messages in thread
From: Theodore Tso @ 2011-11-01 13:32 UTC (permalink / raw)
  To: david
  Cc: Theodore Tso, H. Peter Anvin, Linus Torvalds, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML


On Nov 1, 2011, at 1:03 AM, david@lang.hm wrote:
> 
> my biggest gripe with most IMAP clients is that they try so hard to cache everything locally that you may as well just copy the mail locally to begin with…

I use a combination of mutt and mbsync (isync is the source forge project name), precisely because I want everything locally, so I can read/delete/reply to e-mail while I am off-line, but then be able to get back onto the network and sync everything back to the IMAP server.   I also like this better than copying the mail locally because if my laptop explodes or is stolen in Belgium, I still have a backup on the IMAP server.

One nice thing about mutt by the way is that it has very nice PGP/GPG integration…

-- Ted


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01  3:42                 ` Linus Torvalds
  2011-11-01  4:25                   ` hpanvin@gmail.com
@ 2011-11-01 17:06                   ` Tony Luck
  2011-11-01 17:15                     ` Linus Torvalds
  1 sibling, 1 reply; 101+ messages in thread
From: Tony Luck @ 2011-11-01 17:06 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: H. Peter Anvin, david, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Mon, Oct 31, 2011 at 8:42 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> Maybe it's because I'm used to having all my mail locally, but
> searching all my emails is one of my most common operations. The gmail
> web interface does that fine.

I've had my gmail account subscribed to LKML since the dawn of gmail.

It used to be snappy for searches - but it got slower as my mailbox
filled (a few months ago I had to throw out the oldest couple of years of
LKML because I was getting close to the gmail limit - I'm back up to
86% again now, so getting close to another purge).

So - don't get too used to speed in gmail.

-Tony

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 17:06                   ` Tony Luck
@ 2011-11-01 17:15                     ` Linus Torvalds
  2011-11-12 19:31                       ` Felipe Contreras
  0 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-01 17:15 UTC (permalink / raw)
  To: Tony Luck
  Cc: H. Peter Anvin, david, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Tue, Nov 1, 2011 at 10:06 AM, Tony Luck <tony.luck@gmail.com> wrote:
>
> I've had my gmail account subscribed to LKML since the dawn of gmail.
>
> It used to be snappy for searches - but it got slower as my mailbox
> filled (a few months ago I had to throw out the oldest couple of years of
> LKML because I was getting close to the gmail limit - I'm back up to
> 86% again now, so getting close to another purge).
>
> So - don't get too used to speed in gmail.

The LKML thing is why I really love how gmail works. LKML just goes
into the black hole that is the gmail thing, and I never see it. But
then, when I get cc'd about something in the middle of a thread when a
question comes up - I just magically see the whole thread without
having to look around in other folders or anything like that.

I'm sure 'notmuch' etc does that too, but pine sure never did. If I
made my filters automatically move lkml posts to my lkml folder, by
the time I got cc'd, I had to go back to that folder to see the
history of the thread.

But yeah, I only have about two years of LKML. I'll happily throw out
old lkml email when it gets to be painfully slow to search, though. I
used to have to do lkml folder cleanups every few months even when I
had my email *locally* because it just got too painful with big
folders, so the gmail approach already not only works much better, but
if I have to clean things up every few years, that's much less than I
used to have to do anyway.

                          Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:18           ` Linus Torvalds
  2011-10-31 22:20             ` H. Peter Anvin
@ 2011-11-01 19:47             ` Junio C Hamano
  2011-11-01 21:21               ` Linus Torvalds
  2011-11-02 10:53               ` Michael J Gruber
  1 sibling, 2 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-11-01 19:47 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> But what would be nice is that "git pull" would fetch the tag (based on
> name) *automatically*, and not actually create a tag in my repository at
> all. Instead, if would use the tag to check the signature, and - if we
> do this right - also use the tag contents to populate the merge commit
> message.
>
> In other words, no actual tag would ever be left around as a turd, it
> would simply be used as an automatic communication channel between the
> "git push -s" of the submitter and my subsequent "git pull". Neither
> side would have to do anything special, and the tag would never show
> up in any relevant tree (it could even be in a totally separate
> namespace like "refs/pullmarker/<branchname>" or something).

While I like the "an ephemeral tag is used only for hop-to-hop
communication to carry information to be recorded in the resulting
history" approach, I see a few downsides.

 * The ephemeral tag needs to stay somewhere under refs/ hierarchy of the
   lieutenant's tree until you pick it up, even if they are out of the way
   in refs/pullmarker/$branchname. The next time the same lieutenant makes
   a pull request, either it will be overwritten or multiple versions of
   them refs/pullmarker/$branchname/$serial need to be kept.

   - If the former, this makes forking of the project harder. Suppose a
     pull request is made, you fetch and reject it. The lieutenant reworks
     and makes another pull request. At this point the earlier signature
     is gone. If somebody disagreed with your rejection and wanted to run
     his tree with the initial version you rejected, his tree will not
     carry the signature from the lieutenant.

   - If the latter, then there needs to be a way to expire these pull
     markers when they no longer are useful (i.e. the signature in it is
     transcribed to a merge commit you create) [*1*]. But the party who
     has power to clean them (i.e. the lieutenant who owns the repository)
     is different from the party whose action determines when they no
     longer are necessary (i.e. you). In practice this would lead to these
     pull markers not cleaned at all [*2*].

 * To verify the commit C that was taken from the tip of lieutenant's tree
   some time ago, one has to find the merge commit that has C as a parent,
   and look at the merge commit.  For example "git log --show-signature"
   would either show or not show the authenticity of C depending on where
   the traversal comes from. You certainly can implement it that way, but
   "some child describes an aspect of its parent, but not necessarily all
   children do so" feels philosophically less correct than "the commit has
   data to describe itself".

In your "ephemeral tag", the workflow for a developer (D) and his
integrator (U) would look like this, I think.

 D$ until have something worth sending; do work; done
 D$ git push -s
 Enter passphrase: ...
	- "push" internally creates a pull marker that signs the commit
          object name this is pushing, among other things, and sends it
          along the primary payload
 D$ git pull-request; mail linus

 U$ git pull
 	- "pull" notices the pull marker and fetches it as well;
        - "pull" GPG validates the pull marker;
        - When preparing a merge commit message, the contents of the
          pull marker is included in .git/MERGE_MSG

The "in-commit signature" would give you 100% and your contributors 98% of
that, I think.

 D$ until have something worth sending; do work; done
        - The final round of reworking is concluded with "commit -S",
          which would GPG sign the tip commit itself
 D$ git push
	- Nothing needs to change in the protocol nor "push" itself
 D$ git pull-request; mail linus

 U$ git pull
 	- "pull" GPG validates the tip commit
	- Nothing unusual needs to happen to the resulting "merge" commit

And as a bonus, the code is already there ;-).


[Footnote]

*1* The common ancestor discovery in fetch uses as many refs as it can to
reduce the amount of data that needs to be transferred, and it is known to
hurt performance of the initial advertisement exchange when there are too
many useless refs.

*2* Do casual git users even know how to remove refs in a
remote/publishing repository?

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 19:47             ` Junio C Hamano
@ 2011-11-01 21:21               ` Linus Torvalds
  2011-11-01 21:56                 ` Junio C Hamano
                                   ` (3 more replies)
  2011-11-02 10:53               ` Michael J Gruber
  1 sibling, 4 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-01 21:21 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Tue, Nov 1, 2011 at 12:47 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> While I like the "an ephemeral tag is used only for hop-to-hop
> communication to carry information to be recorded in the resulting
> history" approach, I see a few downsides.

So I do agree.

I'd actually be *happier* with a generic multi-line "branch
description" thing that involves no git objects at all, just a nice
description of what the branch is.

The fact that you could also hide a signed version of the
top-of-branch there would be kind of a side effect, and wouldn't be a
requirement.

I hate how anonymous our branches are. Sure, we can use good names for
them, but it was a mistake to think we should describe the repository
(for gitweb), rather than the branch.

Ok, "hate" is a strong word. I don't "hate" it. I don't even think
it's a major design issue. But I do think that it would have been
nicer if we had had some branch description model.

The only reason I suggest a tag is really because it would fit with
existing tooling - especially the git transport protocol. So it's not
that I actually think that a tag is the right way to describe (and
sign) the branch, it's just that it's the way that wouldn't require
any changes other than in "git push -s" and "git pull".

>  * To verify the commit C that was taken from the tip of lieutenant's tree
>   some time ago, one has to find the merge commit that has C as a parent,
>   and look at the merge commit.  For example "git log --show-signature"
>   would either show or not show the authenticity of C depending on where
>   the traversal comes from. You certainly can implement it that way, but
>   "some child describes an aspect of its parent, but not necessarily all
>   children do so" feels philosophically less correct than "the commit has
>   data to describe itself".

Yeah.

Having thought about it, I'm also not convinced I really want to
pollute the "git log" output with information that realistically
almost nobody cares about. The primary use is just for the person who
pulls things to verify it, after that the information is largely stale
and almost certain to never be interesting to anybody ever again. It's
*theoretically* useful if somebody wants to go back and re-verify, but
at the same time that really isn't expected to be the common case.

So I'm wondering if we want to save it at all. it's quite possible
that realistically speaking "google the mailing list archives" is the
*right* way to look up the signature if it is ever needed later.

Maybe just verifying the email message (with the suggested kind of
change to "git request-pull") is actually the right approach. And what
I should do is to just wrap my "git pull" in some script that I can
just cut-and-paste the gpg-signed thing into, and which just does the
"gpg --verify" on it, and then does the "git pull" after that.

Because in many ways, "git request-pull" is when you do want to sign
stuff. A developer might well want to push out his stuff for some
random internal testing (linux-next, for example), and then only later
decide "Ok, it was all good, now I want to make it 'official' and ask
Linus to pull it", and sign it at *that* time, rather than when
actually pushing it out.

And I suspect signing the pull request fits better into peoples
existing workflow anyway - sending out the email to ask the maintainer
to pull really is the "special event", rather than pushing out the
code itself.

                      Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 21:21               ` Linus Torvalds
@ 2011-11-01 21:56                 ` Junio C Hamano
  2011-11-02 20:04                   ` Linus Torvalds
  2011-11-01 22:39                 ` Ted Ts'o
                                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-01 21:56 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> Having thought about it, I'm also not convinced I really want to
> pollute the "git log" output with information that realistically
> almost nobody cares about. The primary use is just for the person who
> pulls things to verify it, after that the information is largely stale
> and almost certain to never be interesting to anybody ever again. It's
> *theoretically* useful if somebody wants to go back and re-verify, but
> at the same time that really isn't expected to be the common case.
> ...
> So I'm wondering if we want to save it at all. it's quite possible
> that realistically speaking "google the mailing list archives" is the
> *right* way to look up the signature if it is ever needed later.

I'd rather want to hear opinions from people who base their work on public
kernels (e.g. distros, and companies who roll their own prod kernels), on
that.

But my gut feeling is that "usually hidden not to disturb normal users,
but is cast in stone in the history and cannot be lost" strikes the right
balance. Both your "next merge commit records the signature together with
the largely useless merge summary cruft but everybody learned to ignore it
with 'log --no-merges' anyway so it does not hurt to have it there" and
the commit signature topic from the next branch [*1*] that puts the
signature in the object header and teaches '--show-signature' option to
the log family to show it share this property.

> Maybe just verifying the email message (with the suggested kind of
> change to "git request-pull") is actually the right approach. And what
> I should do is to just wrap my "git pull" in some script that I can
> just cut-and-paste the gpg-signed thing into, and which just does the
> "gpg --verify" on it, and then does the "git pull" after that.
>
> Because in many ways, "git request-pull" is when you do want to sign
> stuff. A developer might well want to push out his stuff for some
> random internal testing (linux-next, for example), and then only later
> decide "Ok, it was all good, now I want to make it 'official' and ask
> Linus to pull it", and sign it at *that* time, rather than when
> actually pushing it out.
>
> And I suspect signing the pull request fits better into peoples
> existing workflow anyway - sending out the email to ask the maintainer
> to pull really is the "special event", rather than pushing out the
> code itself.

"I can silently push and re-push or even rewind-and-then-push until I
officially send pull-request out" fits well with the "defer the decision
as much as possible" model Git takes in general, and I find certain
attractiveness in it.

But on the other hand, in many ways, publishing your commit to the outside
world, not necessarily for getting pulled into the final destination
(i.e. your tree) but merely for other people to try it out, is the point
of no return (aka "don't rewind or rebase once you publish").  "pushing
out" might be less special than "please pull", but it still is special.

Also there is nothing lost if you sign commits whenever you push them
out.


[Footnote]

*1* Here are three examples on the same commit that is signed for
illustration.

------------------------------------------------
$ git show -s pu
commit c9d870fceac787fdb1c1c43b136c1a94ab2ab005
Merge: 8367c51 71f45ee
Author: Junio C Hamano <gitster@pobox.com>
Date:   Mon Oct 31 20:06:58 2011 -0700

    Merge branch 'jc/stream-to-pack' into pu
    
    * jc/stream-to-pack:
      Bulk check-in
      finish_tmp_packfile(): a helper function
      create_tmp_packfile(): a helper function
      write_pack_header(): a helper function
------------------------------------------------
$ git show -s --show-signature pu
commit c9d870fceac787fdb1c1c43b136c1a94ab2ab005
gpg: Signature made Mon 31 Oct 2011 08:07:04 PM PDT using RSA key ID 96AFE6CB
gpg: Good signature from "Junio C Hamano <gitster@pobox.com>"
gpg:                 aka "Junio C Hamano <junio@pobox.com>"
gpg:                 aka "Junio C Hamano <jch@google.com>"
Merge: 8367c51 71f45ee
Author: Junio C Hamano <gitster@pobox.com>
Date:   Mon Oct 31 20:06:58 2011 -0700

    Merge branch 'jc/stream-to-pack' into pu
    
    * jc/stream-to-pack:
      Bulk check-in
      finish_tmp_packfile(): a helper function
      create_tmp_packfile(): a helper function
      write_pack_header(): a helper function
------------------------------------------------
$ git cat-file commit pu
tree 9add290d468800c3c51ff68fedfb3d16427872ff
parent 8367c51becc5a225b9a192348b7d7c615fb6d250
parent 71f45eeb8278670257bea83620f7d3eac174eee7
author Junio C Hamano <gitster@pobox.com> 1320116818 -0700
committer Junio C Hamano <gitster@pobox.com> 1320116824 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
gpgsig Version: GnuPG v1.4.10 (GNU/Linux)
gpgsig 
gpgsig ...
gpgsig =c62U
gpgsig -----END PGP SIGNATURE-----

Merge branch 'jc/stream-to-pack' into pu

* jc/stream-to-pack:
  Bulk check-in
  finish_tmp_packfile(): a helper function
  create_tmp_packfile(): a helper function
  write_pack_header(): a helper function
------------------------------------------------


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 21:21               ` Linus Torvalds
  2011-11-01 21:56                 ` Junio C Hamano
@ 2011-11-01 22:39                 ` Ted Ts'o
  2011-11-02 23:34                 ` Junio C Hamano
  2011-11-10 13:52                 ` David Woodhouse
  3 siblings, 0 replies; 101+ messages in thread
From: Ted Ts'o @ 2011-11-01 22:39 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Tue, Nov 01, 2011 at 02:21:59PM -0700, Linus Torvalds wrote:
> So I'm wondering if we want to save it at all. it's quite possible
> that realistically speaking "google the mailing list archives" is the
> *right* way to look up the signature if it is ever needed later.

Given the number of trees that you merge in every merge window (never
mind over an entire release), I don't think "google the mailing list
archives" is going to scale.  Finding some way to keep it along with
the merge window seems the right thing.  I agree that it should hidden
normally, but that's a UI display issue.  Heck, we could just hide
after the terminating NULL in the commit description, per a discussion
on the git list 2-3 weeks ago.  :-)

> Because in many ways, "git request-pull" is when you do want to sign
> stuff. A developer might well want to push out his stuff for some
> random internal testing (linux-next, for example), and then only later
> decide "Ok, it was all good, now I want to make it 'official' and ask
> Linus to pull it", and sign it at *that* time, rather than when
> actually pushing it out.

Sure, the signed content should be buried in the commit that it
describes.  Whether we carry it in an emphemeral tag or in the git
request-pull is not really important from a security perspective.  The
tag is nicer simply because the person doing the pull won't need to
cut and paste the signature information.

One approach which might work is if git request-pull sends the e-mail
message with the git shortlog and diffstat, *and* an MIME attachment
that contained all of the necessary information.  The maintainer would
then save the attachment, and feed it to git, which will display the
git shortlog and diffstat, ask for confirmation, and then embed the
digital signature into the merge commit.

The only problem with that is (a) you'd have to get over your hatred
of attachment (but if you're using Gmail hopefully that's relative
convenient :-), and (b) LKML list filter would have to be taught to
tolerate git-generated attachments.

						- Ted

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-10-31 22:56                       ` Linus Torvalds
@ 2011-11-02  9:11                         ` Ingo Molnar
  2011-11-02 11:20                           ` Jochen Striepe
  0 siblings, 1 reply; 101+ messages in thread
From: Ingo Molnar @ 2011-11-02  9:11 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, H. Peter Anvin, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML


* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> And the receiving side would just do the "git pull" and 
> automatically just get notified that "Yes, this push has been 
> signed by key Xyz Abcdef"

If this approach is used then it would be nice to have a .gitconfig 
switch to require trusted pulls by default: to not allow doing 
non-signed or untrusted pulls accidentally, or for Git to warn in a 
visible, hard to miss way if there's a non-signed pull.

This adds social uncertainty (and an element of a silent alarm) to a 
realistic attack: the attacker wouldnt know exactly how the puller 
checks signed pull requests, it's kept private.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 19:47             ` Junio C Hamano
  2011-11-01 21:21               ` Linus Torvalds
@ 2011-11-02 10:53               ` Michael J Gruber
  2011-11-02 18:58                 ` Junio C Hamano
  1 sibling, 1 reply; 101+ messages in thread
From: Michael J Gruber @ 2011-11-02 10:53 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Linus Torvalds, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Junio C Hamano venit, vidit, dixit 01.11.2011 20:47:
> Linus Torvalds <torvalds@linux-foundation.org> writes:
> 
>> But what would be nice is that "git pull" would fetch the tag (based on
>> name) *automatically*, and not actually create a tag in my repository at
>> all. Instead, if would use the tag to check the signature, and - if we
>> do this right - also use the tag contents to populate the merge commit
>> message.
>>
>> In other words, no actual tag would ever be left around as a turd, it
>> would simply be used as an automatic communication channel between the
>> "git push -s" of the submitter and my subsequent "git pull". Neither
>> side would have to do anything special, and the tag would never show
>> up in any relevant tree (it could even be in a totally separate
>> namespace like "refs/pullmarker/<branchname>" or something).
> 
> While I like the "an ephemeral tag is used only for hop-to-hop
> communication to carry information to be recorded in the resulting
> history" approach, I see a few downsides.
> 
>  * The ephemeral tag needs to stay somewhere under refs/ hierarchy of the
>    lieutenant's tree until you pick it up, even if they are out of the way
>    in refs/pullmarker/$branchname. The next time the same lieutenant makes
>    a pull request, either it will be overwritten or multiple versions of
>    them refs/pullmarker/$branchname/$serial need to be kept.

If we are interested in commit sigs, the easiest tag-based approach is
to name the sig carrying tag by the commit's sha1. Just like the sig is
tied (in)to a commit in Junio's approach, it would be indexed by it. We
can do that now:

git config --global alias.sign '!f() { c=$(git rev-parse "$1") || exit;
shift; git tag -s $@ sigs/$c $c; }; f'

But a different place rather than refs/tags/sigs/<sha1> will be more
appropriate, so that we don't pollute the tag namespace. (Yes, this is
similar to storing them in notes.) tags have a message etc.

With an appropriate refspec, these sigs can be pushed out automatically
(by the lieutenant).

pull-request as in next will list the expected <sha1> at tip.

git pull needs to learn to (fetch and) use refs/<whatever>/<sha1> to
verify that the tip is signed.

git log --show-signature can do the same tricks as with in-commit sigs.

Some things to decide in this approach:
- Should git-pull (pull sigs and) verify by default?
- Should we worry about overwriting existings sigs? We have union-merge
for notes already, and that would be appropriate for sigs. (Yes, our
tags code does verify multiple concatenated sigs.)

The advantage of tags is that they can be added without rewriting the
commit, of course.

Michael

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02  9:11                         ` Ingo Molnar
@ 2011-11-02 11:20                           ` Jochen Striepe
  0 siblings, 0 replies; 101+ messages in thread
From: Jochen Striepe @ 2011-11-02 11:20 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: Linus Torvalds, Junio C Hamano, H. Peter Anvin, git,
	James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

	Hi,

On Wed, Nov 02, 2011 at 10:11:26AM +0100, Ingo Molnar wrote:
> If this approach is used then it would be nice to have a .gitconfig 
> switch to require trusted pulls by default: to not allow doing 
> non-signed or untrusted pulls accidentally, or for Git to warn in a 
> visible, hard to miss way if there's a non-signed pull.
> 
> This adds social uncertainty (and an element of a silent alarm) to a 
> realistic attack: the attacker wouldnt know exactly how the puller 
> checks signed pull requests, it's kept private.

But that way you get a false sense of alarm when someone sent a
perfectly trustable pull request, e.g. by signed email.


Another question: If store the actual pgp/gpg signatures in the git tree,
how do you handle signatures by keys which were valid by the time the
signature was made but expired when checking some time afterwards? AFAICT,
gpg will only tell you the key is expired _now_, and will make no statement
regarding the time the actual signature was made.


Thanks,
Jochen.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02 10:53               ` Michael J Gruber
@ 2011-11-02 18:58                 ` Junio C Hamano
  2011-11-02 21:05                   ` Michael J Gruber
  0 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-02 18:58 UTC (permalink / raw)
  To: Michael J Gruber
  Cc: Linus Torvalds, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Michael J Gruber <git@drmicha.warpmail.net> writes:

> The advantage of tags is that they can be added without rewriting the
> commit, of course.

And you did neither think about the downsides of tags, nor read what
others already explained for you?

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 21:56                 ` Junio C Hamano
@ 2011-11-02 20:04                   ` Linus Torvalds
  2011-11-02 21:13                     ` Junio C Hamano
                                       ` (2 more replies)
  0 siblings, 3 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-02 20:04 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Tue, Nov 1, 2011 at 2:56 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> But on the other hand, in many ways, publishing your commit to the outside
> world, not necessarily for getting pulled into the final destination
> (i.e. your tree) but merely for other people to try it out, is the point
> of no return (aka "don't rewind or rebase once you publish").  "pushing
> out" might be less special than "please pull", but it still is special.

So I really think that signing the top commit itself is fundamentally wrong.

That commit may not even be *yours*. You may have pulled it from a
sub-lieutenant as a fast-forward, or similar. Amending it later would
be actively very very *wrong*.

So quite frankly, I think the stuff in pu (or next?) is completely
mis-designed. Doing it in the commit is wrong for fundamental reasons,
which all boil down to a simple issue:

 - you absolutely *need* to add the signature later. You *cannot* do
it at "git commit" time.

That's a fundamental issue both from a "workflow model" issue (ie you
want to sign stuff after it has passed testing etc, but you may need
to commit it in order to *get* testing), as well as from a
"fundamental git datastructures" issue (ie you would want to sign
commits that aren't yours.

"git commit --amend" is not the answer - that destroys the fundamental
concept of history being immutable, and while it works for your local
commits, it doesn't work for anybody elses commits, or for stuff you
already pushed out.

And "add a fake empty commit just for the signature" is not the answer
either - because that is clearly inferior to the tags we already had.

I dunno. Did I miss something? As far as I can tell, the signed tags
that we've had since day one are *clearly* much better in very
fundamental ways.

                             Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02 18:58                 ` Junio C Hamano
@ 2011-11-02 21:05                   ` Michael J Gruber
  0 siblings, 0 replies; 101+ messages in thread
From: Michael J Gruber @ 2011-11-02 21:05 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Linus Torvalds, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Junio C Hamano venit, vidit, dixit 02.11.2011 19:58:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>> The advantage of tags is that they can be added without rewriting the
>> commit, of course.
> 
> And you did neither think about the downsides of tags, nor read what
> others already explained for you?

We're just weighing things differently here, and no accusations of
"misinformation" or "not thinking" will change this.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02 20:04                   ` Linus Torvalds
@ 2011-11-02 21:13                     ` Junio C Hamano
  2011-11-03  1:02                     ` Shawn Pearce
  2011-11-03 18:29                     ` Junio C Hamano
  2 siblings, 0 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-11-02 21:13 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> And "add a fake empty commit just for the signature" is not the answer
> either - because that is clearly inferior to the tags we already had.
>
> I dunno. Did I miss something? As far as I can tell, the signed tags
> that we've had since day one are *clearly* much better in very
> fundamental ways.

Ok, back to the drawing board (which is not a loss as I wasn't expecting
this to be in the official release in upcoming 1.7.8 anyway).


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 21:21               ` Linus Torvalds
  2011-11-01 21:56                 ` Junio C Hamano
  2011-11-01 22:39                 ` Ted Ts'o
@ 2011-11-02 23:34                 ` Junio C Hamano
  2011-11-02 23:41                   ` david
  2011-11-02 23:42                   ` Linus Torvalds
  2011-11-10 13:52                 ` David Woodhouse
  3 siblings, 2 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-11-02 23:34 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> I hate how anonymous our branches are. Sure, we can use good names for
> them, but it was a mistake to think we should describe the repository
> (for gitweb), rather than the branch.
>
> Ok, "hate" is a strong word. I don't "hate" it. I don't even think
> it's a major design issue. But I do think that it would have been
> nicer if we had had some branch description model.
> ...
> Maybe just verifying the email message (with the suggested kind of
> change to "git request-pull") is actually the right approach. And what
> I should do is to just wrap my "git pull" in some script that I can
> just cut-and-paste the gpg-signed thing into, and which just does the
> "gpg --verify" on it, and then does the "git pull" after that.
>
> Because in many ways, "git request-pull" is when you do want to sign
> stuff. A developer might well want to push out his stuff for some
> random internal testing (linux-next, for example), and then only later
> decide "Ok, it was all good, now I want to make it 'official' and ask
> Linus to pull it", and sign it at *that* time, rather than when
> actually pushing it out.

You keep saying cut-and-paste, but do you mind feeding the e-mail text
itself to a tool, instead of cut-and-paste?

The reason I am wondering about this is because in another topic (also in
'next') cooking there is an extended support for topic description for the
branch that states what the purpose of the topic is why the requestor
wants you to have it (this information can be set and updated with "git
branch --edit-description").

A respond-to-request-pull wrapper you would use could be:

 - Get the e-mail from the standard input;
 - Pick up the signed bits and validate the signature;
 - Perform the requested fetch; and
 - Record the merge (or prepare .git/MERGE_MSG) with both the signed bits.

and the "signed bits" could include:

   - the repository and the branch you were expected to pull;
   - the topic description.

among other things the requestor can edit when request-pull message is
prepared.

That would get us back to your "the lieutenant tip is not so special, but
the merge commit the integrator makes using that tip has the signature for
this particular pull" model.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02 23:34                 ` Junio C Hamano
@ 2011-11-02 23:41                   ` david
  2011-11-02 23:42                   ` Linus Torvalds
  1 sibling, 0 replies; 101+ messages in thread
From: david @ 2011-11-02 23:41 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Linus Torvalds, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Wed, 2 Nov 2011, Junio C Hamano wrote:

> Linus Torvalds <torvalds@linux-foundation.org> writes:
>
>> I hate how anonymous our branches are. Sure, we can use good names for
>> them, but it was a mistake to think we should describe the repository
>> (for gitweb), rather than the branch.
>>
>> Ok, "hate" is a strong word. I don't "hate" it. I don't even think
>> it's a major design issue. But I do think that it would have been
>> nicer if we had had some branch description model.
>> ...
>> Maybe just verifying the email message (with the suggested kind of
>> change to "git request-pull") is actually the right approach. And what
>> I should do is to just wrap my "git pull" in some script that I can
>> just cut-and-paste the gpg-signed thing into, and which just does the
>> "gpg --verify" on it, and then does the "git pull" after that.
>>
>> Because in many ways, "git request-pull" is when you do want to sign
>> stuff. A developer might well want to push out his stuff for some
>> random internal testing (linux-next, for example), and then only later
>> decide "Ok, it was all good, now I want to make it 'official' and ask
>> Linus to pull it", and sign it at *that* time, rather than when
>> actually pushing it out.
>
> You keep saying cut-and-paste, but do you mind feeding the e-mail text
> itself to a tool, instead of cut-and-paste?

think webmail (i.e. gmail), to feed the e-mail itself to a tool you either 
need to cut-n-paste the entire e-mail or you have to first save the mail 
to a text file. both of which are significantly harder than doing a 
cut-n-past of a portion of the message.

David Lang

> The reason I am wondering about this is because in another topic (also in
> 'next') cooking there is an extended support for topic description for the
> branch that states what the purpose of the topic is why the requestor
> wants you to have it (this information can be set and updated with "git
> branch --edit-description").
>
> A respond-to-request-pull wrapper you would use could be:
>
> - Get the e-mail from the standard input;
> - Pick up the signed bits and validate the signature;
> - Perform the requested fetch; and
> - Record the merge (or prepare .git/MERGE_MSG) with both the signed bits.
>
> and the "signed bits" could include:
>
>   - the repository and the branch you were expected to pull;
>   - the topic description.
>
> among other things the requestor can edit when request-pull message is
> prepared.
>
> That would get us back to your "the lieutenant tip is not so special, but
> the merge commit the integrator makes using that tip has the signature for
> this particular pull" model.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02 23:34                 ` Junio C Hamano
  2011-11-02 23:41                   ` david
@ 2011-11-02 23:42                   ` Linus Torvalds
  1 sibling, 0 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-02 23:42 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Wed, Nov 2, 2011 at 4:34 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> You keep saying cut-and-paste, but do you mind feeding the e-mail text
> itself to a tool, instead of cut-and-paste?

Feeding the email to a tool is actually a fair amount of extra work.
It would have worked well in the days when I used text-based email
clients that just had a "pipe email to command" model, but that's long
gone.

In contrast, cut-and-paste to another program is easy - but then you
really can't depend on whitespace or headers or other subtle things.

> A respond-to-request-pull wrapper you would use could be:
>
>  - Get the e-mail from the standard input;
>  - Pick up the signed bits and validate the signature;
>  - Perform the requested fetch; and
>  - Record the merge (or prepare .git/MERGE_MSG) with both the signed bits.

So is there any reason this couldn't be cut-and-paste? Make the signed
part small (*not* including diffstat and shortlog), and make it
whitespace-safe, and I wouldn't mind a tool at all.

If it *can* take the whole email, that would probably be a good design
(so that a "pipe email to command"  model would still work), but it
would be much better if it doesn't require it.

> and the "signed bits" could include:
>
>   - the repository and the branch you were expected to pull;
>   - the topic description.
>
> among other things the requestor can edit when request-pull message is
> prepared.

One thing I'd like is that it would also fire up an editor for the
merge, even if it gets the topic description from the email or
cut-and-paste. I often want to fix up peoples grammar etc. That's a
separate argument for trying to keep the signed part minimal - because
 I really don't want to have to maintain spelin errors just because
they are part of what was signed..

                  Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02 20:04                   ` Linus Torvalds
  2011-11-02 21:13                     ` Junio C Hamano
@ 2011-11-03  1:02                     ` Shawn Pearce
  2011-11-03  1:19                       ` Linus Torvalds
  2011-11-03  2:55                       ` Jeff King
  2011-11-03 18:29                     ` Junio C Hamano
  2 siblings, 2 replies; 101+ messages in thread
From: Shawn Pearce @ 2011-11-03  1:02 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Wed, Nov 2, 2011 at 13:04, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Tue, Nov 1, 2011 at 2:56 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> But on the other hand, in many ways, publishing your commit to the outside
>> world, not necessarily for getting pulled into the final destination
>> (i.e. your tree) but merely for other people to try it out, is the point
>> of no return (aka "don't rewind or rebase once you publish").  "pushing
>> out" might be less special than "please pull", but it still is special.
>
> So I really think that signing the top commit itself is fundamentally wrong.

I really disagree. I like the signed commit approach. It allows for a
lot more workflows than just providing a way for you to validate a
pull from a trusted lieutenant. Debian/Gentoo folks want a way to sign
every commit in their workflow. Just because you don't want that and
think its crazy doesn't mean its not a valid workflow for that
community and is something Git shouldn't support. I never use `git
stash`. I hate the damn command. Yet its still there. I just choose
not to use it. Junio's gpgsig header on each commit is also optional,
and communities/contributors can choose to use (or ignore) the feature
as they need to.

> That commit may not even be *yours*. You may have pulled it from a
> sub-lieutenant as a fast-forward, or similar. Amending it later would
> be actively very very *wrong*.

Obviously you shouldn't amend a commit that would otherwise be a
fast-forward. But why not write a new empty signed commit on top, and
teach `git log` without the verify signatures flag to skip over
commits that have a gpgsig header line, have exactly one parent, and
whose parent tree matches the commit's own tree? This removes these
commits from the normal `git log` revision output, but yet the flow of
changes is still very visible within the history.

As I understand it, the point of multiple Signed-off-by lines in
commit message bodies is to show the flow of a change, who reviewed
and applied a given commit, until it finally lands in a tree where its
commit SHA-1 is frozen in stone and you can later pull it. The empty
signed commit on top of a fast-forward provides that same flow of a
change, readily visible with standard `git log` tools, but doesn't
have to clutter up history if we teach log how to skip this particular
type. Similar to the --no-merges way to skip merges. :-)

> So quite frankly, I think the stuff in pu (or next?) is completely
> mis-designed. Doing it in the commit is wrong for fundamental reasons,
> which all boil down to a simple issue:

Totally disagree. I'm really in favor of embedding these into the
commit headers the way Junio has done.

>  - you absolutely *need* to add the signature later. You *cannot* do
> it at "git commit" time.

Why can't you add it at commit time? What is stopping me from running
`git commit -S` every time I make a commit? Is it that my fingers will
wear out more quickly because I have to type my pass-phrase too often?

What is wrong with making a signed commit on a commit I have a high
level of confidence in, but not signing the others? In my own workflow
I make a lot of commit --amends  / rebases until I am pretty confident
in the code being written and organized the way I think it should be
for distribution to others. But at some point in that workflow I'm
doing an --amend or a rebase to make that last final touch, and during
that commit I can add -S to make it signed, because I'm pretty certain
its ready to go. At that point, barring some horrific bug or reviewer
comments, I am unlikely to change the commit. I know at the time I
make that commit that I am pretty confident in the commit, so I take
the extra few key strokes to sign it.

> That's a fundamental issue both from a "workflow model" issue (ie you
> want to sign stuff after it has passed testing etc,

Why do I have to wait until its tested to sign it? The gpgsig
signature isn't any more special than the Signed-off-by line I put
into my commit message to agree to the developer's certificate of
origin, nor is it any more special than the committer line in the
commit header. Its just a statement on the commit that I have a
reasonable enough confidence in the value of this particular commit
and its ancestors that I should take the time to unlock my GPG key and
sign the content in case I do distribute this to others.

If you are going to spend time testing a commit, its probably going to
take longer to perform that testing than it is to perform the GPG key
unlock and signature. So why are you complaining about the time it
takes to sign something you think is worthy of testing?  If the tests
fail, you'll need to rewind/amend/whatever to address the breakage. If
the tests pass, the commit is already signed and ready for
distribution. If you are spending a lot of time signing commits that
are highly likely to fail tests, well, maybe you should look at other
ways to improve your workflow so that you have a higher level of
confidence in the code you record and assume will be a permanent part
of the project's history.

> but you may need
> to commit it in order to *get* testing),

Maybe consider allowing a ".dirty" suffix like git-core does on
builds? Or if you are submitting the code to a remote test cluster
that auto-compiles the code for you (and that is why you need a
commit), it sounds like the time it takes for that to push, compile,
test, and report back is way higher than the time it takes to make the
signature. So you probably should only be submitting something that
you had a reasonable level of confidence in. So you should go ahead
and sign it before sending it for testing, in case the tests do pass
and you want to publish that commit.

> as well as from a
> "fundamental git datastructures" issue (ie you would want to sign
> commits that aren't yours.

Sure. But this is why you can make an empty commit and sign that.

> "git commit --amend" is not the answer - that destroys the fundamental
> concept of history being immutable, and while it works for your local
> commits, it doesn't work for anybody elses commits, or for stuff you
> already pushed out.

Nobody said you had to amend everything. You can add an empty commit.

> And "add a fake empty commit just for the signature" is not the answer
> either - because that is clearly inferior to the tags we already had.

Really? I disagree. The commit DAG scales quite well. The tag
namespace does not. A refs/signatures/$COMMIT_SHA1 namespace also does
not scale well.

An empty commit with a gpgsig header has about the same object cost as
an annotated tag once packed. But it has the advantage that the damn
thing doesn't clog up the reference space, the reference handling
code, or the advertisements in the native protocol. As history goes
on, older signatures are less relevant, and automatically are
avoided/skipped/bypassed by the normal DAG walking code. Tags don't do
this well because they have no relationship to the project history.

The only downside to an empty commit with the gpgsig header is I
cannot grab an arbitrarily deep ancestor and say "Who has signed a
commit that depends on this"? Today we already have this with git
describe --contains (aka git name-rev) for annotated tags. Its a new
feature we have to teach to some part of the log machinery, but the
algorithm will be easier because it doesn't have to mess with the
mapping table of tag objects. It just has to start digging from roots,
remembering each commit that has a gpgsig on any given branch path,
and then outputting the matches when it finds the commit in question.

The commit approach also has the advantage that your tree
automatically carries any lieutenant's signatures, by virtue of them
already being frozen in the commits.  This allows anyone downstream of
you to verify the same signatures, and check them against their own
keyring contents. If the signatures are all detached in some transient
annotated tag space, its impossible for anyone other than you to
verify pull requests. I would hate to say we have this nice
distributed version control system, but only Linus can prove the pull
requests in his repository are what they claim, and we have to then
implicitly trust you to resign that data without the original
signatures being present. $DAY_JOB would feel a lot better about the
integrity of the Linux kernel repository if _ANYONE_ can validate pull
requests offline after they have happened.

> I dunno. Did I miss something? As far as I can tell, the signed tags
> that we've had since day one are *clearly* much better in very
> fundamental ways.

Completely disagree. :-)

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  1:02                     ` Shawn Pearce
@ 2011-11-03  1:19                       ` Linus Torvalds
  2011-11-03  1:45                         ` Linus Torvalds
  2011-11-03  2:55                       ` Jeff King
  1 sibling, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03  1:19 UTC (permalink / raw)
  To: Shawn Pearce
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Wed, Nov 2, 2011 at 6:02 PM, Shawn Pearce <spearce@spearce.org> wrote:
>>
>> So I really think that signing the top commit itself is fundamentally wrong.
>
> I really disagree. I like the signed commit approach.

If you like it so much, go ahead and use them.

But stop with the crazy excuses for the downsides. I explained exactly
why amending is stupid and wrong, and why empty commits are f*cking
moronic. But even apart from the *technical* problems with the stupid
mis-designed feature, I explained why it was fundamentally broken from
a workflow standpoint too.

I'm not saying that you shouldn't use them - go ahead and use the
feature if you like it. But please spare me your excuses for stupid
workarounds that come from the fact that they aren't a good match for
sane workflows.

                       Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  1:19                       ` Linus Torvalds
@ 2011-11-03  1:45                         ` Linus Torvalds
  2011-11-03  2:14                           ` Shawn Pearce
                                             ` (2 more replies)
  0 siblings, 3 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03  1:45 UTC (permalink / raw)
  To: Shawn Pearce
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Wed, Nov 2, 2011 at 6:19 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> I'm not saying that you shouldn't use them - go ahead and use the
> feature if you like it. But please spare me your excuses for stupid
> workarounds that come from the fact that they aren't a good match for
> sane workflows.

Btw, having now done odd things with signed tags (because we've used
them as a side-band verification mechanism), I can certainly also say
that the signed tags have their set of problems too.

So signed tags aren't perfect. They were designed for making releases,
and that shows very clearly in how git works with them. The default
choices that git makes are very awkward indeed when you use signed
tags as "security tokens".

But unlike the "sign the commit" approach, those are implementation
and UI issues, not "fundamentally broken design" issues.

For example, fetching a single signed tag with git is surprisingly
hard. It *shouldn't* be hard - and there's no underlying technical or
design reason why it would be hard, but it is. Why? Because all the
git actions when it comes to tags are all geared towards one
particular use, that is *not* about the signature checking aspect of
them.

Here's an example: Rusty Russell now makes nice signed tags for the
things he asks me to pull, and then states them in the pull message.
So he will mention that he has a tag named

   rusty@rustcorp.com.au-v3.1-8068-g5087a50

in his git repository at

   git://github.com/rustyrussell/linux.git

and while I don't think his tag names are all that wonderful, it makes
sense from an automated script kind of standpoint.

Now, let's try to get that tag:

  [torvalds@i5 linux]$ git fetch
git://github.com/rustyrussell/linux.git
rusty@rustcorp.com.au-v3.1-8068-g5087a50
  fatal: Couldn't find remote ref rusty@rustcorp.com.au-v3.1-8068-g5087a50

oops. Ok, so his tag naming is *really* akward. Whatever. Let's try again:

   [torvalds@i5 linux]$ git fetch
git://github.com/rustyrussell/linux.git
refs/tags/rusty@rustcorp.com.au-v3.1-8068-g5087a50
   From git://github.com/rustyrussell/linux
    * tag
rusty@rustcorp.com.au-v3.1-8068-g5087a50 -> FETCH_HEAD

Ahh, success!

Oops. Nope. It turns out that git will *peel* the tag when you fetch
it, so FETCH_HEAD actually doesn't contain the tag object at all, but
the commit object that the tag pointed to. MAJOR FAIL.

Quite frankly, I think that's a git bug, but it's a git bug because
"git fetch" was designed to get the commit to merge. Fair enough.
Let's work around it, and rename the tag at the same time:

   [torvalds@i5 linux]$ git fetch
git://github.com/rustyrussell/linux.git
refs/tags/rusty@rustcorp.com.au-v3.1-8068-g5087a50:refs/tags/rusty
   From git://github.com/rustyrussell/linux
    * [new tag]
rusty@rustcorp.com.au-v3.1-8068-g5087a50 -> rusty
    * [new tag]
rusty@rustcorp.com.au-v3.1-2-gb1e4d20 ->
rusty@rustcorp.com.au-v3.1-2-gb1e4d20
    * [new tag]
rusty@rustcorp.com.au-v3.1-4896-g0acf000 ->
rusty@rustcorp.com.au-v3.1-4896-g0acf000
    * [new tag]
rusty@rustcorp.com.au-v3.1-8068-g5087a50 ->
rusty@rustcorp.com.au-v3.1-8068-g5087a50

WTF? Now we finally *did* get the tag, and we can do

   git verify-tag rusty

and that will work. But what the hell happened? We got three other
tags too that we didn't even ask for!

So we have actual git bugs here, that relate to the fact that we've
treated signed tags specially, and have magic code to basically say
"if there's a signed tag that is reachable from the thing you pull,
and you're not just doing a temporary pull into FETCH_HEAD, we'll
fetch that signed tag too".

Again - not a fundamental design mistake in the data structures, and
it actually made sense from a "signed tags are important release
points" standpoint, but it makes it *really* inconvenient to use
signed tags for signature verification.

Also, the fact that the signed tag gets peeled when we do fetch into
FETCH_HEAD also means that we can't actually save the signature in
resulting the merge commit. The merge, instead of being able to
perhaps save the information that we merged a nice trusted signed
point, only has the commit.

But practically, all of these issues should be pretty easily solvable.
So it should be quite easy to make

    git pull <repo> <tag-name>

just do the right thing - including verifying the tag, and adding the
information in the tag into the merge commit message.

So signed tags are not mis-designed from a conceptual standpoint -
they just work really really awkwardly right now for what the kernel
would like to do with them.

With a few UI fixes, I think the signed tag thing would "just work".

That said, I do think that the "signature in the pull request" should
also "just work", and I'm not entirely sure which one is better. It
might be more convenient to get the signature data from the pull
request. So I'm not at all married the the notion of using signed tags
for this.

                       Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  1:45                         ` Linus Torvalds
@ 2011-11-03  2:14                           ` Shawn Pearce
  2011-11-03  2:25                             ` Linus Torvalds
  2011-11-03  2:31                             ` Linus Torvalds
  2011-11-03  2:19                           ` Linus Torvalds
  2011-11-03 18:16                           ` Junio C Hamano
  2 siblings, 2 replies; 101+ messages in thread
From: Shawn Pearce @ 2011-11-03  2:14 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Wed, Nov 2, 2011 at 18:45, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Wed, Nov 2, 2011 at 6:19 PM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>>
>> I'm not saying that you shouldn't use them - go ahead and use the
>> feature if you like it. But please spare me your excuses for stupid
>> workarounds that come from the fact that they aren't a good match for
>> sane workflows.

We often disagree. :-)

> Btw, having now done odd things with signed tags (because we've used
> them as a side-band verification mechanism), I can certainly also say
> that the signed tags have their set of problems too.
...
> But practically, all of these issues should be pretty easily solvable.
> So it should be quite easy to make
>
>    git pull <repo> <tag-name>
>
> just do the right thing - including verifying the tag, and adding the
> information in the tag into the merge commit message.

Uhm, sure.

Quoting you 2 days ago:

On Mon, Oct 31, 2011 at 15:52, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Oct 31, 2011 at 3:44 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> So nobody is worried about this (quoting from my earlier message)?
>
> No, because you haven't been reading what we write.
>
> The tag is useless.
>
> The information *in* the tag is not. But it shouldn't be saved in the
> tag (or note, or whatever). Because that's just an annoying place for
> it to be, with no upside.
>
> Save it in the commit we generate. BAM! Useful, readable, permanent,
> and independently verifiable.

So you propose we put the tag contents into the merge commit message
so it can be verified after the fact? So merges are now going to be
something much more horrific to read, because it will end with Git
object tag cruft, the tag message, and the PGP signature spew that no
human can decode in the head?

Oh, right, tags are almost good enough. Elsewhere in this thread you
also stated we have to redo the way tags are signed so that the tag
message body itself is not part of the signature, allowing you to fix
spelin errors so you are not stuck with them in your commit history.
But I assume we will have to keep the more typical headers of object /
type / tag / tagger fields, as that is the key information the
signature needs to be over to be of any value. So now there will be
two different ways in which a Git annotated tag object will have its
signature created, as certainly you don't mean to remove the tag
message body from the PGP signature content for release tags.

I fail to see how shoving Git object data fields and a complete PGP
signature block into a merge commit message body, which will show by
default in all git log type tools, and exist in cherry-picks or
rebases that might make that data less valuable, is somehow better
than the gpgsig header that neatly tucks it away until requested. I
also fail to see how scraping the message body for the proper fields
in order to implement automated verification of the signature (because
no human can do it themselves and copy-paste sucks) is a good idea.
Everywhere else in Git that we have machine readable formats its very
well structured so that no guessing is required.

> So signed tags are not mis-designed from a conceptual standpoint -
> they just work really really awkwardly right now for what the kernel
> would like to do with them.
>
> With a few UI fixes, I think the signed tag thing would "just work".

Well, UI fixes, protocol changes, improvements to manage a large
reference space which we have previously said is an insane and stupid
workflow, etc. One reason you picked up all of those extra tags was
the include-tag capability kicking on and picking up older tag
history. We now have to disable it in certain cases.

Its not just a few UI fixes. And there is a lot more work to write a
verify for the tag contents+signature that appears in the body of a
merge commit message. Not to mention we now have to do that verify
logic twice, once in the signed pull request tag like but not quite a
tag but uses a tag thing you are advocating, and again for the merge
commit message body that contains the tag object data that we don't
normally show to an end user, but will now be in every merge commit
you make.

Go ahead and call me stupid, but this already is a bigger amount of
surgery to the git-core code, not to mention worse user experience for
the average `git log` reading human, than having a hidden by default
gpgsig header that might ask a contributor to take 2 extra seconds
before making a commit to consider the useful lifespan of that commit.
Or $DEITY forbid, write a new empty commit to record the equivalent of
their Signed-off-by.

Oh, and while I am on that subject...


<rant>
I have never grasped why sometimes a Signed-off-by is added to a
patch, and why sometimes its not. It seems to be this weird function
of "If the commit SHA-1 is already stable DON'T FUCKING TOUCH IT BY
ADDING SIGNED-OFF-BY IT RUINS THE HISTORY", but if you are too far
down the food chain to be fortunate enough for your commit SHA-1 to
remain frozen, the Signed-off-by has to be added to assert that the
code can be contributed. It sounds like the workflow developed around
where it wasn't acceptable to force history rewriting, you suffer by
not having the SOB, but whenever possible you force a history rewrite
on the contributor just so you can add a SOB and feel good about the
fact that the SOB is added to the commit message.

Get over it. Add the fucking empty commit to show the flow of a
change. Stop forcing every fucking contributor to rebase/rewrite his
commits just so someone higher up in the food chain can wank with
their SOB line.

Everyone I talk to that contributes code to the kernel who isn't Linus
or Ted Tso complains about this, and then asks me to fucking fix it.
They want stable SHA-1s so they know their change arrived into Linus'
tree unmolested. Unfortunately, despite their volume of changes, they
aren't high enough in the food chain to be this lucky. Nope, someone
has to wank their SOB in first. And maybe fix a spelin error.
</rant>

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  1:45                         ` Linus Torvalds
  2011-11-03  2:14                           ` Shawn Pearce
@ 2011-11-03  2:19                           ` Linus Torvalds
  2011-11-04 20:16                             ` Junio C Hamano
  2011-11-03 18:16                           ` Junio C Hamano
  2 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03  2:19 UTC (permalink / raw)
  To: Shawn Pearce
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

[-- Attachment #1: Type: text/plain, Size: 1275 bytes --]

On Wed, Nov 2, 2011 at 6:45 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
>   [torvalds@i5 linux]$ git fetch git://github.com/rustyrussell/linux.git  refs/tags/rusty@rustcorp.com.au-v3.1-8068-g5087a50

So this trivial patch removes one line of code, and makes this actually work.

However, it also makes us fail many tests that *test* that we peeled
what we fetched. However, I think the tests are wrong.

If the tag doesn't resolve into a commit, we happily output the SHA1
of the tag itself - and we say that it shouldn't be merged.

And it the tag *does* resolve into a commit, why would we output the
SHA1 of the commit? The tag should be peeled properly later when it
gets used, so peeling it here seems to be just a misfeature that makes
signed tags not work well.

So I suspect we should just apply this patch, but I didn't check
exacty what the failed tests are - except for the first one, that just
compares against a canned response (and the canned response should
just be changed). Maybe there was some reason for the peeling,
although I suspect it was just a fairly mindless case of "make it a
commit, because the merge needs the commit" - never mind that the
merge would peel it anyway.

                           Linus

[-- Attachment #2: patch.diff --]
[-- Type: text/x-patch, Size: 550 bytes --]

 builtin/fetch.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 91731b909aeb..494a7f9976f8 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -436,8 +436,7 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
 		}
 		note[note_len] = '\0';
 		fprintf(fp, "%s\t%s\t%s",
-			sha1_to_hex(commit ? commit->object.sha1 :
-				    rm->old_sha1),
+			sha1_to_hex(rm->old_sha1),
 			rm->merge ? "" : "not-for-merge",
 			note);
 		for (i = 0; i < url_len; ++i)

^ permalink raw reply related	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  2:14                           ` Shawn Pearce
@ 2011-11-03  2:25                             ` Linus Torvalds
  2011-11-03  3:22                               ` Jochen Striepe
  2011-11-03  2:31                             ` Linus Torvalds
  1 sibling, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03  2:25 UTC (permalink / raw)
  To: Shawn Pearce
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Wed, Nov 2, 2011 at 7:14 PM, Shawn Pearce <spearce@spearce.org> wrote:
>
> So you propose we put the tag contents into the merge commit message
> so it can be verified after the fact? So merges are now going to be
> something much more horrific to read, because it will end with Git
> object tag cruft, the tag message, and the PGP signature spew that no
> human can decode in the head?

Actually, I wanted to just drop the damn thing.

To me, the point of the tag is so that the person doing the merge can
verify that he merges something trusted.

However, everybody else seems to disagree, and wants that stupid
signature to live along in the repository. And I can live with that,
although I do agree with you that it's not exactly pretty. I can live
with "ugly signature that I don't care for" way more than "stupid
design".

Because unlike your crazy empty commit, it at least fits the workflow,
and it certainly isn't any uglier that extraneous pointless commit.

You can disagree. You obviously do. I simply don't care. Because I'm right.

(And your claim that it's big UI fixes and protocol changes is pure
and utter garbage. I just sent a patch that cleans the code up,
removes a line that improperly drops information and gets rid of the
biggest problem with our current handling of tags. No protocol changes
involved, no big UI fixup).

                        Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  2:14                           ` Shawn Pearce
  2011-11-03  2:25                             ` Linus Torvalds
@ 2011-11-03  2:31                             ` Linus Torvalds
  1 sibling, 0 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03  2:31 UTC (permalink / raw)
  To: Shawn Pearce
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Wed, Nov 2, 2011 at 7:14 PM, Shawn Pearce <spearce@spearce.org> wrote:
>
> <rant>

I'm answering this separately, because it's a separate rant.

It's also totally bogus, but whatever.

> Get over it. Add the fucking empty commit to show the flow of a
> change. Stop forcing every fucking contributor to rebase/rewrite his
> commits just so someone higher up in the food chain can wank with
> their SOB line.

Shawn, stop using whatever drugs you are using.

NOBODY EVER REBASES ANYTHING FOR SIGNED-OFF-BY.

If they do, they are doing things very very wrong.

Signed-off-by: is *purely* for sending patches by email. No git
operations involved. None. Nada. Zilch. No rebasing involved, because
there's not even a git repository involved, for chissake!

Once something is in git, it's not signed off on - there should be a
sign-off-chain from the author to the committer, and that's it.
Anything else would be crazy.

So stop the crazy rants. Stop with the bad drugs. Seriously. You're
acting crazy.

                          Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  1:02                     ` Shawn Pearce
  2011-11-03  1:19                       ` Linus Torvalds
@ 2011-11-03  2:55                       ` Jeff King
  2011-11-03  3:16                         ` Robin H. Johnson
  1 sibling, 1 reply; 101+ messages in thread
From: Jeff King @ 2011-11-03  2:55 UTC (permalink / raw)
  To: Shawn Pearce
  Cc: Linus Torvalds, Junio C Hamano, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Wed, Nov 02, 2011 at 06:02:37PM -0700, Shawn O. Pearce wrote:

> > So I really think that signing the top commit itself is fundamentally wrong.
> 
> I really disagree. I like the signed commit approach. It allows for a
> lot more workflows than just providing a way for you to validate a
> pull from a trusted lieutenant. Debian/Gentoo folks want a way to sign
> every commit in their workflow. Just because you don't want that and
> think its crazy doesn't mean its not a valid workflow for that
> community and is something Git shouldn't support. I never use `git
> stash`. I hate the damn command. Yet its still there. I just choose
> not to use it. Junio's gpgsig header on each commit is also optional,
> and communities/contributors can choose to use (or ignore) the feature
> as they need to.

Stop for a minute and think about what it _means_ to sign a commit. Is
it saying "I wrote this commit?" Or "I think this commit is good?" Or "I
think all of the history leading to this is good?" It's obviously going
to be a per-project thing, but it's very constricting.  Leaving aside
all of the workflow issues Linus brought up (but which I do agree with),
think about what it would mean for Linus to fetch a commit from a
lieutenant and then sign it. Whatever it means, it can really only be
_one_ thing.

But big projects that are interested in signatures probably want to say
more. They want to say "this developer really wrote this commit". They
want to say "QA passed this commit". They want to say "the history up to
here looks good". And so on.

But they can't say those things without binding some data to the commit
(i.e., making a certificate saying "this commit passed QA").  Data which
might only make sense to assert much later than the commit is written.

So you're going to need to support detached commit signatures in some
form anyway to make everybody happy. Which isn't to say in-commit
signatures are wrong, but they are just one tool in a toolbox.

Personally, I think the only thing that makes sense to assert inside a
commit itself is that you are the author, and the author line of the key
should match the email UID of the signing key. And then anything you
want to say about _other_ people's commits (or even your own commits,
but later) should come in the form of detached signatures with some
content.

That's how signed tags work. It's not just Linus signing a commit. It's
Linus signing a binding between a commit and the statement "this is
v2.6.28". The only thing wrong with the signed tag model for more
general use is that you need some way of naming and organizing large
numbers of tags (e.g., several per commit if you have things like QA
signatures).

-Peff

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  2:55                       ` Jeff King
@ 2011-11-03  3:16                         ` Robin H. Johnson
  0 siblings, 0 replies; 101+ messages in thread
From: Robin H. Johnson @ 2011-11-03  3:16 UTC (permalink / raw)
  To: Jeff King, Git Mailing List
  Cc: Shawn Pearce, Linus Torvalds, Junio C Hamano, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Wed, Nov 02, 2011 at 10:55:32PM -0400,  Jeff King wrote:
> But big projects that are interested in signatures probably want to say
> more. They want to say "this developer really wrote this commit". They
> want to say "QA passed this commit". They want to say "the history up to
> here looks good". And so on.
On the Gentoo side, we've also pondered the question of:
author != committer != pusher
And how to preserve many signatures from sources.

We're on a central repo model, with some ~250 committers.

I was originally primarily after the push certificates/signed-push, and
recording that data in the notes, but that still has the problems of
third-party verification as mentioned in the thread.

If we require that the tip of every push is a signed commit via a hook,
we get knowledge of the pushers. Either your real commit itself is
signed, or you have a signed merge commit on top, or you have a signed
empty commit. In all of the cases, I can verify your signature at the
recv hook. Having signed push in this case has a benefit that you could
ship the data as a bundle, or async from the signing.

The QA value of multiple signatures per commit is also valuable, to
assert SOB WITHOUT altering the commit. I see spearce's rant and the
retort, and really think there needs to be a middle ground - some of
commits that are coming from pulls, and not getting additional SOB,
could really benefit from them being recorded (I see them on mailing
lists, but not introduced since that would break 'stable' IDs).

> But they can't say those things without binding some data to the commit
> (i.e., making a certificate saying "this commit passed QA").  Data which
> might only make sense to assert much later than the commit is written.
> 
> So you're going to need to support detached commit signatures in some
> form anyway to make everybody happy. Which isn't to say in-commit
> signatures are wrong, but they are just one tool in a toolbox.
I was proposing that Git supports _all_ of these models:
- signed commits
- signed pushes (via certs)
- whatever signed lightweight tag idea happens
- existing annotated tags

Choices. Each with their own costs and advantages.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  2:25                             ` Linus Torvalds
@ 2011-11-03  3:22                               ` Jochen Striepe
  2011-11-03  4:13                                 ` Linus Torvalds
  0 siblings, 1 reply; 101+ messages in thread
From: Jochen Striepe @ 2011-11-03  3:22 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Shawn Pearce, Junio C Hamano, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

	Hi,

On Wed, Nov 02, 2011 at 07:25:17PM -0700, Linus Torvalds wrote:
> To me, the point of the tag is so that the person doing the merge can
> verify that he merges something trusted.
> 
> However, everybody else seems to disagree, and wants that stupid
> signature to live along in the repository.

It seems quite useless and leading to false conclusions in several cases
where the merger's gpg output differs from someone's checking later on,
e.g. when

 - the signing key has been revoked in the mean time (for whatever
   reasons)
 - the signing key has expired
 - the public part of the signing key is not available for the general
   public.

AFAIK gpg just gives you an error code and a message like e.g. "Key has
expired" without stating if the key was valid _when signing the commit_.

How do you plan to handle this when keeping the signature in the
repository? Or am I overlooking something?


Thanks,
Jochen.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  3:22                               ` Jochen Striepe
@ 2011-11-03  4:13                                 ` Linus Torvalds
  2011-11-10 13:51                                   ` David Woodhouse
  0 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03  4:13 UTC (permalink / raw)
  To: Jochen Striepe
  Cc: Shawn Pearce, Junio C Hamano, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Wed, Nov 2, 2011 at 8:22 PM, Jochen Striepe <jochen@tolot.escape.de> wrote:
>
> It seems quite useless and leading to false conclusions in several cases
> where the merger's gpg output differs from someone's checking later on,
> e.g. when
>
>  - the signing key has been revoked in the mean time (for whatever
>   reasons)
>  - the signing key has expired
>  - the public part of the signing key is not available for the general
>   public.

So I don't think those are *big* issues. Sure, you'd want the public
key to be public for it to make any real sense to save, but on the
other hand, they *are* generally public. Yes, yes, you might have keys
that are only used - and only made public - within some particular
organization, but in that case the source code that gets signed with
those keys would tend to be private to that organization too, so..

And yes, keys get revoked or they expire, but that's still a pretty
rare event, so it doesn't really invalidate the argument that making
the original signed content available can quite often be useful - even
if it's not guaranteed to *always* be useful.

No, my main objection to saving the data is that it's ugly and it's
redundant. Sure, in practice you can check the signatures later fine
(with the rare exceptions you mention), but even when you can do it,
what's the big upside?

And there are much bigger real downsides, imho.

For example, let's say that we do eventually end up switching from
SHA1 to SHA256 in git, and we do a full re-import of the tree. Guess
what? All those signatures are now just so much garbage. Sure, you can
recreate them (create some trusted script that you agree does a 1:1
transform, and re-sign everything), but in practice you can't ever
really do that - because all those things are tied to the tree, so you
need to have *everybodys* private keys in one place to do so. And the
people who signed things initially would have to be insane to allow
that.

So I'm actually of the opinion that "internal signatures" are bad
design at a rather fundamental level.

In contrast, the "external signed tags" are fine: it's not just that
there are much fewer of them, it's that they are *independent*. So you
can easily re-generate the signed tags, because each signer can
*individually* decide to validate the newly converted tree, and sign
off on the fact that the conversion was done identically using new
external tags with signatures.

This was one of the reasons I made the signed tags work the way they
do. And it wasn't because I was extremely far-sighted and thought of
all the problems that internal signatures have - it's because monotone
had their internal signatures, and every other email on the monotone
list was about all the problems it caused.

> AFAIK gpg just gives you an error code and a message like e.g. "Key has
> expired" without stating if the key was valid _when signing the commit_.
>
> How do you plan to handle this when keeping the signature in the
> repository? Or am I overlooking something?

So see above - I just wouldn't worry about it. The possible few cases
where it would occur are dwarfed by the cases where it *doesn't*
occur, and those are the ones I'd concentrate on. They are the ones
that need to be important enough that it's even worth carrying the
random noise around.

Are they?

So I do think that there are real upsides at the *process* level where
you can use the signatures to verify that what is pulled is pulled
from the person you thought it was. I don't think anybody disputes
those advantages. But outside of that I think it gets very gray, and
there real disadvantages.

That said, I don't care *that* much. I don't mind polluting the merge
commits with information that I don't think is really worth it. So I'd
be willing to carry the signature information around, although I'd
hope to minimize it and have some sane way to hide it.

            Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  1:45                         ` Linus Torvalds
  2011-11-03  2:14                           ` Shawn Pearce
  2011-11-03  2:19                           ` Linus Torvalds
@ 2011-11-03 18:16                           ` Junio C Hamano
  2011-11-03 18:52                             ` Junio C Hamano
                                               ` (2 more replies)
  2 siblings, 3 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-11-03 18:16 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

>   [torvalds@i5 linux]$ git fetch
> git://github.com/rustyrussell/linux.git
> rusty@rustcorp.com.au-v3.1-8068-g5087a50
>   fatal: Couldn't find remote ref rusty@rustcorp.com.au-v3.1-8068-g5087a50
>
> oops. Ok, so his tag naming is *really* akward. Whatever.

It is not "Whatever".

 $ git fetch git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v3.0
 fatal: Couldn't find remote ref v3.0

I do not think we ever DWIMmed fetch refspecs to prefix refs/tags/, so it
is not the naming but fetching tags without saying "git fetch tag v3.0"
(which IIRC was your invention long time ago). 

If we changed this "git fetch $there v3.0" to fetch tag, it would help the
final step in your illustration, and I do not think it would be a huge
regression---the only case it becomes fuzzy is when they have v3.0 branch
at the same time, but the owner of such a repository is already playing
with fire.

>    [torvalds@i5 linux]$ git fetch
> git://github.com/rustyrussell/linux.git
> refs/tags/rusty@rustcorp.com.au-v3.1-8068-g5087a50
>    From git://github.com/rustyrussell/linux
>     * tag
> rusty@rustcorp.com.au-v3.1-8068-g5087a50 -> FETCH_HEAD
>
> Ahh, success!
>
> Oops. Nope. It turns out that git will *peel* the tag when you fetch
> it, so FETCH_HEAD actually doesn't contain the tag object at all, but
> the commit object that the tag pointed to. MAJOR FAIL.
>
> Quite frankly, I think that's a git bug, but it's a git bug because
> "git fetch" was designed to get the commit to merge. Fair enough.

And because FETCH_HEAD started as (and probably still is) an internal
implementation detail of communication between fetch and merge inside
pull. So I do not have any issue in changing it to store tags unpeeled
there.
>    [torvalds@i5 linux]$ git fetch
> git://github.com/rustyrussell/linux.git
> refs/tags/rusty@rustcorp.com.au-v3.1-8068-g5087a50:refs/tags/rusty
>    From git://github.com/rustyrussell/linux
>     * [new tag]
> rusty@rustcorp.com.au-v3.1-8068-g5087a50 -> rusty
>     * [new tag]
> rusty@rustcorp.com.au-v3.1-2-gb1e4d20 ->
> rusty@rustcorp.com.au-v3.1-2-gb1e4d20
>     * [new tag]
> rusty@rustcorp.com.au-v3.1-4896-g0acf000 ->
> rusty@rustcorp.com.au-v3.1-4896-g0acf000
>     * [new tag]
> rusty@rustcorp.com.au-v3.1-8068-g5087a50 ->
> rusty@rustcorp.com.au-v3.1-8068-g5087a50
>
> WTF?

This is not WTF but "fetching a history to store the tip of it in your
refs/ namespace causes tags pointing into the history line followed
automatically", and it exactly is what you want to happen if rusty asked
you to fetch his for-linus branch (which the tag may point at) instead.

> We got three other
> tags too that we didn't even ask for!

We could change the rule to read "fetching a history to store the tip of it
in your refs/heads namespace causes autofollow". I am not sure if that is
what we really want, though.

> Again - not a fundamental design mistake in the data structures, and
> it actually made sense from a "signed tags are important release
> points" standpoint, but it makes it *really* inconvenient to use
> signed tags for signature verification.

We could update three things:

 - DWIM $name in "git fetch $there $name" to refs/tags/$name when it makes
   sense;
 - FETCH_HEAD stores unpeeled object names; and
 - "git pull" learns --verify option.

Then

 $ git pull --verify rusty rusty@rustcorp.com.au-v3.1-8068-g5087a50

could integrate the history leading to that tag to your current branch
while running verify-tag on it.

For this, disabling the tag-auto-following is not necessary, as you are
not storing the retrieved tag anywhere.

That is a longwinded way to say I agree what you said below.

> So signed tags are not mis-designed from a conceptual standpoint -
> they just work really really awkwardly right now for what the kernel
> would like to do with them.
>
> With a few UI fixes, I think the signed tag thing would "just work".
>
> That said, I do think that the "signature in the pull request" should
> also "just work", and I'm not entirely sure which one is better.

I do not think it is necessarily either/or choice.

Either way does not solve anything other than validating the last hop
between the last lieutenant to the integrator without having a way to give
the verification material to third parties.

Your earlier "pull request signature could be copied into the message of
the merge that integrates the pulled history" solves 90% of the "third
party validation" issue.

With the signed tags approach, you could push out these signed tags you
get from lieutenants, but there are quite a few things that need to happen
for it to be usable:

 - You or your lieutenants do not want to keep these tags in your working
   repository, to be listed in "git tag -l". They are ephemeral to you and
   your lieutenant, even though they have to be permanent for third
   party auditors.

 - Normal users of your project do not want to see them in "git tag -l"
   either.

 - Responses to "git fetch" and "git ls-remote" produced by "git
   upload-pack" do need to (optionally) include them to allow third party
   auditors to ask for them.

I wonder if an approach like the following, in addition to the three
things I listed above, may give us a workable solution:

 * "git fetch linus v3.0" called by "git pull --verify linus v3.0" fetches
   the v3.0 unpeeled into FETCH_HEAD, GPG verifies it, creates
   refs/audit/$u, before running "git merge". $u is derived from v3.0
   (given tag), the identity of the GPG signer, and perhaps timestamp to
   make it both identifiable and unique under refs/audit/ hierarchy.

 * You "git push origin". This causes refs/audit/* refs that point at
   commits in the transferred history to auto-follow, just like the
   current "git fetch $there $src:$dst" causes refs/tags/* auto-follow.
   The refs/audit/* hierarchy in your public repository will be populated
   by lieutenant signatures.

 * (Optional) You may have signed "git tag -s 'Linux v3.2' v3.2 master"
   before you push origin out, or you may have not. Currently, you do have
   to "git push origin v3.2" separately if you did. The above auto-follow
   could be extended to push refs/tags/* hierarchy to eliminate this step
   as well.

Note that because of the way "upload-pack" protocol is structured, the
first response from "upload-pack" after it gets connection is the
advertisement of refs, and there is no way for "fetch-pack" to ask for
customized refs advertisement to it. So for this to work without incurring
undue overhead for normal users, we would need to exclude refs/audit/*
from the normal ref advertisement (i.e. "ls-remote" does not see it) so
that "git fetch" by casual users will not have to wait for megabytes of
ref advertisements before issuing its first "want" request. Probably we
can change "upload-pack" to advertise only refs/heads/*, refs/tags/*, and
HEAD by default, and a protocol extension could be added to ask for other
hierarchies for specialized needs like third party auditors.

BUT.

This does not allow third party auditors to audit how sub-subsystem
histories came into your lieutenants' history unless you also fetch from
your lieutenants in "auditor" mode to retrieve their refs/audit/* refs to
be propagated to your public repository, which all of us involved in this
thread know you wouldn't bother if it is an additional manual step (and I
personally do not think I would bother if I were you).

So the audit trail will end at one level unless we have even more complex
arrangements. The auditors know the history up to some point in the past
came from you (your last signed tag at release time, which some people may
feel a bit too sparse for auditing purposes when a security incident like
that one happens in between releases), and they know subhistories of what
you merged came from your direct lieutenants (the refs/audit/* tags the
above change allowed you to forward automatically when you published), but
they have to take the word of your direct lieutenants at face value.

I do not know if that is acceptable for $DAYJOB types, though.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-02 20:04                   ` Linus Torvalds
  2011-11-02 21:13                     ` Junio C Hamano
  2011-11-03  1:02                     ` Shawn Pearce
@ 2011-11-03 18:29                     ` Junio C Hamano
  2 siblings, 0 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-11-03 18:29 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Tue, Nov 1, 2011 at 2:56 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> But on the other hand, in many ways, publishing your commit to the outside
>> world, not necessarily for getting pulled into the final destination
>> (i.e. your tree) but merely for other people to try it out, is the point
>> of no return (aka "don't rewind or rebase once you publish"). "pushing
>> out" might be less special than "please pull", but it still is special.
>
> So I really think that signing the top commit itself is fundamentally wrong.

It merely is a stronger form of the "committer" line in the commit
object. A random repository at Github anybody can create repositories at
can serve you a random commit with any random name on "committer" line,
and the new gpgsig header is a way to let the committer certify it
genuinely is from the committer.

I do not think for that purpose, in-commit signature is fundamentally
wrong. I was hoping it would be more useful than it turned out to be, but
I agree that it just is not suitable as a vehicle to convey "I made that
commit some time ago, and now I want you to pull it for such and such
reasons" in a larger workflow.

The "now I want you to pull it for such and such reasons" part is the pull
request, and if we are to protect them with GPG signatures, and perhaps
copy the signed part in the resulting merge, don't we have a reasonable
solution, without all the downsides the signed tag approach would cause if
we wanted to allow third party auditors to have access to the signatures
for independent auditing purposes (described in a separate message)?

Perhaps what is causing the problem is the desire to allow third party
auditors finer grained audit trail, but after having heard that $DAYJOB
folks went through each and every commit after known release points with
fine-toothed comb, I am not brave/rude/blunt enough to dismiss it as
unimportant.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03 18:16                           ` Junio C Hamano
@ 2011-11-03 18:52                             ` Junio C Hamano
  2011-11-03 19:09                               ` Linus Torvalds
  2011-11-03 19:06                             ` Linus Torvalds
  2011-11-04 21:12                             ` Junio C Hamano
  2 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-03 18:52 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

Junio C Hamano <gitster@pobox.com> writes:

> BUT.

Ahh, sorry for the noise. I realize that we already have a winner, namely,
the proposal outlined in your message I was responding to.

It just didn't click to me that you were replacing "signed material from
pull request copied into the merge" with "contents of signed tag copied
into the merge".

So forget everything I said in the later parts of my response that talks
about refs/audit/*, and the other message except for gpgsig header being a
stronger form of existing committer line.




^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03 18:16                           ` Junio C Hamano
  2011-11-03 18:52                             ` Junio C Hamano
@ 2011-11-03 19:06                             ` Linus Torvalds
  2011-11-04 21:12                             ` Junio C Hamano
  2 siblings, 0 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03 19:06 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Thu, Nov 3, 2011 at 11:16 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> It is not "Whatever".
>
>  $ git fetch git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v3.0
>  fatal: Couldn't find remote ref v3.0
>
> I do not think we ever DWIMmed fetch refspecs to prefix refs/tags/, so it
> is not the naming but fetching tags without saying "git fetch tag v3.0"
> (which IIRC was your invention long time ago).

Ahh. Yeah, and not DWIM'ing tags is probably ok. I'd completely
forgotten about the special "tag" shortcut.

Which probably means it was a bad ui decision to begin with. But once
more, the UI is clearly designed for fetching the tags into your own
tag-space (ie it does "refs/tags/<tag>:refs/tags/<tag>") rather than
fetching the tag just for verification.

> If we changed this "git fetch $there v3.0" to fetch tag, it would help the
> final step in your illustration, and I do not think it would be a huge
> regression---the only case it becomes fuzzy is when they have v3.0 branch
> at the same time, but the owner of such a repository is already playing
> with fire.

Yeah, extending DWIM for remote repos to do the same thing it does for
local repositories is probably the right thing regardless of any other
issues.

We already have the "tag and branch with the same name" issue for
local repositories, and we have perfectly good disambiguation rules
for when disambiguation is necessary. Making the DWIM rules be the
same for a remote case sounds sane.

That said, I don't think it's a big deal either. I was just confused
by the expansion being different, but having to have the refs/tags/
there isn't a dealbreaker by any means.

>> Quite frankly, I think that's a git bug, but it's a git bug because
>> "git fetch" was designed to get the commit to merge. Fair enough.
>
> And because FETCH_HEAD started as (and probably still is) an internal
> implementation detail of communication between fetch and merge inside
> pull.

Well, I certainly don't consider it to be just "an implementation
detail" personally. I use FETCH_HEAD all the time (the same way I use
ORIG_HEAD and just plain HEAD). It's very useful for "fetch and check
what they have", when you want to look at something but you don't want
all the remote tags and crud. So I consider it a honest-to-goodness
real user feature.

>So I do not have any issue in changing it to store tags unpeeled there.

In fact, storing the peeled was really surprising to me, especially
since it actually *says* "tag" in the .git/FETCH_HEAD file. So the
.git/FETCH_HEAD file really currently ends up being actively wrogn and
misleading for tags we fetch: it looks something like

  <sha-of-commit>  tag '<tagname>'  of <reponame>

and says it is a tag, but the SHA1 is of the peeled commit. That's
just crazy, and actually made me think the other end (Rusty, in this
case) had done something wrong initially (ie I quite reasonably - I
thought - blamed it on Rusty using a non-signed tag).

>> WTF?
>
> This is not WTF but "fetching a history to store the tip of it in your
> refs/ namespace causes tags pointing into the history line followed
> automatically", and it exactly is what you want to happen if rusty asked
> you to fetch his for-linus branch (which the tag may point at) instead.

Well, yes and no. But mostly no.

If I just fetch his for-linus branch, I don't get (and I don't want)
his tags. It's only because I fetched it into my ref-space.

And I only fetched it into my ref-space, because otherwise the crazy
git peeling happened if I don't do that.

So I didn't want those other tags, and I really normally wouldn't have
gotten them. Only because I had to do that odd work-around to avoid
the peeling did I get it, because then the totally unrelated logic of
"ok, get the tags too" triggered.

So it's a WTF, because this work-around ends up having the special
side effects - and they make sense when you *really* fetch his branch
and make it part of your name-space, but not when you only did the
"part of my namespace" as a workaround for another git issue.

Obviously, you can use "-n" (--no-tags) to fetch the tag, and that
actually fixes the issue, but that is it's own kind of WTF too: in
order to fetch just *one* tag, you have to specify that you don't want
tags? Not exactly a greatly intuitive use case ;)

Anyway, the one-line rpatch I sent basically avoids all these WTF
moments, by just making "git fetch <repo> <tagname>" work (apart from
the DWIMmery on the tag-name, but that's a totally independent small
detail that doesn't really matter)

>> We got three other
>> tags too that we didn't even ask for!
>
> We could change the rule to read "fetching a history to store the tip of it
> in your refs/heads namespace causes autofollow". I am not sure if that is
> what we really want, though.

No, I think the current "follow tags" rule is fine. It's just that it
didn't really mesh well with "damn, I have to work around this other
git issue".

> We could update three things:
>
>  - DWIM $name in "git fetch $there $name" to refs/tags/$name when it makes
>   sense;
>  - FETCH_HEAD stores unpeeled object names; and
>  - "git pull" learns --verify option.

Yes. I think that would indeed solve everything.

> Then
>
>  $ git pull --verify rusty rusty@rustcorp.com.au-v3.1-8068-g5087a50
>
> could integrate the history leading to that tag to your current branch
> while running verify-tag on it.

Agreed. The only remaining issue then would be how that "yes, I
verified the tag" part would be actually saved for posterity. My
suggestion would be to to just punt that question, and let the user
decide, by simply:

 - start the editor by default with "--verify"

 - output the "gpg --verify" result into the end of the commit file,
along with the tag content (which has the original pgp signature, of
course).

 - let the user decide what part of it he wants to use.

In particular, the "gpg --verify" result may well be something that
the user wants to *act* on - maybe the key didn't exist in the key
ring, or maybe it does exist but doesn't have quite enough trust and
gpg complains about that etc etc. But that's all something that "start
the editor and show the user what is up" would let the user decide on.

> For this, disabling the tag-auto-following is not necessary, as you are
> not storing the retrieved tag anywhere.

Exactly,

>> That said, I do think that the "signature in the pull request" should
>> also "just work", and I'm not entirely sure which one is better.
>
> I do not think it is necessarily either/or choice.

No, I think we can do both, and it actually ends up being just a
matter of convenience which one a particular project ends up using (or
even use both depending on preferences of particular sub-lieutenants
within the project).

> I wonder if an approach like the following, in addition to the three
> things I listed above, may give us a workable solution:
>
>  * "git fetch linus v3.0" called by "git pull --verify linus v3.0" fetches
>   the v3.0 unpeeled into FETCH_HEAD, GPG verifies it, creates
>   refs/audit/$u, before running "git merge". $u is derived from v3.0
>   (given tag), the identity of the GPG signer, and perhaps timestamp to
>   make it both identifiable and unique under refs/audit/ hierarchy.

So far so good, but see above: it may turn out that the user will
*re-verify* the key after having done some gpg action. So..

>  * You "git push origin". This causes refs/audit/* refs that point at
>   commits in the transferred history to auto-follow, just like the
>   current "git fetch $there $src:$dst" causes refs/tags/* auto-follow.
>   The refs/audit/* hierarchy in your public repository will be populated
>   by lieutenant signatures.

So I don't think auto-follow is good here.

I could *easily* see various companies using this for their own
internal audit, without really wanting to expose things outside of the
company. So auto-following sounds like the wrong approach. Make it an
explicit "expose audit checks" thing.

>  * (Optional) You may have signed "git tag -s 'Linux v3.2' v3.2 master"
>   before you push origin out, or you may have not. Currently, you do have
>   to "git push origin v3.2" separately if you did. The above auto-follow
>   could be extended to push refs/tags/* hierarchy to eliminate this step
>   as well.

So far I haven't really had any issues with having to do a "git push
--tags" to push things out.

That said, maybe the auto-push could just be a per-repo option, and
then you can have it both ways.

> Note that because of the way "upload-pack" protocol is structured, the
> first response from "upload-pack" after it gets connection is the
> advertisement of refs, and there is no way for "fetch-pack" to ask for
> customized refs advertisement to it. So for this to work without incurring
> undue overhead for normal users, we would need to exclude refs/audit/*
> from the normal ref advertisement (i.e. "ls-remote" does not see it) so
> that "git fetch" by casual users will not have to wait for megabytes of
> ref advertisements before issuing its first "want" request.

I think that would be a good thing, and make it much more palatable.
After all, th elikelihood is that *nobody* will ever care about the
audit cases at all. They are very much a "..but what if xyz happens"
kind of safety net for the extreme badness, not anything you'd expect
to use.

                         Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03 18:52                             ` Junio C Hamano
@ 2011-11-03 19:09                               ` Linus Torvalds
  2011-11-04 14:59                                 ` Ted Ts'o
                                                   ` (2 more replies)
  0 siblings, 3 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-03 19:09 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Thu, Nov 3, 2011 at 11:52 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Ahh, sorry for the noise. I realize that we already have a winner, namely,
> the proposal outlined in your message I was responding to.

No, no, don't consider my "put in the merge message" a winner at all.

I personally dislike it, and don't really think it's a wonderful thing
at all. I really does have real downsides:

 - internal signatures really *are* a disaster for maintenance. You
can never fix them if they need fixing (and "need fixing" may well be
"you want to re-sign things after a repository format change")

 - they are ugly as heck, and you really don't want to see them in
99.999% of all cases.

So putting those things iin the merge commit message may have some
upsides, but it has tons of downsides too.

I think your refs/audit/ idea should be given real thought, because
maybe that's the right idea.

                           Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03 19:09                               ` Linus Torvalds
@ 2011-11-04 14:59                                 ` Ted Ts'o
  2011-11-04 15:14                                   ` Linus Torvalds
  2011-11-05  6:36                                 ` Junio C Hamano
  2011-11-09 17:26                                 ` Junio C Hamano
  2 siblings, 1 reply; 101+ messages in thread
From: Ted Ts'o @ 2011-11-04 14:59 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Thu, Nov 03, 2011 at 12:09:55PM -0700, Linus Torvalds wrote:
> I personally dislike it, and don't really think it's a wonderful thing
> at all. I really does have real downsides:
> 
>  - internal signatures really *are* a disaster for maintenance. You
> can never fix them if they need fixing (and "need fixing" may well be
> "you want to re-sign things after a repository format change")

Note that a repository format change will break a bunch of other
things as well, including references in commit descriptions ("This
fixes a regression introduced in commit 42DEADBEEF") So if SHA-1 is in
danger of failing in way that would threaten git's use of it (highly
unlikely), we'd probably be well advised to find a way to add a new
crypto checksum (i.e., SHA-256) in parallel, but keep the original
SHA-1 checksum for UI purposes.

>  - they are ugly as heck, and you really don't want to see them in
> 99.999% of all cases.

So we can make them be hidden from "git log" and "gik" by default.
That bit is a bit gross, I agree, but 3rd party verification really is
a good thing, which I'm hoping can be added in a relatively clean
fashion.

						- Ted

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-04 14:59                                 ` Ted Ts'o
@ 2011-11-04 15:14                                   ` Linus Torvalds
  2011-11-07  7:52                                     ` Valdis.Kletnieks
  0 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-04 15:14 UTC (permalink / raw)
  To: Ted Ts'o, Linus Torvalds, Junio C Hamano, Shawn Pearce, git,
	James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Fri, Nov 4, 2011 at 7:59 AM, Ted Ts'o <tytso@mit.edu> wrote:
>
> Note that a repository format change will break a bunch of other
> things as well, including references in commit descriptions ("This
> fixes a regression introduced in commit 42DEADBEEF")

No they won't. Not if you do it right. It's easy enough to
automatically replace the SHA1's in the description, the same way we
replace everything else.

Really.  It's *trivial*.

Maybe some current tools don't do it, but if I were to convert the
kernel tree, I'd absolutely *require* the conversion to be done right.
And "right" means "don't just get the parent SHA1's right, but the
ones hiding in the description too".

Any conversion tool has to keep track of the translation from "old
SHA1 to new SHA1" *anyway* because of all the other issues (ie exactly
things like parent pointers etc), so conversion tools by definition
have the information to do things like this right.

But "internal cryptographic signatures" are fundamentally different. A
conversion tool *cannot* convert them, since it won't have access to
the private keys in question, and thus cannot fix up the signature.

Sure, if I do the conversion, I could make *my* signatures match. And
that is true for every signer out there - individually. But only
individually, never collectively. Sure, we could all meet in one place
and synchronously re-sign things on our private machines with some
"distributed conversion tool", but realistically that really really
doesn't work.

It's a fundamental problem. And it really isn't a theoretical one -
it's one we know will happen *some* day.

I haven't worried about SHA1, exactly because I know it's not a real
problem - we can always convert. But internal signatures very
fundamentally change that.

And it really is about *internal* signatures. The kinds of signed tags
we have now are not a problem. Those can trivially be converted in a
distributed manner, exactly because they are "detatched" from what
they sign. We carry them along with the git repo, but they don't mess
up history, and they can be re-created individually without changing
anything else.

And yes, this was actually a design issue for me, which is why I feel
so strongly about it. I actually *thought* about issues like this
five+ years ago: I wanted to have cryptographic security, but I very
much on purpose wanted it to be "outside" the repo.

(Ok, so the git tag objects can sign other git tag objects
recursively, and in that case you have an ordering issue where a
conversion would first have to get somebody to re-sign their "inner"
tag before the "outer" signature can be re-created, but even if that
were to happen - and I don't think anybody does it - it's a trivial
problem with no real complexity issues).

>>  - they are ugly as heck, and you really don't want to see them in
>> 99.999% of all cases.
>
> So we can make them be hidden from "git log" and "gik" by default.
> That bit is a bit gross, I agree, but 3rd party verification really is
> a good thing, which I'm hoping can be added in a relatively clean
> fashion.

I agree that we can hide them - that's after all what the pgpsig thing
does in the "internal commit signature" that git has in pu/next. That
one hides ie even more specifically, by putting it in the headers of
the commit, but that's just a random implementation detail.

But I really think that "internal signatures" that actually affect the
SHA1 of the object and its history have fundamental design problems.
They may not be "insurmountably bad", but they are definitely real.

                        Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  2:19                           ` Linus Torvalds
@ 2011-11-04 20:16                             ` Junio C Hamano
  2011-11-04 21:22                               ` Junio C Hamano
  0 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-04 20:16 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Shawn Pearce, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> So I suspect we should just apply this patch, but I didn't check
> exacty what the failed tests are - except for the first one, that just
> compares against a canned response (and the canned response should
> just be changed).

After applying your patch and running

$ perl -pi -e 'if (/\ttag /) {
    s/754b754407bf032e9a2f9d5a9ad05ca79a6b228f/6c9dec2b923228c9ff994c6cfe4ae16c12408dc5/;
    s/0567da4d5edd2ff4bb292a465ba9e64dcad9536b/c61a82b60967180544e3c19f819ddbd0c9f89899/;
    s/6134ee8f857693b96ff1cc98d3e2fd62b199e5a8/525b7fb068d59950d185a8779dc957c77eed73ba/;
}' t/t5515/fetch.*

to unpeel the three tags used in the test 5515 that used to expect
FETCH_HEAD to have peeled tags to expect tag objects themselves instead,
all tests passes.

> although I suspect it was just a fairly mindless case of "make it a
> commit, because the merge needs the commit" - never mind that the
> merge would peel it anyway.

I am 100% sure the machinery that comes up with the tree (or half-merged
conflicted state) does not mind being fed tags. After all they need to
peel them down to commits for common ancestor discovery, and they need to
further peel them down to trees to perform three-way merges.

However we would need to audit so that we do not accidentally record the
tag object names in the "parent" headers in the merge commits, which is
what I'll be doing next.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03 18:16                           ` Junio C Hamano
  2011-11-03 18:52                             ` Junio C Hamano
  2011-11-03 19:06                             ` Linus Torvalds
@ 2011-11-04 21:12                             ` Junio C Hamano
  2011-11-04 23:45                               ` Linus Torvalds
  2 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-04 21:12 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

Junio C Hamano <gitster@pobox.com> writes:

> Linus Torvalds <torvalds@linux-foundation.org> writes:
>
>>   [torvalds@i5 linux]$ git fetch
>> git://github.com/rustyrussell/linux.git
>> rusty@rustcorp.com.au-v3.1-8068-g5087a50
>>   fatal: Couldn't find remote ref rusty@rustcorp.com.au-v3.1-8068-g5087a50
>>
>> oops. Ok, so his tag naming is *really* akward. Whatever.
>
> It is not "Whatever".
>
>  $ git fetch git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v3.0
>  fatal: Couldn't find remote ref v3.0
>
> I do not think we ever DWIMmed fetch refspecs to prefix refs/tags/, so it
> is not the naming but fetching tags without saying "git fetch tag v3.0"
> (which IIRC was your invention long time ago). 

If we really wanted to go this route, the attached single-liner should be
sufficient for the DWIMmery.

Note that the DWIMmery rules for "git fetch" and local "git rev-parse" are
still different even after this patch.

"git log frotz" can DWIM to "refs/remotes/frotz/HEAD", but in the remote
access context, "git fetch frotz" to fetch what the other side happened to
have fetched from what it calls 'frotz' (which may not have any relation
to what we consider is 'frotz') the last time would not make much sense,
so the fetch rules table does not include "refs/remotes/%.*s/HEAD".

When the user really wants to, "git fetch $there remotes/frotz/HEAD" would
let her do so anyway, so this is not about safety or security; it merely
is about confusion avoidance and discouraging meaningless usage.

Specifically, it is _not_ about ambiguity avoidance. A name that would
become ambiguous if we use the same rules table for both fetch and local
rev-parse would be ambiguous locally at the remote side.

If we really wanted to, we could 

	#define ref_fetch_rules ref_rev_parse_rules
 
in cache.h and remove the array's declaration from cache.h and its
definition from refs.c to really unify the two, but I haven't thought
things through.

-- >8 --
Subject: [PATCH] fetch: allow "git fetch $there v1.0" to fetch a tag

You can already do so with "git fetch $there tags/v1.0" but if it is not
ambiguous there is no reason to force users to type more.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 refs.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/refs.c b/refs.c
index e69ba26..670a7b3 100644
--- a/refs.c
+++ b/refs.c
@@ -1001,6 +1001,7 @@ const char *ref_rev_parse_rules[] = {
 const char *ref_fetch_rules[] = {
 	"%.*s",
 	"refs/%.*s",
+	"refs/tags/%.*s",
 	"refs/heads/%.*s",
 	NULL
 };
-- 
1.7.8.rc0.108.g71b5ec


^ permalink raw reply related	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-04 20:16                             ` Junio C Hamano
@ 2011-11-04 21:22                               ` Junio C Hamano
  2011-11-04 23:10                                 ` Linus Torvalds
  0 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-04 21:22 UTC (permalink / raw)
  To: git
  Cc: Linus Torvalds, Shawn Pearce, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

Junio C Hamano <gitster@pobox.com> writes:

> However we would need to audit so that we do not accidentally record the
> tag object names in the "parent" headers in the merge commits, which is
> what I'll be doing next.

Just reporting my findings.

builtin/merge.c was updated to use want_commit() that uses peel_to_type()
to commit to make sure we do not get fed anything funky, and also uses
struct commit_list to pass around list of parents to be recorded, so we
should be OK in this department.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-04 21:22                               ` Junio C Hamano
@ 2011-11-04 23:10                                 ` Linus Torvalds
  2011-11-05  3:55                                   ` Jeff King
  2011-11-05  4:37                                   ` Junio C Hamano
  0 siblings, 2 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-04 23:10 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Shawn Pearce, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Fri, Nov 4, 2011 at 2:22 PM, Junio C Hamano <junio@pobox.com> wrote:
>
> builtin/merge.c was updated to use want_commit() that uses peel_to_type()
> to commit to make sure we do not get fed anything funky, and also uses
> struct commit_list to pass around list of parents to be recorded, so we
> should be OK in this department.

I'm pretty sure people have already done "git merge v3.1" kind of
things using local tags (where no peeling of FETCH_HEAD has been
done). See

    git log --merges --grep 'Merge.*v[23]\.[0-9]'

for a ton of examples of this (and there's something odd going on: we
have "Merge commit .." and "Merge tag ..", and I suspect the latter is
people editing it to be correct by hand, but I dunno).

So this has always worked, methinks.

However - exactly beause git apparently makes it do that "Merge commit
" message, I suspect we've peeled things too early and too much. We've
peeled it so early that once again something thinks it's a commit, not
a tag.

So if anything, I suspect "git merge" not only peels, but peels too
much (or at the wrong point). We should probably peel as late as
possible.

But it's a small detail.

                  Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-04 21:12                             ` Junio C Hamano
@ 2011-11-04 23:45                               ` Linus Torvalds
  0 siblings, 0 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-04 23:45 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Fri, Nov 4, 2011 at 2:12 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> If we really wanted to go this route, the attached single-liner should be
> sufficient for the DWIMmery.

My first reaction on reading the patch was "maybe it would be safer to
put the tags case after 'branches', so that the behavior in the
presense of ambiguity would stay the same as with the old case", but
thinking it through I think it's more important to be consistent with
the other lookups, which all prefer tags over branches when ambiguous.

So Ack, patch looks sane to me, and clearly makes it easier to fetch
tags individually.

                  Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-04 23:10                                 ` Linus Torvalds
@ 2011-11-05  3:55                                   ` Jeff King
  2011-11-05  4:37                                   ` Junio C Hamano
  1 sibling, 0 replies; 101+ messages in thread
From: Jeff King @ 2011-11-05  3:55 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, Shawn Pearce, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Fri, Nov 04, 2011 at 04:10:59PM -0700, Linus Torvalds wrote:

> I'm pretty sure people have already done "git merge v3.1" kind of
> things using local tags (where no peeling of FETCH_HEAD has been
> done). See
> 
>     git log --merges --grep 'Merge.*v[23]\.[0-9]'
> 
> for a ton of examples of this (and there's something odd going on: we
> have "Merge commit .." and "Merge tag ..", and I suspect the latter is
> people editing it to be correct by hand, but I dunno).

It looks like fmt-merge-msg looks in FETCH_HEAD to see if each line is
marked as "branch" or "tag". So I get "Merge tag ..." with:

  git pull . tag v1.0

but I get "Merge commit ..." with:

  git merge v1.0

When "git merge" is run, it actually creates a fake FETCH_HEAD in memory
and feeds it to fmt-merge-msg. But that process doesn't seem to bother
looking at tags. I think we just need this:

---
diff --git a/builtin/merge.c b/builtin/merge.c
index dffd5ec..6a44b6d 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -439,10 +439,15 @@ static void merge_name(const char *remote, struct strbuf *msg)
 		if (!prefixcmp(found_ref, "refs/heads/")) {
 			strbuf_addf(msg, "%s\t\tbranch '%s' of .\n",
 				    sha1_to_hex(branch_head), remote);
 			goto cleanup;
 		}
+		if (!prefixcmp(found_ref, "refs/tags/")) {
+			strbuf_addf(msg, "%s\t\ttag '%s' of .\n",
+				    sha1_to_hex(branch_head), remote);
+			goto cleanup;
+		}
 		if (!prefixcmp(found_ref, "refs/remotes/")) {
 			strbuf_addf(msg, "%s\t\tremote-tracking branch '%s' of .\n",
 				    sha1_to_hex(branch_head), remote);
 			goto cleanup;
 		}

where the result of merge_name is just fed to fmt-merge-msg eventually.

-Peff

^ permalink raw reply related	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-04 23:10                                 ` Linus Torvalds
  2011-11-05  3:55                                   ` Jeff King
@ 2011-11-05  4:37                                   ` Junio C Hamano
  1 sibling, 0 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-11-05  4:37 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: git, Shawn Pearce, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> However - exactly beause git apparently makes it do that "Merge commit
> " message, I suspect we've peeled things too early and too much. We've
> peeled it so early that once again something thinks it's a commit, not
> a tag.

And you are right.

I am working on a larger series that should sit on top of 89587fa (Split
GPG interface into its own helper library, 2011-09-07), which is the first
commit in jc/signed-commit topic.

-- >8 --
Subject: [PATCH] merge: notice local merging of tags and keep it unwrapped

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 builtin/merge.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index f61b367..ce3b4f8 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -428,6 +428,11 @@ static void merge_name(const char *remote, struct strbuf *msg)
 				    sha1_to_hex(branch_head), remote);
 			goto cleanup;
 		}
+		if (!prefixcmp(found_ref, "refs/tags/")) {
+			strbuf_addf(msg, "%s\t\ttag '%s' of .\n",
+				    sha1_to_hex(branch_head), remote);
+			goto cleanup;
+		}
 		if (!prefixcmp(found_ref, "refs/remotes/")) {
 			strbuf_addf(msg, "%s\t\tremote-tracking branch '%s' of .\n",
 				    sha1_to_hex(branch_head), remote);
-- 
1.7.8.rc0.108.g71b5ec


^ permalink raw reply related	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03 19:09                               ` Linus Torvalds
  2011-11-04 14:59                                 ` Ted Ts'o
@ 2011-11-05  6:36                                 ` Junio C Hamano
  2011-11-05 16:41                                   ` Linus Torvalds
  2011-11-09 17:26                                 ` Junio C Hamano
  2 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-05  6:36 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Thu, Nov 3, 2011 at 11:52 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>
>> Ahh, sorry for the noise. I realize that we already have a winner, namely,
>> the proposal outlined in your message I was responding to.
>
> No, no, don't consider my "put in the merge message" a winner at all.
>
> I personally dislike it, and don't really think it's a wonderful thing
> at all. I really does have real downsides:
>
>  - internal signatures really *are* a disaster for maintenance. You
> can never fix them if they need fixing (and "need fixing" may well be
> "you want to re-sign things after a repository format change")
>
>  - they are ugly as heck, and you really don't want to see them in
> 99.999% of all cases.
>
> So putting those things iin the merge commit message may have some
> upsides, but it has tons of downsides too.
>
> I think your refs/audit/ idea should be given real thought, because
> maybe that's the right idea.

While I agree that re-signing is a problem, I do not see it as a huge
issue. In your "SHA-1 to SHA-256 transtion" scenario, the conversion is a
flag day event in the hopefully fairly distant (in the git timescale)
future, and I am reasonably sure that by that time we would already have
infrastructure updates necessary to support huge number of refs, including
the "lazily scan only the refs necessary" and the "some refs are optional
in advertisement" topics that are useful for other purposes.

In the worst case, even if we used your "merge commit records the merged
tag as the record of requested pull" design today, we could choose not to
rewrite these in-merge-commit signatures when the conversion becomes
necessary. Instead, the conversion procedure can prepare a mapping table
between the old SHA-1 and the rewritten SHA-256, and contributors can
prepare detached signature for the mappings of their own commits after
verifying that the conversion produced what they are happy with. And then
we store concatenation of these detached signatures in a blob to help
future third party auditors to audit these (by-then) historical commits.

About the ugliness of the merge commit log messages, you have already
learned to ignore them with "log --no-merges" ;-) and the material the
patch series I sent out adds are at the end, so "/^commit.*$" in less
would hopefully work well enough in "log --no-merges" as well.

Because the refs/audit/ approach requires too much infrastructure we still
do not have today, and workflow elements are not fully worked out
(e.g. propagating audit trails fully from sub-sub-sub-...-lieutenants
upwards is tricky as I outlined in the other message), I think we should
start from a design that we can see how it would work now.

With the posted series, the workflow would become something like this:

  contributor$ work work work

  contributor$ git tag -s -m 'Signed pull

  This series is to allow the integrator to pull from contributors
  by specifying a signed tag, not the tip of the branch, and verify
  the authenticity of the series while merging' for-linus

  contributor$ git push public for-linus

  contributor$ git request-pull origin \
          $(git config remote.public.url) for-linus >msg
  contributor$ edit msg
  contributor$ mail torvalds@...

  integrator$ mail ;# read the pull request
  integrator$ git pull git://github.com/contributor/linux.git for-linus
   ... editor opens with the usual merge message, but with
   ... the contents of the tag and the "GPG verify" result at
   ... the end.

It might make sense to also teach the "git tag" part somehow use branch
description of the tip of branch being tagged to prime the tag message.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-05  6:36                                 ` Junio C Hamano
@ 2011-11-05 16:41                                   ` Linus Torvalds
  2011-11-05 23:49                                     ` Junio C Hamano
  0 siblings, 1 reply; 101+ messages in thread
From: Linus Torvalds @ 2011-11-05 16:41 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Fri, Nov 4, 2011 at 11:36 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> About the ugliness of the merge commit log messages, you have already
> learned to ignore them with "log --no-merges" ;-)

Absolutely not. I look at merges all the time. I never use
"--no-merges" except when I'm doing certain statistics (ie "How many
real changes do we have") or when I do release files.

But I actually think it's important that people write *good* merge
messages. I've berated some people for it when they just have

    Merge branch 'origin'

in their commit message, because I think a merge commit should say why
it happened or what it brought in.

> and the material the
> patch series I sent out adds are at the end, so "/^commit.*$" in less
> would hopefully work well enough in "log --no-merges" as well.

I agree that being at the end helps, but I do a lot of "git log
ORIG_HEAD.." etc, and I don't do a lot of "/^commit" searching.

The "/commit" thing I do tends to be because I do "git log -p" to see
patches, but at the same time am not going to read through
everything..

So I'd really like some way to not see it.

Ted suggested a NUL character in the commit message in front of the
"hidden content". What do you think?

                Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-05 16:41                                   ` Linus Torvalds
@ 2011-11-05 23:49                                     ` Junio C Hamano
  2011-11-06  0:53                                       ` Linus Torvalds
  0 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-05 23:49 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> So I'd really like some way to not see it.
>
> Ted suggested a NUL character in the commit message in front of the
> "hidden content". What do you think?

You do not have to resort to NUL; we could just stuff whatever you do not
need to see but needs to be left *intact* in the new header fields just
like the embedded GPG signatures are stored in signed commits.

By the time the integrator is presented the merge commit template, we
would have:

 1. The merge title (e.g. "Merge tag for-linus of git://.../rusty.git/");

 2. Payload of the signed tag (or just "annotated tag"), which is used to
    convey meaningful topic description from the lieutenant;

 3. The signature in the tag, if the tag is not just merely annotated, but
    is signed;

 4. The output from GPG verification of the above (only when 3. is
    available); and

 5. The traditional "merge summary", if merge.log is enabled.

The 10-patch series I sent earlier appends 2 and 3 with "tag:" prefix and
4 with "# " prefix in the commit log template, but it does not have to be
that way. We could arrange things so that we put only 1, 2, 4 (still with
"# " prefix because this is meant to help you verify the authenticity, not
for later third-party audit, and to be stripped away with stripspace
before the commit is made) and 5 in the commit log template, and the
original signed tag contents (only when the tag is signed, not merely
annotated) in a separate file MERGE_SIG in $GIT_DIR/ next to MERGE_MSG,
and teach "git commit" to pick it up and stuff it in a new header field.

That way, the integrator can use the message 2 for the commit log message
and is free to typofix it, without breaking later third-party audit which
would use what is taken literally from the signed tag and stored in the
new header field, because the integrator's editor would never touch the
latter.








^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-05 23:49                                     ` Junio C Hamano
@ 2011-11-06  0:53                                       ` Linus Torvalds
  0 siblings, 0 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-06  0:53 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Sat, Nov 5, 2011 at 4:49 PM, Junio C Hamano <junio@pobox.com> wrote:
>
> You do not have to resort to NUL; we could just stuff whatever you do not
> need to see but needs to be left *intact* in the new header fields just
> like the embedded GPG signatures are stored in signed commits.

Agreed, [ details removed ] that sounds perfect. And makes it easy to
get at if you want to with just "git cat-file commit" - without ever
really being visible to people who don't care. And having it visible
in the editor with '#' means that the user who does the merge gets to
see what actually ended up being put in there, along with the fact
that yes, it verified correctly.

So I think I really like that approach - it seems to solve all problems.

               Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-04 15:14                                   ` Linus Torvalds
@ 2011-11-07  7:52                                     ` Valdis.Kletnieks
  2011-11-07 16:24                                       ` Linus Torvalds
  0 siblings, 1 reply; 101+ messages in thread
From: Valdis.Kletnieks @ 2011-11-07  7:52 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, Junio C Hamano, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

On Fri, 04 Nov 2011 08:14:52 PDT, Linus Torvalds said:
> On Fri, Nov 4, 2011 at 7:59 AM, Ted Ts'o <tytso@mit.edu> wrote:
> > Note that a repository format change will break a bunch of other
> > things as well, including references in commit descriptions ("This
> > fixes a regression introduced in commit 42DEADBEEF")

> No they won't. Not if you do it right. It's easy enough to
> automatically replace the SHA1's in the description, the same way we
> replace everything else.

OK.. I'll bite.  How do you disambiguate a '42deadbeef' in the changelog part
of a commit as being a commit ID, as opposed to being an address in a traceback
or something similar? Yes, I know you only change the ones that actually map to
a commit ID, but I'd not be surprised if by now we've got enough commits and
stack tracebacks in the git history that we'll birthday-paradox ourselves into
a false-positive in an automatic replacement.

(And it's OK to say "the 3 stack tracebacks in changelogs we just mangled can
just go jump", but it does need at least a few seconds consideration..)


[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-07  7:52                                     ` Valdis.Kletnieks
@ 2011-11-07 16:24                                       ` Linus Torvalds
  0 siblings, 0 replies; 101+ messages in thread
From: Linus Torvalds @ 2011-11-07 16:24 UTC (permalink / raw)
  To: Valdis.Kletnieks
  Cc: Ted Ts'o, Junio C Hamano, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Sun, Nov 6, 2011 at 11:52 PM,  <Valdis.Kletnieks@vt.edu> wrote:
>
> OK.. I'll bite.  How do you disambiguate a '42deadbeef' in the changelog part
> of a commit as being a commit ID, as opposed to being an address in a traceback
> or something similar? Yes, I know you only change the ones that actually map to
> a commit ID, but I'd not be surprised if by now we've got enough commits and
> stack tracebacks in the git history that we'll birthday-paradox ourselves into
> a false-positive in an automatic replacement.

I don't think we are quite there yet. And (sadly) most of the commit
ID's in the history are 7 hex characters, because that used to be the
default git abbreviation. So there is unlikely to be any real
conflicts.

If we do miss one or two, that will be sad and embarrassing, but is
not a real problem in practice.

We probably could add various heuristics (the SHA1 values are *often*
preceded by the string "commit"), and a really good import would also
have somebody at least visually inspecting ones that other heuristics
say might be debatable (for example - because they have 8 hex digits
and there are other numbers around them that were *not* converted),
but in the end perfection is the enemy of good. It's not really worth
the headache to worry about *all* the cases, if you can cheaply and
simply get 99+% right.

And I think the 99% is almost trivial. While the last 1% may or may
not be worth worrying about.

               Linus

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03 19:09                               ` Linus Torvalds
  2011-11-04 14:59                                 ` Ted Ts'o
  2011-11-05  6:36                                 ` Junio C Hamano
@ 2011-11-09 17:26                                 ` Junio C Hamano
  2011-11-10  8:02                                   ` Johan Herland
  2 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-09 17:26 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Ted Ts'o, Shawn Pearce, git, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

Linus Torvalds <torvalds@linux-foundation.org> writes:

> No, no, don't consider my "put in the merge message" a winner at all.
>
> I personally dislike it, and don't really think it's a wonderful thing
> at all. I really does have real downsides:
>
>  - internal signatures really *are* a disaster for maintenance. You
> can never fix them if they need fixing (and "need fixing" may well be
> "you want to re-sign things after a repository format change")
>
>  - they are ugly as heck, and you really don't want to see them in
> 99.999% of all cases.
>
> So putting those things iin the merge commit message may have some
> upsides, but it has tons of downsides too.
>
> I think your refs/audit/ idea should be given real thought, because
> maybe that's the right idea.

With the latest round of touch-ups, modulo a few bugs I will be fixing
before the 1.7.8 final, I think what we have is more or less OK in the
shorter term and should be ready for general consumption. The ugliness is
gone, but the issue around internal signatures may remain to be solved in
the longer term. At least, by storing the full contents of the tag today
in an extended header, when we figure out how a detached signature should
really work, we could convert by extracting them from the history.

In a separate message earlier in the thread, you raised another issue.

> I hate how anonymous our branches are. Sure, we can use good names for
> them, but it was a mistake to think we should describe the repository
> (for gitweb), rather than the branch.
> 
> Ok, "hate" is a strong word. I don't "hate" it. I don't even think
> it's a major design issue. But I do think that it would have been
> nicer if we had had some branch description model.

At the first glance, our branch model is indeed peculiar in that a branch
does not have a global identity. The scope of its name is local to the
repository, and it is just a pointer into the history. A "note" [*1*] that
can annotate a commit long after the commit is made is not a good way to
describe what a branch is about, because the tip of the branch can advance
beyond the commit that is annotated by such a note. A commit on a branch
does not serve as a good anchoring point to describe the branch.

However, a commit that merges the history of a branch, whether the merged
branch is from a local repository or from a remote one, does serve as a
good anchoring point. The work on a branch is finished as complete as
possible at the time of the merge, and the committer who merges the branch
agrees with both the objective and the implementation of the work done on
the branch, and that is why the merge is made [*2*]. Describing what the
history of the side branch was about in the resulting merge is a perfectly
sensible way to explain the branch. So in that sense, I am very happy with
the way the merge message template uses the pull request tag to let the
lieutenant explain and defend the history behind the tag used for the pull
request. Such an explanation does not have to be keyed with anybody's
local branch name (e.g. "for-linus" would mean different things for
different pull requests even from the same person), but keying it with the
resulting merge commit is a sensible way to leave the record in the
history.

After justifying with the above two paragraphs that it is perfectly
sensible to record the annotations on commits and not on "branch names", I
do agree that we would eventually want to be able to have such annotations
on commits after the fact. Neither "tags" nor "notes" is necessarily a
very good mechanism, however, for the purpose of "signed pull requests"
and "signed commits" [*3*]. Here are some pros and cons:

 - tags must be named, but the only thing we need is to be able to look
   the contents (with signature if signed) up given a commit object.
   Unlike the usual "I want to check out v3.0 release" look-up that goes
   from tag names to the commits, annotation look-ups go the other way, do
   not have to have a tagname, and having tagname does not help our
   look-up in any way. If we want to use tag to annotate various commits
   by various people and keep them around, we would need global namespace
   that would not cause them to crash (we can work this around by using
   the object name of the tag, e.g. renaming 'for-linus' tag to $(git
   rev-parse tags/for-linus), but that is merely a workaround of having to
   name things that do not have to be named in the first place). As a
   local storage machinery for annotations, tags hanging below refs/tags/
   (or refs/audit for that matter) hierarchy with their own names is an
   inappropriate model.

 + tags can auto-follow the commits when object transfer happens (at least
   in the fetch direction), and for the purpose of "signed pull requests"
   and "signed commits", this is a desirable property. When a repository
   gains a commit, the annotations attached to the commit that are missing
   from the receiving repository are automatically transferred from the
   place the commit comes from. Annotations given to other commits that
   are not transferred into the repository do not come to the repository.

 - "git notes" is represented as a commit that records a tree that holds
   the entire mapping from commit to its annotations, and the only way to
   transferr it is to send it together with its history as a whole. It
   does not have the nice auto-following property that transfers only the
   relevant annotations.

 + "git notes" maps the commits to its annotations in the right direction;
   the object name of an annotated object to its annotation.

In the longer term, I think we would need to extend the system in the
following way:

 - Introduce a mapping machanism that can be locally used to map names of
   the objects being annotated to names of other objects (most likely
   blobs but there is nothing that fundamentally prevents you from
   annotating a commit with a tree). The current "git notes" might be a
   perfectly suitable representation of this, or it may turn out to be
   lacking (I haven't thought things through), but the important point is
   that this "mapping store" is _local_. fsck, repack and prune need to be
   told that objects that store the annotation are reachable from the
   annotated objects.

 - Introduce a protocol extension to transfer this mapping information for
   objects being transferred in an efficient way. When "rev-list --objects
   have..want" tells us that the receiving end (in either fetch/push
   direction) would have an object at the end of the primary transfer
   (note that I did not say "an object will be sent in this transfer
   transaction"; "have" does not come into the picture), we make sure that
   missing annotations attached to the object is also transferred, and new
   mapping is registered at the receiving end.

The detailed design for the latter needs more thought. The auto-following
of tags works even if nothing is being fetched in the primary transfer
(i.e. "git fetch" && "git fetch" back to back to update our origin/master
with the master at the origin) when a new tag is added to ancient part of
the history that leads to the master at the origin, but this is exactly
because the sending end advertises all the available tags and the objects
they point at so that we can tell what new tags added to an old object is
missing from the receiving end. This obviously would not scale well when
we have tens of thousands of objects to annotate. Perhaps an entry in the
"mapping store" would record:

 - The object name of the object being annotated;

 - The object name of the annotation;

 - The "timestamp", i.e. when the association between the above two was
   made--this can be local to the repository and a simple counter would
   do.

and also maintain the last "timestamp" this repository sent annotations to
the remote (one timestamp per remote repository). When we push, we would
send annotations pertaining to the object reachable from what we are
pushing (not limited by what they already have, as the whole point of this
exercise is to allow us to transfer annotations added to an object long
after the object was created and sent to the remote) that is newer than
that "timestamp". Similarly, when fetching, we would send the "timestamp"
this repository last fetched annotations from the other end (which means
we would need one such "timestamp" per remote repository) and let the
remote side decide the set of new annotations they added since we last
synched that are on objects reachable from what we "want".

Or something like that.

[Footnote]

*1* By this word, I do not necessarily mean what the "git notes" command
manipulates. A tag that points at a commit is also equally a good vehicle
to annotate a commit after the fact.

*2* For this reason, it may make sense to "commit -S" such a merge
commit. The "mergetag" asserts the authenticity of the pull request from
the lieutenant whose history is being integrated, and the "gpgsig" asserts
the authenticity of the merge itself--the fact that it was made by the
integrator.

*3* I do not mean what "git commit -S" parked in 'pu' produces, which is
to store the signature in the commit. Adding "Signed-off-by:" after the
fact to an existing commit by many people is a more appropriate example.


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-09 17:26                                 ` Junio C Hamano
@ 2011-11-10  8:02                                   ` Johan Herland
  2011-11-10 15:15                                     ` Junio C Hamano
  0 siblings, 1 reply; 101+ messages in thread
From: Johan Herland @ 2011-11-10  8:02 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Linus Torvalds, Ted Ts'o, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Wed, Nov 9, 2011 at 18:26, Junio C Hamano <gitster@pobox.com> wrote:
>  - "git notes" is represented as a commit that records a tree that holds
>   the entire mapping from commit to its annotations, and the only way to
>   transferr it is to send it together with its history as a whole. It
>   does not have the nice auto-following property that transfers only the
>   relevant annotations.

True. However, consider these mitigating factors:

 - The annotations in question (the "signing" of commits) are all intended to
   be merged eventually (i.e. there is no reason for a developer to (after the
   fact) sign a commit that will never end up in the public record). Therefore,
   most or all of the notes in the notes tree are already relevant, or
will become
   relevant in the near future (when the associated commits are merged).

 - Additionally, you could organize these notes into two (or more) notes trees,
   one for merged/official annotations, and one for unmerged/pending
annotations.
   Then make the relevant tools (e.g. "git merge") transfer notes from one tree
   to the other, thereby making sure that the "official" record only contains
   notes that are relevant to the merged history.

 - Finally, there's always "git notes prune" to purge annotations for commits
   that ended up never being merged.

My point is that although "notes" might end up transferring more annotations
than strictly necessary, I believe that in practice all the notes
being transferred
are already (or will soon become) relevant.

>  + "git notes" maps the commits to its annotations in the right direction;
>   the object name of an annotated object to its annotation.
>
> In the longer term, I think we would need to extend the system in the
> following way:
>
>  - Introduce a mapping machanism that can be locally used to map names of
>   the objects being annotated to names of other objects (most likely
>   blobs but there is nothing that fundamentally prevents you from
>   annotating a commit with a tree). The current "git notes" might be a
>   perfectly suitable representation of this, or it may turn out to be
>   lacking (I haven't thought things through), but the important point is
>   that this "mapping store" is _local_. fsck, repack and prune need to be
>   told that objects that store the annotation are reachable from the
>   annotated objects.

IMHO this is precisely what "git notes" does today.

>  - Introduce a protocol extension to transfer this mapping information for
>   objects being transferred in an efficient way. When "rev-list --objects
>   have..want" tells us that the receiving end (in either fetch/push
>   direction) would have an object at the end of the primary transfer
>   (note that I did not say "an object will be sent in this transfer
>   transaction"; "have" does not come into the picture), we make sure that
>   missing annotations attached to the object is also transferred, and new
>   mapping is registered at the receiving end.
>
> The detailed design for the latter needs more thought. The auto-following
> of tags works even if nothing is being fetched in the primary transfer
> (i.e. "git fetch" && "git fetch" back to back to update our origin/master
> with the master at the origin) when a new tag is added to ancient part of
> the history that leads to the master at the origin, but this is exactly
> because the sending end advertises all the available tags and the objects
> they point at so that we can tell what new tags added to an old object is
> missing from the receiving end. This obviously would not scale well when
> we have tens of thousands of objects to annotate. Perhaps an entry in the
> "mapping store" would record:
>
>  - The object name of the object being annotated;
>
>  - The object name of the annotation;
>
>  - The "timestamp", i.e. when the association between the above two was
>   made--this can be local to the repository and a simple counter would
>   do.
>
> and also maintain the last "timestamp" this repository sent annotations to
> the remote (one timestamp per remote repository). When we push, we would
> send annotations pertaining to the object reachable from what we are
> pushing (not limited by what they already have, as the whole point of this
> exercise is to allow us to transfer annotations added to an object long
> after the object was created and sent to the remote) that is newer than
> that "timestamp". Similarly, when fetching, we would send the "timestamp"
> this repository last fetched annotations from the other end (which means
> we would need one such "timestamp" per remote repository) and let the
> remote side decide the set of new annotations they added since we last
> synched that are on objects reachable from what we "want".
>
> Or something like that.

You would also have to keep track of deleted annotations, to enable the local
side to delete an annotation corresponding to an already-deleted annotation
on the remote side.

Pretty soon, you end up having to record something similar to a DAG,
describing the history of manipulating these annotations. At that point, your
"timestamp" calculation starts to look very similar to the "have..want"
calculation already done when transferring "regular" refs. At which point you
have a system that is very similar to what "git notes" does today...


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-03  4:13                                 ` Linus Torvalds
@ 2011-11-10 13:51                                   ` David Woodhouse
  2011-11-10 15:23                                     ` Marc Branchaud
  0 siblings, 1 reply; 101+ messages in thread
From: David Woodhouse @ 2011-11-10 13:51 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Jochen Striepe, Shawn Pearce, Junio C Hamano, git,
	James Bottomley, Jeff Garzik, Andrew Morton, linux-ide, LKML

On Wed, 2011-11-02 at 21:13 -0700, Linus Torvalds wrote:
> No, my main objection to saving the data is that it's ugly and it's
> redundant. Sure, in practice you can check the signatures later fine
> (with the rare exceptions you mention), but even when you can do it,
> what's the big upside? 

Another objection (although it may not be insurmountable) is that it's
not necessarily *entirely* clear what's being signed.

In the simple case where I clone your tree, make a few commits with my
Signed-off-by:, sign a tag and then ask you to pull, that's easy enough.
I'm vouching for what I committed, and not for everything that was in
your tree beforehand.

But what if I'm working on top of someone else's published git tree?
Does a signed tag at the top of *my* work imply that I'm vouching for
all of theirs too?

In the case where the signature is ephemeral and only used for you to
trust my pull request, the answer is simple: If that other work wasn't
in your tree yet at the time I send my pull request, I'd damn well
better be vouching for it when I ask you to pull it. Nothing new there.

But if we're keeping signatures around for auditing purposes, we'd
better have a coherent answer to that question. One that isn't "a
signature cover everything since the last commit with torvalds@ as the
committer", if we want it to be useful for the general case.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 21:21               ` Linus Torvalds
                                   ` (2 preceding siblings ...)
  2011-11-02 23:34                 ` Junio C Hamano
@ 2011-11-10 13:52                 ` David Woodhouse
  3 siblings, 0 replies; 101+ messages in thread
From: David Woodhouse @ 2011-11-10 13:52 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Junio C Hamano, git, James Bottomley, Jeff Garzik, Andrew Morton,
	linux-ide, LKML

On Tue, 2011-11-01 at 14:21 -0700, Linus Torvalds wrote:
> I hate how anonymous our branches are. Sure, we can use good names for
> them, but it was a mistake to think we should describe the repository
> (for gitweb), rather than the branch.
> 
> Ok, "hate" is a strong word. I don't "hate" it. I don't even think
> it's a major design issue. But I do think that it would have been
> nicer if we had had some branch description model. 

I actually quite like it. I take it as a hint: if the contents of a
branch are *so* wildly different from the main repository that they need
a different description, perhaps I should be using a separate repository
instead of just a branch.

-- 
dwmw2


^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-10  8:02                                   ` Johan Herland
@ 2011-11-10 15:15                                     ` Junio C Hamano
  2011-11-10 16:03                                       ` Johan Herland
  0 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-10 15:15 UTC (permalink / raw)
  To: Johan Herland
  Cc: Linus Torvalds, Ted Ts'o, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

Johan Herland <johan@herland.net> writes:

> On Wed, Nov 9, 2011 at 18:26, Junio C Hamano <gitster@pobox.com> wrote:
>>  - "git notes" is represented as a commit that records a tree that holds
>>   the entire mapping from commit to its annotations, and the only way to
>>   transferr it is to send it together with its history as a whole. It
>>   does not have the nice auto-following property that transfers only the
>>   relevant annotations.
>
> True. However, consider these mitigating factors:
> ...
>
> My point is that although "notes" might end up transferring more
> annotations than strictly necessary, I believe that in practice all the
> notes being transferred are already (or will soon become) relevant.

Sorry, but I do not think you are considering what would happen when you
have many branches with different purposes, whose commits near tips will
never get merged with each other. "automatic following" semantics like
what "git fetch" does for signed tags is absolutely necessary in such a
case, and the above are not mitigating factors at all in that context.



^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-10 13:51                                   ` David Woodhouse
@ 2011-11-10 15:23                                     ` Marc Branchaud
  0 siblings, 0 replies; 101+ messages in thread
From: Marc Branchaud @ 2011-11-10 15:23 UTC (permalink / raw)
  To: David Woodhouse
  Cc: Linus Torvalds, Jochen Striepe, Shawn Pearce, Junio C Hamano,
	git, James Bottomley, Jeff Garzik, Andrew Morton, linux-ide,
	LKML

On 11-11-10 08:51 AM, David Woodhouse wrote:
> On Wed, 2011-11-02 at 21:13 -0700, Linus Torvalds wrote:
>> No, my main objection to saving the data is that it's ugly and it's
>> redundant. Sure, in practice you can check the signatures later fine
>> (with the rare exceptions you mention), but even when you can do it,
>> what's the big upside? 
> 
> Another objection (although it may not be insurmountable) is that it's
> not necessarily *entirely* clear what's being signed.

I think this is a non-issue as far as the implementation is concerned.  That
is, the question exists regardless of what actual bits get (hashed and)
encrypted by a private key.  Furthermore, the answer will depend on who's
using the signatures and in what context, and it's not appropriate for the
git tool to make assumptions about those things.

> In the simple case where I clone your tree, make a few commits with my
> Signed-off-by:, sign a tag and then ask you to pull, that's easy enough.
> I'm vouching for what I committed, and not for everything that was in
> your tree beforehand.
> 
> But what if I'm working on top of someone else's published git tree?
> Does a signed tag at the top of *my* work imply that I'm vouching for
> all of theirs too?

<philosophy>

It all depends on what you mean by "vouch for".

You obviously thought that the 3rd-party repo was good for something,
otherwise why did you base your work on it in the first place?  So maybe
you're just vouching for the 3rd-party repo being good enough for what you're
trying to do.

Or, maybe you've done a thorough analysis of the 3rd-party code and are ready
to certify it as completely memory-leak-free or something.

Or or, maybe you're only making a statement about the commits that you've
authored yourself.  (You probably want to individually sign each of those
commits in this case.)

These sorts of issues have been debated on PKI mailing lists ad nauseum.  I
think the best approach is that if you want your signature to have a
particular meaning, then put that into some text that's part of what's being
signed.  Let other humans read that text and make their own decisions.

</philosophy>

And whatever the case, the software that makes and validates the signatures
shouldn't make any assertions about how to interpret good or bad signatures.
 (Yes, other software could interpret meanings according to some criteria,
and that software could exist alongside or be incorporated into the basic
digital signature software, but the interpretation software is doing a
different job.)

		M.

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-10 15:15                                     ` Junio C Hamano
@ 2011-11-10 16:03                                       ` Johan Herland
  2011-11-10 17:18                                         ` Junio C Hamano
  0 siblings, 1 reply; 101+ messages in thread
From: Johan Herland @ 2011-11-10 16:03 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Linus Torvalds, Ted Ts'o, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Thu, Nov 10, 2011 at 16:15, Junio C Hamano <junio@pobox.com> wrote:
> Johan Herland <johan@herland.net> writes:
>> On Wed, Nov 9, 2011 at 18:26, Junio C Hamano <gitster@pobox.com> wrote:
>>>  - "git notes" is represented as a commit that records a tree that holds
>>>   the entire mapping from commit to its annotations, and the only way to
>>>   transferr it is to send it together with its history as a whole. It
>>>   does not have the nice auto-following property that transfers only the
>>>   relevant annotations.
>>
>> True. However, consider these mitigating factors:
>> ...
>>
>> My point is that although "notes" might end up transferring more
>> annotations than strictly necessary, I believe that in practice all the
>> notes being transferred are already (or will soon become) relevant.
>
> Sorry, but I do not think you are considering what would happen when you
> have many branches with different purposes, whose commits near tips will
> never get merged with each other. "automatic following" semantics like
> what "git fetch" does for signed tags is absolutely necessary in such a
> case, and the above are not mitigating factors at all in that context.

What about having one notes ref per branch? If/when the branch is merged,
the associated notes ref containing the annotations for the commits on that
branch would be merged as well (using "git notes merge").

Sure, using one notes ref per branch is more expensive than a single notes
ref, but it's still cheaper than one ref per signed commit (which is what we
get when using annotated tags). And it prevents the added code and
complexity of the timestamped mapping approach.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-10 16:03                                       ` Johan Herland
@ 2011-11-10 17:18                                         ` Junio C Hamano
  2011-11-11  1:17                                           ` Johan Herland
  0 siblings, 1 reply; 101+ messages in thread
From: Junio C Hamano @ 2011-11-10 17:18 UTC (permalink / raw)
  To: Johan Herland
  Cc: Linus Torvalds, Ted Ts'o, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

Johan Herland <johan@herland.net> writes:

> What about having one notes ref per branch? If/when the branch is merged,
> the associated notes ref containing the annotations for the commits on that
> branch would be merged as well (using "git notes merge").

That is a crude workaround that you could (with help from users) make it
work, but it does not change the fact that the current mechanism to
transfer and integrate notes across repositories is a bad match for what
the "signed commit" type annotations wants to achieve. In fact, the need
for such a workaround is an illustration of how bad a match the mechanism
is.

When you merge a history that has commit A into another history that did
not have that commit, the act of creating a merge commit itself should be
enough to make the resulting history to contain that commit. The commit
DAG already expresses it, and if a parallel "notes" mechanism needs to be
futzed with to match that DAG, and command like "merge" needs to be told
to help that process, that is a shortcoming of the "notes" mechanism.





^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-10 17:18                                         ` Junio C Hamano
@ 2011-11-11  1:17                                           ` Johan Herland
  2011-11-11  5:26                                             ` Junio C Hamano
  0 siblings, 1 reply; 101+ messages in thread
From: Johan Herland @ 2011-11-11  1:17 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Linus Torvalds, Ted Ts'o, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

On Thu, Nov 10, 2011 at 18:18, Junio C Hamano <junio@pobox.com> wrote:
> Johan Herland <johan@herland.net> writes:
>
>> What about having one notes ref per branch? If/when the branch is merged,
>> the associated notes ref containing the annotations for the commits on that
>> branch would be merged as well (using "git notes merge").
>
> That is a crude workaround that you could (with help from users) make it
> work, but it does not change the fact that the current mechanism to
> transfer and integrate notes across repositories is a bad match for what
> the "signed commit" type annotations wants to achieve. In fact, the need
> for such a workaround is an illustration of how bad a match the mechanism
> is.
>
> When you merge a history that has commit A into another history that did
> not have that commit, the act of creating a merge commit itself should be
> enough to make the resulting history to contain that commit. The commit
> DAG already expresses it, and if a parallel "notes" mechanism needs to be
> futzed with to match that DAG, and command like "merge" needs to be told
> to help that process, that is a shortcoming of the "notes" mechanism.

[ ...and from elsewhere in this thread: ]

> Note that in this thread, I am not saying that "git notes" mechanism is
> not good for anything. A tree whose node names encode an object name is a
> valid way to store the mapping from that object to a set of other objects,
> and we already agreed that as the "local" storage mechanism, "git notes"
> may be used as-is for the purpose of this thread.
>
> But the transfer and merge semantics "git notes" mechanism offers treats
> the entire "notes" that appear in _one_ repository and merging that set to
> the entire "notes" in another repository and it is not a good match for
> the purpose of this thread.

Ok. Point taken.

Given that we need an alternative way to transfer annotations between
repos (using auto-follow to select the relevant set of annotations, and
then transferring only those annotations): Can we leverage existing
functionality in "notes" where useful (e.g. using existing notes merge
strategies to deal with colliding annotations), while at the same time
extending the current "notes" feature with this alternative transfer
mechanism? FWIW, I expect there are other "notes" use cases that
would also prefer the auto-follow only-relevant transfer behavior.

So, how can we use "notes" to better support the transfer semantics you
suggest? The mapping from the object being annotated to the annotation
object is already contained in the notes tree, but the "timestamp" you
describe (needed to efficiently calculate the set of annotations to
auto-follow) is not [1]. However, we could easily enough add a sorted
list of (timestamp,  annotated object name) pairs, to allow fast lookup
of annotations created after a given timestamp. We could even store this
list in a blob or tree object referenced directly from the notes tree [2].


Have fun! :)

...Johan


[1]: Although I did at some point experiment with using timestamps in the
internal organization of the notes tree (see for example
http://article.gmane.org/gmane.comp.version-control.git/127966 ), I ended
up using only the annotated object name (with flexible fanout). I don't
think that reintroducing timestamps in the notes tree organization will
pay off, because we need both lookup by annotated SHA1 and lookup by
newer-than-given-timestamp to be fast, and there's AFAIK no way to get
both from a single notes tree organzation.

[2]: E.g. accessible with "git cat-file refs/notes/foo:timestamps". When
a notes tree contains an entry that is obviously not an object name (SHA1),
the notes code will leave it alone/untouched in the tree (see "struct
non_note" and associated code in notes.c for further details).

-- 
Johan Herland, <johan@herland.net>
www.herland.net

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-11  1:17                                           ` Johan Herland
@ 2011-11-11  5:26                                             ` Junio C Hamano
  0 siblings, 0 replies; 101+ messages in thread
From: Junio C Hamano @ 2011-11-11  5:26 UTC (permalink / raw)
  To: Johan Herland
  Cc: Linus Torvalds, Ted Ts'o, Shawn Pearce, git, James Bottomley,
	Jeff Garzik, Andrew Morton, linux-ide, LKML

Johan Herland <johan@herland.net> writes:

> Given that we need an alternative way to transfer annotations between
> repos (using auto-follow to select the relevant set of annotations, and
> then transferring only those annotations): Can we leverage existing
> functionality in "notes" where useful (e.g. using existing notes merge
> strategies to deal with colliding annotations), while at the same time
> extending the current "notes" feature with this alternative transfer
> mechanism? FWIW, I expect there are other "notes" use cases that
> would also prefer the auto-follow only-relevant transfer behavior.
>
> So, how can we use "notes" to better support the transfer semantics you
> suggest? The mapping from the object being annotated to the annotation
> object is already contained in the notes tree, but the "timestamp" you
> describe (needed to efficiently calculate the set of annotations to
> auto-follow) is not [1].

Please do not take the "timestamp" part too seriously.

I am starting to think that what we want in this context actually is very
close to annotated tags. I said we want a mapping from an annotated object
to "a set of other objects" that annotate it, but it was an unnecessary
and premature generalization. There is no reason that these annotations
have to be structured "Git" objects such as blobs and trees.

A set of annotated tags that have the same value on their "object" field
is a perfect match for "a set of annotations attached to a given object".

We already know that using the real tags has its own problems coming from
having to give each and every one of them unique names somewhere in the
refs hierarchy (be it refs/tags/ or refs/audit/), but imagine if we
somehow had a way to:

 - keep these annotated tags in the object store;

 - keep them from getting pruned even if they are not referenced from
   anywhere in refs/ hierarchy;

 - given an object, efficiently enumerate such annotate tags that refer to
   the object.

And then imagine that we are pushing history leading to a commit from one
repository to another. Both repositories store these "anonymous" (that is
what they are---they do not have a name in the refs/ hierarchy) tags.

The two repositories can individually enumerate all these "anonymous" tags
that annotate commits in the history that is being exchanged, and run a
set reconciliation algorithm (e.g. [*1*]) to find out the anonymous tags
that are missing from the recipient repository.

Such an approach does not require any timestamp.

My point is _not_ that the alternative in this message is superiour to the
handwaving in my other message, but is that I think it may not be the best
approach to think what needs to be added to "notes" to make it applicable
for the problem we are solving.

Rather, I think we should design how the overall system should look like
(i.e. what property the resulting system should have) and then find out
what is necessary in each part of the resulting solution (i.e. the list of
"somehow had a way to..." above, plus "efficient set reconciliation").


[Footnote]

*1* What's the Difference? Efficient Set Reconciliation without Prior
Context http://cseweb.ucsd.edu/~fuyeda/papers/sigcomm2011.pdf

^ permalink raw reply	[flat|nested] 101+ messages in thread

* Re: [git patches] libata updates, GPG signed (but see admin notes)
  2011-11-01 17:15                     ` Linus Torvalds
@ 2011-11-12 19:31                       ` Felipe Contreras
  0 siblings, 0 replies; 101+ messages in thread
From: Felipe Contreras @ 2011-11-12 19:31 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Tony Luck, H. Peter Anvin, david, James Bottomley, Jeff Garzik,
	Andrew Morton, linux-ide, LKML

On Tue, Nov 1, 2011 at 7:15 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> The LKML thing is why I really love how gmail works. LKML just goes
> into the black hole that is the gmail thing, and I never see it. But
> then, when I get cc'd about something in the middle of a thread when a
> question comes up - I just magically see the whole thread without
> having to look around in other folders or anything like that.
>
> I'm sure 'notmuch' etc does that too, but pine sure never did. If I
> made my filters automatically move lkml posts to my lkml folder, by
> the time I got cc'd, I had to go back to that folder to see the
> history of the thread.

'notmuch' is the only mail client that I know does that (because it
has tags). So I do exactly the same in Gmail than in notmuch; "mark as
archived", and then I don't see the messages in my inbox, but the
actual mail never really moves, it just doesn't have the 'inbox' tag
any more.

Moreover, searching all your mail is very fast, just like in Gmail.

One benefit is that it's open source, so you add your own key
bindings, or whatever you like, and unlike projects like GNOME, they
have similar development practices as in git; you send patches to the
mailing list, no bugzilla crap.

Of course, it only works with local mail (maildir), but you could use
mbsync (as Theodore Tso suggested). That's what I do in my mail setup;
I use mbsync to synchronize ~/onlinemail with the IMAP server, and
after I archive mail, I run a script that moves that mail to ~/mail,
so it's removed from the server on the next sync.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 101+ messages in thread

end of thread, other threads:[~2011-11-12 19:31 UTC | newest]

Thread overview: 101+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-10-26 20:22 [git patches] libata updates, GPG signed (but see admin notes) Jeff Garzik
2011-10-29 19:13 ` Valdis.Kletnieks
2011-10-29 19:27   ` Jeff Garzik
2011-10-30 10:05 ` James Bottomley
2011-10-30 17:12   ` Linus Torvalds
2011-10-31  8:19     ` James Bottomley
2011-10-31 15:53       ` Linus Torvalds
2011-10-31 18:23         ` Junio C Hamano
2011-10-31 22:18           ` Linus Torvalds
2011-10-31 22:20             ` H. Peter Anvin
2011-10-31 22:30               ` Linus Torvalds
2011-10-31 22:33                 ` H. Peter Anvin
2011-10-31 22:38                   ` Linus Torvalds
2011-10-31 22:51                     ` Junio C Hamano
2011-10-31 22:56                       ` Linus Torvalds
2011-11-02  9:11                         ` Ingo Molnar
2011-11-02 11:20                           ` Jochen Striepe
2011-10-31 23:09                       ` Junio C Hamano
2011-10-31 22:44                   ` Junio C Hamano
2011-10-31 22:47                     ` H. Peter Anvin
2011-10-31 22:49                     ` Ted Ts'o
2011-10-31 22:51                       ` H. Peter Anvin
2011-10-31 22:52                     ` Linus Torvalds
2011-10-31 22:54                       ` H. Peter Anvin
2011-10-31 23:03                         ` Linus Torvalds
2011-11-01  5:39                       ` James Bottomley
2011-10-31 23:55                     ` Jeff Garzik
2011-11-01  0:42                       ` H. Peter Anvin
2011-10-31 22:33               ` Jiri Kosina
2011-11-01 19:47             ` Junio C Hamano
2011-11-01 21:21               ` Linus Torvalds
2011-11-01 21:56                 ` Junio C Hamano
2011-11-02 20:04                   ` Linus Torvalds
2011-11-02 21:13                     ` Junio C Hamano
2011-11-03  1:02                     ` Shawn Pearce
2011-11-03  1:19                       ` Linus Torvalds
2011-11-03  1:45                         ` Linus Torvalds
2011-11-03  2:14                           ` Shawn Pearce
2011-11-03  2:25                             ` Linus Torvalds
2011-11-03  3:22                               ` Jochen Striepe
2011-11-03  4:13                                 ` Linus Torvalds
2011-11-10 13:51                                   ` David Woodhouse
2011-11-10 15:23                                     ` Marc Branchaud
2011-11-03  2:31                             ` Linus Torvalds
2011-11-03  2:19                           ` Linus Torvalds
2011-11-04 20:16                             ` Junio C Hamano
2011-11-04 21:22                               ` Junio C Hamano
2011-11-04 23:10                                 ` Linus Torvalds
2011-11-05  3:55                                   ` Jeff King
2011-11-05  4:37                                   ` Junio C Hamano
2011-11-03 18:16                           ` Junio C Hamano
2011-11-03 18:52                             ` Junio C Hamano
2011-11-03 19:09                               ` Linus Torvalds
2011-11-04 14:59                                 ` Ted Ts'o
2011-11-04 15:14                                   ` Linus Torvalds
2011-11-07  7:52                                     ` Valdis.Kletnieks
2011-11-07 16:24                                       ` Linus Torvalds
2011-11-05  6:36                                 ` Junio C Hamano
2011-11-05 16:41                                   ` Linus Torvalds
2011-11-05 23:49                                     ` Junio C Hamano
2011-11-06  0:53                                       ` Linus Torvalds
2011-11-09 17:26                                 ` Junio C Hamano
2011-11-10  8:02                                   ` Johan Herland
2011-11-10 15:15                                     ` Junio C Hamano
2011-11-10 16:03                                       ` Johan Herland
2011-11-10 17:18                                         ` Junio C Hamano
2011-11-11  1:17                                           ` Johan Herland
2011-11-11  5:26                                             ` Junio C Hamano
2011-11-03 19:06                             ` Linus Torvalds
2011-11-04 21:12                             ` Junio C Hamano
2011-11-04 23:45                               ` Linus Torvalds
2011-11-03  2:55                       ` Jeff King
2011-11-03  3:16                         ` Robin H. Johnson
2011-11-03 18:29                     ` Junio C Hamano
2011-11-01 22:39                 ` Ted Ts'o
2011-11-02 23:34                 ` Junio C Hamano
2011-11-02 23:41                   ` david
2011-11-02 23:42                   ` Linus Torvalds
2011-11-10 13:52                 ` David Woodhouse
2011-11-02 10:53               ` Michael J Gruber
2011-11-02 18:58                 ` Junio C Hamano
2011-11-02 21:05                   ` Michael J Gruber
2011-10-31  8:40     ` Ingo Molnar
2011-10-31 21:46     ` H. Peter Anvin
2011-10-31 22:21       ` Linus Torvalds
2011-10-31 22:23         ` H. Peter Anvin
2011-10-31 22:34           ` Linus Torvalds
2011-11-01  2:17             ` david
2011-11-01  3:25               ` H. Peter Anvin
2011-11-01  3:42                 ` Linus Torvalds
2011-11-01  4:25                   ` hpanvin@gmail.com
2011-11-01  5:19                     ` James Bottomley
2011-11-01 13:13                       ` Henrique de Moraes Holschuh
2011-11-01 17:06                   ` Tony Luck
2011-11-01 17:15                     ` Linus Torvalds
2011-11-12 19:31                       ` Felipe Contreras
2011-11-01  5:03                 ` david
2011-11-01 13:32                   ` Theodore Tso
2011-11-01  3:31               ` Linus Torvalds
2011-11-01  4:58                 ` david
     [not found]     ` <20111031084048.GA11807__21610.4542407722$1320051469$gmane$org@elte.hu>
2011-10-31 22:03       ` Junio C Hamano

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).