Regression caused by commit bc976233a872, ethernet r8169 stops working after system S3

Regression caused by commit bc976233a872, ethernet r8169 stops working after system S3

[Kai-Heng Feng]

Hi Thomas,

A user reported [1] that the Realtek ethernet r8169 stops working after S3  
since v4.15-rc6. The issue still exists in Linus' tree:

[ 150.877998] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready
[ 150.944101] do_IRQ: 3.37 No irq handler for vector
[ 150.944105] r8169 0000:01:00.0 enp1s0: link down
[ 150.944180] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready

My desktop also has this device, which is also affected by this regression.  
So I did a bisect, and this is the bad commit:
bc976233a872 genirq/msi x86/vector: Prevent reservation mode for non  
maskable MSI

After this commit gets reverted, the issue is gone.

Let me know if you need more debug information.

Kai-Heng

[1] https://bugs.launchpad.net/bugs/1752772

Re: Regression caused by commit bc976233a872, ethernet r8169 stops working after system S3

[Thomas Gleixner]

On Sat, 31 Mar 2018, Kai-Heng Feng wrote:
> A user reported [1] that the Realtek ethernet r8169 stops working after S3
> since v4.15-rc6. The issue still exists in Linus' tree:
> 
> [ 150.877998] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready
> [ 150.944101] do_IRQ: 3.37 No irq handler for vector
> [ 150.944105] r8169 0000:01:00.0 enp1s0: link down
> [ 150.944180] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready
> 
> My desktop also has this device, which is also affected by this regression. So
> I did a bisect, and this is the bad commit:
> bc976233a872 genirq/msi x86/vector: Prevent reservation mode for non maskable
> MSI
> 
> After this commit gets reverted, the issue is gone.

Sigh. Can you please apply the debug patch below and provide the output of

# cat /proc/interrupts

and full dmesg after resume.

Thanks,

	tglx

8<------------------
--- a/kernel/irq/cpuhotplug.c
+++ b/kernel/irq/cpuhotplug.c
@@ -134,6 +134,10 @@ static bool migrate_one_irq(struct irq_d
 		brokeaff = false;
 	}
 
+	pr_info("IRQ%u: New affinity: %*pbl effective: %*pbl\n",
+		cpumask_pr_args(affinity),
+		irq_data_get_effective_affinity_mask(d));
+
 	if (maskchip && chip->irq_unmask)
 		chip->irq_unmask(d);
 

Re: Regression caused by commit bc976233a872, ethernet r8169 stops working after system S3

[Kai-Heng Feng]

[-- Attachment #1: Type: text/plain, Size: 1406 bytes --]

Thomas Gleixner <tglx@linutronix.de> wrote:

> On Sat, 31 Mar 2018, Kai-Heng Feng wrote:
>> A user reported [1] that the Realtek ethernet r8169 stops working after S3
>> since v4.15-rc6. The issue still exists in Linus' tree:
>>
>> [ 150.877998] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready
>> [ 150.944101] do_IRQ: 3.37 No irq handler for vector
>> [ 150.944105] r8169 0000:01:00.0 enp1s0: link down
>> [ 150.944180] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready
>>
>> My desktop also has this device, which is also affected by this  
>> regression. So
>> I did a bisect, and this is the bad commit:
>> bc976233a872 genirq/msi x86/vector: Prevent reservation mode for non  
>> maskable
>> MSI
>>
>> After this commit gets reverted, the issue is gone.
>
> Sigh. Can you please apply the debug patch below and provide the output of
>
> # cat /proc/interrupts
>
> and full dmesg after resume.

Weird. I don't see this issue with this debug patch.

dmesg after resume attached.

>
> Thanks,
>
> 	tglx
>
> 8<------------------
> --- a/kernel/irq/cpuhotplug.c
> +++ b/kernel/irq/cpuhotplug.c
> @@ -134,6 +134,10 @@ static bool migrate_one_irq(struct irq_d
>  		brokeaff = false;
>  	}
>
> +	pr_info("IRQ%u: New affinity: %*pbl effective: %*pbl\n",
> +		cpumask_pr_args(affinity),
> +		irq_data_get_effective_affinity_mask(d));
> +
>  	if (maskchip && chip->irq_unmask)
>  		chip->irq_unmask(d);


[-- Attachment #2: dmesg --]
[-- Type: application/octet-stream, Size: 39043 bytes --]

[  555.959240] PM: suspend entry (deep)
[  555.959244] PM: Syncing filesystems ... done.
[  556.793867] Freezing user space processes ... (elapsed 0.002 seconds) done.
[  556.796442] OOM killer disabled.
[  556.796443] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[  556.797832] Suspending console(s) (use no_console_suspend to debug)
[  556.816739] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[  556.816879] sd 0:0:0:0: [sda] Stopping disk
[  557.975398] ACPI: EC: interrupt blocked
[  558.014366] ACPI: Preparing to enter system sleep state S3
[  558.014563] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
[  558.016597] ACPI: EC: event blocked
[  558.016599] ACPI: EC: EC stopped
[  558.016600] PM: Saving platform NVS memory
[  558.016930] Disabling non-boot CPUs ...
[  558.229333] IRQ6: New affinity: 0,64-69,128-133,257,259-261,263,267-269,273,275-277,281-286,290-295,298-299,304,306,308,311-313,315-316,384-389,449,452,454,456,460-461,465,467-469,473-478,482-487,490-491,496,498,500,503-505,507-508,704,710,712,720-723,725-727,833,836,838-839,844-845,849,851-853,857-862,866-871,874-875,880,882,884,887-889,891-892,1029,1037,1040,1042-1043,1045,1049,1053-1054,1058,1060-1062,1065,1067,1080-1081,1083-1084,1088,1090-1091,1093-1094,1096,1098,1101-1102,1104,1106-1107,1109-1110,1112-1115,1117-1118,1121,1124-1126,1128,1131-1134,1138,1140-1141,1153,1157,1161,1165,1177-1182,1186-1191,1194-1195,1200,1202,1204,1207-1209,1211-1212,1280,1408-1409,1413-1416,1418-1439,1441-1444,1446-1447,1449-1450,1452-1458,1460-1467,1469-1471,1536-1539,1543-1551,1554-1558,1560-1564,1566-1570,1572-1575,1577-1591,1593-1599,1601-1602,1604,1607-1608,1610,1612-1614,1616,1618-1619,1621-1622,1624-1625,1630,1641,1643,1645,1662-1664,1669,1671,1673-1681,1683,1685-1688,1692,1694-1700,1703-1704,170
[  558.297381] IRQ6: New affinity: 0,64-65,128,130,133-134,140-142,144-148,150,155-157,160,164-165,176,178,180,183-185,187-188,256-257,261-262,264,266,268-270,273,276-278,280-281,284-286,288,290-291,293,296,300-301,312-313,315-316,323-325,328,331-333,337,339-341,344,348-349,352-353,356-357,368,370,372,375-377,379-380,387-389,393,395-397,404-405,418-423,426-427,432,434,436,439-441,443-444,451-453,456,459-461,465,467-469,472,476-477,480,482,484-485,496,498,500,503-505,507-508,514,516-517,520,522-523,525,528,532-533,546-551,554-555,560,562,564,567-569,571-572,578,581-582,585,588-590,592,594-595,597-598,610-615,618-619,624,626,628,631-633,635-636,648,674-679,682-683,688,690,692,695-697,699-700,704-706,708-709,712,714-715,717,724-725,729,731-733,736,740-741,745-747,749,756-757,768-769,772-774,776-778,781-782,788-789,802-807,810-811,816,818,820,823-825,827-828,836-837,900-902,904-907,909-910,912-915,917-918,922-923,925-926,938-939,944,946,948,951-953,955-956,960,964,969,971-973,978-979,985-986,9
[  558.366184] IRQ6: New affinity: 2,68-69,73,75-77,84-85,89,91-93,100-101,105,107-109,116-117,128,131,133-134,137,140-141,144-145,149-150,152,154-155,157,160,164-165,168,170,172-173,184-185,187-188,193-195,197,202,205-206,208,213-214,218,220-222,224,229-230,240,242,244,247-249,251-252,259-261,264,266-267,269,276-277,281,283-285,288,292-293,297-299,301,308-309,320,326,333,336,338,341,384-385,388-389,393-394,396-397,409-414,418-423,426-427,432,434,436,439-441,443-444,448-449,452-453,456,459-461,473-478,482-487,490-491,496,498,500,503-505,507-508,512,515,517-518,521,524-525,528-529,533-534,536,538-539,541,546,548-549,560,562,564,567-569,571-572,577-579,581,586,589-590,592,597-598,602,604-606,608,613-614,624,626,628,631-633,635-636,640,645-646,648,650,652-654,658,660-662,664-667,669-670,682-683,688,690,692,695-697,699-700,704-705,707,709-710,712,714,717-718,721,724-726,729-731,733-734,736,738,741-742,746-747,749-750,760-761,763-764,768,771,773-774,777,780-781,784-785,789-790,792,794-795,797,8
[  558.366188] IRQ 39: no longer affine to CPU1
[  558.367186] ------------[ cut here ]------------
[  558.367188] field width 1845707312 too large
[  558.367190] WARNING: CPU: 1 PID: 15 at lib/vsprintf.c:2170 set_field_width+0x73/0x80
[  558.367191] Modules linked in: rfcomm ccm kvm_amd arc4 cmac kvm iwlmvm bnep mac80211 snd_hda_codec_hdmi snd_hda_codec_realtek irqbypass crct10dif_pclmul crc32_pclmul snd_hda_codec_generic ghash_clmulni_intel snd_hda_intel nls_iso8859_1 pcbc aesni_intel snd_hda_codec snd_hwdep snd_hda_core iwlwifi btusb btrtl btbcm btintel aes_x86_64 snd_pcm bluetooth snd_seq crypto_simd cryptd cfg80211 glue_helper joydev input_leds snd_timer snd_seq_device ecdh_generic eeepc_wmi asus_wmi snd wmi_bmof k10temp fam15h_power sparse_keymap soundcore shpchp mac_hid sch_fq_codel parport_pc ppdev lp parport ip_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq nouveau hid_microsoft hid_generic usbhid hid video i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mxm_wmi ahci r8169 i2c_piix4
[  558.367242]  libahci mii wmi
[  558.367246] CPU: 1 PID: 15 Comm: migration/1 Not tainted 4.16.0-rc7+ #10
[  558.367248] Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 LE R2.0, BIOS 2701 03/24/2016
[  558.367249] RIP: 0010:set_field_width+0x73/0x80
[  558.367250] RSP: 0018:ffffb03701947bb8 EFLAGS: 00010086
[  558.367252] RAX: 0000000000000000 RBX: ffffb03701947bf0 RCX: 0000000000000002
[  558.367254] RDX: 0000000000000020 RSI: ffffffff9fb79640 RDI: ffff9ccc7ec58508
[  558.367255] RBP: ffffb03701947bc8 R08: 0000000000000000 R09: abcc77118461cefd
[  558.367257] R10: ffffb03701947d38 R11: ffff9ccc7ec5855d R12: 000000006e034230
[  558.367258] R13: 00000000000003e0 R14: ffffffff9fab6fa9 R15: ffffffff9fab6fa9
[  558.367260] FS:  0000000000000000(0000) GS:ffff9ccc7ec40000(0000) knlGS:0000000000000000
[  558.367261] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  558.367262] CR2: 00005650b8dd99c0 CR3: 000000041e82a000 CR4: 00000000000406e0
[  558.367263] Call Trace:
[  558.367264]  vsnprintf+0x317/0x4e0
[  558.367265]  vscnprintf+0xd/0x30
[  558.367266]  vprintk_emit+0x94/0x450
[  558.367267]  vprintk_default+0x29/0x50
[  558.367268]  vprintk_func+0x27/0x60
[  558.367269]  printk+0x52/0x6e
[  558.367270]  irq_migrate_all_off_this_cpu+0x29c/0x320
[  558.367271]  fixup_irqs+0x34/0x100
[  558.367272]  ? __switch_to_asm+0x34/0x70
[  558.367273]  ? __switch_to_asm+0x40/0x70
[  558.367274]  cpu_disable_common+0x240/0x260
[  558.367275]  native_cpu_disable+0x25/0x30
[  558.367276]  take_cpu_down+0x35/0xa0
[  558.367277]  multi_cpu_stop+0x74/0x100
[  558.367278]  ? cpu_stopper_thread+0x110/0x110
[  558.367279]  cpu_stopper_thread+0x97/0x110
[  558.367280]  smpboot_thread_fn+0xfc/0x170
[  558.367281]  kthread+0x121/0x140
[  558.367282]  ? sort_range+0x30/0x30
[  558.367283]  ? kthread_create_worker_on_cpu+0x70/0x70
[  558.367284]  ret_from_fork+0x22/0x40
[  558.367285] Code: ff ff 7f 00 0f 4f c2 88 43 01 88 63 02 c1 e8 10 88 43 03 5b 41 5c 5d c3 48 c7 c7 28 96 b7 9f c6 05 a1 36 a9 00 01 e8 5d 7c 72 ff <0f> 0b eb bb 66 0f 1f 84 00 00 00 00 00 89 f0 40 88 77 06 88 67 
[  558.367319] ---[ end trace 738a38570d9970e9 ]---
[  558.369394] smpboot: CPU 1 is now offline
[  558.547799] IRQ6: New affinity: 3,65-67,69-70,72-75,77-78,82,85-86,88,90,93-94,100-101,112,114,116,119-121,123-124,129-131,133-134,136-139,141-142,146,149-150,152,154,157-158,164-165,176,178,180,183-185,187-188,196-197,204-205,212-213,220-221,225,227-229,236-237,241,244-245,258,260-261,264,266-267,269,276-277,281,283-285,288,292-293,297-299,301,308-309,321-323,325-326,328-331,333-334,338,341-342,344,346,349-350,356-357,368,370,372,375-377,379-380,384,386-387,389-390,392-395,397-398,402,405-406,408,410,412-414,418-419,421-422,424,426,429-430,440-441,443-444,449-451,453-454,456-459,461-462,466,469-470,472,474,477-478,484-485,496,498,500,503-505,507-508,512-513,517-518,524-526,528,530,532-534,537,540-541,554-555,560,562,564,567-569,571-572,576,580-581,588-589,601-606,610-615,618-619,624,626,628,631-633,635-636,640-641,644,646,648-649,652,654,658,662,666,668,670,682-683,688,690,692,695-697,699-700,705-707,709-710,712-715,717-718,722,725-726,728,730,733-734,740-741,752,754,756,759-761,763-76
[  558.695486] IRQ6: New affinity: 3,68-69,81,83-85,89-94,98-103,106-107,112,114,116,119-121,123-124,130,132-133,145,147-149,153-158,162-167,170-171,176,178,180,183-185,187-188,192-193,196-198,202,204-206,208,213-214,218,220-222,224,226,229-230,233,236-237,248-249,251-252,258,260-262,266,268-270,272,275-278,280-281,284,286,292-293,304,306,308,311-313,315-316,320-321,325-326,332-334,336,338,340-342,348-349,362-363,368,370,372,375-377,379-380,386,388-390,394,396-398,400,403-406,410,412-413,420-421,432,434,436,439-441,443-444,448,452-453,457,459-461,464-466,468-469,482-487,490-491,496,498,500,503-505,507-508,514,516-517,521,523-525,529,532-533,546-551,554-555,560,562,564,567-569,571-572,576,580-581,593,595-597,601-606,610-615,618-619,624,626,628,631-633,635-636,640,642,644-645,657,659-661,665-670,674-679,682-683,688,690,692,695-697,699-700,706,708-710,714,716-718,720,723-726,729,732-733,746-747,752,754,756,759-761,763-764,770,772-774,778,780-782,784,787-790,792,796-797,810-811,816,818,820,823
[  558.764959] IRQ6: New affinity: 3,67-69,72,75-77,81,83-85,88,92-93,106-107,112,114,116,119-121,123-124,128,131,133-134,137,140-141,144-145,149-150,152,154-155,157,164-165,176,178,180,183-185,187-188,192,197-198,202,204-206,208,213-214,216,220-221,234-235,240,242,244,247-249,251-252,257-259,261,264-265,268-270,272,275-278,280,282-283,285-286,290,292-294,296,301-302,305,309-310,321-323,325-326,328-331,333-334,336,338,340-342,345-346,348-350,352,354,357-358,360,365-366,368,370,372-374,384,386,388-390,392-393,396-398,401,405-406,408,411-413,426-427,432,434,436,439-441,443-444,448-449,452-453,458,460-461,473-478,482-487,490-491,496,498,500,503-505,507-508,512-513,516-517,520,524-525,537-542,546-551,554-555,560,562,564,567-569,571-572,576-577,580-581,587-589,601-606,610-615,618-619,624,626,628,631-633,635-636,640,642,645-646,652-654,656-660,662,667-669,673,676-677,688,690,692,695-697,699-700,711,714,717-721,724-725,729,731-732,734,738-739,742-743,746-748,751-768,771,773-774,777,780-781,784-78
[  558.834917] IRQ6: New affinity: 4,68-69,73,75-77,84-85,89,91-93,100-101,105,107-109,116-117,128,131,133-134,137,140-141,144-145,149-150,152,154-155,157,160,164-165,168,170,172-173,184-185,187-188,193-195,197,202,205-206,208,213-214,218,220-222,224,229-230,240,242,244,247-249,251-252,259-261,264,266-267,269,276-277,281,283-285,288,292-293,297-299,301,308-309,320,326,333,336,338,341,384-385,388-389,393-394,396-397,409-414,418-423,426-427,432,434,436,439-441,443-444,448-449,452-453,456,459-461,473-478,482-487,490-491,496,498,500,503-505,507-508,512,515,517-518,521,524-525,528-529,533-534,536,538-539,541,546,548-549,560,562,564,567-569,571-572,577-579,581,586,589-590,592,597-598,602,604-606,608,613-614,624,626,628,631-633,635-636,640,645-646,648,650,652-654,658,660-662,664-667,669-670,682-683,688,690,692,695-697,699-700,704-705,707,709-710,712,714,717-718,721,724-726,729-731,733-734,736,738,741-742,746-747,749-750,760-761,763-764,768,771,773-774,777,780-781,784-785,789-790,792,794-795,797,8
[  558.837969] smpboot: CPU 2 is now offline
[  558.982744] IRQ6: New affinity: 0,68,70,73-75,78,84,86,92-93,97,100-101,108-109,116-117,132,134,137-139,142,148,150,156-157,160-161,166,172-173,177,180-181,193-195,197-198,200-203,205-206,210,213-214,216,218,221-222,228-229,240,242,244,247-249,251-252,257-259,261-262,264-267,269-270,274,277-278,280,282,285-286,292-293,304,306,308,311-313,315-316,320,324-325,331-333,345-350,354-359,362-363,368,370,372,375-377,379-380,384-385,389-390,396-398,400,402,404-406,408-409,412-413,426-427,432,434,436,439-441,443-444,452-453,460-461,468-469,476-477,481,483-485,492-493,496,498,500-501,513-515,517-518,520-523,525-526,530,533-534,536,538,541-542,548-549,560,562,564,567-569,571-572,576-577,581-582,588-590,592,594,596-598,604-605,618-619,624,626,628,631-633,635-636,641-643,645-646,648-651,653-654,658,661-662,664,666,669-670,676-677,688,690,692,695-697,699-700,705-706,708-709,712,714-715,717,724-725,729,731-733,736,740-741,745-747,749,756-757,775,785-788,794-795,797-798,802-803,806-807,810-812,815-831,8
[  558.982749] IRQ 17: no longer affine to CPU3
[  559.109146] IRQ6: New affinity: 4,65-67,69-70,72-75,77-78,82,85-86,88,90,93-94,100-101,112,114,116,119-121,123-124,129-131,133-134,136-139,141-142,146,149-150,152,154,157-158,164-165,176,178,180,183-185,187-188,196-197,204-205,212-213,220-221,225,227-229,236-237,241,244-245,258,260-261,264,266-267,269,276-277,281,283-285,288,292-293,297-299,301,308-309,321-323,325-326,328-331,333-334,338,341-342,344,346,349-350,356-357,368,370,372,375-377,379-380,384,386-387,389-390,392-395,397-398,402,405-406,408,410,412-414,418-419,421-422,424,426,429-430,440-441,443-444,449-451,453-454,456-459,461-462,466,469-470,472,474,477-478,484-485,496,498,500,503-505,507-508,512-513,517-518,524-526,528,530,532-534,537,540-541,554-555,560,562,564,567-569,571-572,576,580-581,588-589,601-606,610-615,618-619,624,626,628,631-633,635-636,640-641,644,646,648-649,652,654,658,662,666,668,670,682-683,688,690,692,695-697,699-700,705-707,709-710,712-715,717-718,722,725-726,728,730,733-734,740-741,752,754,756,759-761,763-76
[  559.109149] IRQ 21: no longer affine to CPU3
[  559.235272] IRQ6: New affinity: 5,65-67,69-70,72-75,77-78,82,85-86,88,90,93-94,100-101,112,114,116,119-121,123-124,129-131,133-134,136-139,141-142,146,149-150,152,154,157-158,164-165,176,178,180,183-185,187-188,224,228,256,259,262,265-266,268,270,273,276,278,280-281,284,286,298-299,304,306,308,311-313,315-316,320,322,324-326,328-329,332-334,337,341-342,344-345,348-349,362-363,368,370,372,375-377,379-380,385-387,389-390,392-395,397-398,402,405-406,408,410,413-414,420-421,432,434,436,439-441,443-444,452,454,457-459,462,468,470,476-477,480-481,486,492-493,496,500-501,516,518,521-523,526,532,534,540-541,544,548-549,556-557,564-565,580,582,585-587,590,596,598,604-605,608-609,612-613,620-621,625,630,640,644-645,648,650-651,653,660-661,665,667-669,672,676-677,681-683,685,692-693,704-705,708-710,713,716-718,720-724,726,728,730-731,733-734,736-739,741-742,746,749-750,760-761,763-764,768,770,772-773,785,787-789,793-798,802-807,810-811,816,818,820,823-825,827-828,833-835,837-838,840-843,845-846,85
[  559.235274] IRQ 23: no longer affine to CPU3
[  559.355852] IRQ6: New affinity: 0,68-69,81,83-85,89-94,98-103,106-107,112,114,116,119-121,123-124,130,132-133,145,147-149,153-158,162-167,170-171,176,178,180,183-185,187-188,192-193,196-198,202,204-206,208,213-214,218,220-222,224,226,229-230,233,236-237,248-249,251-252,258,260-262,266,268-270,272,275-278,280-281,284,286,292-293,304,306,308,311-313,315-316,320-321,325-326,332-334,336,338,340-342,348-349,362-363,368,370,372,375-377,379-380,386,388-390,394,396-398,400,403-406,410,412-413,420-421,432,434,436,439-441,443-444,448,452-453,457,459-461,464-466,468-469,482-487,490-491,496,498,500,503-505,507-508,514,516-517,521,523-525,529,532-533,546-551,554-555,560,562,564,567-569,571-572,576,580-581,593,595-597,601-606,610-615,618-619,624,626,628,631-633,635-636,640,642,644-645,657,659-661,665-670,674-679,682-683,688,690,692,695-697,699-700,706,708-710,714,716-718,720,723-726,729,732-733,746-747,752,754,756,759-761,763-764,770,772-774,778,780-782,784,787-790,792,796-797,810-811,816,818,820,823
[  559.411069] IRQ6: New affinity: 4,67-69,72,75-77,81,83-85,88,92-93,106-107,112,114,116,119-121,123-124,128,131,133-134,137,140-141,144-145,149-150,152,154-155,157,164-165,176,178,180,183-185,187-188,192,197-198,202,204-206,208,213-214,216,220-221,234-235,240,242,244,247-249,251-252,257-259,261,264-265,268-270,272,275-278,280,282-283,285-286,290,292-294,296,301-302,305,309-310,321-323,325-326,328-331,333-334,336,338,340-342,345-346,348-350,352,354,357-358,360,365-366,368,370,372-374,384,386,388-390,392-393,396-398,401,405-406,408,411-413,426-427,432,434,436,439-441,443-444,448-449,452-453,458,460-461,473-478,482-487,490-491,496,498,500,503-505,507-508,512-513,516-517,520,524-525,537-542,546-551,554-555,560,562,564,567-569,571-572,576-577,580-581,587-589,601-606,610-615,618-619,624,626,628,631-633,635-636,640,642,645-646,652-654,656-660,662,667-669,673,676-677,688,690,692,695-697,699-700,711,714,717-721,724-725,729,731-732,734,738-739,742-743,746-748,751-768,771,773-774,777,780-781,784-78
[  559.411071] IRQ 31: no longer affine to CPU3
[  559.538768] IRQ6: New affinity: 5,68-70,72,75,77-78,81-83,85-86,88-92,94,96-97,101-102,105-106,109-110,112-114,117-118,132-133,136-138,140-141,153-158,162-167,170-171,176,178,180,183-185,187-188,200,203,207,209-210,213-214,220,222,227-228,237-257,260-262,264,267,269-270,273-275,277-278,280-282,285-286,290-291,293-294,296,298,301-302,312-313,315-316,320-324,326,328-332,334,340-342,344,349-350,353,356-358,360,365-366,368,370-371,373-374,388-389,392,394,397-398,409-414,418-423,426-427,432,434,436,439-441,443-444,449-451,453,457,461-462,464-465,468-470,472-473,476-478,490-491,496,498,500,503-505,507-508,513,517-518,520-522,524-525,529,531-533,540-541,554-555,560,562,564,567-569,571-572,577-579,581,586,589-590,592,597-598,602,604-606,608,613-614,624,626,628,631-633,635-636,646,650-651,654-655,658-662,668,670,675-676,685-703,769-771,773,776-777,780-782,784,787-790,792,794-795,797-798,802,804-806,808,813-814,817,821-822,833-835,837,842,845-846,848,853-854,858,860-862,864,869-870,880,882,884,88
[  559.538771] IRQ 41: no longer affine to CPU3
[  559.539814] smpboot: CPU 3 is now offline
[  559.761110] IRQ6: New affinity: 0,64-128,130-131,133-134,136,138,141-142,144,146-147,149-150,152-155,157-158,161,164-166,168,171-174,176-177,180-181,192,194-195,197-198,200,202,205-206,208,210-211,213-214,216-219,221-222,225,228-230,232,235-238,244-245,256,258-259,261-262,264,266,269-270,272,274-275,277-278,280-283,285-286,289,292-294,296,299-302,305-306,308-309,330,333-334,341,346-347,349-350,354-355,358-359,362-364,367-383,641,644-645,648,650-653,657,659-661,665-670,674-679,682-683,688,690,692,695-697,699-700,705,707,711,714-717,721,723-725,729-734,738-743,746-747,752,754,756,759-761,763-764,897,899,901-903,905-906,908-909,913,915-917,921-926,930-935,938-939,944,946,948,951-953,955-956,960,962-963,965-966,968,970,973-974,976,978-979,981-982,984-987,989-990,993,996-998,1000,1003-1006,1008-1009,1012-1013,1029,1089,1091,1094,1097-1101,1105,1107-1109,1113-1118,1122-1127,1130-1131,1136,1138,1140,1143-1145,1147-1148,1186-1191,1194-1195,1200,1202,1204,1207-1209,1211-1212,1217,1219-1220,1227-
[  559.829259] IRQ6: New affinity: 5,64,67,69-70,73,76-77,80-81,85-86,88,90-91,93,97-98,100-101,112,114,116,119-121,123-124,194,196-197,200,202-203,205,208,212-213,217,219-221,224,228-229,233-235,237,244-245,258,260-261,264,266-267,269,272,276-277,290-295,298-299,304,306,308,311-313,315-316,320,323,325-326,329,332-333,336-337,341-342,344,346-347,349,356-357,368,370,372,375-377,379-380,452-453,457,459-461,468-469,473,475-477,484-485,489,491-493,500-501,512,515,517-518,521,524-525,528-529,533-534,536,538-539,541,544,548-549,554,556-557,568-569,571-572,578,581-582,585,588-590,592,594-595,597-598,610-615,618-619,624,626,628,631-633,635-636,640,643,645-646,649,652-653,656-657,661-662,664,666-667,669,672,676-677,688,690,692,695-697,699-700,707,709-710,712,714,717-718,720,725-726,730,733-734,736,738-739,741,745,748-749,760-761,763-764,768,774,777,780,782,785,788-789,794,796-797,802-807,810-811,816,818,820,823-825,827-828,833-835,837,841-843,845,848-851,853,856-857,860-862,864-866,869-870,876-877,
[  559.829263] IRQ 19: no longer affine to CPU4
[  559.984691] IRQ6: New affinity: 5,64,66,68-69,72,74-75,77,84-85,89,91-93,96,100-101,105-107,109,116-117,129-131,133-134,136-139,141-142,146,149-150,152,154,157-158,164-165,176,178,180,183-185,187-188,196-198,200-201,204-206,210,212-214,216-219,221-222,225,228-230,232,234,237-238,248-249,251-252,264,290-295,298-299,304,306,308,311-313,315-316,320,324-325,331-333,336,339-341,345,347-349,356-357,368,370,372,375-377,379-380,388,390,393-395,398,404,406,412-413,416-417,422,428-429,433-434,438,449,452-453,456,458-459,461,468-469,473,475-477,480,484-485,489-491,493,500-501,512-513,515,517-518,522,525-526,529,533-534,546-551,554-555,560,562,564,567-569,571-572,577-579,581-582,584-587,589-590,594,597-598,600,602,605-606,612-613,624,626,628,631-633,635-636,640-641,643,645-646,648-650,653-654,658,661-662,665,669-670,672-675,677-678,680-681,685-686,696-697,699-700,708-709,716-717,724-725,732-733,737,739-741,748-749,752-753,756-757,769-770,772-773,785,787-789,793-798,802-807,810-811,816,818,820,823-8
[  560.140307] IRQ6: New affinity: 0,65-67,69-70,72-75,77-78,82,85-86,88,90,93-94,100-101,112,114,116,119-121,123-124,129-131,133-134,136-139,141-142,146,149-150,152,154,157-158,164-165,176,178,180,183-185,187-188,196-197,204-205,212-213,220-221,225,227-229,236-237,241,244-245,258,260-261,264,266-267,269,276-277,281,283-285,288,292-293,297-299,301,308-309,321-323,325-326,328-331,333-334,338,341-342,344,346,349-350,356-357,368,370,372,375-377,379-380,384,386-387,389-390,392-395,397-398,402,405-406,408,410,412-414,418-419,421-422,424,426,429-430,440-441,443-444,449-451,453-454,456-459,461-462,466,469-470,472,474,477-478,484-485,496,498,500,503-505,507-508,512-513,517-518,524-526,528,530,532-534,537,540-541,554-555,560,562,564,567-569,571-572,576,580-581,588-589,601-606,610-615,618-619,624,626,628,631-633,635-636,640-641,644,646,648-649,652,654,658,662,666,668,670,682-683,688,690,692,695-697,699-700,705-707,709-710,712-715,717-718,722,725-726,728,730,733-734,740-741,752,754,756,759-761,763-76
[  560.209817] IRQ6: New affinity: 5,67-69,72,75-77,81,83-85,91-93,106-107,112,114,116,119-121,123-124,130,132-133,136,138-139,141,145,148-149,162-167,170-171,176,178,180,183-185,187-188,192-193,196-197,202,204-205,217-222,226-231,234-235,240,242,244,247-249,251-252,256,259,261-262,265,268-269,272-273,277-278,280,282-283,285,291-293,304,306,308,311-313,315-316,320,323,325-326,329,332-333,336-337,341-342,344,346-347,349,352,356-357,361,364-365,376-377,379-380,384,387,389-390,393,396-397,400-401,405-406,408,410-411,413,416-418,420-421,432,434,436,439-441,443-444,449,452-453,457,460-461,473-478,482-487,490-491,496,498,500,503-505,507-508,513-515,517-518,520-523,525-526,528,530,532-534,537-538,540-542,544,546,549-550,552,557-558,560,562,564-566,576,578,581-582,588-590,592-596,598,604-605,612-613,624,626,628,631-633,635-636,640,645-646,650,652-654,656,661-662,664,668-669,682-683,688,690,692,695-697,699-700,768-772,774,776-780,782,788-790,792,797-798,801,804-806,808,813-814,816,818-819,821-822,8
[  560.278035] IRQ6: New affinity: 0,67-69,72,75-77,81,83-85,88,92-93,106-107,112,114,116,119-121,123-124,128,131,133-134,137,140-141,144-145,149-150,152,154-155,157,164-165,176,178,180,183-185,187-188,192,197-198,202,204-206,208,213-214,216,220-221,234-235,240,242,244,247-249,251-252,257-259,261,264-265,268-270,272,275-278,280,282-283,285-286,290,292-294,296,301-302,305,309-310,321-323,325-326,328-331,333-334,336,338,340-342,345-346,348-350,352,354,357-358,360,365-366,368,370,372-374,384,386,388-390,392-393,396-398,401,405-406,408,411-413,426-427,432,434,436,439-441,443-444,448-449,452-453,458,460-461,473-478,482-487,490-491,496,498,500,503-505,507-508,512-513,516-517,520,524-525,537-542,546-551,554-555,560,562,564,567-569,571-572,576-577,580-581,587-589,601-606,610-615,618-619,624,626,628,631-633,635-636,640,642,645-646,652-654,656-660,662,667-669,673,676-677,688,690,692,695-697,699-700,711,714,717-721,724-725,729,731-732,734,738-739,742-743,746-748,751-768,771,773-774,777,780-781,784-78
[  560.348235] IRQ6: New affinity: 0,68-69,73,75-77,84-85,89,91-93,100-101,105,107-109,116-117,128,131,133-134,137,140-141,144-145,149-150,152,154-155,157,160,164-165,168,170,172-173,184-185,187-188,193-195,197,202,205-206,208,213-214,218,220-222,224,229-230,240,242,244,247-249,251-252,259-261,264,266-267,269,276-277,281,283-285,288,292-293,297-299,301,308-309,320,326,333,336,338,341,384-385,388-389,393-394,396-397,409-414,418-423,426-427,432,434,436,439-441,443-444,448-449,452-453,456,459-461,473-478,482-487,490-491,496,498,500,503-505,507-508,512,515,517-518,521,524-525,528-529,533-534,536,538-539,541,546,548-549,560,562,564,567-569,571-572,577-579,581,586,589-590,592,597-598,602,604-606,608,613-614,624,626,628,631-633,635-636,640,645-646,648,650,652-654,658,660-662,664-667,669-670,682-683,688,690,692,695-697,699-700,704-705,707,709-710,712,714,717-718,721,724-726,729-731,733-734,736,738,741-742,746-747,749-750,760-761,763-764,768,771,773-774,777,780-781,784-785,789-790,792,794-795,797,8
[  560.351283] smpboot: CPU 4 is now offline
[  560.492727] IRQ6: New affinity: 0,64,66,68-70,72-73,76-78,81,85-86,88,90,92-93,106-107,112,114,116,119-121,123-124,128-133,256,258,260-262,264-265,268-270,273,277-278,281,283-285,291,293-294,296,298,300-302,305,309-310,320,324-325,337,339-341,345-350,354-359,362-363,368,370,372,375-377,379-380,384,386,388-390,392-393,396-398,401,405-406,408-409,413-414,416-419,421-422,425,428-430,432,434,437-438,448-479,512-543,577-579,581-582,584-587,589-590,594,597-598,600,602,605-606,612-613,624,626,628,631-633,635-636,640-641,645-646,652-654,656,658,660-662,664,667,669-670,674,677-678,682-683,685-686,688,690,693-694,704-705,707,709-710,712,714,717-718,721,724-726,729-731,733-734,736,738,741-742,746-747,749-750,760-761,763-764,772-773,780-781,788-789,796-797,801,803-805,812-813,820-821,833,837-838,840,845-846,850,852-854,858,860-862,864,866,869-870,873,876-878,880,883-886,896,900-901,907-909,912,915-917,921,923-925,928-929,932-933,939-941,946,948-949,960,964-965,971-973,976,979-981,985,987-989,993,99
[  560.492731] IRQ 18: no longer affine to CPU5
[  560.547925] IRQ6: New affinity: 0,64,67,69-70,73,76-77,80-81,85-86,88,90-91,93,97-98,100-101,112,114,116,119-121,123-124,194,196-197,200,202-203,205,208,212-213,217,219-221,224,228-229,233-235,237,244-245,258,260-261,264,266-267,269,272,276-277,290-295,298-299,304,306,308,311-313,315-316,320,323,325-326,329,332-333,336-337,341-342,344,346-347,349,356-357,368,370,372,375-377,379-380,452-453,457,459-461,468-469,473,475-477,484-485,489,491-493,500-501,512,515,517-518,521,524-525,528-529,533-534,536,538-539,541,544,548-549,554,556-557,568-569,571-572,578,581-582,585,588-590,592,594-595,597-598,610-615,618-619,624,626,628,631-633,635-636,640,643,645-646,649,652-653,656-657,661-662,664,666-667,669,672,676-677,688,690,692,695-697,699-700,707,709-710,712,714,717-718,720,725-726,730,733-734,736,738-739,741,745,748-749,760-761,763-764,768,774,777,780,782,785,788-789,794,796-797,802-807,810-811,816,818,820,823-825,827-828,833-835,837,841-843,845,848-851,853,856-857,860-862,864-866,869-870,876-877,
[  560.673735] IRQ6: New affinity: 0,64,66,68-69,72,74-75,77,84-85,89,91-93,96,100-101,105-107,109,116-117,129-131,133-134,136-139,141-142,146,149-150,152,154,157-158,164-165,176,178,180,183-185,187-188,196-198,200-201,204-206,210,212-214,216-219,221-222,225,228-230,232,234,237-238,248-249,251-252,264,290-295,298-299,304,306,308,311-313,315-316,320,324-325,331-333,336,339-341,345,347-349,356-357,368,370,372,375-377,379-380,388,390,393-395,398,404,406,412-413,416-417,422,428-429,433-434,438,449,452-453,456,458-459,461,468-469,473,475-477,480,484-485,489-491,493,500-501,512-513,515,517-518,522,525-526,529,533-534,546-551,554-555,560,562,564,567-569,571-572,577-579,581-582,584-587,589-590,594,597-598,600,602,605-606,612-613,624,626,628,631-633,635-636,640-641,643,645-646,648-650,653-654,658,661-662,665,669-670,672-675,677-678,680-681,685-686,696-697,699-700,708-709,716-717,724-725,732-733,737,739-741,748-749,752-753,756-757,769-770,772-773,785,787-789,793-798,802-807,810-811,816,818,820,823-8
[  560.673737] IRQ 20: no longer affine to CPU5
[  560.728832] IRQ6: New affinity: 0,66,68-69,72,74-75,77,80,84-85,89,91-93,96,100-101,105-107,109,116-117,131,133-134,138,141-142,144,146-147,149-150,152,155,157-158,160,162-163,165-166,171,173-174,177,179-182,256,259,261-262,265,268-269,272-273,277-278,280,282-283,285,288,292-293,300-301,312-313,315-316,320-321,324-325,328,332-333,345-350,354-359,362-363,368,370,372,375-377,379-380,384,387,389-390,393,396-397,400-401,405-406,408,410-411,413,418,420-421,432,434,436,439-441,443-444,448-449,451,453-454,456,458,461-462,465,468-470,473-475,477-478,480,482,485-486,490-491,493-494,504-505,507-508,512,515,517-518,521,524-525,528-529,533-534,536,538-539,541,547-549,560,562,564,567-569,571-572,576,579,581-582,585,588-589,592-593,597-598,600,602-603,605,608,612-613,618,620-621,632-633,635-636,645,647,649,674-679,682-683,688,690,692,695-697,699-700,709,768,832-833,896,898,901-902,908-910,912-916,918,923-925,928,932-933,944,946,948,951-953,955-956,1024-1025,1029-1030,1032,1034,1036-1038,1041,1044-104
[  560.728834] IRQ 22: no longer affine to CPU5
[  560.857904] IRQ6: New affinity: 0,65-67,69-70,72-75,77-78,82,85-86,88,90,93-94,100-101,112,114,116,119-121,123-124,129-131,133-134,136-139,141-142,146,149-150,152,154,157-158,164-165,176,178,180,183-185,187-188,224,228,256,259,262,265-266,268,270,273,276,278,280-281,284,286,298-299,304,306,308,311-313,315-316,320,322,324-326,328-329,332-334,337,341-342,344-345,348-349,362-363,368,370,372,375-377,379-380,385-387,389-390,392-395,397-398,402,405-406,408,410,413-414,420-421,432,434,436,439-441,443-444,452,454,457-459,462,468,470,476-477,480-481,486,492-493,496,500-501,516,518,521-523,526,532,534,540-541,544,548-549,556-557,564-565,580,582,585-587,590,596,598,604-605,608-609,612-613,620-621,625,630,640,644-645,648,650-651,653,660-661,665,667-669,672,676-677,681-683,685,692-693,704-705,708-710,713,716-718,720-724,726,728,730-731,733-734,736-739,741-742,746,749-750,760-761,763-764,768,770,772-773,785,787-789,793-798,802-807,810-811,816,818,820,823-825,827-828,833-835,837-838,840-843,845-846,85
[  560.912931] IRQ6: New affinity: 0,67-69,72,75-77,81,83-85,91-93,106-107,112,114,116,119-121,123-124,130,132-133,136,138-139,141,145,148-149,162-167,170-171,176,178,180,183-185,187-188,192-193,196-197,202,204-205,217-222,226-231,234-235,240,242,244,247-249,251-252,256,259,261-262,265,268-269,272-273,277-278,280,282-283,285,291-293,304,306,308,311-313,315-316,320,323,325-326,329,332-333,336-337,341-342,344,346-347,349,352,356-357,361,364-365,376-377,379-380,384,387,389-390,393,396-397,400-401,405-406,408,410-411,413,416-418,420-421,432,434,436,439-441,443-444,449,452-453,457,460-461,473-478,482-487,490-491,496,498,500,503-505,507-508,513-515,517-518,520-523,525-526,528,530,532-534,537-538,540-542,544,546,549-550,552,557-558,560,562,564-566,576,578,581-582,588-590,592-596,598,604-605,612-613,624,626,628,631-633,635-636,640,645-646,650,652-654,656,661-662,664,668-669,682-683,688,690,692,695-697,699-700,768-772,774,776-780,782,788-790,792,797-798,801,804-806,808,813-814,816,818-819,821-822,8
[  560.967999] IRQ6: New affinity: 0,64,128-129,192,194,197-198,204-206,208-212,214,219-221,224,228-229,240,242,244,247-249,251-252,320-321,325-326,328,330,332-334,337,340-342,344-345,348-350,352,354-355,357,360,364-365,376-377,379-380,387-389,392,395-397,401,403-405,408,412-413,416-417,420-421,432,434,436,439-441,443-444,451-453,457,459-461,468-469,482-487,490-491,496,498,500,503-505,507-508,515-517,520,523-525,529,531-533,536,540-541,544,546,548-549,560,562,564,567-569,571-572,578,580-581,584,586-587,589,592,596-597,610-615,618-619,624,626,628,631-633,635-636,642,645-646,649,652-654,656,658-659,661-662,674-679,682-683,688,690,692,695-697,699-700,712,738-743,746-747,752,754,756,759-761,763-764,768-770,772-773,776,778-779,781,788-789,793,795-797,800,804-805,809-811,813,820-821,832-833,836-838,840-842,845-846,852-853,866-871,874-875,880,882,884,887-889,891-892,896,964-966,968-971,973-974,976-979,981-982,986-987,989-990,1002-1003,1008,1010,1012,1015-1017,1019-1020,1024,1028,1033,1035-1037,10
[  561.024815] IRQ6: New affinity: 0,66,68-69,72,74-75,77,81,84-85,89,91-93,96,100-101,105-107,109,116-117,128-129,132-133,137,140-141,153-158,162-167,170-171,176,178,180,183-185,187-188,192,195,197-198,201,204-205,208-209,213-214,216,218-219,221,224-226,228-229,240,242,244,247-249,251-252,256,259,261-262,265,268-269,272-273,277-278,280,282-283,285,290,292-293,304,306,308,311-313,315-316,321,323-324,327,329-333,337,339-341,345-350,354-359,362-363,368,370,372,375-377,379-380,384,387,389-390,393,396-397,400-401,405-406,408,410-411,413,417-418,420-421,432,434,436,439-441,443-444,448,450,453-454,460-462,464-468,470,475-477,480,484-485,496,498,500,503-505,507-508,520-521,523,528,546,640-641,644-645,649-650,652-653,665-670,674-679,682-683,688,690,692,695-697,699-700,704-705,708-709,712-713,716-717,729-734,738-743,746-747,752,754,756,759-761,763-764,768-769,772-773,776,778,780-781,793-798,802-807,810-811,816,818,820,823-825,827-828,832,835,837-838,841,844-845,848-849,853-854,856,858-859,861,864,8
[  561.080495] IRQ6: New affinity: 0,65,68-70,74,76-78,80-81,85-86,92-93,106-107,112,114,116,119-121,123-124,195-197,200,203-205,209,211-213,216,218,220-221,234-235,240,242,244,247-249,251-252,256,258,261-262,268-270,272-276,278,283-285,288,292-293,304,306,308,311-313,315-316,321,325-326,328,333-334,337-339,341-342,344-345,347,349-350,353,356-357,368,370,372,375-377,379-380,384,387,389-390,393,396-397,400-401,405-406,408,410-411,413,416,420-421,425-426,428-429,440-441,443-444,448,451,453-454,457,460-461,464-465,469-470,472,474-475,477,480,482,484-485,496,498,500,503-505,507-508,512-513,516-518,522,525-526,528,533-534,536,540-541,554-555,560,562,564,567-569,571-572,576,579,581-582,585,588-589,592-593,597-598,600,602-603,605,608,612-613,618,620-621,632-633,635-636,648-649,651,656,705-707,709,712-713,716-718,722,724-726,729,732-734,738,740-742,744,749-750,753,757-758,768-769,772-774,776-778,781-782,788-789,802-807,810-811,816,818,820,823-825,827-828,832,835,837-838,841,844-845,848-849,853-854
[  561.208184] IRQ6: New affinity: 0,68-70,72,75,77-78,81-83,85-86,88-92,94,96-97,101-102,105-106,109-110,112-114,117-118,132-133,136-138,140-141,153-158,162-167,170-171,176,178,180,183-185,187-188,200,203,207,209-210,213-214,220,222,227-228,237-257,260-262,264,267,269-270,273-275,277-278,280-282,285-286,290-291,293-294,296,298,301-302,312-313,315-316,320-324,326,328-332,334,340-342,344,349-350,353,356-358,360,365-366,368,370-371,373-374,388-389,392,394,397-398,409-414,418-423,426-427,432,434,436,439-441,443-444,449-451,453,457,461-462,464-465,468-470,472-473,476-478,490-491,496,498,500,503-505,507-508,513,517-518,520-522,524-525,529,531-533,540-541,554-555,560,562,564,567-569,571-572,577-579,581,586,589-590,592,597-598,602,604-606,608,613-614,624,626,628,631-633,635-636,646,650-651,654-655,658-662,668,670,675-676,685-703,769-771,773,776-777,780-782,784,787-790,792,794-795,797-798,802,804-806,808,813-814,817,821-822,833-835,837,842,845-846,848,853-854,858,860-862,864,869-870,880,882,884,88
[  561.209236] smpboot: CPU 5 is now offline
[  561.212293] ACPI: Low-level resume complete
[  561.212329] ACPI: EC: EC started
[  561.212329] PM: Restoring platform NVS memory
[  561.293704] LVT offset 0 assigned for vector 0x400
[  561.293958] Enabling non-boot CPUs ...
[  561.294003] x86: Booting SMP configuration:
[  561.294004] smpboot: Booting Node 0 Processor 1 APIC 0x11
[  561.296957]  cache: parent cpu1 should not be sleeping
[  561.297047] microcode: CPU1: patch_level=0x06000822
[  561.297168] CPU1 is up
[  561.297196] smpboot: Booting Node 0 Processor 2 APIC 0x12
[  561.300025]  cache: parent cpu2 should not be sleeping
[  561.300120] microcode: CPU2: patch_level=0x06000822
[  561.300276] CPU2 is up
[  561.300292] smpboot: Booting Node 0 Processor 3 APIC 0x13
[  561.303240]  cache: parent cpu3 should not be sleeping
[  561.303330] microcode: CPU3: patch_level=0x06000822
[  561.303498] CPU3 is up
[  561.303517] smpboot: Booting Node 0 Processor 4 APIC 0x14
[  561.306336]  cache: parent cpu4 should not be sleeping
[  561.306424] microcode: CPU4: patch_level=0x06000822
[  561.306606] CPU4 is up
[  561.306621] smpboot: Booting Node 0 Processor 5 APIC 0x15
[  561.309596]  cache: parent cpu5 should not be sleeping
[  561.309691] microcode: CPU5: patch_level=0x06000822
[  561.309893] CPU5 is up
[  561.318395] ACPI: Waking up from system sleep state S3
[  561.319268] ACPI: EC: interrupt unblocked
[  561.357500] ACPI: EC: event unblocked
[  561.357551] usb usb8: root hub lost power or was reset
[  561.357553] usb usb9: root hub lost power or was reset
[  561.386088] sd 0:0:0:0: [sda] Starting disk
[  561.641574] r8169 0000:02:00.0 enp2s0: link down
[  561.724548] usb 7-4: reset full-speed USB device number 2 using ohci-pci
[  561.916155] OOM killer enabled.
[  561.916156] Restarting tasks ... done.
[  561.923794] thermal thermal_zone0: failed to read out thermal zone (-61)
[  561.923796] PM: suspend exit
[  561.925610] Bluetooth: hci0: Bootloader revision 0.0 build 2 week 52 2014
[  561.933602] Bluetooth: hci0: Device revision is 5
[  561.933603] Bluetooth: hci0: Secure boot is enabled
[  561.933604] Bluetooth: hci0: OTP lock is enabled
[  561.933604] Bluetooth: hci0: API lock is enabled
[  561.933605] Bluetooth: hci0: Debug lock is disabled
[  561.933606] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[  562.967477] r8169 0000:02:00.0 enp2s0: link up
[  569.120012] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[  569.259229] ata1.00: configured for UDMA/133
[  569.397561] Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi
[  569.560141] IPv6: ADDRCONF(NETDEV_UP): enp2s0: link is not ready
[  569.674757] r8169 0000:02:00.0 enp2s0: link down
[  569.674775] r8169 0000:02:00.0 enp2s0: link down
[  569.674844] IPv6: ADDRCONF(NETDEV_UP): enp2s0: link is not ready
[  569.676116] IPv6: ADDRCONF(NETDEV_UP): wlp4s0: link is not ready
[  569.912690] IPv6: ADDRCONF(NETDEV_UP): wlp4s0: link is not ready
[  569.981068] IPv6: ADDRCONF(NETDEV_UP): wlp4s0: link is not ready
[  571.181885] r8169 0000:02:00.0 enp2s0: link up
[  571.181896] IPv6: ADDRCONF(NETDEV_CHANGE): enp2s0: link becomes ready
[  573.393202] wlp4s0: authenticate with dc:9b:9c:f1:07:f5
[  573.400697] wlp4s0: send auth to dc:9b:9c:f1:07:f5 (try 1/3)
[  573.405784] wlp4s0: authenticated
[  573.407664] wlp4s0: associate with dc:9b:9c:f1:07:f5 (try 1/3)
[  573.408647] wlp4s0: RX AssocResp from dc:9b:9c:f1:07:f5 (capab=0x1511 status=0 aid=3)
[  573.410433] wlp4s0: associated
[  573.425944] IPv6: ADDRCONF(NETDEV_CHANGE): wlp4s0: link becomes ready
[  573.463392] wlp4s0: Limiting TX power to 30 (30 - 0) dBm as advertised by dc:9b:9c:f1:07:f5
[  574.194776] Bluetooth: hci0: Waiting for firmware download to complete
[  574.195685] Bluetooth: hci0: Firmware loaded in 11984884 usecs
[  574.195798] Bluetooth: hci0: Waiting for device to boot
[  574.208675] Bluetooth: hci0: Device booted in 12638 usecs
[  574.208796] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-11-5.ddc
[  574.216727] Bluetooth: hci0: Applying Intel DDC parameters completed

Re: Regression caused by commit bc976233a872, ethernet r8169 stops working after system S3

[Thomas Gleixner]

On Sat, 31 Mar 2018, Kai-Heng Feng wrote:
> Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> > On Sat, 31 Mar 2018, Kai-Heng Feng wrote:
> > > A user reported [1] that the Realtek ethernet r8169 stops working after S3
> > > since v4.15-rc6. The issue still exists in Linus' tree:
> > > 
> > > [ 150.877998] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready
> > > [ 150.944101] do_IRQ: 3.37 No irq handler for vector
> > > [ 150.944105] r8169 0000:01:00.0 enp1s0: link down
> > > [ 150.944180] IPv6: ADDRCONF(NETDEV_UP): enp1s0: link is not ready
> > > 
> > > My desktop also has this device, which is also affected by this
> > > regression. So
> > > I did a bisect, and this is the bad commit:
> > > bc976233a872 genirq/msi x86/vector: Prevent reservation mode for non
> > > maskable
> > > MSI
> > > 
> > > After this commit gets reverted, the issue is gone.
> > 
> > Sigh. Can you please apply the debug patch below and provide the output of
> > 
> > # cat /proc/interrupts
> > 
> > and full dmesg after resume.
> 
> Weird. I don't see this issue with this debug patch.

Bah. The patch is broken. New version written with brain awake below.

Thanks,

	tglx

8<---------------------
--- a/kernel/irq/cpuhotplug.c
+++ b/kernel/irq/cpuhotplug.c
@@ -134,6 +134,10 @@ static bool migrate_one_irq(struct irq_d
 		brokeaff = false;
 	}
 
+	pr_info("IRQ%u: New affinity: %*pbl effective: %*pbl\n",
+		d->irq, cpumask_pr_args(affinity),
+		cpumask_pr_args(irq_data_get_effective_affinity_mask(d)));
+
 	if (maskchip && chip->irq_unmask)
 		chip->irq_unmask(d);
 

Re: Regression caused by commit bc976233a872, ethernet r8169 stops working after system S3

[Kai Heng Feng]

Hi,

> On Apr 3, 2018, at 4:51 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> Bah. The patch is broken. New version written with brain awake below.

Actually I can't reproduce this issue anymore on latest Linus' tree.

I'll do a bisect and ask linux-stable maintainer to include the commit.

Thanks for your help!

Kai-Heng

>
> Thanks,
>
> 	tglx
>
> 8<---------------------
> --- a/kernel/irq/cpuhotplug.c
> +++ b/kernel/irq/cpuhotplug.c
> @@ -134,6 +134,10 @@ static bool migrate_one_irq(struct irq_d
>  		brokeaff = false;
>  	}
>
> +	pr_info("IRQ%u: New affinity: %*pbl effective: %*pbl\n",
> +		d->irq, cpumask_pr_args(affinity),
> +		cpumask_pr_args(irq_data_get_effective_affinity_mask(d)));
> +
>  	if (maskchip && chip->irq_unmask)
>  		chip->irq_unmask(d);