CP110 Comphy Issue

CP110 Comphy Issue

[Sven Auhagen]

Hi Miquel,

I am testing Kernel 5.4 RC8 with the new CP110 comphy initialization and it does not work for our custom board on CP 1 PCIE0.
It hangs on phy up indefinitely during boot.

CP0 PCIE0 works fine.

I am using:
+&cp1_pcie0 {
+              pinctrl-names = "default";
+              pinctrl-0 = <&cp0_pci1_reset_pins>;
+              num-lanes = <1>;
+              //num-viewport = <8>;
+              reset-gpios = <&cp0_gpio2 1 GPIO_ACTIVE_LOW>;
+              //rx-disable-gpio = <&cp0_gpio2 20 GPIO_ACTIVE_LOW>;
+              phys = <&cp1_comphy0 0>;
+              phy-names = "cp1-pcie0-x1-phy";
+              status = "okay";
+};

And this one works

+&cp0_pcie0 {
+              pinctrl-names = "default";
+              pinctrl-0 = <&cp0_pci0_reset_pins>;
+              num-lanes = <1>;
+              //num-viewport = <8>;
+              reset-gpios = <&cp0_gpio2 0 GPIO_ACTIVE_LOW>;
+              //rx-disable-gpio = <&cp0_gpio2 19 GPIO_ACTIVE_LOW>;
+              phys = <&cp0_comphy0 0>;
+              phy-names = "cp0-pcie0-x1-phy";
+              status = "okay";
+};
+

I am using u-boot 2019.04 and ATF 2.1.
PCIe is initialized by U-Boot.

I was using a ported version of the comphy code from the marvell linux github before which did not have this problem.

Let me know if I can debug anything here on my end.

Best
Sven


Beste Grüße/Best regards

Sven Auhagen
Dipl. Math. oec., M.Sc.
Voleatech GmbH
HRB: B 754643
USTID: DE303643180
Grathwohlstr. 5
72762 Reutlingen
Tel: +49 7121539550
Fax: +49 7121539551
E-Mail: sven.auhagen@voleatech.de
www.voleatech.de<https://www.voleatech.de>
Diese Information ist ausschließlich für den Adressaten bestimmt und kann vertraulich oder gesetzlich geschützte Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Anderen als dem bestimmungsgemäßen Adressaten ist es untersagt, diese E-Mail zu lesen, zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden. Für den Adressaten sind die Informationen in dieser Mail nur zum persönlichen Gebrauch. Eine Weiterleitung darf nur nach Rücksprache mit dem Absender erfolgen. Wir verwenden aktuelle Virenschutzprogramme. Für Schäden, die dem Empfänger gleichwohl durch von uns zugesandte mit Viren befallene E-Mails entstehen, schließen wir jede Haftung aus.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Baruch Siach]

Hi Sven, Miquel,

On Sun, Nov 24 2019, Sven Auhagen wrote:
> I am testing Kernel 5.4 RC8 with the new CP110 comphy initialization
> and it does not work for our custom board on CP 1 PCIE0.
>
> It hangs on phy up indefinitely during boot.
>
> CP0 PCIE0 works fine.
>
> I am using:
> +&cp1_pcie0 {
> +              pinctrl-names = "default";
> +              pinctrl-0 = <&cp0_pci1_reset_pins>;
> +              num-lanes = <1>;
> +              //num-viewport = <8>;
> +              reset-gpios = <&cp0_gpio2 1 GPIO_ACTIVE_LOW>;
> +              //rx-disable-gpio = <&cp0_gpio2 20 GPIO_ACTIVE_LOW>;
> +              phys = <&cp1_comphy0 0>;
> +              phy-names = "cp1-pcie0-x1-phy";
> +              status = "okay";
> +};
>
> And this one works
>
> +&cp0_pcie0 {
> +              pinctrl-names = "default";
> +              pinctrl-0 = <&cp0_pci0_reset_pins>;
> +              num-lanes = <1>;
> +              //num-viewport = <8>;
> +              reset-gpios = <&cp0_gpio2 0 GPIO_ACTIVE_LOW>;
> +              //rx-disable-gpio = <&cp0_gpio2 19 GPIO_ACTIVE_LOW>;
> +              phys = <&cp0_comphy0 0>;
> +              phy-names = "cp0-pcie0-x1-phy";
> +              status = "okay";
> +};
> +
>
> I am using u-boot 2019.04 and ATF 2.1.
> PCIe is initialized by U-Boot.
>
> I was using a ported version of the comphy code from the marvell linux
> github before which did not have this problem.
>
> Let me know if I can debug anything here on my end.

One important bit of missing information is that the board is based on
Armada 8040.

baruch

--
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch@tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Miquel Raynal]

Hi Sven,

Baruch Siach <baruch@tkos.co.il> wrote on Sun, 24 Nov 2019 09:31:06
+0200:

> Hi Sven, Miquel,
> 
> On Sun, Nov 24 2019, Sven Auhagen wrote:
> > I am testing Kernel 5.4 RC8 with the new CP110 comphy initialization
> > and it does not work for our custom board on CP 1 PCIE0.
> >
> > It hangs on phy up indefinitely during boot.
> >
> > CP0 PCIE0 works fine.
> >
> > I am using:
> > +&cp1_pcie0 {
> > +              pinctrl-names = "default";
> > +              pinctrl-0 = <&cp0_pci1_reset_pins>;
> > +              num-lanes = <1>;
> > +              //num-viewport = <8>;
> > +              reset-gpios = <&cp0_gpio2 1 GPIO_ACTIVE_LOW>;
> > +              //rx-disable-gpio = <&cp0_gpio2 20 GPIO_ACTIVE_LOW>;
> > +              phys = <&cp1_comphy0 0>;
> > +              phy-names = "cp1-pcie0-x1-phy";
> > +              status = "okay";
> > +};
> >
> > And this one works
> >
> > +&cp0_pcie0 {
> > +              pinctrl-names = "default";
> > +              pinctrl-0 = <&cp0_pci0_reset_pins>;
> > +              num-lanes = <1>;
> > +              //num-viewport = <8>;
> > +              reset-gpios = <&cp0_gpio2 0 GPIO_ACTIVE_LOW>;
> > +              //rx-disable-gpio = <&cp0_gpio2 19 GPIO_ACTIVE_LOW>;
> > +              phys = <&cp0_comphy0 0>;
> > +              phy-names = "cp0-pcie0-x1-phy";
> > +              status = "okay";
> > +};
> > +
> >
> > I am using u-boot 2019.04 and ATF 2.1.
> > PCIe is initialized by U-Boot.
> >
> > I was using a ported version of the comphy code from the marvell linux
> > github before which did not have this problem.
> >
> > Let me know if I can debug anything here on my end.  
> 
> One important bit of missing information is that the board is based on
> Armada 8040.

Can you share the logs? How does it fail? Please share the full log.

Also, are you sure your ATF has COMPHY support? Recent kernels use ARM
SMC convention to request COMPHY initialization, which was probably not
the case before with the code ported from Marvell's BSP. Maybe if you
are using an official ATF release you don't have it (yet?).

Thanks,
Miquèl

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Sven Auhagen]

Hi Miquèl,

Yes, I am sure the code uses ARM SMC and my ATF is very new, upstream version and supports it.
I am not using the marvell version.

I am also calling mvebu_comphy_power_off before initializing COMPHY and I can visually see my NICs resetting (LEDs off and back on) at Linux Kernel start.

The Kernel 5.4 RC8 boot hangs on PCIe Phy up.
I will send you logs tomorrow morning, when I have the device.

You can also see the marvell patches here:

https://voleatech-my.sharepoint.com/:u:/g/personal/sven_auhagen_voleatech_de/EcBFSVWdyWZNtI_4HjgEEN0B43WKQj_JwBYZgEbzFJaT_Q?e=h83gEy

Best
Sven

From: Miquel Raynal <miquel.raynal@bootlin.com>
Organisation: Bootlin
Date: Sunday, 24. November 2019 at 3:18 PM
To: Baruch Siach <baruch@tkos.co.il>
Cc: Sven Auhagen <sven.auhagen@voleatech.de>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
Subject: Re: CP110 Comphy Issue

Hi Sven,

Baruch Siach <mailto:baruch@tkos.co.il> wrote on Sun, 24 Nov 2019 09:31:06
+0200:

Hi Sven, Miquel,
On Sun, Nov 24 2019, Sven Auhagen wrote:
> I am testing Kernel 5.4 RC8 with the new CP110 comphy initialization
> and it does not work for our custom board on CP 1 PCIE0.
>
> It hangs on phy up indefinitely during boot.
>
> CP0 PCIE0 works fine.
>
> I am using:
> +&cp1_pcie0 {
> +              pinctrl-names = "default";
> +              pinctrl-0 = <&cp0_pci1_reset_pins>;
> +              num-lanes = <1>;
> +              //num-viewport = <8>;
> +              reset-gpios = <&cp0_gpio2 1 GPIO_ACTIVE_LOW>;
> +              //rx-disable-gpio = <&cp0_gpio2 20 GPIO_ACTIVE_LOW>;
> +              phys = <&cp1_comphy0 0>;
> +              phy-names = "cp1-pcie0-x1-phy";
> +              status = "okay";
> +};
>
> And this one works
>
> +&cp0_pcie0 {
> +              pinctrl-names = "default";
> +              pinctrl-0 = <&cp0_pci0_reset_pins>;
> +              num-lanes = <1>;
> +              //num-viewport = <8>;
> +              reset-gpios = <&cp0_gpio2 0 GPIO_ACTIVE_LOW>;
> +              //rx-disable-gpio = <&cp0_gpio2 19 GPIO_ACTIVE_LOW>;
> +              phys = <&cp0_comphy0 0>;
> +              phy-names = "cp0-pcie0-x1-phy";
> +              status = "okay";
> +};
> +
>
> I am using u-boot 2019.04 and ATF 2.1.
> PCIe is initialized by U-Boot.
>
> I was using a ported version of the comphy code from the marvell linux
> github before which did not have this problem.
>
> Let me know if I can debug anything here on my end.
One important bit of missing information is that the board is based on
Armada 8040.

Can you share the logs? How does it fail? Please share the full log.

Also, are you sure your ATF has COMPHY support? Recent kernels use ARM
SMC convention to request COMPHY initialization, which was probably not
the case before with the code ported from Marvell's BSP. Maybe if you
are using an official ATF release you don't have it (yet?).

Thanks,
Miquèl



Beste Grüße/Best regards

Sven Auhagen
Dipl. Math. oec., M.Sc.
Voleatech GmbH
HRB: B 754643
USTID: DE303643180
Grathwohlstr. 5
72762 Reutlingen
Tel: +49 7121539550
Fax: +49 7121539551
E-Mail: sven.auhagen@voleatech.de
www.voleatech.de<https://www.voleatech.de>
Diese Information ist ausschließlich für den Adressaten bestimmt und kann vertraulich oder gesetzlich geschützte Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Anderen als dem bestimmungsgemäßen Adressaten ist es untersagt, diese E-Mail zu lesen, zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden. Für den Adressaten sind die Informationen in dieser Mail nur zum persönlichen Gebrauch. Eine Weiterleitung darf nur nach Rücksprache mit dem Absender erfolgen. Wir verwenden aktuelle Virenschutzprogramme. Für Schäden, die dem Empfänger gleichwohl durch von uns zugesandte mit Viren befallene E-Mails entstehen, schließen wir jede Haftung aus.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Sven Auhagen]

Hi Miquèl,

Here the log of the boot:

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd081]
[    0.000000] Linux version 5.4.0-rc8-vtair-arm64 (tech@voleatech.de) (gcc version 8.3.0 (Debian 8.3.0-2)) #1 SMP Debian 5.4~rc8-1~vtair (2019-11-25)
[    0.000000] Machine model: VT AIR 300
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] cma: Reserved 64 MiB at 0x00000000bc000000
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x000000013fffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x13f7cf800-0x13f7d0fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000013fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000003ffffff]
[    0.000000]   node   0: [mem 0x0000000004200000-0x00000000bfffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000013fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000013fffffff]
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 32 pages/cpu s93464 r8192 d29416 u131072
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1031688
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line:  root=UUID=55c232c9-ae76-43cd-bd6d-291cd5ea8951 ro
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0xb8000000-0xbc000000] (64MB)
[    0.000000] Memory: 3931124K/4192256K available (10812K kernel code, 1826K rwdata, 3852K rodata, 5312K init, 553K bss, 195596K reserved, 65536K cma-reserved)
[    0.000000] random: get_random_u64 called from __kmem_cache_create+0x50/0x5d0 with crng_init=0
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] ftrace: allocating 38551 entries in 151 pages
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Adjusting CPU interface base to 0x00000000f022f000
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:160, num:32)
[    0.000000] GICv2m: range[mem 0xf0280000-0xf0280fff], SPI[160:191]
[    0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:192, num:32)
[    0.000000] GICv2m: range[mem 0xf0290000-0xf0290fff], SPI[192:223]
[    0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:224, num:32)
[    0.000000] GICv2m: range[mem 0xf02a0000-0xf02a0fff], SPI[224:255]
[    0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:256, num:32)
[    0.000000] GICv2m: range[mem 0xf02b0000-0xf02b0fff], SPI[256:287]
[    0.000000] arch_timer: cp15 timer(s) running at 25.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x5c40939b5, max_idle_ns: 440795202646 ns
[    0.000002] sched_clock: 56 bits at 25MHz, resolution 40ns, wraps every 4398046511100ns
[    0.000253] Console: colour dummy device 80x25
[    0.000427] printk: console [tty0] enabled
[    0.000478] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=100000)
[    0.000488] pid_max: default: 32768 minimum: 301
[    0.000549] LSM: Security Framework initializing
[    0.000566] Yama: disabled by default; enable with sysctl kernel.yama.*
[    0.000612] AppArmor: AppArmor initialized
[    0.000620] TOMOYO Linux initialized
[    0.000662] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.000682] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001452] ASID allocator initialised with 32768 entries
[    0.001508] rcu: Hierarchical SRCU implementation.
[    0.002606] EFI services will not be available.
[    0.002814] smp: Bringing up secondary CPUs ...
[    0.003178] Detected PIPT I-cache on CPU1
[    0.003216] CPU1: Booted secondary processor 0x0000000001 [0x410fd081]
[    0.003625] Detected PIPT I-cache on CPU2
[    0.003650] CPU2: Booted secondary processor 0x0000000100 [0x410fd081]
[    0.004048] Detected PIPT I-cache on CPU3
[    0.004066] CPU3: Booted secondary processor 0x0000000101 [0x410fd081]
[    0.004117] smp: Brought up 1 node, 4 CPUs
[    0.004142] SMP: Total of 4 processors activated.
[    0.004148] CPU features: detected: 32-bit EL0 Support
[    0.004154] CPU features: detected: CRC32 instructions
[    0.015463] CPU: All CPU(s) started at EL2
[    0.015496] alternatives: patching kernel code
[    0.016216] devtmpfs: initialized
[    0.018910] Registered cp15_barrier emulation handler
[    0.018923] Registered setend emulation handler
[    0.019076] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.019090] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.019801] pinctrl core: initialized pinctrl subsystem
[    0.020194] DMI not present or invalid.
[    0.020382] NET: Registered protocol family 16
[    0.020946] DMA: preallocated 256 KiB pool for atomic allocations
[    0.020956] audit: initializing netlink subsys (disabled)
[    0.021062] audit: type=2000 audit(0.020:1): state=initialized audit_enabled=0 res=1
[    0.021526] cpuidle: using governor ladder
[    0.021542] cpuidle: using governor menu
[    0.021593] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.021754] Serial: AMBA PL011 UART driver
[    0.029818] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.029828] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.029834] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.029840] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.281670] ACPI: Interpreter disabled.
[    0.281810] iommu: Default domain type: Translated
[    0.281930] vgaarb: loaded
[    0.282116] EDAC MC: Ver: 3.0.0
[    0.282794] clocksource: Switched to clocksource arch_sys_counter
[    0.305969] VFS: Disk quotas dquot_6.6.0
[    0.306013] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.306329] AppArmor: AppArmor Filesystem Enabled
[    0.306494] pnp: PnP ACPI: disabled
[    0.309691] thermal_sys: Registered thermal governor 'fair_share'
[    0.309692] thermal_sys: Registered thermal governor 'step_wise'
[    0.309700] thermal_sys: Registered thermal governor 'power_allocator'
[    0.310142] NET: Registered protocol family 2
[    0.310404] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.310441] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.310578] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.310896] TCP: Hash tables configured (established 32768 bind 32768)
[    0.310973] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.311028] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.311162] NET: Registered protocol family 1
[    0.311177] NET: Registered protocol family 44
[    0.311186] PCI: CLS 0 bytes, default 64
[    0.311269] Trying to unpack rootfs image as initramfs...
[    1.047190] Freeing initrd memory: 26604K
[    1.047623] hw perfevents: unable to count PMU IRQs
[    1.047633] hw perfevents: /ap806/config-space@f0000000/pmu: failed to register PMU devices!
[    1.047819] kvm [1]: IPA Size Limit: 44bits
[    1.048142] kvm [1]: vgic interrupt IRQ1
[    1.048220] kvm [1]: Hyp mode initialized successfully
[    1.048934] Initialise system trusted keyrings
[    1.048955] Key type blacklist registered
[    1.049059] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    1.051356] zbud: loaded
[    1.051726] Platform Keyring initialized
[    1.051735] Key type asymmetric registered
[    1.051740] Asymmetric key parser 'x509' registered
[    1.051762] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[    1.051854] io scheduler mq-deadline registered
[    1.054051] debugfs: Directory 'dummy-system-controller@440000' with parent 'regmap' already present!
[    1.054630] armada-ap806-pinctrl f06f4000.system-controller:pinctrl: registered pinctrl driver
[    1.054988] armada-cp110-pinctrl f2440000.system-controller:pinctrl: registered pinctrl driver
[    1.055223] armada-cp110-pinctrl f4440000.system-controller:pinctrl: registered pinctrl driver
[    1.056392] GPIO line 66 (lte_reset) hogged as output/low
[    1.056408] GPIO line 85 (lte_power_off) hogged as output/low
[    1.056421] GPIO line 83 (pcie0_power_off) hogged as output/low
[    1.056434] GPIO line 84 (pcie1_power_off) hogged as output/low
[    1.057235] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    1.060926] mv_xor_v2 f0400000.xor: Marvell Version 2 XOR driver
[    1.061173] mv_xor_v2 f0420000.xor: Marvell Version 2 XOR driver
[    1.061448] mv_xor_v2 f0440000.xor: Marvell Version 2 XOR driver
[    1.061692] mv_xor_v2 f0460000.xor: Marvell Version 2 XOR driver
[    1.062005] mv_xor_v2 f26a0000.xor: Marvell Version 2 XOR driver
[    1.062267] mv_xor_v2 f26c0000.xor: Marvell Version 2 XOR driver
[    1.062649] mv_xor_v2 f46a0000.xor: Marvell Version 2 XOR driver
[    1.062937] mv_xor_v2 f46c0000.xor: Marvell Version 2 XOR driver
[    1.064254] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    1.085308] f0512000.serial: ttyS0 at MMIO 0xf0512000 (irq = 8, base_baud = 12500000) is a 16550A
[    2.004387] printk: console [ttyS0] enabled
[    2.008939] Serial: AMBA driver
[    2.012276] msm_serial: driver initialized
[    2.017947] libphy: Fixed MDIO Bus: probed
[    2.022292] orion-mdio f212a200.mdio: IRQ index 0 not found
[    2.027943] libphy: orion_mdio_bus: probed
[    2.040727] orion-mdio f412a200.mdio: IRQ index 0 not found
[    2.046383] libphy: orion_mdio_bus: probed
[    2.058137] mvpp2 f2000000.ethernet: using 8 per-cpu buffers
[    2.070379] mvpp2 f4000000.ethernet: using 8 per-cpu buffers
[    2.083464] mousedev: PS/2 mouse device common for all mice
[    2.089498] armada38x-rtc f4284000.rtc: registered as rtc0
[    2.096532] sbsa-gwdt f0610000.watchdog: Initialized with 10s timeout @ 25000000 Hz, action=0.
[    2.105538] armada_8k_cpufreq: Cannot get clock for CPU 0
[    2.112110] DT idle-states: AP does not support CPU IDLE
[    2.117598] sdhci: Secure Digital Host Controller Interface driver
[    2.123810] sdhci: Copyright(c) Pierre Ossman
[    2.128223] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.134014] xenon-sdhci f06e0000.sdhci: AP SDHC is running in slow mode
[    2.141003] ledtrig-cpu: registered to indicate activity on CPUs
[    2.147923] drop_monitor: Initializing network drop monitor service
[    2.164464] xt_time: kernel timezone is -0000
[    2.169098] ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully
[    2.175726] NET: Registered protocol family 10
[    2.202050] Segment Routing with IPv6
[    2.205802] mip6: Mobile IPv6
[    2.209028] NET: Registered protocol family 17
[    2.213566] mpls_gso: MPLS GSO support
[    2.217528] registered taskstats version 1
[    2.221652] Loading compiled-in X.509 certificates
[    2.286634] Loaded X.509 cert 'Debian Secure Boot CA: 6ccece7e4c6c0d1f6149f3dd27dfcc5cbb419ea1'
[    2.295417] Loaded X.509 cert 'Debian Secure Boot Signer: 00a7468def'
[    2.301949] zswap: loaded using pool lzo/zbud
[    2.306521] Key type ._fscrypt registered
[    2.310561] Key type .fscrypt registered
[    2.314543] AppArmor: AppArmor sha1 policy hashing enabled
[    2.322370] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    2.331616] armada8k-pcie f2600000.pcie: host bridge /cp0/pcie@f2600000 ranges:
[    2.338987] armada8k-pcie f2600000.pcie:    IO 0xf9000000..0xf900ffff -> 0xf9000000
[    2.346690] armada8k-pcie f2600000.pcie:   MEM 0xf6000000..0xf6efffff -> 0xf6000000
[    2.354445] armada8k-pcie f2600000.pcie: Link up
[    2.359147] armada8k-pcie f2600000.pcie: PCI host bridge to bus 0000:00
[    2.365798] pci_bus 0000:00: root bus resource [bus 00-ff]
[    2.371313] pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0xf9000000-0xf900ffff])
[    2.380841] pci_bus 0000:00: root bus resource [mem 0xf6000000-0xf6efffff]
[    2.387763] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    2.393815] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.400161] pci 0000:00:00.0: supports D1 D2
[    2.404454] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.411388] pci 0000:01:00.0: [8086:1539] type 00 class 0x020000
[    2.417581] pci 0000:01:00.0: reg 0x10: [mem 0xf6000000-0xf601ffff]
[    2.423991] pci 0000:01:00.0: reg 0x18: [io  0xffffffe0-0xffffffff]
[    2.430346] pci 0000:01:00.0: reg 0x1c: [mem 0xf6020000-0xf6023fff]
[    2.437242] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    2.455760] pci 0000:00:00.0: BAR 0: assigned [mem 0xf6000000-0xf60fffff]
[    2.462595] pci 0000:00:00.0: BAR 14: assigned [mem 0xf6100000-0xf61fffff]
[    2.469507] pci 0000:00:00.0: BAR 13: assigned [io  0x1000-0x1fff]
[    2.475722] pci 0000:01:00.0: BAR 0: assigned [mem 0xf6100000-0xf611ffff]
[    2.482564] pci 0000:01:00.0: BAR 3: assigned [mem 0xf6120000-0xf6123fff]
[    2.489403] pci 0000:01:00.0: BAR 2: assigned [io  0x1000-0x101f]
[    2.495546] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.500800] pci 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
[    2.506926] pci 0000:00:00.0:   bridge window [mem 0xf6100000-0xf61fffff]
[    2.513922] pcieport 0000:00:00.0: PME: Signaling with IRQ 32
[    2.519815] pcieport 0000:00:00.0: AER: enabled with IRQ 32
[    2.525992] libphy: SFP I2C Bus: probed
[    2.529939] sfp sfp-cp0-eth0: Host maximum power 2.0W
[    2.535493] libphy: SFP I2C Bus: probed
[    2.539417] sfp sfp-cp1-eth0: Host maximum power 2.0W
[    2.545254] mvpp2 f2000000.ethernet: using 8 per-cpu buffers
[    2.560258] mvpp2 f2000000.ethernet eth0: Using firmware node mac address 70:b3:d5:da:30:1c
[    2.570313] mvpp2 f2000000.ethernet eth1: Using firmware node mac address 70:b3:d5:da:30:19
[    2.580413] mvpp2 f2000000.ethernet eth2: Using firmware node mac address 70:b3:d5:da:30:18
[    2.600055] mvpp2 f4000000.ethernet: using 8 per-cpu buffers
[    2.613263] xenon-sdhci f06e0000.sdhci: AP SDHC is running in slow mode
[    2.621462] mvpp2 f4000000.ethernet: using 8 per-cpu buffers
[    2.634592] xenon-sdhci f06e0000.sdhci: AP SDHC is running in slow mode
[    2.642197] armada38x-rtc f4284000.rtc: setting system clock to 2019-11-25T07:39:14 UTC (1574667554)
[    2.654779] Freeing unused kernel memory: 5312K
[    2.666852] Run /init as init process
Loading, please wait...
Starting version 241
[    2.831430] mvpp2 f2000000.ethernet eno0: renamed from eth2
[    2.837198] usb_common: module verification failed: signature and/or required key missing - tainting kernel
[    2.848531] SCSI subsystem initialized
[    2.855136] usbcore: registered new interface driver usbfs
[    2.860719] usbcore: registered new interface driver hub
[    2.860894] pps_core: LinuxPPS API ver. 1 registered
[    2.867400] usbcore: registered new device driver usb
[    2.871092] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    2.871376] mvpp2 f2000000.ethernet eno4: renamed from eth0
[    2.876484] PTP clock support registered
[    2.896126] mvpp2 f4000000.ethernet: using 8 per-cpu buffers
[    2.904389] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.6.0-k
[    2.911435] igb: Copyright (c) 2007-2014 Intel Corporation.
[    2.911866] xhci-hcd f2500000.usb3: xHCI Host Controller
[    2.913779] xenon-sdhci f06e0000.sdhci: AP SDHC is running in slow mode
[    2.918131] input: keys as /devices/platform/keys/input/input0
[    2.922715] xhci-hcd f2500000.usb3: new USB bus registered, assigned bus number 1
[    2.935887] mvpp2 f2000000.ethernet eno1: renamed from eth1
[    2.942731] xhci-hcd f2500000.usb3: hcc params 0x0a000990 hci version 0x100 quirks 0x0000000000010010
[    2.957609] xhci-hcd f2500000.usb3: irq 52, io mem 0xf2500000
[    2.957935] ahci f2540000.sata: f2540000.sata supply ahci not found, using dummy regulator
[    2.958811] mmc0: SDHCI controller on f06e0000.sdhci [f06e0000.sdhci] using ADMA 64-bit
[    2.963575] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.04
[    2.971837] ahci f2540000.sata: f2540000.sata supply phy not found, using dummy regulator
[    2.979767] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    2.979770] usb usb1: Product: xHCI Host Controller
[    2.988451] platform f2540000.sata:sata-port@0: f2540000.sata:sata-port@0 supply target not found, using dummy regulator
[    2.990347] mmc1: SDHCI controller on f2780000.sdhci [f2780000.sdhci] using ADMA 64-bit
[    2.991104] armada8k-pcie f4600000.pcie: host bridge /cp1/pcie@f4600000 ranges:
[    2.991121] armada8k-pcie f4600000.pcie:    IO 0xfd000000..0xfd00ffff -> 0xfd000000
[    2.991129] armada8k-pcie f4600000.pcie:   MEM 0xfa000000..0xfaefffff -> 0xfa000000
[    2.991184] armada8k-pcie f4600000.pcie: Link up
[    2.991261] armada8k-pcie f4600000.pcie: PCI host bridge to bus 0001:00
[    2.991265] pci_bus 0001:00: root bus resource [bus 00-ff]
[    2.991269] pci_bus 0001:00: root bus resource [io  0x10000-0x1ffff] (bus address [0xfd000000-0xfd00ffff])
[    2.991271] pci_bus 0001:00: root bus resource [mem 0xfa000000-0xfaefffff]
[    2.991291] pci 0001:00:00.0: [11ab:0110] type 01 class 0x060400
[    2.991309] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.991363] pci 0001:00:00.0: supports D1 D2
[    2.991366] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[    2.996300] usb usb1: Manufacturer: Linux 5.4.0-rc8-vtair-arm64 xhci-hcd
[    2.996302] usb usb1: SerialNumber: f2500000.usb3
[    2.997143] hub 1-0:1.0: USB hub found
[    3.003932] platform f2540000.sata:sata-port@1: f2540000.sata:sata-port@1 supply target not found, using dummy regulator
[    3.016172] hub 1-0:1.0: 1 port detected
[    3.020593] ahci f2540000.sata: masking port_map 0x3 -> 0x3
[    3.027664] xhci-hcd f2500000.usb3: xHCI Host Controller
[    3.034862] ahci f2540000.sata: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl platform mode
[    3.042529] xhci-hcd f2500000.usb3: new USB bus registered, assigned bus number 2
[    3.050221] ahci f2540000.sata: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs
[    3.054866] xhci-hcd f2500000.usb3: Host supports USB 3.0 SuperSpeed

When I remove the PCIe card from CP1 the boot continues just fine.

Also I looked through the code you probably want to change this, or you will try to access phys that are none for PCIes that have less than 4 lanes.

--- a/drivers/pci/controller/dwc/pcie-armada8k.c
+++ b/drivers/pci/controller/dwc/pcie-armada8k.c
@@ -75,7 +75,7 @@
 {
 int i;

-for (i = 0; i < ARMADA8K_PCIE_MAX_LANES; i++) {
+for (i = 0; i < pcie->phy_count; i++) {
 phy_power_off(pcie->phy[i]);
 phy_exit(pcie->phy[i]);
 }
@@ -86,7 +86,7 @@
 int ret;
 int i;

-for (i = 0; i < ARMADA8K_PCIE_MAX_LANES; i++) {
+for (i = 0; i < pcie->phy_count; i++) {
 ret = phy_init(pcie->phy[i]);
 if (ret)
 return ret;


I will compile a debug kernel later today and try to follow the comphy code.

Best
Sven

From: Sven Auhagen <sven.auhagen@voleatech.de>
Date: Sunday, 24. November 2019 at 5:21 PM
To: Miquel Raynal <miquel.raynal@bootlin.com>, Baruch Siach <baruch@tkos.co.il>
Cc: "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
Subject: Re: CP110 Comphy Issue

Hi Miquèl,

Yes, I am sure the code uses ARM SMC and my ATF is very new, upstream version and supports it.
I am not using the marvell version.

I am also calling mvebu_comphy_power_off before initializing COMPHY and I can visually see my NICs resetting (LEDs off and back on) at Linux Kernel start.

The Kernel 5.4 RC8 boot hangs on PCIe Phy up.
I will send you logs tomorrow morning, when I have the device.

You can also see the marvell patches here:

https://voleatech-my.sharepoint.com/:u:/g/personal/sven_auhagen_voleatech_de/EcBFSVWdyWZNtI_4HjgEEN0B43WKQj_JwBYZgEbzFJaT_Q?e=h83gEy

Best
Sven

From: Miquel Raynal <mailto:miquel.raynal@bootlin.com>
Organisation: Bootlin
Date: Sunday, 24. November 2019 at 3:18 PM
To: Baruch Siach <mailto:baruch@tkos.co.il>
Cc: Sven Auhagen <mailto:sven.auhagen@voleatech.de>, "mailto:linux-arm-kernel@lists.infradead.org" <mailto:linux-arm-kernel@lists.infradead.org>
Subject: Re: CP110 Comphy Issue

Hi Sven,

Baruch Siach <mailto:baruch@tkos.co.il> wrote on Sun, 24 Nov 2019 09:31:06
+0200:

Hi Sven, Miquel,
On Sun, Nov 24 2019, Sven Auhagen wrote:
I am testing Kernel 5.4 RC8 with the new CP110 comphy initialization
and it does not work for our custom board on CP 1 PCIE0.

It hangs on phy up indefinitely during boot.

CP0 PCIE0 works fine.

I am using:
+&cp1_pcie0 {
+              pinctrl-names = "default";
+              pinctrl-0 = <&cp0_pci1_reset_pins>;
+              num-lanes = <1>;
+              //num-viewport = <8>;
+              reset-gpios = <&cp0_gpio2 1 GPIO_ACTIVE_LOW>;
+              //rx-disable-gpio = <&cp0_gpio2 20 GPIO_ACTIVE_LOW>;
+              phys = <&cp1_comphy0 0>;
+              phy-names = "cp1-pcie0-x1-phy";
+              status = "okay";
+};

And this one works

+&cp0_pcie0 {
+              pinctrl-names = "default";
+              pinctrl-0 = <&cp0_pci0_reset_pins>;
+              num-lanes = <1>;
+              //num-viewport = <8>;
+              reset-gpios = <&cp0_gpio2 0 GPIO_ACTIVE_LOW>;
+              //rx-disable-gpio = <&cp0_gpio2 19 GPIO_ACTIVE_LOW>;
+              phys = <&cp0_comphy0 0>;
+              phy-names = "cp0-pcie0-x1-phy";
+              status = "okay";
+};
+

I am using u-boot 2019.04 and ATF 2.1.
PCIe is initialized by U-Boot.

I was using a ported version of the comphy code from the marvell linux
github before which did not have this problem.

Let me know if I can debug anything here on my end.
One important bit of missing information is that the board is based on
Armada 8040.

Can you share the logs? How does it fail? Please share the full log.

Also, are you sure your ATF has COMPHY support? Recent kernels use ARM
SMC convention to request COMPHY initialization, which was probably not
the case before with the code ported from Marvell's BSP. Maybe if you
are using an official ATF release you don't have it (yet?).

Thanks,
Miquèl





Beste Grüße/Best regards

Sven Auhagen
Dipl. Math. oec., M.Sc.
Voleatech GmbH
HRB: B 754643
USTID: DE303643180
Grathwohlstr. 5
72762 Reutlingen
Tel: +49 7121539550
Fax: +49 7121539551
E-Mail: sven.auhagen@voleatech.de
www.voleatech.de<https://www.voleatech.de>
Diese Information ist ausschließlich für den Adressaten bestimmt und kann vertraulich oder gesetzlich geschützte Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Anderen als dem bestimmungsgemäßen Adressaten ist es untersagt, diese E-Mail zu lesen, zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden. Für den Adressaten sind die Informationen in dieser Mail nur zum persönlichen Gebrauch. Eine Weiterleitung darf nur nach Rücksprache mit dem Absender erfolgen. Wir verwenden aktuelle Virenschutzprogramme. Für Schäden, die dem Empfänger gleichwohl durch von uns zugesandte mit Viren befallene E-Mails entstehen, schließen wir jede Haftung aus.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Miquel Raynal]

Hi Sven,


> When I remove the PCIe card from CP1 the boot continues just fine.

Maybe you can have a look at the reset code.

Maybe also check the clock direction set in ATF -which might differ for
CP1 depending on the SaR registers.

> Also I looked through the code you probably want to change this, or you will try to access phys that are none for PCIes that have less than 4 lanes.

AFAIK this is not a problem but might be more elegant, please send a
patch to the ML.


Thanks,
Miquèl

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Miquel Raynal]

Hi Sven,

Sven Auhagen <sven.auhagen@voleatech.de> wrote on Wed, 27 Nov 2019
06:08:16 +0000:

> Hi Miquèl,
> 
> I looked through the ATF code and as far as I can tell the pcie clock direction is set based on the sar register in cp110_pcie_clk_cfg
> I am not sure how this can lead to the problem of the comphy not coming up.

Clocks are often responsible of stalls when accessing registers.

I don't know what your problem is, I am giving you suggestions.

> I can also see that the PCIe works in uboot and the Intel NIC is recognized just fine there so the intial comphy setup must be working.

Is U-Boot doing SMC calls too? I suppose it does not.

> I could not find the reset code, can you send me the file where it is?

I thought we had some in the comphy driver, apparently not.

I don't have more inputs so far. Is your debugging giving interesting
infos? Have you checked in ->set_mode() everything looks fine?


Thanks,
Miquèl

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Sven Auhagen]

Hi Miquèl,

I appreciate your help, thank you.

U-Boot is not using SMC calls, correct.

The marvell github version does and I am checking the code against the upstream code currently.
The comphy initialization for PCIe produces the same values, I will debug it further the next days.

Best and thanks
Sven

From: Miquel Raynal <miquel.raynal@bootlin.com>
Organisation: Bootlin
Date: Wednesday, 27. November 2019 at 12:06 PM
To: Sven Auhagen <sven.auhagen@voleatech.de>
Cc: Baruch Siach <baruch@tkos.co.il>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
Subject: Re: CP110 Comphy Issue

Hi Sven,

Sven Auhagen <mailto:sven.auhagen@voleatech.de> wrote on Wed, 27 Nov 2019
06:08:16 +0000:

Hi Miquèl,
I looked through the ATF code and as far as I can tell the pcie clock direction is set based on the sar register in cp110_pcie_clk_cfg
I am not sure how this can lead to the problem of the comphy not coming up.

Clocks are often responsible of stalls when accessing registers.

I don't know what your problem is, I am giving you suggestions.

I can also see that the PCIe works in uboot and the Intel NIC is recognized just fine there so the intial comphy setup must be working.

Is U-Boot doing SMC calls too? I suppose it does not.

I could not find the reset code, can you send me the file where it is?

I thought we had some in the comphy driver, apparently not.

I don't have more inputs so far. Is your debugging giving interesting
infos? Have you checked in ->set_mode() everything looks fine?


Thanks,
Miquèl



Beste Grüße/Best regards

Sven Auhagen
Dipl. Math. oec., M.Sc.
Voleatech GmbH
HRB: B 754643
USTID: DE303643180
Grathwohlstr. 5
72762 Reutlingen
Tel: +49 7121539550
Fax: +49 7121539551
E-Mail: sven.auhagen@voleatech.de
www.voleatech.de<https://www.voleatech.de>
Diese Information ist ausschließlich für den Adressaten bestimmt und kann vertraulich oder gesetzlich geschützte Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Anderen als dem bestimmungsgemäßen Adressaten ist es untersagt, diese E-Mail zu lesen, zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden. Für den Adressaten sind die Informationen in dieser Mail nur zum persönlichen Gebrauch. Eine Weiterleitung darf nur nach Rücksprache mit dem Absender erfolgen. Wir verwenden aktuelle Virenschutzprogramme. Für Schäden, die dem Empfänger gleichwohl durch von uns zugesandte mit Viren befallene E-Mails entstehen, schließen wir jede Haftung aus.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: CP110 Comphy Issue

[Sven Auhagen]

Hi Miquèl,

I found the problem, it was the USB line that is also connected to the same M.2 PCIe which caused the stall.
I was able to fix that with a device tree change.

Thank you for your help.

Best
Sven

From: Miquel Raynal <miquel.raynal@bootlin.com>
Organisation: Bootlin
Date: Wednesday, 27. November 2019 at 12:06 PM
To: Sven Auhagen <sven.auhagen@voleatech.de>
Cc: Baruch Siach <baruch@tkos.co.il>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
Subject: Re: CP110 Comphy Issue

Hi Sven,

Sven Auhagen <mailto:sven.auhagen@voleatech.de> wrote on Wed, 27 Nov 2019
06:08:16 +0000:

Hi Miquèl,
I looked through the ATF code and as far as I can tell the pcie clock direction is set based on the sar register in cp110_pcie_clk_cfg
I am not sure how this can lead to the problem of the comphy not coming up.

Clocks are often responsible of stalls when accessing registers.

I don't know what your problem is, I am giving you suggestions.

I can also see that the PCIe works in uboot and the Intel NIC is recognized just fine there so the intial comphy setup must be working.

Is U-Boot doing SMC calls too? I suppose it does not.

I could not find the reset code, can you send me the file where it is?

I thought we had some in the comphy driver, apparently not.

I don't have more inputs so far. Is your debugging giving interesting
infos? Have you checked in ->set_mode() everything looks fine?


Thanks,
Miquèl



Beste Grüße/Best regards

Sven Auhagen
Dipl. Math. oec., M.Sc.
Voleatech GmbH
HRB: B 754643
USTID: DE303643180
Grathwohlstr. 5
72762 Reutlingen
Tel: +49 7121539550
Fax: +49 7121539551
E-Mail: sven.auhagen@voleatech.de
www.voleatech.de<https://www.voleatech.de>
Diese Information ist ausschließlich für den Adressaten bestimmt und kann vertraulich oder gesetzlich geschützte Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Anderen als dem bestimmungsgemäßen Adressaten ist es untersagt, diese E-Mail zu lesen, zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden. Für den Adressaten sind die Informationen in dieser Mail nur zum persönlichen Gebrauch. Eine Weiterleitung darf nur nach Rücksprache mit dem Absender erfolgen. Wir verwenden aktuelle Virenschutzprogramme. Für Schäden, die dem Empfänger gleichwohl durch von uns zugesandte mit Viren befallene E-Mails entstehen, schließen wir jede Haftung aus.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel