* Re: next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) [not found] <5d700b15.1c69fb81.2abcd.479b@mx.google.com> @ 2019-09-04 19:27 ` Mark Brown 2019-09-05 0:15 ` Kevin Hilman 0 siblings, 1 reply; 7+ messages in thread From: Mark Brown @ 2019-09-04 19:27 UTC (permalink / raw) To: khilman, Matthias Brugger, linux-mediatek, linux-arm-kernel Cc: kernel-build-reports [-- Attachment #1.1: Type: text/plain, Size: 959 bytes --] On Wed, Sep 04, 2019 at 12:05:57PM -0700, kernelci.org bot wrote: Since 30th August -next fails to boot with no kernel output on mt7622-rfb1: > arm64: > defconfig: > gcc-8: > mt7622-rfb1: 1 failed lab > > defconfig+CONFIG_RANDOMIZE_BASE=y: > gcc-8: > mt7622-rfb1: 1 failed lab There's logging from ATF so it looks like we try to boot the kernel: Starting kernel ... [ATF][ 36.199793]save kernel info [ATF][ 36.202824]Kernel_EL2 [ATF][ 36.205580]Kernel is 64Bit [ATF][ 36.208768]pc=0x40080000, r0=0x5cf48000, r1=0x0 INFO: BL3-1: Preparing for EL3 exit to normal world, Kernel INFO: BL3-1: Next image address = 0x40080000 INFO: BL3-1: Next image spsr = 0x3c9 [ATF][ 36.227037]el3_exit but no output. More details including full logs at: https://kernelci.org/boot/id/5d6fe70059b514164ef1224d/ https://kernelci.org/boot/id/5d6fe6e259b514164ef12243/ [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) 2019-09-04 19:27 ` next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) Mark Brown @ 2019-09-05 0:15 ` Kevin Hilman 2019-09-05 15:39 ` Mark Brown 0 siblings, 1 reply; 7+ messages in thread From: Kevin Hilman @ 2019-09-05 0:15 UTC (permalink / raw) To: Matthias Brugger, Steven Liu Cc: , Mark Brown, linux-mediatek, linux-arm-kernel, kernel-build-reports [ + Steven Liu who donated this board to my kernelCI lab ] Mark Brown <broonie@kernel.org> writes: > On Wed, Sep 04, 2019 at 12:05:57PM -0700, kernelci.org bot wrote: > > Since 30th August -next fails to boot with no kernel output on > mt7622-rfb1: > >> arm64: >> defconfig: >> gcc-8: >> mt7622-rfb1: 1 failed lab >> >> defconfig+CONFIG_RANDOMIZE_BASE=y: >> gcc-8: >> mt7622-rfb1: 1 failed lab > > There's logging from ATF so it looks like we try to boot the kernel: > > Starting kernel ... > > [ATF][ 36.199793]save kernel info > [ATF][ 36.202824]Kernel_EL2 > [ATF][ 36.205580]Kernel is 64Bit > [ATF][ 36.208768]pc=0x40080000, r0=0x5cf48000, r1=0x0 > INFO: BL3-1: Preparing for EL3 exit to normal world, Kernel > INFO: BL3-1: Next image address = 0x40080000 > INFO: BL3-1: Next image spsr = 0x3c9 > [ATF][ 36.227037]el3_exit > > but no output. More details including full logs at: > > https://kernelci.org/boot/id/5d6fe70059b514164ef1224d/ > https://kernelci.org/boot/id/5d6fe6e259b514164ef12243/ Bisected down to this commit[1], full bisect log here[2]. It didn't revert cleanly on top of next-20190904, so I didn't get any further. Kevin [1] 419e2f1838819e954071dfa1d1f820ab3386ada1 is the first bad commit commit 419e2f1838819e954071dfa1d1f820ab3386ada1 Author: Christoph Hellwig <hch@lst.de> Date: Mon Aug 26 09:03:44 2019 +0200 dma-mapping: remove arch_dma_mmap_pgprot arch_dma_mmap_pgprot is used for two things: 1) to override the "normal" uncached page attributes for mapping memory coherent to devices that can't snoop the CPU caches 2) to provide the special DMA_ATTR_WRITE_COMBINE semantics on older arm systems and some mips platforms Replace one with the pgprot_dmacoherent macro that is already provided by arm and much simpler to use, and lift the DMA_ATTR_WRITE_COMBINE handling to common code with an explicit arch opt-in. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> # m68k Acked-by: Paul Burton <paul.burton@mips.com> # mips arch/arm/Kconfig | 2 +- arch/arm/mm/dma-mapping.c | 6 ------ arch/arm64/Kconfig | 1 - arch/arm64/include/asm/pgtable.h | 4 ++++ arch/arm64/mm/dma-mapping.c | 6 ------ arch/m68k/Kconfig | 1 - arch/m68k/include/asm/pgtable_mm.h | 3 +++ arch/m68k/kernel/dma.c | 3 +-- arch/mips/Kconfig | 2 +- arch/mips/mm/dma-noncoherent.c | 8 -------- include/linux/dma-noncoherent.h | 13 +++++++++++-- kernel/dma/Kconfig | 12 +++++++++--- kernel/dma/mapping.c | 8 +++++--- 13 files changed, 35 insertions(+), 34 deletions(-) bisect run success [2] $ git bisect log git bisect start # bad: [35394d031b710e832849fca60d0f53b513f0c390] Add linux-next specific files for 20190904 git bisect bad 35394d031b710e832849fca60d0f53b513f0c390 # good: [089cf7f6ecb266b6a4164919a2e69bd2f938374a] Linux 5.3-rc7 git bisect good 089cf7f6ecb266b6a4164919a2e69bd2f938374a # bad: [493424779be81b03fb4aca89cc05ba95e9fc0c31] Merge remote-tracking branch 'crypto/master' git bisect bad 493424779be81b03fb4aca89cc05ba95e9fc0c31 # bad: [58a02f90fddfdc9e4dfbd6579ff788ffdc22afe4] Merge remote-tracking branch 'hid/for-next' git bisect bad 58a02f90fddfdc9e4dfbd6579ff788ffdc22afe4 # bad: [27c3f6e1d84b47def9060fd481be92838d819a9b] Merge remote-tracking branch 'csky/linux-next' git bisect bad 27c3f6e1d84b47def9060fd481be92838d819a9b # good: [f119c164735d85f2a41d14503cb9933d219c539e] arm-soc: document merges git bisect good f119c164735d85f2a41d14503cb9933d219c539e # bad: [e87b432d6c45697defc03eb69261661060c85245] Merge remote-tracking branch 'actions/for-next' git bisect bad e87b432d6c45697defc03eb69261661060c85245 # good: [1ab97157f64dadb44d029096c6a92305d6631ab2] Merge remote-tracking branch 'kbuild/for-next' git bisect good 1ab97157f64dadb44d029096c6a92305d6631ab2 # good: [ac12cf85d682a2c1948210c65f7fb21ef01dd9f6] Merge branches 'for-next/52-bit-kva', 'for-next/cpu-topology', 'for-next/error-injection', 'for-next/perf', 'for-next/psci-cpuidle', 'for-next/rng', 'for-next/smpboot', 'for-next/tbi' and 'for-next/tlbi' into for-next/core git bisect good ac12cf85d682a2c1948210c65f7fb21ef01dd9f6 # bad: [4934d349f6e5afc9345a44acb0daa3066594088a] Merge remote-tracking branch 'asm-generic/master' git bisect bad 4934d349f6e5afc9345a44acb0daa3066594088a # good: [5251a1c90f7f4e458dc3154920e09624311f54b6] Merge remote-tracking branch 'compiler-attributes/compiler-attributes' git bisect good 5251a1c90f7f4e458dc3154920e09624311f54b6 # skip: [38c38cb73223218f6eedf485280917af1f8a0af2] mmc: queue: use bigger segments if DMA MAP layer can merge the segments git bisect skip 38c38cb73223218f6eedf485280917af1f8a0af2 # bad: [419e2f1838819e954071dfa1d1f820ab3386ada1] dma-mapping: remove arch_dma_mmap_pgprot git bisect bad 419e2f1838819e954071dfa1d1f820ab3386ada1 # good: [5518ea1ad2c0c7f38d067f621d9349e6a11c8879] unicore32: remove the unused pgprot_dmacoherent define git bisect good 5518ea1ad2c0c7f38d067f621d9349e6a11c8879 # good: [b898e50f9f49f7d90f3bca94ac046145072034a2] arm-nommu: remove the unused pgprot_dmacoherent define git bisect good b898e50f9f49f7d90f3bca94ac046145072034a2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) 2019-09-05 0:15 ` Kevin Hilman @ 2019-09-05 15:39 ` Mark Brown 2019-09-05 15:40 ` Christoph Hellwig 2019-09-05 17:02 ` Catalin Marinas 0 siblings, 2 replies; 7+ messages in thread From: Mark Brown @ 2019-09-05 15:39 UTC (permalink / raw) To: Kevin Hilman, Christoph Hellwig Cc: Steven Liu, kernel-build-reports, Catalin Marinas, linux-mediatek, Matthias Brugger, Will Deacon, linux-arm-kernel [-- Attachment #1.1: Type: text/plain, Size: 5671 bytes --] On Wed, Sep 04, 2019 at 05:15:53PM -0700, Kevin Hilman wrote: > [ + Steven Liu who donated this board to my kernelCI lab ] Also adding Christoph since this was bisected to his commit and Catalin and Will since this was an architecture change. > Mark Brown <broonie@kernel.org> writes: > > On Wed, Sep 04, 2019 at 12:05:57PM -0700, kernelci.org bot wrote: > > Since 30th August -next fails to boot with no kernel output on > > mt7622-rfb1: ... > > There's logging from ATF so it looks like we try to boot the kernel: > > > > Starting kernel ... > > > > [ATF][ 36.199793]save kernel info > > [ATF][ 36.202824]Kernel_EL2 > > [ATF][ 36.205580]Kernel is 64Bit > > [ATF][ 36.208768]pc=0x40080000, r0=0x5cf48000, r1=0x0 > > INFO: BL3-1: Preparing for EL3 exit to normal world, Kernel > > INFO: BL3-1: Next image address = 0x40080000 > > INFO: BL3-1: Next image spsr = 0x3c9 > > [ATF][ 36.227037]el3_exit > > but no output. More details including full logs at: > > > > https://kernelci.org/boot/id/5d6fe70059b514164ef1224d/ > > https://kernelci.org/boot/id/5d6fe6e259b514164ef12243/ > > Bisected down to this commit[1], full bisect log here[2]. It didn't > revert cleanly on top of next-20190904, so I didn't get any further. > [1] > 419e2f1838819e954071dfa1d1f820ab3386ada1 is the first bad commit > commit 419e2f1838819e954071dfa1d1f820ab3386ada1 > Author: Christoph Hellwig <hch@lst.de> > Date: Mon Aug 26 09:03:44 2019 +0200 > > dma-mapping: remove arch_dma_mmap_pgprot > > arch_dma_mmap_pgprot is used for two things: > > 1) to override the "normal" uncached page attributes for mapping > memory coherent to devices that can't snoop the CPU caches > 2) to provide the special DMA_ATTR_WRITE_COMBINE semantics on older > arm systems and some mips platforms > > Replace one with the pgprot_dmacoherent macro that is already provided > by arm and much simpler to use, and lift the DMA_ATTR_WRITE_COMBINE > handling to common code with an explicit arch opt-in. > > Signed-off-by: Christoph Hellwig <hch@lst.de> > Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> # m68k > Acked-by: Paul Burton <paul.burton@mips.com> # mips > > arch/arm/Kconfig | 2 +- > arch/arm/mm/dma-mapping.c | 6 ------ > arch/arm64/Kconfig | 1 - > arch/arm64/include/asm/pgtable.h | 4 ++++ > arch/arm64/mm/dma-mapping.c | 6 ------ > arch/m68k/Kconfig | 1 - > arch/m68k/include/asm/pgtable_mm.h | 3 +++ > arch/m68k/kernel/dma.c | 3 +-- > arch/mips/Kconfig | 2 +- > arch/mips/mm/dma-noncoherent.c | 8 -------- > include/linux/dma-noncoherent.h | 13 +++++++++++-- > kernel/dma/Kconfig | 12 +++++++++--- > kernel/dma/mapping.c | 8 +++++--- > 13 files changed, 35 insertions(+), 34 deletions(-) > bisect run success > > [2] > $ git bisect log > git bisect start > # bad: [35394d031b710e832849fca60d0f53b513f0c390] Add linux-next specific files for 20190904 > git bisect bad 35394d031b710e832849fca60d0f53b513f0c390 > # good: [089cf7f6ecb266b6a4164919a2e69bd2f938374a] Linux 5.3-rc7 > git bisect good 089cf7f6ecb266b6a4164919a2e69bd2f938374a > # bad: [493424779be81b03fb4aca89cc05ba95e9fc0c31] Merge remote-tracking branch 'crypto/master' > git bisect bad 493424779be81b03fb4aca89cc05ba95e9fc0c31 > # bad: [58a02f90fddfdc9e4dfbd6579ff788ffdc22afe4] Merge remote-tracking branch 'hid/for-next' > git bisect bad 58a02f90fddfdc9e4dfbd6579ff788ffdc22afe4 > # bad: [27c3f6e1d84b47def9060fd481be92838d819a9b] Merge remote-tracking branch 'csky/linux-next' > git bisect bad 27c3f6e1d84b47def9060fd481be92838d819a9b > # good: [f119c164735d85f2a41d14503cb9933d219c539e] arm-soc: document merges > git bisect good f119c164735d85f2a41d14503cb9933d219c539e > # bad: [e87b432d6c45697defc03eb69261661060c85245] Merge remote-tracking branch 'actions/for-next' > git bisect bad e87b432d6c45697defc03eb69261661060c85245 > # good: [1ab97157f64dadb44d029096c6a92305d6631ab2] Merge remote-tracking branch 'kbuild/for-next' > git bisect good 1ab97157f64dadb44d029096c6a92305d6631ab2 > # good: [ac12cf85d682a2c1948210c65f7fb21ef01dd9f6] Merge branches 'for-next/52-bit-kva', 'for-next/cpu-topology', 'for-next/error-injection', 'for-next/perf', 'for-next/psci-cpuidle', 'for-next/rng', 'for-next/smpboot', 'for-next/tbi' and 'for-next/tlbi' into for-next/core > git bisect good ac12cf85d682a2c1948210c65f7fb21ef01dd9f6 > # bad: [4934d349f6e5afc9345a44acb0daa3066594088a] Merge remote-tracking branch 'asm-generic/master' > git bisect bad 4934d349f6e5afc9345a44acb0daa3066594088a > # good: [5251a1c90f7f4e458dc3154920e09624311f54b6] Merge remote-tracking branch 'compiler-attributes/compiler-attributes' > git bisect good 5251a1c90f7f4e458dc3154920e09624311f54b6 > # skip: [38c38cb73223218f6eedf485280917af1f8a0af2] mmc: queue: use bigger segments if DMA MAP layer can merge the segments > git bisect skip 38c38cb73223218f6eedf485280917af1f8a0af2 > # bad: [419e2f1838819e954071dfa1d1f820ab3386ada1] dma-mapping: remove arch_dma_mmap_pgprot > git bisect bad 419e2f1838819e954071dfa1d1f820ab3386ada1 > # good: [5518ea1ad2c0c7f38d067f621d9349e6a11c8879] unicore32: remove the unused pgprot_dmacoherent define > git bisect good 5518ea1ad2c0c7f38d067f621d9349e6a11c8879 > # good: [b898e50f9f49f7d90f3bca94ac046145072034a2] arm-nommu: remove the unused pgprot_dmacoherent define > git bisect good b898e50f9f49f7d90f3bca94ac046145072034a2 [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) 2019-09-05 15:39 ` Mark Brown @ 2019-09-05 15:40 ` Christoph Hellwig 2019-09-05 15:43 ` Mark Brown 2019-09-05 17:02 ` Catalin Marinas 1 sibling, 1 reply; 7+ messages in thread From: Christoph Hellwig @ 2019-09-05 15:40 UTC (permalink / raw) To: Mark Brown Cc: Steven Liu, kernel-build-reports, Kevin Hilman, linux-mediatek, Catalin Marinas, Matthias Brugger, Will Deacon, Christoph Hellwig, linux-arm-kernel On Thu, Sep 05, 2019 at 04:39:38PM +0100, Mark Brown wrote: > On Wed, Sep 04, 2019 at 05:15:53PM -0700, Kevin Hilman wrote: > > [ + Steven Liu who donated this board to my kernelCI lab ] > > Also adding Christoph since this was bisected to his commit and Catalin > and Will since this was an architecture change. Given Will in CC the problem is on arm64? _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) 2019-09-05 15:40 ` Christoph Hellwig @ 2019-09-05 15:43 ` Mark Brown 0 siblings, 0 replies; 7+ messages in thread From: Mark Brown @ 2019-09-05 15:43 UTC (permalink / raw) To: Christoph Hellwig Cc: Steven Liu, kernel-build-reports, Kevin Hilman, linux-mediatek, Catalin Marinas, Matthias Brugger, Will Deacon, linux-arm-kernel [-- Attachment #1.1: Type: text/plain, Size: 590 bytes --] On Thu, Sep 05, 2019 at 05:40:53PM +0200, Christoph Hellwig wrote: > On Thu, Sep 05, 2019 at 04:39:38PM +0100, Mark Brown wrote: > > On Wed, Sep 04, 2019 at 05:15:53PM -0700, Kevin Hilman wrote: > > > [ + Steven Liu who donated this board to my kernelCI lab ] > > Also adding Christoph since this was bisected to his commit and Catalin > > and Will since this was an architecture change. > Given Will in CC the problem is on arm64? Yes, this is an arm64 system. It seems to be the only one in KernelCI that's been affected so it's not like the entire architecture exploded or anything. [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] [-- Attachment #2: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) 2019-09-05 15:39 ` Mark Brown 2019-09-05 15:40 ` Christoph Hellwig @ 2019-09-05 17:02 ` Catalin Marinas 2019-09-05 19:43 ` Kevin Hilman 1 sibling, 1 reply; 7+ messages in thread From: Catalin Marinas @ 2019-09-05 17:02 UTC (permalink / raw) To: Mark Brown Cc: Steven Liu, kernel-build-reports, Kevin Hilman, linux-mediatek, Matthias Brugger, Will Deacon, Christoph Hellwig, linux-arm-kernel On Thu, Sep 05, 2019 at 04:39:38PM +0100, Mark Brown wrote: > On Wed, Sep 04, 2019 at 05:15:53PM -0700, Kevin Hilman wrote: > > [ + Steven Liu who donated this board to my kernelCI lab ] > > Also adding Christoph since this was bisected to his commit and Catalin > and Will since this was an architecture change. Thanks. > > Bisected down to this commit[1], full bisect log here[2]. It didn't > > revert cleanly on top of next-20190904, so I didn't get any further. > > > [1] > > 419e2f1838819e954071dfa1d1f820ab3386ada1 is the first bad commit > > commit 419e2f1838819e954071dfa1d1f820ab3386ada1 > > Author: Christoph Hellwig <hch@lst.de> > > Date: Mon Aug 26 09:03:44 2019 +0200 I stared at this commit and the ones around it for a while but couldn't see anything obvious. AFAICT this commit only replaces some macros, no functional change. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) 2019-09-05 17:02 ` Catalin Marinas @ 2019-09-05 19:43 ` Kevin Hilman 0 siblings, 0 replies; 7+ messages in thread From: Kevin Hilman @ 2019-09-05 19:43 UTC (permalink / raw) To: Catalin Marinas, Mark Brown Cc: Steven Liu, kernel-build-reports, linux-mediatek, Matthias Brugger, Will Deacon, Christoph Hellwig, linux-arm-kernel Catalin Marinas <catalin.marinas@arm.com> writes: > On Thu, Sep 05, 2019 at 04:39:38PM +0100, Mark Brown wrote: >> On Wed, Sep 04, 2019 at 05:15:53PM -0700, Kevin Hilman wrote: >> > [ + Steven Liu who donated this board to my kernelCI lab ] >> >> Also adding Christoph since this was bisected to his commit and Catalin >> and Will since this was an architecture change. > > Thanks. > >> > Bisected down to this commit[1], full bisect log here[2]. It didn't >> > revert cleanly on top of next-20190904, so I didn't get any further. >> >> > [1] >> > 419e2f1838819e954071dfa1d1f820ab3386ada1 is the first bad commit >> > commit 419e2f1838819e954071dfa1d1f820ab3386ada1 >> > Author: Christoph Hellwig <hch@lst.de> >> > Date: Mon Aug 26 09:03:44 2019 +0200 > > I stared at this commit and the ones around it for a while but couldn't > see anything obvious. AFAICT this commit only replaces some macros, no > functional change. This is a an arm64 mediatek SoC (MT7622) so hopefully the folks on linux-mediatek can take a closer look and see what's going on here. If someone can suggest how I can enable an earlcon on this platform, I'd be happy to give that a try to see if we can get any kernel output before it hangs. Kevin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-09-05 19:43 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <5d700b15.1c69fb81.2abcd.479b@mx.google.com>
2019-09-04 19:27 ` next/master boot: 310 boots: 11 failed, 292 passed with 6 offline, 1 untried/unknown (next-20190904) Mark Brown
2019-09-05 0:15 ` Kevin Hilman
2019-09-05 15:39 ` Mark Brown
2019-09-05 15:40 ` Christoph Hellwig
2019-09-05 15:43 ` Mark Brown
2019-09-05 17:02 ` Catalin Marinas
2019-09-05 19:43 ` Kevin Hilman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).