From: Michael Mueller <mimu@linux.ibm.com> To: Cornelia Huck <cohuck@redhat.com>, Halil Pasic <pasic@linux.ibm.com> Cc: KVM Mailing List <kvm@vger.kernel.org>, Linux-S390 Mailing List <linux-s390@vger.kernel.org>, Sebastian Ott <sebott@linux.ibm.com>, Heiko Carstens <heiko.carstens@de.ibm.com>, virtualization@lists.linux-foundation.org, "Michael S . Tsirkin" <mst@redhat.com>, Christoph Hellwig <hch@infradead.org>, Thomas Huth <thuth@redhat.com>, Christian Borntraeger <borntraeger@de.ibm.com>, Viktor Mihajlovski <mihajlov@linux.ibm.com>, Vasily Gorbik <gor@linux.ibm.com>, Janosch Frank <frankja@linux.ibm.com>, Claudio Imbrenda <imbrenda@linux.ibm.com>, Farhan Ali <alifm@linux.ibm.com>, Eric Farman <farman@linux.ibm.com>, Pierre Morel <pmorel@linux.ibm.com> Subject: Re: [PATCH v3 2/8] s390/cio: introduce DMA pools to cio Date: Mon, 3 Jun 2019 14:09:02 +0200 [thread overview] Message-ID: <035b4bd3-5856-e8e5-91bf-ba0b5c7c3736@linux.ibm.com> (raw) In-Reply-To: <20190603133745.240c00a7.cohuck@redhat.com> On 03.06.19 13:37, Cornelia Huck wrote: > On Wed, 29 May 2019 14:26:51 +0200 > Michael Mueller <mimu@linux.ibm.com> wrote: > >> From: Halil Pasic <pasic@linux.ibm.com> >> >> To support protected virtualization cio will need to make sure the >> memory used for communication with the hypervisor is DMA memory. >> >> Let us introduce one global pool for cio. >> >> Our DMA pools are implemented as a gen_pool backed with DMA pages. The >> idea is to avoid each allocation effectively wasting a page, as we >> typically allocate much less than PAGE_SIZE. >> >> Signed-off-by: Halil Pasic <pasic@linux.ibm.com> >> Reviewed-by: Sebastian Ott <sebott@linux.ibm.com> >> Signed-off-by: Michael Mueller <mimu@linux.ibm.com> >> --- >> arch/s390/Kconfig | 1 + >> arch/s390/include/asm/cio.h | 11 ++++ >> drivers/s390/cio/css.c | 120 ++++++++++++++++++++++++++++++++++++++++++-- >> 3 files changed, 128 insertions(+), 4 deletions(-) > > (...) > >> diff --git a/arch/s390/include/asm/cio.h b/arch/s390/include/asm/cio.h >> index 1727180e8ca1..43c007d2775a 100644 >> --- a/arch/s390/include/asm/cio.h >> +++ b/arch/s390/include/asm/cio.h >> @@ -328,6 +328,17 @@ static inline u8 pathmask_to_pos(u8 mask) >> void channel_subsystem_reinit(void); >> extern void css_schedule_reprobe(void); >> >> +extern void *cio_dma_zalloc(size_t size); >> +extern void cio_dma_free(void *cpu_addr, size_t size); >> +extern struct device *cio_get_dma_css_dev(void); >> + >> +struct gen_pool; > > That forward declaration is a bit ugly... I guess the alternative was > include hell? That's an easy one. #include <linux/bitops.h> +#include <linux/genalloc.h> #include <asm/types.h> > >> +void *cio_gp_dma_zalloc(struct gen_pool *gp_dma, struct device *dma_dev, >> + size_t size); >> +void cio_gp_dma_free(struct gen_pool *gp_dma, void *cpu_addr, size_t size); >> +void cio_gp_dma_destroy(struct gen_pool *gp_dma, struct device *dma_dev); >> +struct gen_pool *cio_gp_dma_create(struct device *dma_dev, int nr_pages); >> + >> /* Function from drivers/s390/cio/chsc.c */ >> int chsc_sstpc(void *page, unsigned int op, u16 ctrl, u64 *clock_delta); >> int chsc_sstpi(void *page, void *result, size_t size); >> diff --git a/drivers/s390/cio/css.c b/drivers/s390/cio/css.c >> index aea502922646..b97618497848 100644 >> --- a/drivers/s390/cio/css.c >> +++ b/drivers/s390/cio/css.c >> @@ -20,6 +20,8 @@ >> #include <linux/reboot.h> >> #include <linux/suspend.h> >> #include <linux/proc_fs.h> >> +#include <linux/genalloc.h> >> +#include <linux/dma-mapping.h> >> #include <asm/isc.h> >> #include <asm/crw.h> >> >> @@ -224,6 +226,8 @@ struct subchannel *css_alloc_subchannel(struct subchannel_id schid, >> INIT_WORK(&sch->todo_work, css_sch_todo); >> sch->dev.release = &css_subchannel_release; >> device_initialize(&sch->dev); > > It might be helpful to add a comment why you use 31 bit here... @Halil, please let me know what comment you prefere here... > >> + sch->dev.coherent_dma_mask = DMA_BIT_MASK(31); >> + sch->dev.dma_mask = &sch->dev.coherent_dma_mask; >> return sch; >> >> err: >> @@ -899,6 +903,8 @@ static int __init setup_css(int nr) >> dev_set_name(&css->device, "css%x", nr); >> css->device.groups = cssdev_attr_groups; >> css->device.release = channel_subsystem_release; > > ...and 64 bit here. and here. > >> + css->device.coherent_dma_mask = DMA_BIT_MASK(64); >> + css->device.dma_mask = &css->device.coherent_dma_mask; >> >> mutex_init(&css->mutex); >> css->cssid = chsc_get_cssid(nr); > > (...) > >> @@ -1059,16 +1168,19 @@ static int __init css_bus_init(void) >> if (ret) >> goto out_unregister; >> ret = register_pm_notifier(&css_power_notifier); >> - if (ret) { >> - unregister_reboot_notifier(&css_reboot_notifier); >> - goto out_unregister; >> - } >> + if (ret) >> + goto out_unregister_rn; >> + ret = cio_dma_pool_init(); >> + if (ret) >> + goto out_unregister_rn; > > Don't you also need to unregister the pm notifier on failure here? Mmh, that was the original intention. Thanks! > > Other than that, I noticed only cosmetic issues; seems reasonable to me. > >> css_init_done = 1; >> >> /* Enable default isc for I/O subchannels. */ >> isc_register(IO_SCH_ISC); >> >> return 0; >> +out_unregister_rn: >> + unregister_reboot_notifier(&css_reboot_notifier); >> out_unregister: >> while (i-- > 0) { >> struct channel_subsystem *css = channel_subsystems[i]; > Thanks, Michael
WARNING: multiple messages have this Message-ID (diff)
From: Michael Mueller <mimu@linux.ibm.com> To: Cornelia Huck <cohuck@redhat.com>, Halil Pasic <pasic@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com>, Linux-S390 Mailing List <linux-s390@vger.kernel.org>, Thomas Huth <thuth@redhat.com>, Claudio Imbrenda <imbrenda@linux.ibm.com>, KVM Mailing List <kvm@vger.kernel.org>, Sebastian Ott <sebott@linux.ibm.com>, "Michael S . Tsirkin" <mst@redhat.com>, Pierre Morel <pmorel@linux.ibm.com>, Farhan Ali <alifm@linux.ibm.com>, Heiko Carstens <heiko.carstens@de.ibm.com>, Eric Farman <farman@linux.ibm.com>, virtualization@lists.linux-foundation.org, Christoph Hellwig <hch@infradead.org>, Christian Borntraeger <borntraeger@de.ibm.com>, Viktor Mihajlovski <mihajlov@linux.ibm.com>, Janosch Frank <frankja@linux.ibm.com> Subject: Re: [PATCH v3 2/8] s390/cio: introduce DMA pools to cio Date: Mon, 3 Jun 2019 14:09:02 +0200 [thread overview] Message-ID: <035b4bd3-5856-e8e5-91bf-ba0b5c7c3736@linux.ibm.com> (raw) In-Reply-To: <20190603133745.240c00a7.cohuck@redhat.com> On 03.06.19 13:37, Cornelia Huck wrote: > On Wed, 29 May 2019 14:26:51 +0200 > Michael Mueller <mimu@linux.ibm.com> wrote: > >> From: Halil Pasic <pasic@linux.ibm.com> >> >> To support protected virtualization cio will need to make sure the >> memory used for communication with the hypervisor is DMA memory. >> >> Let us introduce one global pool for cio. >> >> Our DMA pools are implemented as a gen_pool backed with DMA pages. The >> idea is to avoid each allocation effectively wasting a page, as we >> typically allocate much less than PAGE_SIZE. >> >> Signed-off-by: Halil Pasic <pasic@linux.ibm.com> >> Reviewed-by: Sebastian Ott <sebott@linux.ibm.com> >> Signed-off-by: Michael Mueller <mimu@linux.ibm.com> >> --- >> arch/s390/Kconfig | 1 + >> arch/s390/include/asm/cio.h | 11 ++++ >> drivers/s390/cio/css.c | 120 ++++++++++++++++++++++++++++++++++++++++++-- >> 3 files changed, 128 insertions(+), 4 deletions(-) > > (...) > >> diff --git a/arch/s390/include/asm/cio.h b/arch/s390/include/asm/cio.h >> index 1727180e8ca1..43c007d2775a 100644 >> --- a/arch/s390/include/asm/cio.h >> +++ b/arch/s390/include/asm/cio.h >> @@ -328,6 +328,17 @@ static inline u8 pathmask_to_pos(u8 mask) >> void channel_subsystem_reinit(void); >> extern void css_schedule_reprobe(void); >> >> +extern void *cio_dma_zalloc(size_t size); >> +extern void cio_dma_free(void *cpu_addr, size_t size); >> +extern struct device *cio_get_dma_css_dev(void); >> + >> +struct gen_pool; > > That forward declaration is a bit ugly... I guess the alternative was > include hell? That's an easy one. #include <linux/bitops.h> +#include <linux/genalloc.h> #include <asm/types.h> > >> +void *cio_gp_dma_zalloc(struct gen_pool *gp_dma, struct device *dma_dev, >> + size_t size); >> +void cio_gp_dma_free(struct gen_pool *gp_dma, void *cpu_addr, size_t size); >> +void cio_gp_dma_destroy(struct gen_pool *gp_dma, struct device *dma_dev); >> +struct gen_pool *cio_gp_dma_create(struct device *dma_dev, int nr_pages); >> + >> /* Function from drivers/s390/cio/chsc.c */ >> int chsc_sstpc(void *page, unsigned int op, u16 ctrl, u64 *clock_delta); >> int chsc_sstpi(void *page, void *result, size_t size); >> diff --git a/drivers/s390/cio/css.c b/drivers/s390/cio/css.c >> index aea502922646..b97618497848 100644 >> --- a/drivers/s390/cio/css.c >> +++ b/drivers/s390/cio/css.c >> @@ -20,6 +20,8 @@ >> #include <linux/reboot.h> >> #include <linux/suspend.h> >> #include <linux/proc_fs.h> >> +#include <linux/genalloc.h> >> +#include <linux/dma-mapping.h> >> #include <asm/isc.h> >> #include <asm/crw.h> >> >> @@ -224,6 +226,8 @@ struct subchannel *css_alloc_subchannel(struct subchannel_id schid, >> INIT_WORK(&sch->todo_work, css_sch_todo); >> sch->dev.release = &css_subchannel_release; >> device_initialize(&sch->dev); > > It might be helpful to add a comment why you use 31 bit here... @Halil, please let me know what comment you prefere here... > >> + sch->dev.coherent_dma_mask = DMA_BIT_MASK(31); >> + sch->dev.dma_mask = &sch->dev.coherent_dma_mask; >> return sch; >> >> err: >> @@ -899,6 +903,8 @@ static int __init setup_css(int nr) >> dev_set_name(&css->device, "css%x", nr); >> css->device.groups = cssdev_attr_groups; >> css->device.release = channel_subsystem_release; > > ...and 64 bit here. and here. > >> + css->device.coherent_dma_mask = DMA_BIT_MASK(64); >> + css->device.dma_mask = &css->device.coherent_dma_mask; >> >> mutex_init(&css->mutex); >> css->cssid = chsc_get_cssid(nr); > > (...) > >> @@ -1059,16 +1168,19 @@ static int __init css_bus_init(void) >> if (ret) >> goto out_unregister; >> ret = register_pm_notifier(&css_power_notifier); >> - if (ret) { >> - unregister_reboot_notifier(&css_reboot_notifier); >> - goto out_unregister; >> - } >> + if (ret) >> + goto out_unregister_rn; >> + ret = cio_dma_pool_init(); >> + if (ret) >> + goto out_unregister_rn; > > Don't you also need to unregister the pm notifier on failure here? Mmh, that was the original intention. Thanks! > > Other than that, I noticed only cosmetic issues; seems reasonable to me. > >> css_init_done = 1; >> >> /* Enable default isc for I/O subchannels. */ >> isc_register(IO_SCH_ISC); >> >> return 0; >> +out_unregister_rn: >> + unregister_reboot_notifier(&css_reboot_notifier); >> out_unregister: >> while (i-- > 0) { >> struct channel_subsystem *css = channel_subsystems[i]; > Thanks, Michael
next prev parent reply other threads:[~2019-06-03 12:09 UTC|newest] Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-29 12:26 [PATCH v3 0/8] s390: virtio: support protected virtualization Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-05-29 12:26 ` [PATCH v3 1/8] s390/mm: force swiotlb for " Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-05-29 12:26 ` [PATCH v3 2/8] s390/cio: introduce DMA pools to cio Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-06-03 11:37 ` Cornelia Huck 2019-06-03 11:37 ` Cornelia Huck 2019-06-03 12:09 ` Michael Mueller [this message] 2019-06-03 12:09 ` Michael Mueller 2019-06-03 12:57 ` Halil Pasic 2019-06-03 12:57 ` Halil Pasic 2019-06-03 13:34 ` Cornelia Huck 2019-06-03 13:34 ` Cornelia Huck 2019-06-03 13:43 ` Michael Mueller 2019-06-03 13:43 ` Michael Mueller 2019-06-03 14:04 ` Halil Pasic 2019-06-03 14:04 ` Halil Pasic 2019-06-03 14:22 ` Halil Pasic 2019-06-03 14:22 ` Halil Pasic 2019-06-03 12:47 ` Halil Pasic 2019-06-03 12:47 ` Halil Pasic 2019-06-03 13:40 ` Cornelia Huck 2019-06-03 13:40 ` Cornelia Huck 2019-05-29 12:26 ` [PATCH v3 3/8] s390/cio: add basic protected virtualization support Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-06-03 12:06 ` Cornelia Huck 2019-06-03 12:06 ` Cornelia Huck 2019-06-03 12:45 ` Michael Mueller 2019-06-03 12:45 ` Michael Mueller 2019-06-03 13:42 ` Cornelia Huck 2019-06-03 13:42 ` Cornelia Huck 2019-06-03 13:52 ` Michael Mueller 2019-06-03 13:52 ` Michael Mueller 2019-05-29 12:26 ` [PATCH v3 4/8] s390/airq: use DMA memory for adapter interrupts Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-06-03 15:27 ` Cornelia Huck 2019-06-03 15:27 ` Cornelia Huck 2019-06-04 13:22 ` Halil Pasic 2019-06-04 13:22 ` Halil Pasic 2019-06-04 14:51 ` Cornelia Huck 2019-06-04 14:51 ` Cornelia Huck 2019-06-04 15:06 ` Halil Pasic 2019-06-04 15:06 ` Halil Pasic 2019-05-29 12:26 ` [PATCH v3 5/8] virtio/s390: use cacheline aligned airq bit vectors Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-06-03 9:10 ` Christian Borntraeger 2019-06-03 9:10 ` Christian Borntraeger 2019-06-03 15:40 ` Cornelia Huck 2019-06-03 15:40 ` Cornelia Huck 2019-05-29 12:26 ` [PATCH v3 6/8] virtio/s390: add indirection to indicators access Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-06-03 15:55 ` Cornelia Huck 2019-06-03 15:55 ` Cornelia Huck 2019-06-03 17:55 ` Halil Pasic 2019-06-03 17:55 ` Halil Pasic 2019-05-29 12:26 ` [PATCH v3 7/8] virtio/s390: use DMA memory for ccw I/O and classic notifiers Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-06-03 16:17 ` Cornelia Huck 2019-06-03 16:17 ` Cornelia Huck 2019-06-04 13:08 ` Halil Pasic 2019-06-04 13:08 ` Halil Pasic 2019-06-04 13:36 ` Cornelia Huck 2019-06-04 13:36 ` Cornelia Huck 2019-06-04 14:29 ` Halil Pasic 2019-06-04 14:29 ` Halil Pasic 2019-05-29 12:26 ` [PATCH v3 8/8] virtio/s390: make airq summary indicators DMA Michael Mueller 2019-05-29 12:26 ` Michael Mueller 2019-06-03 16:03 ` Cornelia Huck 2019-06-03 16:03 ` Cornelia Huck 2019-06-03 17:54 ` Halil Pasic 2019-06-03 17:54 ` Halil Pasic
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=035b4bd3-5856-e8e5-91bf-ba0b5c7c3736@linux.ibm.com \ --to=mimu@linux.ibm.com \ --cc=alifm@linux.ibm.com \ --cc=borntraeger@de.ibm.com \ --cc=cohuck@redhat.com \ --cc=farman@linux.ibm.com \ --cc=frankja@linux.ibm.com \ --cc=gor@linux.ibm.com \ --cc=hch@infradead.org \ --cc=heiko.carstens@de.ibm.com \ --cc=imbrenda@linux.ibm.com \ --cc=kvm@vger.kernel.org \ --cc=linux-s390@vger.kernel.org \ --cc=mihajlov@linux.ibm.com \ --cc=mst@redhat.com \ --cc=pasic@linux.ibm.com \ --cc=pmorel@linux.ibm.com \ --cc=sebott@linux.ibm.com \ --cc=thuth@redhat.com \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.