From: malte.vesper@postgrad.manchester.ac.uk (Malte Vesper)
To: kernelnewbies@lists.kernelnewbies.org
Subject: Propper use of pci_map_sg
Date: Mon, 02 Feb 2015 19:49:56 +0000 [thread overview]
Message-ID: <54CFD4E4.80607@postgrad.manchester.ac.uk> (raw)
In-Reply-To: <54CA8F1C.50909@postgrad.manchester.ac.uk>
Since my last mail went unanswered I did some further investigation. I
compiled a kernel with the DMA-API debug fs support and got the dmesg
output below. Note how I run the program 3 times and it only reports on
the third run that the page mapping count is off. The thing confusing me
the most about this, is that I can't see where I go wrong. Since I call
pci_map_sg and pci_unmap_sg directly after each other there is no room
for parameter modification. (I have only one device, and the interrupt
handler has output when invoked). So what is wrong with:
mapped =*pci_map_sg*(pcidev, scatterlist, pinned,
DMA_BIDIRECTIONAL);
*pci_unmap_sg*(pcidev, scatterlist, pinned, DMA_BIDIRECTIONAL);
Do I have to recreate the vanilla scatterlist for pci_unmap_sg?
Any hints/help is highly appreciated.
Ted
Dmesg output:
[ 2087.171247] SCAS FPGA 0000:01:00.0: [fpga+]: Removing device 0
[ 2087.171771] [fpga+]: SCAS MAJOR UNREGISTERED: 100
[ 2087.180070] [fpga+]: fpga+ registered with major number 100
[ 2087.180096] SCAS FPGA 0000:01:00.0: Probing
[ 2087.180163] SCAS FPGA 0000:01:00.0: irq 66 for MSI/MSI-X
[ 2087.180182] SCAS FPGA 0000:01:00.0: [fpga+]: Board tied to slot 0
[ 2092.359203] [fpga+]: IOCTL 2148040970
[ 2092.359206] [fpga+]: MINOR: 0
[ 2092.359207] [fpga+]: ARG: 140736686734672
[ 2092.359210] SCAS FPGA 0000:01:00.0: [fpga+]: IOCTL:
IOCTL_FPGA_PIN_PAGE
[ 2092.359211] Testers hell
[ 2092.359215] pinned 2
[ 2092.359216] setting page: 0
[ 2092.359217] setting page: 1
[ 2092.359223] Mapped: 2
[ 2092.359227] Done
[ 2117.846740] [fpga+]: IOCTL 2148040970
[ 2117.846742] [fpga+]: MINOR: 0
[ 2117.846743] [fpga+]: ARG: 140735106116944
[ 2117.846746] SCAS FPGA 0000:01:00.0: [fpga+]: IOCTL:
IOCTL_FPGA_PIN_PAGE
[ 2117.846746] Testers hell
[ 2117.846749] pinned 2
[ 2117.846749] setting page: 0
[ 2117.846750] setting page: 1
[ 2117.846755] Mapped: 2
[ 2117.846759] Done
[ 2124.136132] [fpga+]: IOCTL 2148040970
[ 2124.136134] [fpga+]: MINOR: 0
[ 2124.136135] [fpga+]: ARG: 140735560133008
[ 2124.136138] SCAS FPGA 0000:01:00.0: [fpga+]: IOCTL:
IOCTL_FPGA_PIN_PAGE
[ 2124.136138] Testers hell
[ 2124.136141] pinned 2
[ 2124.136142] setting page: 0
[ 2124.136142] setting page: 1
[ 2124.136147] Mapped: 2
[ 2124.136152] Done
[ 2124.136251] BUG: Bad page map in process dma pte:8000000f7b983867
pmd:fa9d31067
[ 2124.136254] page:ffffea003dee60c0 count:1 mapcount:-30705
mapping:ffff880fadddb901 index:0xc0054000afddc901
[ 2124.136254] flags:
0x2ffff000008006c(referenced|uptodate|lru|active|swapbacked)
[ 2124.136257] page dumped because: bad pte
[ 2124.136258] addr:0000000001382000 vm_flags:08100073
anon_vma:ffff880fadddb900 mapping: (null) index:1382
[ 2124.136261] CPU: 6 PID: 6902 Comm: dma Tainted: G B W OE
3.18.5hotdog #2
[ 2124.136261] Hardware name: System manufacturer System Product
Name/RAMPAGE IV BLACK EDITION, BIOS 0701 06/04/2014
[ 2124.136262] ffff880fb842eac8 ffff880fd4093c58 ffffffff8176ab0c
0000000000000000
[ 2124.136264] 0000000001382000 ffff880fd4093ca8 ffffffff8119215a
8000000f7b983867
[ 2124.136265] 0000000000001382 ffff880fd4093ca8 ffff880fa9d31c10
ffffea003dee60c0
[ 2124.136267] Call Trace:
[ 2124.136270] [<ffffffff8176ab0c>] dump_stack+0x46/0x58
[ 2124.136274] [<ffffffff8119215a>] print_bad_pte+0x1aa/0x250
[ 2124.136275] [<ffffffff8119367f>] unmap_single_vma+0x76f/0x8f0
[ 2124.136278] [<ffffffff81175cf0>] ? put_pages_list+0x70/0x70
[ 2124.136279] [<ffffffff811942d1>] unmap_vmas+0x51/0xa0
[ 2124.136281] [<ffffffff8119d48c>] exit_mmap+0x9c/0x170
[ 2124.136284] [<ffffffff8106c1c4>] mmput+0x64/0x130
[ 2124.136286] [<ffffffff810716ec>] do_exit+0x27c/0xa80
[ 2124.136287] [<ffffffff81071f7f>] do_group_exit+0x3f/0xa0
[ 2124.136290] [<ffffffff81071ff4>] SyS_exit_group+0x14/0x20
[ 2124.136292] [<ffffffff81772f6d>] system_call_fastpath+0x16/0x1b
[ 2124.136319] BUG: Bad page state in process dma pfn:f7b983
[ 2124.136320] page:ffffea003dee60c0 count:0 mapcount:-30705
mapping: (null) index:0xc0054000afddc901
[ 2124.136321] flags: 0x2ffff000008000c(referenced|uptodate|swapbacked)
[ 2124.136322] page dumped because: nonzero mapcount
[ 2124.136323] Modules linked in: fpga+(OE) hid_generic nfsv3
rpcsec_gss_krb5 nfsv4 vmw_vsock_vmci_transport vsock vmw_vmci
autofs4 nfsd auth_rpcgss nfs_acl nfs snd_hda_codec_realtek
snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel
snd_hda_controller snd_hda_codec x86_pkg_temp_thermal
intel_powerclamp snd_hwdep coretemp rfcomm snd_pcm kvm_intel kvm
bnep lockd bluetooth snd_seq_midi snd_seq_midi_event snd_rawmidi
snd_seq grace sunrpc eeepc_wmi asus_wmi snd_seq_device snd_timer
sparse_keymap joydev video mei_me sb_edac snd edac_core mei
serio_raw lpc_ich soundcore fscache binfmt_misc nls_iso8859_1
parport_pc ppdev mac_hid tpm_infineon cp210x usbserial lp parport
dm_crypt hid_microsoft usbhid hid radeon i2c_algo_bit ttm
drm_kms_helper mxm_wmi drm crct10dif_pclmul crc32_pclmul e1000e
ghash_clmulni_intel aesni_intel psmouse aes_x86_64 lrw gf128mul
glue_helper ablk_helper cryptd aacraid ahci ptp libahci pps_core wmi
[last unloaded: fpga+]
[ 2124.136351] CPU: 6 PID: 6902 Comm: dma Tainted: G B W OE
3.18.5hotdog #2
[ 2124.136352] Hardware name: System manufacturer System Product
Name/RAMPAGE IV BLACK EDITION, BIOS 0701 06/04/2014
[ 2124.136353] ffffffff81aa55c7 ffff880fd4093bb8 ffffffff8176ab0c
0000000000000000
[ 2124.136354] ffffea003dee60c0 ffff880fd4093be8 ffffffff81767527
ff0000ffffffffff
[ 2124.136355] ffffea003dee60c0 0000000000000000 00ffff0000000000
ffff880fd4093c38
[ 2124.136356] Call Trace:
[ 2124.136358] [<ffffffff8176ab0c>] dump_stack+0x46/0x58
[ 2124.136360] [<ffffffff81767527>] bad_page.part.50+0xe0/0xfe
[ 2124.136362] [<ffffffff8116c91d>] free_pages_prepare+0x16d/0x190
[ 2124.136363] [<ffffffff8116f1a5>] free_hot_cold_page+0x35/0x180
[ 2124.136365] [<ffffffff8116f33e>] free_hot_cold_page_list+0x4e/0xa0
[ 2124.136366] [<ffffffff81176076>] release_pages+0x1f6/0x270
[ 2124.136368] [<ffffffff811a88bd>] free_pages_and_swap_cache+0x8d/0xa0
[ 2124.136370] [<ffffffff81191b04>] tlb_flush_mmu_free+0x34/0x60
[ 2124.136371] [<ffffffff81192524>] tlb_flush_mmu+0x24/0x30
[ 2124.136373] [<ffffffff81192544>] tlb_finish_mmu+0x14/0x40
[ 2124.136374] [<ffffffff8119d4bb>] exit_mmap+0xcb/0x170
[ 2124.136376] [<ffffffff8106c1c4>] mmput+0x64/0x130
[ 2124.136378] [<ffffffff810716ec>] do_exit+0x27c/0xa80
[ 2124.136379] [<ffffffff81071f7f>] do_group_exit+0x3f/0xa0
[ 2124.136381] [<ffffffff81071ff4>] SyS_exit_group+0x14/0x20
[ 2124.136382] [<ffffffff81772f6d>] system_call_fastpath+0x16/0x1b
ted at ted-headlessBrute:~$
On 29/01/15 19:50, Malte Vesper wrote:
> Hi,
> I am trying to use pci_map_sg in combination with get_user_pages_fast
> for a driver. However my code screws the processes memory map over.
> For easier testing I bundled the piinin/unpinning and mapping
> unmapping into one ioctl call. The following code misses all the
> checks on return values, since it is only inteneded as a MWE.
>
> When I run my code I get a "*BUG: Bad page map in process ...*", after
> "Done" is printed.
>
> I tried following DMA-API-howto.txt and looking at other code, but I
> fail to see where I go wrong.
>
> Regards, Ted
>
>
> Code from the IOCTL handler:
>
> case IOCTL_FPGA_PIN_PAGE:
> {
> struct pageInfo pageInfo;
>
> dev_dbg(&pcidev->dev, pr_fmt("IOCTL: IOCTL_FPGA_PIN_PAGE\n"));
>
> if(!copy_from_user(&pageInfo, (void*)arg, sizeof(struct
> pageInfo))) {
> //horrible test
> const int noPages = pageInfo.size/PAGE_SIZE;
> int pinned;
> int mapped = 0;
> struct scatterlist* scatterlist;
> printk("Test start\n");
> //userspacestartpointer, nopages, write?, page* array
> struct page** pages=kmalloc(sizeof(struct
> page*)*noPages, GFP_KERNEL);
> pinned=*get_user_pages_fast*((unsigned
> long)pageInfo.start, noPages, 1, pages);
>
> scatterlist = kmalloc(sizeof(struct
> scatterlist)*pinned, GFP_KERNEL);
>
> for(int i=0; i<pinned; ++i) {
> sg_set_page(&scatterlist[i], pages[i], PAGE_SIZE, 0);
> }
>
> mapped =*pci_map_sg*(pcidev, scatterlist, pinned,
> DMA_BIDIRECTIONAL);
> *pci_unmap_sg*(pcidev, scatterlist, pinned, DMA_BIDIRECTIONAL);
>
> for(int i=0; i<pinned; ++i) {
> *put_page*(pages[i]); //I did place a print here and got two pages
> unpinned as I expected
> }
> kfree(scatterlist);
> kfree(pages);
> printk("Done\n");
> /*pageInfo.pageId = getPageId(getArea(&pageInfo));
> copy_to_user((void*)arg, &pageInfo, sizeof(struct
> pageInfo));*/
> return pageInfo.pageId;
> } else {
> return -EFAULT;
> }
> }
> break;
>
> P.S.: I used the following code to allocate the memory in userspace,
> and I use chunk and chunksize to fill pageInfo.start and pageInfo.size
> respectivly:
>
> const int pagesize = sysconf(_SC_PAGESIZE);
> const int chunksize = 2*pagesize;
> void* chunk;
>
> if(int error = posix_memalign(&chunk, pagesize, chunksize)) {
> std::cout << "Could not get a page." << std::endl;
> return error;
> }
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150202/2f1b79da/attachment-0001.html
next prev parent reply other threads:[~2015-02-02 19:49 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-01-29 19:50 Propper use of pci_map_sg Malte Vesper
2015-02-02 19:49 ` Malte Vesper [this message]
2015-02-03 18:13 ` Malte Vesper
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54CFD4E4.80607@postgrad.manchester.ac.uk \
--to=malte.vesper@postgrad.manchester.ac.uk \
--cc=kernelnewbies@lists.kernelnewbies.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.