[v2] drm/i915/kvmgt: Check the pfn got from vfio_pin_pages

Submitted by changbin.du@intel.com on March 30, 2018, 7:35 a.m.

Details

Message ID 1522395319-6414-1-git-send-email-changbin.du@intel.com
State New
Headers show
Series "drm/i915/kvmgt: Check the pfn got from vfio_pin_pages" ( rev: 2 ) in Intel GVT devel

Not browsing as part of any series.

Commit Message

changbin.du@intel.com March 30, 2018, 7:35 a.m.
From: Changbin Du <changbin.du@intel.com>

This can fix below oops. The target pfn must be mem backed.

[ 3639.109674] BUG: unable to handle kernel paging request at ffff8c44832a3000
[ 3639.109681] IP: memcpy_erms+0x6/0x10
[ 3639.109682] PGD 0 P4D 0
[ 3639.109685] Oops: 0000 1 SMP PTI
[ 3639.109726] CPU: 2 PID: 1724 Comm: qemu-system-x86 Not tainted 4.16.0-rc5+ #1
[ 3639.109727] Hardware name: /NUC7i7BNB, BIOS BNKBL357.86A.0050.2017.0816.2002 08/16/2017
[ 3639.109729] RIP: 0010:memcpy_erms+0x6/0x10
[ 3639.109730] RSP: 0018:ffffb1b7c3fbbbf0 EFLAGS: 00010246
[ 3639.109731] RAX: ffff8a44b6460000 RBX: 0000000036460000 RCX: 0000000000001000
[ 3639.109732] RDX: 0000000000001000 RSI: ffff8c44832a3000 RDI: ffff8a44b6460000
[ 3639.109733] RBP: 000000000006c8c0 R08: ffff8a44b6460000 R09: 0000000000000000
[ 3639.109734] R10: ffffb1b7c3fbbcd0 R11: ffff8a4d102018c0 R12: 0000000000000000
[ 3639.109734] R13: 0000000000000002 R14: 0000000000200000 R15: 0000000000000000
[ 3639.109736] FS: 00007f37f6d09700(0000) GS:ffff8a4d36d00000(0000) knlGS:0000000000000000
[ 3639.109737] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3639.109738] CR2: ffff8c44832a3000 CR3: 000000088b7b8004 CR4: 00000000003626e0
[ 3639.109739] Call Trace:
[ 3639.109743] swiotlb_tbl_map_single+0x2bb/0x300
[ 3639.109746] map_single+0x30/0x80
[ 3639.109748] swiotlb_map_page+0x87/0x150
[ 3639.109751] kvmgt_dma_map_guest_page+0x329/0x3a0 [kvmgt]
[ 3639.109764] ? kvm_write_guest_offset_cached+0x84/0xe0 [kvm]
[ 3639.109789] intel_vgpu_emulate_ggtt_mmio_write+0x1f4/0x250 [i915]
[ 3639.109808] intel_vgpu_emulate_mmio_write+0x162/0x230 [i915]
[ 3639.109811] intel_vgpu_rw+0x1fc/0x240 [kvmgt]
[ 3639.109813] intel_vgpu_write+0x164/0x1f0 [kvmgt]
[ 3639.109816] __vfs_write+0x33/0x170
[ 3639.109818] ? do_vfs_ioctl+0x9f/0x5f0
[ 3639.109820] vfs_write+0xb3/0x1a0
[ 3639.109822] SyS_pwrite64+0x90/0xb0
[ 3639.109825] do_syscall_64+0x68/0x120
[ 3639.109827] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[ 3639.109829] RIP: 0033:0x7f3802b2d873
[ 3639.109830] RSP: 002b:00007f37f6d08670 EFLAGS: 00000293 ORIG_RAX: 0000000000000012
[ 3639.109831] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3802b2d873
[ 3639.109832] RDX: 0000000000000008 RSI: 00007f37f6d086a0 RDI: 000000000000001a
[ 3639.109833] RBP: 00007f37f6d086c0 R08: 0000000000000008 R09: ffffffffffffffff
[ 3639.109834] R10: 00000000008041c8 R11: 0000000000000293 R12: 00007ffd8bbf92ae
[ 3639.109835] R13: 00007ffd8bbf92af R14: 00007f37f6d09700 R15: 00007f37f6d099c0

v2: add Fixes tag.

Signed-off-by: Changbin Du <changbin.du@intel.com>
Fixes: cf4ee73 ("drm/i915/gvt: Fix guest vGPU hang caused by very high dma setup overhead")
---
 drivers/gpu/drm/i915/gvt/kvmgt.c | 6 ++++++
 1 file changed, 6 insertions(+)

Patch hide | download patch | download mbox

diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 9d1c948..d80e817 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -123,6 +123,12 @@  static int gvt_dma_map_page(struct intel_vgpu *vgpu, unsigned long gfn,
 		return -EINVAL;
 	}
 
+	if (!pfn_valid(pfn)) {
+		gvt_vgpu_err("pfn 0x%lx is not mem backed\n", pfn);
+		vfio_unpin_pages(mdev_dev(vgpu->vdev.mdev), &gfn, 1);
+		return -EINVAL;
+	}
+
 	/* Setup DMA mapping. */
 	page = pfn_to_page(pfn);
 	*dma_addr = dma_map_page(dev, page, 0, PAGE_SIZE,

Comments

On 2018.03.30 15:35:19 +0800, changbin.du@intel.com wrote:
> From: Changbin Du <changbin.du@intel.com>
> 
> This can fix below oops. The target pfn must be mem backed.
> 
> [ 3639.109674] BUG: unable to handle kernel paging request at ffff8c44832a3000
> [ 3639.109681] IP: memcpy_erms+0x6/0x10
> [ 3639.109682] PGD 0 P4D 0
> [ 3639.109685] Oops: 0000 1 SMP PTI
> [ 3639.109726] CPU: 2 PID: 1724 Comm: qemu-system-x86 Not tainted 4.16.0-rc5+ #1
> [ 3639.109727] Hardware name: /NUC7i7BNB, BIOS BNKBL357.86A.0050.2017.0816.2002 08/16/2017
> [ 3639.109729] RIP: 0010:memcpy_erms+0x6/0x10
> [ 3639.109730] RSP: 0018:ffffb1b7c3fbbbf0 EFLAGS: 00010246
> [ 3639.109731] RAX: ffff8a44b6460000 RBX: 0000000036460000 RCX: 0000000000001000
> [ 3639.109732] RDX: 0000000000001000 RSI: ffff8c44832a3000 RDI: ffff8a44b6460000
> [ 3639.109733] RBP: 000000000006c8c0 R08: ffff8a44b6460000 R09: 0000000000000000
> [ 3639.109734] R10: ffffb1b7c3fbbcd0 R11: ffff8a4d102018c0 R12: 0000000000000000
> [ 3639.109734] R13: 0000000000000002 R14: 0000000000200000 R15: 0000000000000000
> [ 3639.109736] FS: 00007f37f6d09700(0000) GS:ffff8a4d36d00000(0000) knlGS:0000000000000000
> [ 3639.109737] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3639.109738] CR2: ffff8c44832a3000 CR3: 000000088b7b8004 CR4: 00000000003626e0
> [ 3639.109739] Call Trace:
> [ 3639.109743] swiotlb_tbl_map_single+0x2bb/0x300
> [ 3639.109746] map_single+0x30/0x80
> [ 3639.109748] swiotlb_map_page+0x87/0x150
> [ 3639.109751] kvmgt_dma_map_guest_page+0x329/0x3a0 [kvmgt]
> [ 3639.109764] ? kvm_write_guest_offset_cached+0x84/0xe0 [kvm]
> [ 3639.109789] intel_vgpu_emulate_ggtt_mmio_write+0x1f4/0x250 [i915]
> [ 3639.109808] intel_vgpu_emulate_mmio_write+0x162/0x230 [i915]
> [ 3639.109811] intel_vgpu_rw+0x1fc/0x240 [kvmgt]
> [ 3639.109813] intel_vgpu_write+0x164/0x1f0 [kvmgt]
> [ 3639.109816] __vfs_write+0x33/0x170
> [ 3639.109818] ? do_vfs_ioctl+0x9f/0x5f0
> [ 3639.109820] vfs_write+0xb3/0x1a0
> [ 3639.109822] SyS_pwrite64+0x90/0xb0
> [ 3639.109825] do_syscall_64+0x68/0x120
> [ 3639.109827] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
> [ 3639.109829] RIP: 0033:0x7f3802b2d873
> [ 3639.109830] RSP: 002b:00007f37f6d08670 EFLAGS: 00000293 ORIG_RAX: 0000000000000012
> [ 3639.109831] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3802b2d873
> [ 3639.109832] RDX: 0000000000000008 RSI: 00007f37f6d086a0 RDI: 000000000000001a
> [ 3639.109833] RBP: 00007f37f6d086c0 R08: 0000000000000008 R09: ffffffffffffffff
> [ 3639.109834] R10: 00000000008041c8 R11: 0000000000000293 R12: 00007ffd8bbf92ae
> [ 3639.109835] R13: 00007ffd8bbf92af R14: 00007f37f6d09700 R15: 00007f37f6d099c0
> 
> v2: add Fixes tag.
> 
> Signed-off-by: Changbin Du <changbin.du@intel.com>
> Fixes: cf4ee73 ("drm/i915/gvt: Fix guest vGPU hang caused by very high dma setup overhead")
> ---
>  drivers/gpu/drm/i915/gvt/kvmgt.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
> index 9d1c948..d80e817 100644
> --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> @@ -123,6 +123,12 @@ static int gvt_dma_map_page(struct intel_vgpu *vgpu, unsigned long gfn,
>  		return -EINVAL;
>  	}
>  
> +	if (!pfn_valid(pfn)) {
> +		gvt_vgpu_err("pfn 0x%lx is not mem backed\n", pfn);
> +		vfio_unpin_pages(mdev_dev(vgpu->vdev.mdev), &gfn, 1);
> +		return -EINVAL;
> +	}
> +
>  	/* Setup DMA mapping. */
>  	page = pfn_to_page(pfn);
>  	*dma_addr = dma_map_page(dev, page, 0, PAGE_SIZE,
> -- 

Applied this, thanks!

btw, we might still need to double check the reason that non mem backed pfn returned,
which should be avoided in case of gpu usage.
On Tue, Apr 17, 2018 at 10:44:34AM +0800, Zhenyu Wang wrote:
> On 2018.03.30 15:35:19 +0800, changbin.du@intel.com wrote:
> > From: Changbin Du <changbin.du@intel.com>
> > 
> > This can fix below oops. The target pfn must be mem backed.
> > 
> > [ 3639.109674] BUG: unable to handle kernel paging request at ffff8c44832a3000
> > [ 3639.109681] IP: memcpy_erms+0x6/0x10
> > [ 3639.109682] PGD 0 P4D 0
> > [ 3639.109685] Oops: 0000 1 SMP PTI
> > [ 3639.109726] CPU: 2 PID: 1724 Comm: qemu-system-x86 Not tainted 4.16.0-rc5+ #1
> > [ 3639.109727] Hardware name: /NUC7i7BNB, BIOS BNKBL357.86A.0050.2017.0816.2002 08/16/2017
> > [ 3639.109729] RIP: 0010:memcpy_erms+0x6/0x10
> > [ 3639.109730] RSP: 0018:ffffb1b7c3fbbbf0 EFLAGS: 00010246
> > [ 3639.109731] RAX: ffff8a44b6460000 RBX: 0000000036460000 RCX: 0000000000001000
> > [ 3639.109732] RDX: 0000000000001000 RSI: ffff8c44832a3000 RDI: ffff8a44b6460000
> > [ 3639.109733] RBP: 000000000006c8c0 R08: ffff8a44b6460000 R09: 0000000000000000
> > [ 3639.109734] R10: ffffb1b7c3fbbcd0 R11: ffff8a4d102018c0 R12: 0000000000000000
> > [ 3639.109734] R13: 0000000000000002 R14: 0000000000200000 R15: 0000000000000000
> > [ 3639.109736] FS: 00007f37f6d09700(0000) GS:ffff8a4d36d00000(0000) knlGS:0000000000000000
> > [ 3639.109737] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 3639.109738] CR2: ffff8c44832a3000 CR3: 000000088b7b8004 CR4: 00000000003626e0
> > [ 3639.109739] Call Trace:
> > [ 3639.109743] swiotlb_tbl_map_single+0x2bb/0x300
> > [ 3639.109746] map_single+0x30/0x80
> > [ 3639.109748] swiotlb_map_page+0x87/0x150
> > [ 3639.109751] kvmgt_dma_map_guest_page+0x329/0x3a0 [kvmgt]
> > [ 3639.109764] ? kvm_write_guest_offset_cached+0x84/0xe0 [kvm]
> > [ 3639.109789] intel_vgpu_emulate_ggtt_mmio_write+0x1f4/0x250 [i915]
> > [ 3639.109808] intel_vgpu_emulate_mmio_write+0x162/0x230 [i915]
> > [ 3639.109811] intel_vgpu_rw+0x1fc/0x240 [kvmgt]
> > [ 3639.109813] intel_vgpu_write+0x164/0x1f0 [kvmgt]
> > [ 3639.109816] __vfs_write+0x33/0x170
> > [ 3639.109818] ? do_vfs_ioctl+0x9f/0x5f0
> > [ 3639.109820] vfs_write+0xb3/0x1a0
> > [ 3639.109822] SyS_pwrite64+0x90/0xb0
> > [ 3639.109825] do_syscall_64+0x68/0x120
> > [ 3639.109827] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
> > [ 3639.109829] RIP: 0033:0x7f3802b2d873
> > [ 3639.109830] RSP: 002b:00007f37f6d08670 EFLAGS: 00000293 ORIG_RAX: 0000000000000012
> > [ 3639.109831] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3802b2d873
> > [ 3639.109832] RDX: 0000000000000008 RSI: 00007f37f6d086a0 RDI: 000000000000001a
> > [ 3639.109833] RBP: 00007f37f6d086c0 R08: 0000000000000008 R09: ffffffffffffffff
> > [ 3639.109834] R10: 00000000008041c8 R11: 0000000000000293 R12: 00007ffd8bbf92ae
> > [ 3639.109835] R13: 00007ffd8bbf92af R14: 00007f37f6d09700 R15: 00007f37f6d099c0
> > 
> > v2: add Fixes tag.
> > 
> > Signed-off-by: Changbin Du <changbin.du@intel.com>
> > Fixes: cf4ee73 ("drm/i915/gvt: Fix guest vGPU hang caused by very high dma setup overhead")
> > ---
> >  drivers/gpu/drm/i915/gvt/kvmgt.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
> > index 9d1c948..d80e817 100644
> > --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> > +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> > @@ -123,6 +123,12 @@ static int gvt_dma_map_page(struct intel_vgpu *vgpu, unsigned long gfn,
> >  		return -EINVAL;
> >  	}
> >  
> > +	if (!pfn_valid(pfn)) {
> > +		gvt_vgpu_err("pfn 0x%lx is not mem backed\n", pfn);
> > +		vfio_unpin_pages(mdev_dev(vgpu->vdev.mdev), &gfn, 1);
> > +		return -EINVAL;
> > +	}
> > +
> >  	/* Setup DMA mapping. */
> >  	page = pfn_to_page(pfn);
> >  	*dma_addr = dma_map_page(dev, page, 0, PAGE_SIZE,
> > -- 
> 
> Applied this, thanks!
> 
> btw, we might still need to double check the reason that non mem backed pfn returned,
> which should be avoided in case of gpu usage.
>
Not sure if guest drivers still ensure init-modify seqeunce for gtt entries. We
still can see invalid gfn erros with reported bugs. Will keep an eye on it.

> -- 
> Open Source Technology Center, Intel ltd.
> 
> $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827