drm/i915/gvt: force to active the high-performance mode during vGPU busy

Submitted by Weinan Li on April 18, 2018, 2:31 a.m.

Details

Message ID 1524018693-31241-1-git-send-email-weinan.z.li@intel.com
State New
Headers show
Series "drm/i915/gvt: force to active the high-performance mode during vGPU busy" ( rev: 1 ) in Intel GVT devel

Not browsing as part of any series.

Commit Message

Weinan Li April 18, 2018, 2:31 a.m.
With the RPS interrupt, KMD can adjust the GPU frequency dynamically for
power saving. It works well in the non-virtualized environment, but there
is more latency imported by VMM and virtual interrupt handler which may
break the RPS policy work model, and GPU works in inefficient mode. Here
we force to active the high-performance mode when detect vgpu is busy until
the GPU runs into idle.

Signed-off-by: Weinan Li <weinan.z.li@intel.com>
---
 drivers/gpu/drm/i915/gvt/scheduler.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

Patch hide | download patch | download mbox

diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c
index d71e3de..7ba8703 100644
--- a/drivers/gpu/drm/i915/gvt/scheduler.c
+++ b/drivers/gpu/drm/i915/gvt/scheduler.c
@@ -222,6 +222,17 @@  static void save_ring_hw_state(struct intel_vgpu *vgpu, int ring_id)
 	vgpu_vreg(vgpu, i915_mmio_reg_offset(reg)) = I915_READ_FW(reg);
 }
 
+static int active_high_performance_mode(struct drm_i915_private *dev_priv)
+{
+	struct intel_rps rps = dev_priv->gt_pm.rps;
+
+	if (rps.cur_freq != rps.max_freq || rps.interrupts_enabled) {
+		intel_set_rps(dev_priv, rps.max_freq);
+		gen6_disable_rps_interrupts(dev_priv);
+		trace_printk("active high performance mode for GVT-g.\n");
+	}
+}
+
 static int shadow_context_status_change(struct notifier_block *nb,
 		unsigned long action, void *data)
 {
@@ -263,6 +274,7 @@  static int shadow_context_status_change(struct notifier_block *nb,
 			gvt_dbg_sched("skip ring %d mmio switch for vgpu%d\n",
 				      ring_id, workload->vgpu->id);
 		spin_unlock_irqrestore(&scheduler->mmio_context_lock, flags);
+		active_high_performance_mode(gvt->dev_priv);
 		atomic_set(&workload->shadow_ctx_active, 1);
 		break;
 	case INTEL_CONTEXT_SCHEDULE_OUT:

Comments

On Wed, Apr 18, 2018 at 10:31:33AM +0800, Weinan Li wrote:
> With the RPS interrupt, KMD can adjust the GPU frequency dynamically for
> power saving. It works well in the non-virtualized environment, but there
> is more latency imported by VMM and virtual interrupt handler which may
> break the RPS policy work model, and GPU works in inefficient mode. Here
> we force to active the high-performance mode when detect vgpu is busy until
> the GPU runs into idle.
> 
> Signed-off-by: Weinan Li <weinan.z.li@intel.com>
> ---
>  drivers/gpu/drm/i915/gvt/scheduler.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c
> index d71e3de..7ba8703 100644
> --- a/drivers/gpu/drm/i915/gvt/scheduler.c
> +++ b/drivers/gpu/drm/i915/gvt/scheduler.c
> @@ -222,6 +222,17 @@ static void save_ring_hw_state(struct intel_vgpu *vgpu, int ring_id)
>  	vgpu_vreg(vgpu, i915_mmio_reg_offset(reg)) = I915_READ_FW(reg);
>  }
>  
> +static int active_high_performance_mode(struct drm_i915_private *dev_priv)
> +{
> +	struct intel_rps rps = dev_priv->gt_pm.rps;
> +
> +	if (rps.cur_freq != rps.max_freq || rps.interrupts_enabled) {
> +		intel_set_rps(dev_priv, rps.max_freq);
> +		gen6_disable_rps_interrupts(dev_priv);
> +		trace_printk("active high performance mode for GVT-g.\n");
trace_printk() should not be used. It will cause non-production warning.

> +	}
> +}
> +
>  static int shadow_context_status_change(struct notifier_block *nb,
>  		unsigned long action, void *data)
>  {
> @@ -263,6 +274,7 @@ static int shadow_context_status_change(struct notifier_block *nb,
>  			gvt_dbg_sched("skip ring %d mmio switch for vgpu%d\n",
>  				      ring_id, workload->vgpu->id);
>  		spin_unlock_irqrestore(&scheduler->mmio_context_lock, flags);
> +		active_high_performance_mode(gvt->dev_priv);
>  		atomic_set(&workload->shadow_ctx_active, 1);
>  		break;
>  	case INTEL_CONTEXT_SCHEDULE_OUT:
> -- 
> 1.9.1
> 
> _______________________________________________
> intel-gvt-dev mailing list
> intel-gvt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
On 2018.04.18 10:31:33 +0800, Weinan Li wrote:
> With the RPS interrupt, KMD can adjust the GPU frequency dynamically for
> power saving. It works well in the non-virtualized environment, but there
> is more latency imported by VMM and virtual interrupt handler which may
> break the RPS policy work model, and GPU works in inefficient mode. Here
> we force to active the high-performance mode when detect vgpu is busy until
> the GPU runs into idle.
>
> Signed-off-by: Weinan Li <weinan.z.li@intel.com>
> ---
>  drivers/gpu/drm/i915/gvt/scheduler.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c b/drivers/gpu/drm/i915/gvt/scheduler.c
> index d71e3de..7ba8703 100644
> --- a/drivers/gpu/drm/i915/gvt/scheduler.c
> +++ b/drivers/gpu/drm/i915/gvt/scheduler.c
> @@ -222,6 +222,17 @@ static void save_ring_hw_state(struct intel_vgpu *vgpu, int ring_id)
>  	vgpu_vreg(vgpu, i915_mmio_reg_offset(reg)) = I915_READ_FW(reg);
>  }
>  
> +static int active_high_performance_mode(struct drm_i915_private *dev_priv)
> +{
> +	struct intel_rps rps = dev_priv->gt_pm.rps;
> +
> +	if (rps.cur_freq != rps.max_freq || rps.interrupts_enabled) {
> +		intel_set_rps(dev_priv, rps.max_freq);
> +		gen6_disable_rps_interrupts(dev_priv);
> +		trace_printk("active high performance mode for GVT-g.\n");
> +	}
> +}

I'm not sure what's current rps policy model? rps should work across all
workload to gpu which determine from render engine power unit, you force
to boost freq for gvt workload would bring unfair for host? And you don't
hold proper lock to set rps. But I still like to understand what's the
actual reason to make hw believe that render freq could be lower, caused
by i915 idle handler or just by hw?


> +
>  static int shadow_context_status_change(struct notifier_block *nb,
>  		unsigned long action, void *data)
>  {
> @@ -263,6 +274,7 @@ static int shadow_context_status_change(struct notifier_block *nb,
>  			gvt_dbg_sched("skip ring %d mmio switch for vgpu%d\n",
>  				      ring_id, workload->vgpu->id);
>  		spin_unlock_irqrestore(&scheduler->mmio_context_lock, flags);
> +		active_high_performance_mode(gvt->dev_priv);
>  		atomic_set(&workload->shadow_ctx_active, 1);
>  		break;
>  	case INTEL_CONTEXT_SCHEDULE_OUT:
> -- 
> 1.9.1
> 
> _______________________________________________
> intel-gvt-dev mailing list
> intel-gvt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
> -----Original Message-----
> From: Zhenyu Wang [mailto:zhenyuw@linux.intel.com]
> Sent: Wednesday, April 18, 2018 12:30 PM
> To: Li, Weinan Z <weinan.z.li@intel.com>
> Cc: intel-gvt-dev@lists.freedesktop.org
> Subject: Re: [PATCH] drm/i915/gvt: force to active the high-performance
> mode during vGPU busy
> 
> On 2018.04.18 10:31:33 +0800, Weinan Li wrote:
> > With the RPS interrupt, KMD can adjust the GPU frequency dynamically
> > for power saving. It works well in the non-virtualized environment,
> > but there is more latency imported by VMM and virtual interrupt
> > handler which may break the RPS policy work model, and GPU works in
> > inefficient mode. Here we force to active the high-performance mode
> > when detect vgpu is busy until the GPU runs into idle.
> >
> > Signed-off-by: Weinan Li <weinan.z.li@intel.com>
> > ---
> >  drivers/gpu/drm/i915/gvt/scheduler.c | 12 ++++++++++++
> >  1 file changed, 12 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c
> > b/drivers/gpu/drm/i915/gvt/scheduler.c
> > index d71e3de..7ba8703 100644
> > --- a/drivers/gpu/drm/i915/gvt/scheduler.c
> > +++ b/drivers/gpu/drm/i915/gvt/scheduler.c
> > @@ -222,6 +222,17 @@ static void save_ring_hw_state(struct intel_vgpu
> *vgpu, int ring_id)
> >  	vgpu_vreg(vgpu, i915_mmio_reg_offset(reg)) =
> I915_READ_FW(reg);  }
> >
> > +static int active_high_performance_mode(struct drm_i915_private
> > +*dev_priv) {
> > +	struct intel_rps rps = dev_priv->gt_pm.rps;
> > +
> > +	if (rps.cur_freq != rps.max_freq || rps.interrupts_enabled) {
> > +		intel_set_rps(dev_priv, rps.max_freq);
> > +		gen6_disable_rps_interrupts(dev_priv);
> > +		trace_printk("active high performance mode for GVT-g.\n");
> > +	}
> > +}
> 
> I'm not sure what's current rps policy model? rps should work across all
> workload to gpu which determine from render engine power unit, you force
> to boost freq for gvt workload would bring unfair for host? And you don't
> hold proper lock to set rps. But I still like to understand what's the actual
> reason to make hw believe that render freq could be lower, caused by i915
> idle handler or just by hw?
> 
Hi Zhenyu, If the workloads come from host, then host can work well with the rps policy, but if the workloads come from vGPU, then GVT-g trap the workloads and submit it to execution queue, this process should import latency, then hardware execute this workload, GVT-g gets completed notification, then inject virtual interrupts into vGPU, vGPU may submit the next workload after the virtual interrupt handle is finished. This process also imports latency. Since the latency imported by GVT-g, in some scenarios, it causes GPU hardware believe the workload is so light that lower the freq. 
> 
> > +
> >  static int shadow_context_status_change(struct notifier_block *nb,
> >  		unsigned long action, void *data)
> >  {
> > @@ -263,6 +274,7 @@ static int shadow_context_status_change(struct
> notifier_block *nb,
> >  			gvt_dbg_sched("skip ring %d mmio switch for
> vgpu%d\n",
> >  				      ring_id, workload->vgpu->id);
> >  		spin_unlock_irqrestore(&scheduler->mmio_context_lock,
> flags);
> > +		active_high_performance_mode(gvt->dev_priv);
> >  		atomic_set(&workload->shadow_ctx_active, 1);
> >  		break;
> >  	case INTEL_CONTEXT_SCHEDULE_OUT:
> > --
> > 1.9.1
> >
> > _______________________________________________
> > intel-gvt-dev mailing list
> > intel-gvt-dev@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev
> 
> --
> Open Source Technology Center, Intel ltd.
> 
> $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827
On 2018.05.03 06:43:04 +0000, Li, Weinan Z wrote:
> > On 2018.04.18 10:31:33 +0800, Weinan Li wrote:
> > > With the RPS interrupt, KMD can adjust the GPU frequency dynamically
> > > for power saving. It works well in the non-virtualized environment,
> > > but there is more latency imported by VMM and virtual interrupt
> > > handler which may break the RPS policy work model, and GPU works in
> > > inefficient mode. Here we force to active the high-performance mode
> > > when detect vgpu is busy until the GPU runs into idle.
> > >
> > > Signed-off-by: Weinan Li <weinan.z.li@intel.com>
> > > ---
> > >  drivers/gpu/drm/i915/gvt/scheduler.c | 12 ++++++++++++
> > >  1 file changed, 12 insertions(+)
> > >
> > > diff --git a/drivers/gpu/drm/i915/gvt/scheduler.c
> > > b/drivers/gpu/drm/i915/gvt/scheduler.c
> > > index d71e3de..7ba8703 100644
> > > --- a/drivers/gpu/drm/i915/gvt/scheduler.c
> > > +++ b/drivers/gpu/drm/i915/gvt/scheduler.c
> > > @@ -222,6 +222,17 @@ static void save_ring_hw_state(struct intel_vgpu
> > *vgpu, int ring_id)
> > >  	vgpu_vreg(vgpu, i915_mmio_reg_offset(reg)) =
> > I915_READ_FW(reg);  }
> > >
> > > +static int active_high_performance_mode(struct drm_i915_private
> > > +*dev_priv) {
> > > +	struct intel_rps rps = dev_priv->gt_pm.rps;
> > > +
> > > +	if (rps.cur_freq != rps.max_freq || rps.interrupts_enabled) {
> > > +		intel_set_rps(dev_priv, rps.max_freq);
> > > +		gen6_disable_rps_interrupts(dev_priv);
> > > +		trace_printk("active high performance mode for GVT-g.\n");
> > > +	}
> > > +}
> > 
> > I'm not sure what's current rps policy model? rps should work across all
> > workload to gpu which determine from render engine power unit, you force
> > to boost freq for gvt workload would bring unfair for host? And you don't
> > hold proper lock to set rps. But I still like to understand what's the actual
> > reason to make hw believe that render freq could be lower, caused by i915
> > idle handler or just by hw?
> > 
> Hi Zhenyu, If the workloads come from host, then host can work well with the rps policy, but if the workloads come from vGPU, then GVT-g trap the workloads and submit it to execution queue, this process should import latency, then hardware execute this workload, GVT-g gets completed notification, then inject virtual interrupts into vGPU, vGPU may submit the next workload after the virtual interrupt handle is finished. This process also imports latency. Since the latency imported by GVT-g, in some scenarios, it causes GPU hardware believe the workload is so light that lower the freq. 
> > 

Does i915 idle handler kick in to handle that? which try to lower the
frequence? We can't stick to max freq for gvt-g instance as it
doesn't apply to all cases. Maybe we could tell i915 there're more
requests to submit so don't enter lower freq?