[Spice-devel] Very poor video performance using Virt Viewer compared to RDP

Submitted by Brad Wilson on Oct. 13, 2016, 6 p.m.

Details

Message ID 1092928130.5247750.1476381610832@mail.yahoo.com
State New
Headers show
Series "Very poor video performance using Virt Viewer compared to RDP" ( rev: 2 ) in Spice

Not browsing as part of any series.

Commit Message

Brad Wilson Oct. 13, 2016, 6 p.m.
So one thing I notice is that there is a multimedia audio device that has a bad driver in the device manager... maybe that is screwing things up, forcing the video to wait for audio that will never come... I'll see if i can get that working properly and maybe it will help. 

      From: Frediano Ziglio <fziglio@redhat.com>
 To: Brad Wilson <bradwilsonstl@yahoo.com> 
Cc: spice-devel@lists.freedesktop.org
 Sent: Thursday, October 13, 2016 10:08 AM
 Subject: Re: [Spice-devel] Very poor video performance using Virt Viewer compared to RDP
   
> I have a Windows 7 VM running via Qemu on Gentoo Linux using a product called
> Foss-Cloud.
> I have spice tools and the QXL video driver installed
> Using Virt-Viewer The Vm shows a little lag when typing (always takes a
> second to catch up when typing my password to log in and such). In general
> pulling up folders and text heavy web pages is pretty snappy and doesn’t tax
> the cpu or memory at all. If look at a site that has a lot of pictures I
> start seeing frame loss and the mouse gets a little laggy when scrolling. If
> I pull up a video I see the cpu jump a little and the video is extremely
> laggy and I see about ever 10 th frame of the video, at the same time I lose
> sight of the mouse pointer for a second at a time and it becomes hard to
> control.
> While connecting to this vm using Virt-Viewer I also notice very low
> bandwidth usage, well below 500Kbps which is strange for a remote session
> sending video traffic.
> Using Windows RDP I was able to pull up all the same videos and web pages
> with no problem, the videos were clear and ran smooth and the websites with
> a lot of pictures scrolled normally. I did notice that the network traffic
> using RDP was closer to 10mbps and spiked to 14mbps so it’s understandable
> that I would get a better experience.

I'm not really familiar with the client code and I don't know if this
is really related to your problem but it share the fact of the lag and
the video.

While I was trying to code Virgl remote support I noted that in some
conditions I had EXTREME (like 1/2 seconds) lags. This was worsened
with low bandwidth. I started putting some debug code to understand
if server was queueing too data or was just too slow to handle the
encoding. And if was fast enough to handle all encoded frames and
the code in the streaming to deal with low bandwidth was apparently
doing it's job and the network queue (you can see with netstat) was
low. I then started putting timestamps in the frames trying to
understand where the lag was coming from and how much time take from
the encoding to the final client screen (you can see a video at
https://www.youtube.com/watch?v=D_DCs2sriu0 to understand the
time stamping, the green time in the video was written in the frames
before encoding in the server, not by the client).

I discovered where the main lag came quite by "accident". To test
I used one small utility and this utility don't have a way to change
parameters (like bandwidth) so knowing how the tcp work I stopped
the utility and started again as fast as I can with different
parameters. This of course create packets loss and some delay but
the connection works fine as tcp retransmit the packets lost.
But surprisingly the delay introduced by this accident was maintained
in the video! I then removed all Virgl stuff, tried again with a video
and the lag is still present. Also interrupting the utility and
starting again was increasing the lag (I managed to have about
3 seconds lag!).

Now... I have a patch for the client but as I said I'm not familiar
with the client too much and it's really terrible. It mainly remove
the synchronization with the audio (with still lags) and play the
video as fast as possible. By the way, I think is time to send as
someone could be able to make a sensible version of this patch
(perhaps fixing the sound too)


From 31364b1b1b1e5041ba64441f18104f7f1e978af5 Mon Sep 17 00:00:00 2001
From: Frediano Ziglio <fziglio@redhat.com>
Date: Thu, 13 Oct 2016 15:40:49 +0100
Subject: [spice-gtk] Try to reduce video lag

---
 src/channel-display-gst.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

Patch hide | download patch | download mbox

diff --git a/src/channel-display-gst.c b/src/channel-display-gst.c
index 430bb95..c1ebfeb 100644
--- a/src/channel-display-gst.c
+++ b/src/channel-display-gst.c
@@ -48,6 +48,7 @@  typedef struct SpiceGstDecoder {
    GQueue *decoding_queue;
    GQueue *display_queue;
    guint timer_id;
+    guint unique_id;
 } SpiceGstDecoder;
 
 
@@ -94,13 +95,15 @@  static gboolean display_frame(gpointer video_decoder)
    GstBuffer *buffer;
    GstMapInfo mapinfo;
 
-    decoder->timer_id = 0;
-
+ again:
    g_mutex_lock(&decoder->queues_mutex);
+    decoder->timer_id = 0;
    frame = g_queue_pop_head(decoder->display_queue);
    g_mutex_unlock(&decoder->queues_mutex);
    /* If the queue is empty we don't even need to reschedule */
-    g_return_val_if_fail(frame, G_SOURCE_REMOVE);
+    if (!frame) {
+        return G_SOURCE_REMOVE;
+    }
 
    if (!frame->sample) {
        spice_warning("got a frame without a sample!");
@@ -132,26 +135,28 @@  static gboolean display_frame(gpointer video_decoder)
 
  error:
    free_frame(frame);
-    schedule_frame(decoder);
-    return G_SOURCE_REMOVE;
+    goto again;
 }
 
 /* main loop or GStreamer streaming thread */
 static void schedule_frame(SpiceGstDecoder *decoder)
 {
-    guint32 now = stream_get_time(decoder->base.stream);
+//    guint32 now = stream_get_time(decoder->base.stream);
    g_mutex_lock(&decoder->queues_mutex);
 
+    if (decoder->timer_id)
+        printf("not scheduling, already a timer\n");
    while (!decoder->timer_id) {
        SpiceFrame *frame = g_queue_peek_head(decoder->display_queue);
        if (!frame) {
            break;
        }
 
-        SpiceStreamDataHeader *op = spice_msg_in_parsed(frame->msg);
+//        SpiceStreamDataHeader *op = spice_msg_in_parsed(frame->msg);
+        decoder->timer_id = g_timeout_add(0, display_frame, decoder);
+#if 0
        if (now < op->multi_media_time) {
-            decoder->timer_id = g_timeout_add(op->multi_media_time - now,
-                                              display_frame, decoder);
+            decoder->timer_id = g_timeout_add(0, display_frame, decoder);
        } else if (g_queue_get_length(decoder->display_queue) == 1) {
            /* Still attempt to display the least out of date frame so the
              * video is not completely frozen for an extended period of time.
@@ -165,6 +170,7 @@  static void schedule_frame(SpiceGstDecoder *decoder)
            g_queue_pop_head(decoder->display_queue);
            free_frame(frame);
        }
+#endif
    }
 
    g_mutex_unlock(&decoder->queues_mutex);
@@ -415,7 +421,7 @@  static void spice_gst_decoder_queue_frame(VideoDecoder *video_decoder,
 
    GST_BUFFER_DURATION(buffer) = GST_CLOCK_TIME_NONE;
    GST_BUFFER_DTS(buffer) = GST_CLOCK_TIME_NONE;
-    GST_BUFFER_PTS(buffer) = gst_clock_get_time(decoder->clock) - gst_element_get_base_time(decoder->pipeline) + ((uint64_t)MAX(0, latency)) * 1000 * 1000;
+    GST_BUFFER_PTS(buffer) = ++decoder->unique_id;
 
    g_mutex_lock(&decoder->queues_mutex);
    g_queue_push_tail(decoder->decoding_queue, create_frame(buffer, frame_msg));

Comments

Hi,

On Thu, Oct 13, 2016 at 06:00:10PM +0000, Brad Wilson wrote:
> So one thing I notice is that there is a multimedia audio device that
> has a bad driver in the device manager...

Can you elaborate in the issue you see/hear? Might be this one:
https://bugzilla.redhat.com/show_bug.cgi?id=1203507

> maybe that is screwing things up, forcing the video to wait for audio
> that will never come...  I'll see if i can get that working properly
> and maybe it will help. 

I think Frediano can have sync issues even on Linux Guest.

  toso

>
>       From: Frediano Ziglio <fziglio@redhat.com>
>  To: Brad Wilson <bradwilsonstl@yahoo.com> 
> Cc: spice-devel@lists.freedesktop.org
>  Sent: Thursday, October 13, 2016 10:08 AM
>  Subject: Re: [Spice-devel] Very poor video performance using Virt Viewer compared to RDP
>  
> > I have a Windows 7 VM running via Qemu on Gentoo Linux using a product called
> > Foss-Cloud.
> > I have spice tools and the QXL video driver installed
> > Using Virt-Viewer The Vm shows a little lag when typing (always takes a
> > second to catch up when typing my password to log in and such). In general
> > pulling up folders and text heavy web pages is pretty snappy and doesn’t tax
> > the cpu or memory at all. If look at a site that has a lot of pictures I
> > start seeing frame loss and the mouse gets a little laggy when scrolling. If
> > I pull up a video I see the cpu jump a little and the video is extremely
> > laggy and I see about ever 10 th frame of the video, at the same time I lose
> > sight of the mouse pointer for a second at a time and it becomes hard to
> > control.
> > While connecting to this vm using Virt-Viewer I also notice very low
> > bandwidth usage, well below 500Kbps which is strange for a remote session
> > sending video traffic.
> > Using Windows RDP I was able to pull up all the same videos and web pages
> > with no problem, the videos were clear and ran smooth and the websites with
> > a lot of pictures scrolled normally. I did notice that the network traffic
> > using RDP was closer to 10mbps and spiked to 14mbps so it’s understandable
> > that I would get a better experience.
> 
> I'm not really familiar with the client code and I don't know if this
> is really related to your problem but it share the fact of the lag and
> the video.
> 
> While I was trying to code Virgl remote support I noted that in some
> conditions I had EXTREME (like 1/2 seconds) lags. This was worsened
> with low bandwidth. I started putting some debug code to understand
> if server was queueing too data or was just too slow to handle the
> encoding. And if was fast enough to handle all encoded frames and
> the code in the streaming to deal with low bandwidth was apparently
> doing it's job and the network queue (you can see with netstat) was
> low. I then started putting timestamps in the frames trying to
> understand where the lag was coming from and how much time take from
> the encoding to the final client screen (you can see a video at
> https://www.youtube.com/watch?v=D_DCs2sriu0 to understand the
> time stamping, the green time in the video was written in the frames
> before encoding in the server, not by the client).
> 
> I discovered where the main lag came quite by "accident". To test
> I used one small utility and this utility don't have a way to change
> parameters (like bandwidth) so knowing how the tcp work I stopped
> the utility and started again as fast as I can with different
> parameters. This of course create packets loss and some delay but
> the connection works fine as tcp retransmit the packets lost.
> But surprisingly the delay introduced by this accident was maintained
> in the video! I then removed all Virgl stuff, tried again with a video
> and the lag is still present. Also interrupting the utility and
> starting again was increasing the lag (I managed to have about
> 3 seconds lag!).
> 
> Now... I have a patch for the client but as I said I'm not familiar
> with the client too much and it's really terrible. It mainly remove
> the synchronization with the audio (with still lags) and play the
> video as fast as possible. By the way, I think is time to send as
> someone could be able to make a sensible version of this patch
> (perhaps fixing the sound too)
> 
> 
> From 31364b1b1b1e5041ba64441f18104f7f1e978af5 Mon Sep 17 00:00:00 2001
> From: Frediano Ziglio <fziglio@redhat.com>
> Date: Thu, 13 Oct 2016 15:40:49 +0100
> Subject: [spice-gtk] Try to reduce video lag
> 
> ---
>  src/channel-display-gst.c | 26 ++++++++++++++++----------
>  1 file changed, 16 insertions(+), 10 deletions(-)
> 
> diff --git a/src/channel-display-gst.c b/src/channel-display-gst.c
> index 430bb95..c1ebfeb 100644
> --- a/src/channel-display-gst.c
> +++ b/src/channel-display-gst.c
> @@ -48,6 +48,7 @@ typedef struct SpiceGstDecoder {
>     GQueue *decoding_queue;
>     GQueue *display_queue;
>     guint timer_id;
> +    guint unique_id;
>  } SpiceGstDecoder;
>  
>  
> @@ -94,13 +95,15 @@ static gboolean display_frame(gpointer video_decoder)
>     GstBuffer *buffer;
>     GstMapInfo mapinfo;
>  
> -    decoder->timer_id = 0;
> -
> + again:
>     g_mutex_lock(&decoder->queues_mutex);
> +    decoder->timer_id = 0;
>     frame = g_queue_pop_head(decoder->display_queue);
>     g_mutex_unlock(&decoder->queues_mutex);
>     /* If the queue is empty we don't even need to reschedule */
> -    g_return_val_if_fail(frame, G_SOURCE_REMOVE);
> +    if (!frame) {
> +        return G_SOURCE_REMOVE;
> +    }
>  
>     if (!frame->sample) {
>         spice_warning("got a frame without a sample!");
> @@ -132,26 +135,28 @@ static gboolean display_frame(gpointer video_decoder)
>  
>   error:
>     free_frame(frame);
> -    schedule_frame(decoder);
> -    return G_SOURCE_REMOVE;
> +    goto again;
>  }
>  
>  /* main loop or GStreamer streaming thread */
>  static void schedule_frame(SpiceGstDecoder *decoder)
>  {
> -    guint32 now = stream_get_time(decoder->base.stream);
> +//    guint32 now = stream_get_time(decoder->base.stream);
>     g_mutex_lock(&decoder->queues_mutex);
>  
> +    if (decoder->timer_id)
> +        printf("not scheduling, already a timer\n");
>     while (!decoder->timer_id) {
>         SpiceFrame *frame = g_queue_peek_head(decoder->display_queue);
>         if (!frame) {
>             break;
>         }
>  
> -        SpiceStreamDataHeader *op = spice_msg_in_parsed(frame->msg);
> +//        SpiceStreamDataHeader *op = spice_msg_in_parsed(frame->msg);
> +        decoder->timer_id = g_timeout_add(0, display_frame, decoder);
> +#if 0
>         if (now < op->multi_media_time) {
> -            decoder->timer_id = g_timeout_add(op->multi_media_time - now,
> -                                              display_frame, decoder);
> +            decoder->timer_id = g_timeout_add(0, display_frame, decoder);
>         } else if (g_queue_get_length(decoder->display_queue) == 1) {
>             /* Still attempt to display the least out of date frame so the
>               * video is not completely frozen for an extended period of time.
> @@ -165,6 +170,7 @@ static void schedule_frame(SpiceGstDecoder *decoder)
>             g_queue_pop_head(decoder->display_queue);
>             free_frame(frame);
>         }
> +#endif
>     }
>  
>     g_mutex_unlock(&decoder->queues_mutex);
> @@ -415,7 +421,7 @@ static void spice_gst_decoder_queue_frame(VideoDecoder *video_decoder,
>  
>     GST_BUFFER_DURATION(buffer) = GST_CLOCK_TIME_NONE;
>     GST_BUFFER_DTS(buffer) = GST_CLOCK_TIME_NONE;
> -    GST_BUFFER_PTS(buffer) = gst_clock_get_time(decoder->clock) - gst_element_get_base_time(decoder->pipeline) + ((uint64_t)MAX(0, latency)) * 1000 * 1000;
> +    GST_BUFFER_PTS(buffer) = ++decoder->unique_id;
>  
>     g_mutex_lock(&decoder->queues_mutex);
>     g_queue_push_tail(decoder->decoding_queue, create_frame(buffer, frame_msg));
> -- 
> 2.7.4
> 
> 
> Frediano
> 
>    

> _______________________________________________
> Spice-devel mailing list
> Spice-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/spice-devel