kdrive/ephyr: Fix crash on 24bpp host framebuffer

Submitted by Adam Jackson on July 12, 2010, 9:07 p.m.

Details

Message ID 1278943632-25197-1-git-send-email-ajax@redhat.com
State Superseded, archived
Headers show

Not browsing as part of any series.

Commit Message

Adam Jackson July 12, 2010, 9:07 p.m.
bytes_per_line in the XImage will (more or less) reflect the pixmap
format of the server.  On 24bpp servers it will be approximately 3*width
(plus scanline padding); we were assuming depth 24 always meant 32bpp,
so the exposure painting in root window map would walk off the end of
the image and crash.

Signed-off-by: Adam Jackson <ajax@redhat.com>
---
 hw/kdrive/ephyr/ephyr.c |    9 ++++++---
 hw/kdrive/ephyr/hostx.c |    4 +++-
 hw/kdrive/ephyr/hostx.h |    2 +-
 3 files changed, 10 insertions(+), 5 deletions(-)

Patch hide | download patch | download mbox

diff --git a/hw/kdrive/ephyr/ephyr.c b/hw/kdrive/ephyr/ephyr.c
index 14ab591..a6057db 100644
--- a/hw/kdrive/ephyr/ephyr.c
+++ b/hw/kdrive/ephyr/ephyr.c
@@ -234,8 +234,6 @@  ephyrMapFramebuffer (KdScreenInfo *screen)
   KdComputePointerMatrix (&m, scrpriv->randr, screen->width, screen->height);
   KdSetPointerMatrix (&m);
   
-  priv->bytes_per_line = ((screen->width * screen->fb[0].bitsPerPixel + 31) >> 5) << 2;
-  
   /* point the framebuffer to the data in an XImage */
   /* If fakexa is enabled, allocate a larger buffer so that fakexa has space to
    * put offscreen pixmaps.
@@ -245,7 +243,12 @@  ephyrMapFramebuffer (KdScreenInfo *screen)
   else
     buffer_height = 3 * screen->height;
 
-  priv->base = hostx_screen_init (screen, screen->width, screen->height, buffer_height);
+  priv->base = hostx_screen_init (screen, screen->width, screen->height, buffer_height, &priv->bytes_per_line);
+
+  /* laaaaaame */
+  if (screen->fb[0].depth == 24)
+    if (priv->bytes_per_line < (screen->width * 4))
+      screen->fb[0].bitsPerPixel = 24;
 
   screen->memory_base  = (CARD8 *) (priv->base);
   screen->memory_size  = priv->bytes_per_line * buffer_height;
diff --git a/hw/kdrive/ephyr/hostx.c b/hw/kdrive/ephyr/hostx.c
index cdb019d..874c754 100644
--- a/hw/kdrive/ephyr/hostx.c
+++ b/hw/kdrive/ephyr/hostx.c
@@ -639,7 +639,7 @@  hostx_set_cmap_entry(unsigned char idx,
 void*
 hostx_screen_init (EphyrScreenInfo screen,
                    int width, int height,
-                   int buffer_height)
+                   int buffer_height, int *bytes_per_line)
 {
   int         bitmap_pad;
   Bool        shm_success = False;
@@ -727,6 +727,8 @@  hostx_screen_init (EphyrScreenInfo screen,
               malloc (host_screen->ximg->bytes_per_line * buffer_height);
     }
 
+  *bytes_per_line = host_screen->ximg->bytes_per_line;
+
   XResizeWindow (HostX.dpy, host_screen->win, width, height);
 
   XMapWindow(HostX.dpy, host_screen->win);
diff --git a/hw/kdrive/ephyr/hostx.h b/hw/kdrive/ephyr/hostx.h
index 69e3ceb..b0627da 100644
--- a/hw/kdrive/ephyr/hostx.h
+++ b/hw/kdrive/ephyr/hostx.h
@@ -203,7 +203,7 @@  hostx_set_cmap_entry(unsigned char idx,
 void*
 hostx_screen_init (EphyrScreenInfo screen,
                    int width, int height,
-                   int buffer_height);
+                   int buffer_height, int *bytes_per_line);
 
 void
 hostx_paint_rect(EphyrScreenInfo screen,

Comments

On Mon, 12 Jul 2010 10:07:12 -0400, Adam Jackson <ajax@redhat.com> wrote:

> +  /* laaaaaame */
> +  if (screen->fb[0].depth == 24)
> +    if (priv->bytes_per_line < (screen->width * 4))
> +      screen->fb[0].bitsPerPixel = 24;

Please get this from the underlying XImage somehow...
Was there ever an updated version of this patch?  This one seems to have bitrot.

On Jul 12, 2010, at 09:15, Keith Packard wrote:

> On Mon, 12 Jul 2010 10:07:12 -0400, Adam Jackson <ajax@redhat.com> wrote:
> 
>> +  /* laaaaaame */
>> +  if (screen->fb[0].depth == 24)
>> +    if (priv->bytes_per_line < (screen->width * 4))
>> +      screen->fb[0].bitsPerPixel = 24;
> 
> Please get this from the underlying XImage somehow...
> 
> -- 
> keith.packard@intel.com
> _______________________________________________
> xorg-devel@lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: http://lists.x.org/mailman/listinfo/xorg-devel