[PATCH_v2] virtual-keyboard: Add new virtual keyboard protocol

Submitted by Dorota Czaplejewicz on May 24, 2018, 6:27 p.m.

Details

Message ID 20180524182729.18790-1-dorota.czaplejewicz@puri.sm
State New
Series "virtual-keyboard: Add new virtual keyboard protocol"
Headers show

Commit Message

Dorota Czaplejewicz May 24, 2018, 6:27 p.m.
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.

The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.

Changes done:
- fixed typos
- enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
- added errors

The test client is at https://code.puri.sm/Librem5/weston/src/virtual_keyboard - please use the weston-keyboard program:

./configure --enable-clients
./weston-keyboard

A working wlroots implementation is available here:

https://github.com/swaywm/wlroots/pull/999

Thanks for feedback,
Dorota Czaplejewicz


 Makefile.am                                        |   1 +
 unstable/virtual-keyboard/README                   |   2 +
 .../virtual-keyboard-unstable-v1.xml               | 113 +++++++++++++++++++++
 3 files changed, 116 insertions(+)
 create mode 100644 unstable/virtual-keyboard/README
 create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml

Patch hide | download patch | download mbox

diff --git a/Makefile.am b/Makefile.am
index 4b9a901..fcd4572 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@  unstable_protocols =								\
 	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
 	unstable/xdg-output/xdg-output-unstable-v1.xml				\
 	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
+	unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
 	$(NULL)
 
 stable_protocols =								\
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@ 
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..df4d01c
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,113 @@ 
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+  <copyright>
+    Copyright © 2008-2011  Kristian Høgsberg
+    Copyright © 2010-2013  Intel Corporation
+    Copyright © 2012-2013  Collabora, Ltd.
+    Copyright © 2018       Purism SPC
+
+    Permission is hereby granted, free of charge, to any person obtaining a
+    copy of this software and associated documentation files (the "Software"),
+    to deal in the Software without restriction, including without limitation
+    the rights to use, copy, modify, merge, publish, distribute, sublicense,
+    and/or sell copies of the Software, and to permit persons to whom the
+    Software is furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice (including the next
+    paragraph) shall be included in all copies or substantial portions of the
+    Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+    DEALINGS IN THE SOFTWARE.
+  </copyright>
+
+  <interface name="zwp_virtual_keyboard_v1" version="1">
+    <description summary="virtual keyboard">
+      The virtual keyboard provides an application with requests which emulate
+      the behaviour of a physical keyboard.
+
+      This interface can be used by clients on its own to provide raw input
+      events, or it can accompany the input method protocol.
+    </description>
+
+    <request name="keymap">
+      <description summary="keyboard mapping">
+        Provide a file descriptor to the compositor which can be
+        memory-mapped to provide a keyboard mapping description.
+
+        Format carries a value from the keymap_format enumeration.
+      </description>
+      <arg name="format" type="uint" summary="keymap format"/>
+      <arg name="fd" type="fd" summary="keymap file descriptor"/>
+      <arg name="size" type="uint" summary="keymap size, in bytes"/>
+    </request>
+    
+    <enum name="error">
+      <entry name="no_keymap" value="0" summary="No keymap was set"/>
+    </enum>
+
+    <request name="key">
+      <description summary="key event">
+        A key was pressed or released.
+        The time argument is a timestamp with millisecond granularity, with an
+        undefined base. All requests regarding a single object must share the
+        same clock.
+
+        Keymap must be set before issuing this request.
+
+        State carries a value from the key_state enumeration.
+      </description>
+      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+      <arg name="key" type="uint" summary="key that produced the event"/>
+      <arg name="state" type="uint" summary="physical state of the key"/>
+    </request>
+
+    <request name="modifiers">
+      <description summary="modifier and group state">
+        Notifies the compositor that the modifier and/or group state has
+        changed, and it should update state.
+
+        The client should use wl_keyboard.modifiers event to synchronize its
+        internal state with seat state.
+
+        Keymap must be set before issuing this request.
+      </description>
+      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+      <arg name="group" type="uint" summary="keyboard layout"/>
+    </request>
+
+    <request name="destroy" type="destructor" since="1">
+      <description summary="destroy the virtual keyboard keyboard object"/>
+    </request>
+  </interface>
+
+  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+    <description summary="virtual keyboard manager">
+      A virtual keyboard manager allows an application to provide keyboard
+      input events as if they came from a physical keyboard.
+    </description>
+
+    <enum name="error">
+      <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
+    </enum>
+
+    <request name="create_virtual_keyboard">
+      <description summary="Create a new virtual keyboard">
+        Creates a new virtual keyboard associated to a seat.
+
+        If the compositor enables a keyboard to perform arbitrary actions, it
+        should present an error when an untrusted client requests a new
+        keyboard.
+      </description>
+      <arg name="seat" type="object" interface="wl_seat"/>
+      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+    </request>
+  </interface>
+</protocol>

Comments

Simon Ser May 29, 2018, 5:42 p.m.
On May 24, 2018 7:27 PM, Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> wrote:
> Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
>
> The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
> ---
> This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.
>
> Changes done:
> - fixed typos
> - enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
> - added errors
>
> The test client is at https://code.puri.sm/Librem5/weston/src/virtual_keyboard - please use the weston-keyboard program:
>
> ./configure --enable-clients
> ./weston-keyboard
>
> A working wlroots implementation is available here:
>
> https://github.com/swaywm/wlroots/pull/999
>
> Thanks for feedback,
> Dorota Czaplejewicz
>
>
>  Makefile.am                                        |   1 +
>  unstable/virtual-keyboard/README                   |   2 +
>  .../virtual-keyboard-unstable-v1.xml               | 113 +++++++++++++++++++++
>  3 files changed, 116 insertions(+)
>  create mode 100644 unstable/virtual-keyboard/README
>  create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
>
> diff --git a/Makefile.am b/Makefile.am
> index 4b9a901..fcd4572 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -17,6 +17,7 @@ unstable_protocols =								\
>  	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
>  	unstable/xdg-output/xdg-output-unstable-v1.xml				\
>  	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
> +	unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
>  	$(NULL)
>
>  stable_protocols =								\
> diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
> new file mode 100644
> index 0000000..a2c646d
> --- /dev/null
> +++ b/unstable/virtual-keyboard/README
> @@ -0,0 +1,2 @@
> +Virtual keyboard protocol
> +
> diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> new file mode 100644
> index 0000000..df4d01c
> --- /dev/null
> +++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> @@ -0,0 +1,113 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<protocol name="virtual_keyboard_unstable_v1">
> +  <copyright>
> +    Copyright © 2008-2011  Kristian Høgsberg
> +    Copyright © 2010-2013  Intel Corporation
> +    Copyright © 2012-2013  Collabora, Ltd.
> +    Copyright © 2018       Purism SPC
> +
> +    Permission is hereby granted, free of charge, to any person obtaining a
> +    copy of this software and associated documentation files (the "Software"),
> +    to deal in the Software without restriction, including without limitation
> +    the rights to use, copy, modify, merge, publish, distribute, sublicense,
> +    and/or sell copies of the Software, and to permit persons to whom the
> +    Software is furnished to do so, subject to the following conditions:
> +
> +    The above copyright notice and this permission notice (including the next
> +    paragraph) shall be included in all copies or substantial portions of the
> +    Software.
> +
> +    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> +    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> +    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> +    DEALINGS IN THE SOFTWARE.
> +  </copyright>
> +
> +  <interface name="zwp_virtual_keyboard_v1" version="1">
> +    <description summary="virtual keyboard">
> +      The virtual keyboard provides an application with requests which emulate
> +      the behaviour of a physical keyboard.
> +
> +      This interface can be used by clients on its own to provide raw input
> +      events, or it can accompany the input method protocol.
> +    </description>
> +
> +    <request name="keymap">
> +      <description summary="keyboard mapping">
> +        Provide a file descriptor to the compositor which can be
> +        memory-mapped to provide a keyboard mapping description.
> +
> +        Format carries a value from the keymap_format enumeration.

Is it possible to specify the interface name too, to make it easier to find the
enum? ("wl_keyboard.keymap_format")

Anyway, it doesn't matter a lot, because a fix for references to foreign enums
will be merged soon (hopefully).

> +      </description>
> +      <arg name="format" type="uint" summary="keymap format"/>
> +      <arg name="fd" type="fd" summary="keymap file descriptor"/>
> +      <arg name="size" type="uint" summary="keymap size, in bytes"/>
> +    </request>
> +
> +    <enum name="error">
> +      <entry name="no_keymap" value="0" summary="No keymap was set"/>
> +    </enum>
> +
> +    <request name="key">
> +      <description summary="key event">
> +        A key was pressed or released.
> +        The time argument is a timestamp with millisecond granularity, with an
> +        undefined base. All requests regarding a single object must share the
> +        same clock.
> +
> +        Keymap must be set before issuing this request.
> +
> +        State carries a value from the key_state enumeration.

(Ditto)

> +      </description>
> +      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
> +      <arg name="key" type="uint" summary="key that produced the event"/>
> +      <arg name="state" type="uint" summary="physical state of the key"/>
> +    </request>
> +
> +    <request name="modifiers">
> +      <description summary="modifier and group state">
> +        Notifies the compositor that the modifier and/or group state has
> +        changed, and it should update state.
> +
> +        The client should use wl_keyboard.modifiers event to synchronize its
> +        internal state with seat state.
> +
> +        Keymap must be set before issuing this request.
> +      </description>
> +      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
> +      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
> +      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
> +      <arg name="group" type="uint" summary="keyboard layout"/>
> +    </request>
> +
> +    <request name="destroy" type="destructor" since="1">

No need for `since="1"`, this is implicit.

> +      <description summary="destroy the virtual keyboard keyboard object"/>
> +    </request>
> +  </interface>
> +
> +  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
> +    <description summary="virtual keyboard manager">
> +      A virtual keyboard manager allows an application to provide keyboard
> +      input events as if they came from a physical keyboard.
> +    </description>
> +
> +    <enum name="error">
> +      <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
> +    </enum>
> +
> +    <request name="create_virtual_keyboard">
> +      <description summary="Create a new virtual keyboard">
> +        Creates a new virtual keyboard associated to a seat.
> +
> +        If the compositor enables a keyboard to perform arbitrary actions, it
> +        should present an error when an untrusted client requests a new
> +        keyboard.

Which error?

Maybe this could just be reworded as something among the lines of "it should
only allow trusted clients to use this request", implying it can either
disconnect clients or (and this is probably preferred) don't expose the global
by default.

> +      </description>
> +      <arg name="seat" type="object" interface="wl_seat"/>
> +      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
> +    </request>

Is it possible to add a request to destroy the manager? Quoting pq from the
latest DRM lease protocol review:

>This interface is missing a destroy request. Interfaces must always have a
>destroy request unless there is a very good reason to not have one. In any
>case, every object must be destroyable somehow.

> +  </interface>
> +</protocol>
> --
> 2.14.3
> _______________________________________________
> wayland-devel mailing list
> wayland-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/wayland-devel

Otherwise, this protocol looks pretty good to me.

---
Simon Ser
https://emersion.fr
Peter Hutterer May 30, 2018, 4:28 a.m.
On Thu, May 24, 2018 at 08:27:29PM +0200, Dorota Czaplejewicz wrote:
> Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
> 
> The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
> ---
> This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.
> 
> Changes done:
> - fixed typos
> - enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
> - added errors
> 
> The test client is at https://code.puri.sm/Librem5/weston/src/virtual_keyboard - please use the weston-keyboard program:
> 
> ./configure --enable-clients
> ./weston-keyboard
> 
> A working wlroots implementation is available here:
> 
> https://github.com/swaywm/wlroots/pull/999
> 
> Thanks for feedback,
> Dorota Czaplejewicz

just one nitpick over Simon's comments, the copyright list starting at 2008
seems excessive.

However, I wonder what you're really trying to achieve here. For virtual
keyboards the mapping to a normal keyboard's physical buttons, then mapped
to the glyph elsewhere seems strange and limiting. On a virtual keyboard,
I'm not hitting KEY_A, I'm clicking on the button currently labelled with
'A'. There's a million things we could do to virtual keyboard that make the
assumption of virtual keyboard == physical keyboards go boom quite quickly.
e.g. on my phone once the emoticons are selected I don't get the normal
qwerty layout anymore but just a row/column arrangement of smileys.

I read your v1 email about splitting input-method and virtual-keyboard but
it's still not quite clear to me, sorry.

Cheers,
   Peter

 
>  Makefile.am                                        |   1 +
>  unstable/virtual-keyboard/README                   |   2 +
>  .../virtual-keyboard-unstable-v1.xml               | 113 +++++++++++++++++++++
>  3 files changed, 116 insertions(+)
>  create mode 100644 unstable/virtual-keyboard/README
>  create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> 
> diff --git a/Makefile.am b/Makefile.am
> index 4b9a901..fcd4572 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -17,6 +17,7 @@ unstable_protocols =								\
>  	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
>  	unstable/xdg-output/xdg-output-unstable-v1.xml				\
>  	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
> +	unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
>  	$(NULL)
>  
>  stable_protocols =								\
> diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
> new file mode 100644
> index 0000000..a2c646d
> --- /dev/null
> +++ b/unstable/virtual-keyboard/README
> @@ -0,0 +1,2 @@
> +Virtual keyboard protocol
> +
> diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> new file mode 100644
> index 0000000..df4d01c
> --- /dev/null
> +++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> @@ -0,0 +1,113 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<protocol name="virtual_keyboard_unstable_v1">
> +  <copyright>
> +    Copyright © 2008-2011  Kristian Høgsberg
> +    Copyright © 2010-2013  Intel Corporation
> +    Copyright © 2012-2013  Collabora, Ltd.
> +    Copyright © 2018       Purism SPC
> +
> +    Permission is hereby granted, free of charge, to any person obtaining a
> +    copy of this software and associated documentation files (the "Software"),
> +    to deal in the Software without restriction, including without limitation
> +    the rights to use, copy, modify, merge, publish, distribute, sublicense,
> +    and/or sell copies of the Software, and to permit persons to whom the
> +    Software is furnished to do so, subject to the following conditions:
> +
> +    The above copyright notice and this permission notice (including the next
> +    paragraph) shall be included in all copies or substantial portions of the
> +    Software.
> +
> +    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> +    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> +    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> +    DEALINGS IN THE SOFTWARE.
> +  </copyright>
> +
> +  <interface name="zwp_virtual_keyboard_v1" version="1">
> +    <description summary="virtual keyboard">
> +      The virtual keyboard provides an application with requests which emulate
> +      the behaviour of a physical keyboard.
> +
> +      This interface can be used by clients on its own to provide raw input
> +      events, or it can accompany the input method protocol.
> +    </description>
> +
> +    <request name="keymap">
> +      <description summary="keyboard mapping">
> +        Provide a file descriptor to the compositor which can be
> +        memory-mapped to provide a keyboard mapping description.
> +
> +        Format carries a value from the keymap_format enumeration.
> +      </description>
> +      <arg name="format" type="uint" summary="keymap format"/>
> +      <arg name="fd" type="fd" summary="keymap file descriptor"/>
> +      <arg name="size" type="uint" summary="keymap size, in bytes"/>
> +    </request>
> +    
> +    <enum name="error">
> +      <entry name="no_keymap" value="0" summary="No keymap was set"/>
> +    </enum>
> +
> +    <request name="key">
> +      <description summary="key event">
> +        A key was pressed or released.
> +        The time argument is a timestamp with millisecond granularity, with an
> +        undefined base. All requests regarding a single object must share the
> +        same clock.
> +
> +        Keymap must be set before issuing this request.
> +
> +        State carries a value from the key_state enumeration.
> +      </description>
> +      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
> +      <arg name="key" type="uint" summary="key that produced the event"/>
> +      <arg name="state" type="uint" summary="physical state of the key"/>
> +    </request>
> +
> +    <request name="modifiers">
> +      <description summary="modifier and group state">
> +        Notifies the compositor that the modifier and/or group state has
> +        changed, and it should update state.
> +
> +        The client should use wl_keyboard.modifiers event to synchronize its
> +        internal state with seat state.
> +
> +        Keymap must be set before issuing this request.
> +      </description>
> +      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
> +      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
> +      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
> +      <arg name="group" type="uint" summary="keyboard layout"/>
> +    </request>
> +
> +    <request name="destroy" type="destructor" since="1">
> +      <description summary="destroy the virtual keyboard keyboard object"/>
> +    </request>
> +  </interface>
> +
> +  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
> +    <description summary="virtual keyboard manager">
> +      A virtual keyboard manager allows an application to provide keyboard
> +      input events as if they came from a physical keyboard.
> +    </description>
> +
> +    <enum name="error">
> +      <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
> +    </enum>
> +
> +    <request name="create_virtual_keyboard">
> +      <description summary="Create a new virtual keyboard">
> +        Creates a new virtual keyboard associated to a seat.
> +
> +        If the compositor enables a keyboard to perform arbitrary actions, it
> +        should present an error when an untrusted client requests a new
> +        keyboard.
> +      </description>
> +      <arg name="seat" type="object" interface="wl_seat"/>
> +      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
> +    </request>
> +  </interface>
> +</protocol>
> -- 
> 2.14.3
> 
> _______________________________________________
> wayland-devel mailing list
> wayland-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/wayland-devel
>
Dorota Czaplejewicz May 30, 2018, 6:43 a.m.
On Wed, 30 May 2018 14:28:12 +1000
Peter Hutterer <peter.hutterer@who-t.net> wrote:

> On Thu, May 24, 2018 at 08:27:29PM +0200, Dorota Czaplejewicz wrote:
> > Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
> > 
> > The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
> > ---
> > This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.
> > 
> > Changes done:
> > - fixed typos
> > - enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
> > - added errors
> > 
> > The test client is at https://code.puri.sm/Librem5/weston/src/virtual_keyboard - please use the weston-keyboard program:
> > 
> > ./configure --enable-clients
> > ./weston-keyboard
> > 
> > A working wlroots implementation is available here:
> > 
> > https://github.com/swaywm/wlroots/pull/999
> > 
> > Thanks for feedback,
> > Dorota Czaplejewicz  
> 
> just one nitpick over Simon's comments, the copyright list starting at 2008
> seems excessive.
> 
> However, I wonder what you're really trying to achieve here. For virtual
> keyboards the mapping to a normal keyboard's physical buttons, then mapped
> to the glyph elsewhere seems strange and limiting. On a virtual keyboard,
> I'm not hitting KEY_A, I'm clicking on the button currently labelled with
> 'A'. There's a million things we could do to virtual keyboard that make the
> assumption of virtual keyboard == physical keyboards go boom quite quickly.
> e.g. on my phone once the emoticons are selected I don't get the normal
> qwerty layout anymore but just a row/column arrangement of smileys.
> 
> I read your v1 email about splitting input-method and virtual-keyboard but
> it's still not quite clear to me, sorry.
> 
> Cheers,
>    Peter
> 

Hi Peter,

copyrights go back to 2008 because I used chunks of input-method and wayland.xml as the base.

This virtual keyboard protocol is just that - a way to provide an emulated input device. It's not expected to be a complete solution for what an actual on-screen keyboard is doing, but it's a necessary part. It allows for keyboard shortcuts to be used to interact with the compositor for example.

The second part of responsibilities of an on-screen keyboard is providing text composition capabilities, which covers the emojis you mentioned, but is also limited to text composition scenarios. As an example, it doesn't make sense to control the window manager by sending it text but they usually care about keyboard shortcuts.

These use cases both look like "a keyboard" as they were traditionally done with keyboards, but are really something completely different. To exaggerate the difference, the virtual-keyboard protocol is similar to using a game controller, and input-method is much closer to handwriting recognition than pressing buttons.

I hope that cleared it up a little. Either way, I will be submitting an input-method update soon to show the implementation.

Cheers,
Dorota
>  
> >  Makefile.am                                        |   1 +
> >  unstable/virtual-keyboard/README                   |   2 +
> >  .../virtual-keyboard-unstable-v1.xml               | 113 +++++++++++++++++++++
> >  3 files changed, 116 insertions(+)
> >  create mode 100644 unstable/virtual-keyboard/README
> >  create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > 
> > diff --git a/Makefile.am b/Makefile.am
> > index 4b9a901..fcd4572 100644
> > --- a/Makefile.am
> > +++ b/Makefile.am
> > @@ -17,6 +17,7 @@ unstable_protocols =								\
> >  	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
> >  	unstable/xdg-output/xdg-output-unstable-v1.xml				\
> >  	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
> > +	unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
> >  	$(NULL)
> >  
> >  stable_protocols =								\
> > diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
> > new file mode 100644
> > index 0000000..a2c646d
> > --- /dev/null
> > +++ b/unstable/virtual-keyboard/README
> > @@ -0,0 +1,2 @@
> > +Virtual keyboard protocol
> > +
> > diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > new file mode 100644
> > index 0000000..df4d01c
> > --- /dev/null
> > +++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > @@ -0,0 +1,113 @@
> > +<?xml version="1.0" encoding="UTF-8"?>
> > +<protocol name="virtual_keyboard_unstable_v1">
> > +  <copyright>
> > +    Copyright © 2008-2011  Kristian Høgsberg
> > +    Copyright © 2010-2013  Intel Corporation
> > +    Copyright © 2012-2013  Collabora, Ltd.
> > +    Copyright © 2018       Purism SPC
> > +
> > +    Permission is hereby granted, free of charge, to any person obtaining a
> > +    copy of this software and associated documentation files (the "Software"),
> > +    to deal in the Software without restriction, including without limitation
> > +    the rights to use, copy, modify, merge, publish, distribute, sublicense,
> > +    and/or sell copies of the Software, and to permit persons to whom the
> > +    Software is furnished to do so, subject to the following conditions:
> > +
> > +    The above copyright notice and this permission notice (including the next
> > +    paragraph) shall be included in all copies or substantial portions of the
> > +    Software.
> > +
> > +    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> > +    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> > +    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> > +    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> > +    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> > +    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> > +    DEALINGS IN THE SOFTWARE.
> > +  </copyright>
> > +
> > +  <interface name="zwp_virtual_keyboard_v1" version="1">
> > +    <description summary="virtual keyboard">
> > +      The virtual keyboard provides an application with requests which emulate
> > +      the behaviour of a physical keyboard.
> > +
> > +      This interface can be used by clients on its own to provide raw input
> > +      events, or it can accompany the input method protocol.
> > +    </description>
> > +
> > +    <request name="keymap">
> > +      <description summary="keyboard mapping">
> > +        Provide a file descriptor to the compositor which can be
> > +        memory-mapped to provide a keyboard mapping description.
> > +
> > +        Format carries a value from the keymap_format enumeration.
> > +      </description>
> > +      <arg name="format" type="uint" summary="keymap format"/>
> > +      <arg name="fd" type="fd" summary="keymap file descriptor"/>
> > +      <arg name="size" type="uint" summary="keymap size, in bytes"/>
> > +    </request>
> > +    
> > +    <enum name="error">
> > +      <entry name="no_keymap" value="0" summary="No keymap was set"/>
> > +    </enum>
> > +
> > +    <request name="key">
> > +      <description summary="key event">
> > +        A key was pressed or released.
> > +        The time argument is a timestamp with millisecond granularity, with an
> > +        undefined base. All requests regarding a single object must share the
> > +        same clock.
> > +
> > +        Keymap must be set before issuing this request.
> > +
> > +        State carries a value from the key_state enumeration.
> > +      </description>
> > +      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
> > +      <arg name="key" type="uint" summary="key that produced the event"/>
> > +      <arg name="state" type="uint" summary="physical state of the key"/>
> > +    </request>
> > +
> > +    <request name="modifiers">
> > +      <description summary="modifier and group state">
> > +        Notifies the compositor that the modifier and/or group state has
> > +        changed, and it should update state.
> > +
> > +        The client should use wl_keyboard.modifiers event to synchronize its
> > +        internal state with seat state.
> > +
> > +        Keymap must be set before issuing this request.
> > +      </description>
> > +      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
> > +      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
> > +      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
> > +      <arg name="group" type="uint" summary="keyboard layout"/>
> > +    </request>
> > +
> > +    <request name="destroy" type="destructor" since="1">
> > +      <description summary="destroy the virtual keyboard keyboard object"/>
> > +    </request>
> > +  </interface>
> > +
> > +  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
> > +    <description summary="virtual keyboard manager">
> > +      A virtual keyboard manager allows an application to provide keyboard
> > +      input events as if they came from a physical keyboard.
> > +    </description>
> > +
> > +    <enum name="error">
> > +      <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
> > +    </enum>
> > +
> > +    <request name="create_virtual_keyboard">
> > +      <description summary="Create a new virtual keyboard">
> > +        Creates a new virtual keyboard associated to a seat.
> > +
> > +        If the compositor enables a keyboard to perform arbitrary actions, it
> > +        should present an error when an untrusted client requests a new
> > +        keyboard.
> > +      </description>
> > +      <arg name="seat" type="object" interface="wl_seat"/>
> > +      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
> > +    </request>
> > +  </interface>
> > +</protocol>
> > -- 
> > 2.14.3
> > 
> > _______________________________________________
> > wayland-devel mailing list
> > wayland-devel@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/wayland-devel
> >
Jan Arne Petersen May 30, 2018, 8:05 a.m.
Hey,

On Wed, May 30, 2018 at 9:33 AM Peter Hutterer <peter.hutterer@who-t.net>
wrote:

> On Thu, May 24, 2018 at 08:27:29PM +0200, Dorota Czaplejewicz wrote:
> > Provides the ability to emulate keyboards by applications.
Complementary to input-method protocol.
> >
> > The interface is a mirror copy of wl_keyboard, with removed serials,
and added seat binding.
> > ---

I do not really understand for what this protocol is useful, trying to map
a virtual keyboard to a physical keyboard seems to be extremely limiting
and I do not see really use-cases for it. If you want to be able to send
non-text from a virtual keyboard to an application it makes more sense to
have some protocol to send keysyms for example.

Regards
Jan Arne Petersen
Dorota Czaplejewicz May 31, 2018, 2:25 p.m.
On Wed, 30 May 2018 10:05:52 +0200
Jan Arne Petersen <janarne@gmail.com> wrote:

> Hey,
> 
> On Wed, May 30, 2018 at 9:33 AM Peter Hutterer <peter.hutterer@who-t.net>
> wrote:
> 
> > On Thu, May 24, 2018 at 08:27:29PM +0200, Dorota Czaplejewicz wrote:  
> > > Provides the ability to emulate keyboards by applications.  
> Complementary to input-method protocol.
> > >
> > > The interface is a mirror copy of wl_keyboard, with removed serials,  
> and added seat binding.
> > > ---  
> 
> I do not really understand for what this protocol is useful, trying to map
> a virtual keyboard to a physical keyboard seems to be extremely limiting
> and I do not see really use-cases for it. If you want to be able to send
> non-text from a virtual keyboard to an application it makes more sense to
> have some protocol to send keysyms for example.
> 
> Regards
> Jan Arne Petersen

Hi,

the immediate need is a virtual keyboard indeed. Keeping the protocol simple and general can make it useful for automating input for example, so I think it's worth to stick to the current form.

It's not a suboptimal form for a virtual keyboard either. I have considered keysyms, gathered opinions, and decided against them. The reason is simple. If we tried to send scan codes, then the compositor would have to do the scan code->keysym translation anyway, but with less information. Either that, or applications would have to support yet another protocol (in addition to wl_keyboard and text-input) to receive generic keyboard events, defeating the point of having a generic protocol.

On the other hand, raw scan codes can be trivially piped via wl_keyboard to any existing application that cares about keyboard input, even if it doesn't implement text-input.

Cheers,
Dorota Czaplejewicz