virtual-keyboard: Add new virtual keyboard protocol

Submitted by Dorota Czaplejewicz on May 17, 2018, 4:53 p.m.

Details

Message ID 20180517165310.7358-1-dorota.czaplejewicz@puri.sm
State New
Series "virtual-keyboard: Add new virtual keyboard protocol"
Headers show

Commit Message

Dorota Czaplejewicz May 17, 2018, 4:53 p.m.
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.

The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---

This proposal is another one needed by Purism to support on screen keyboards on a phone screen.

Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.

Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.

Input welcome.

Cheers,
Dorota

 Makefile.am                                        |  1 +
 .../virtual-keyboard-unstable-v1.xml               | 97 ++++++++++++++++++++++
 2 files changed, 98 insertions(+)
 create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml

Patch hide | download patch | download mbox

diff --git a/Makefile.am b/Makefile.am
index 4b9a901..51a47a2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@  unstable_protocols =								\
 	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
 	unstable/xdg-output/xdg-output-unstable-v1.xml				\
 	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
+	nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
 	$(NULL)
 
 stable_protocols =								\
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..18130e2
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,97 @@ 
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+  <copyright>
+    Copyright © 2008-2011  Kristian Høgsberg
+    Copyright © 2010-2013  Intel Corporation
+    Copyright © 2012-2013  Collabora, Ltd.
+    Copyright © 2018       Purism SPC
+
+    Permission is hereby granted, free of charge, to any person obtaining a
+    copy of this software and associated documentation files (the "Software"),
+    to deal in the Software without restriction, including without limitation
+    the rights to use, copy, modify, merge, publish, distribute, sublicense,
+    and/or sell copies of the Software, and to permit persons to whom the
+    Software is furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice (including the next
+    paragraph) shall be included in all copies or substantial portions of the
+    Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+    DEALINGS IN THE SOFTWARE.
+  </copyright>
+
+  <interface name="zwp_virtual_keyboard_v1" version="1">
+    <description summary="virtual keyboard">
+      The virtual keyboard provides an application with requests which emulate
+      the behaviour of a physical keyboard.
+
+      This interface can be used by clients on its own to provide raw input
+      events, or it can accompany the input method protocol.
+    </description>
+
+    <request name="keymap">
+      <description summary="keyboard mapping">
+        Provide a file descriptor to the compositor which can be
+        memory-mapped to provide a keyboard mapping description.
+      </description>
+      <arg name="format" type="uint" enum="keymap_format" summary="keymap format"/>
+      <arg name="fd" type="fd" summary="keymap file descriptor"/>
+      <arg name="size" type="uint" summary="keymap size, in bytes"/>
+    </request>
+
+    <request name="key">
+      <description summary="key event">
+        A key was pressed or released.
+        The time argument is a timestamp with millisecond granularity, with an
+        undefined base. All requests regarding a single object must share the
+        same clock.
+      </description>
+      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+      <arg name="key" type="uint" summary="key that produced the event"/>
+      <arg name="state" type="uint" enum="key_state" summary="physical state of the key"/>
+    </request>
+
+    <request name="modifiers">
+      <description summary="modifier and group state">
+        Notifies the compositor that the modifier and/or group state has
+        changed, and it should update state.
+
+        The client should use wl_keyboard.modifiers event to synchronize its
+        internal state with seat state.
+      </description>
+      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+      <arg name="group" type="uint" summary="keyboard layout"/>
+    </request>
+
+    <request name="destroy" type="destructor" since="1">
+      <description summary="destroy the virtual keyboard keyboard object"/>
+    </request>
+  </interface>
+
+  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+    <description summary="virtual keyboard manager">
+      A virtual keyboard manager allows an application to provide keyboard
+      input events as if they came from a physical keyboard.
+    </description>
+
+    <request name="create_virtual_keyboard">
+      <description summary="Create a new virtual keyboard">
+        Creates a new virtual keyboard associated to a seat.
+
+        If the compositor enables a keyboard to perform arbitrary actions, it
+        should present an error when an untrusted client requests a new
+        keyboard.
+      </description>
+      <arg name="seat" type="object" interface="wl_seat"/>
+      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+    </request>
+  </interface>
+</protocol>

Comments

Silvan Jegen May 18, 2018, 6:31 p.m.
Hi

Just a typo I found below.

On Thu, May 17, 2018 at 06:53:10PM +0200, Dorota Czaplejewicz wrote:
> Provides the ability to emulate keyboards by
> applications. Complementary to input-method protocol.
> 
> The interface is a mirror copy of wl_keyboard, with removed serials,
> and added seat binding.
> ---
> 
> This proposal is another one needed by Purism to support on screen
> keyboards on a phone screen.
> 
> Virtual-keyboard is a protocol to let applications send arbitrary
> keyboard events. The need for this protocol comes from the fact
> that the input-method protocol combines two separate input
> responsibilities. It currently deals both with text input and raw
> keyboard events. I hope to split input-method along this line into
> virtual-keyboard and the rest of input-method. I'm going to submit the
> updated input-method for review soon.
> 
> Applications should be able to control both interfaces at the same
> time. A screen keyboard supporting autocorrect (input-method) still
> wants to send arrow keys (vityual-keyboard) correctly. Because of
> this, both kinds of events at minimum must be sent to the same seat. I
> made the seat binding explicitly done by the application, taking
> inspiration from text-input protocol, which assumes per-seat binding
> as well.
> 
> Input welcome.
> 
> Cheers,
> Dorota
> 
>  Makefile.am                                        |  1 +
>  .../virtual-keyboard-unstable-v1.xml               | 97 ++++++++++++++++++++++
>  2 files changed, 98 insertions(+)
>  create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> 
> diff --git a/Makefile.am b/Makefile.am
> index 4b9a901..51a47a2 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -17,6 +17,7 @@ unstable_protocols =								\
>  	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
>  	unstable/xdg-output/xdg-output-unstable-v1.xml				\
>  	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
> +	nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\

There is a 'u' missing in the beginning of the file location.


Cheers,

Silvan
Roderick.Colenbrander@sony.com May 18, 2018, 7:23 p.m.
Hi Dorota,

Thanks for sharing your proposal. Ourselves we have interest in this 
kind of capability as well. Looking at our own use cases, I wonder if 
this proposal goes far enough.

We are basically interested in the ability to inject keyboard, mouse, 
touch (and gamepad). On regular X some of that worked through XTest 
protocol. From my perspective any virtual keyboard proposal, would make 
sense to be flexible enough to handle other input devices as well.

On the other hand some may bring up, why virtual device support should 
be in Wayland as input devices can also be spoofed through uinput.

Thanks,

Roderick Colenbrander
Sr Manager Hardware & Systems Engineering
Sony Interactive Entertainment

On 05/17/2018 09:55 AM, Dorota Czaplejewicz wrote:
> Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
> 
> The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
> ---
> 
> This proposal is another one needed by Purism to support on screen keyboards on a phone screen.
> 
> Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.
> 
> Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.
> 
> Input welcome.
> 
> Cheers,
> Dorota
> 
>   Makefile.am                                        |  1 +
>   .../virtual-keyboard-unstable-v1.xml               | 97 ++++++++++++++++++++++
>   2 files changed, 98 insertions(+)
>   create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> 
> diff --git a/Makefile.am b/Makefile.am
> index 4b9a901..51a47a2 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -17,6 +17,7 @@ unstable_protocols =								\
>   	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
>   	unstable/xdg-output/xdg-output-unstable-v1.xml				\
>   	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
> +	nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
>   	$(NULL)
>   
>   stable_protocols =								\
> diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> new file mode 100644
> index 0000000..18130e2
> --- /dev/null
> +++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> @@ -0,0 +1,97 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<protocol name="virtual_keyboard_unstable_v1">
> +  <copyright>
> +    Copyright © 2008-2011  Kristian Høgsberg
> +    Copyright © 2010-2013  Intel Corporation
> +    Copyright © 2012-2013  Collabora, Ltd.
> +    Copyright © 2018       Purism SPC
> +
> +    Permission is hereby granted, free of charge, to any person obtaining a
> +    copy of this software and associated documentation files (the "Software"),
> +    to deal in the Software without restriction, including without limitation
> +    the rights to use, copy, modify, merge, publish, distribute, sublicense,
> +    and/or sell copies of the Software, and to permit persons to whom the
> +    Software is furnished to do so, subject to the following conditions:
> +
> +    The above copyright notice and this permission notice (including the next
> +    paragraph) shall be included in all copies or substantial portions of the
> +    Software.
> +
> +    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> +    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> +    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> +    DEALINGS IN THE SOFTWARE.
> +  </copyright>
> +
> +  <interface name="zwp_virtual_keyboard_v1" version="1">
> +    <description summary="virtual keyboard">
> +      The virtual keyboard provides an application with requests which emulate
> +      the behaviour of a physical keyboard.
> +
> +      This interface can be used by clients on its own to provide raw input
> +      events, or it can accompany the input method protocol.
> +    </description>
> +
> +    <request name="keymap">
> +      <description summary="keyboard mapping">
> +        Provide a file descriptor to the compositor which can be
> +        memory-mapped to provide a keyboard mapping description.
> +      </description>
> +      <arg name="format" type="uint" enum="keymap_format" summary="keymap format"/>
> +      <arg name="fd" type="fd" summary="keymap file descriptor"/>
> +      <arg name="size" type="uint" summary="keymap size, in bytes"/>
> +    </request>
> +
> +    <request name="key">
> +      <description summary="key event">
> +        A key was pressed or released.
> +        The time argument is a timestamp with millisecond granularity, with an
> +        undefined base. All requests regarding a single object must share the
> +        same clock.
> +      </description>
> +      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
> +      <arg name="key" type="uint" summary="key that produced the event"/>
> +      <arg name="state" type="uint" enum="key_state" summary="physical state of the key"/>
> +    </request>
> +
> +    <request name="modifiers">
> +      <description summary="modifier and group state">
> +        Notifies the compositor that the modifier and/or group state has
> +        changed, and it should update state.
> +
> +        The client should use wl_keyboard.modifiers event to synchronize its
> +        internal state with seat state.
> +      </description>
> +      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
> +      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
> +      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
> +      <arg name="group" type="uint" summary="keyboard layout"/>
> +    </request>
> +
> +    <request name="destroy" type="destructor" since="1">
> +      <description summary="destroy the virtual keyboard keyboard object"/>
> +    </request>
> +  </interface>
> +
> +  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
> +    <description summary="virtual keyboard manager">
> +      A virtual keyboard manager allows an application to provide keyboard
> +      input events as if they came from a physical keyboard.
> +    </description>
> +
> +    <request name="create_virtual_keyboard">
> +      <description summary="Create a new virtual keyboard">
> +        Creates a new virtual keyboard associated to a seat.
> +
> +        If the compositor enables a keyboard to perform arbitrary actions, it
> +        should present an error when an untrusted client requests a new
> +        keyboard.
> +      </description>
> +      <arg name="seat" type="object" interface="wl_seat"/>
> +      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
> +    </request>
> +  </interface>
> +</protocol>
>
Dorota Czaplejewicz May 20, 2018, 11:12 p.m.
On Thu, 17 May 2018 18:53:10 +0200
Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> wrote:

> Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
> 
> The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
> ---
> 
> This proposal is another one needed by Purism to support on screen keyboards on a phone screen.
> 
> Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.
> 
> Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.
> 
> Input welcome.
> 
> Cheers,
> Dorota


Apart from the typo that Silvan spotted, I have also encountered the issue where the .c/.h generator complained about undefined key_state and keymap_format enums which are defined in wayland.xml. I'm not sure what the correct way to solve this - should they be copied into this protocol?

Cheers,
Dorota
Simon Ser May 21, 2018, 7:19 a.m.
On May 21, 2018 12:12 AM, Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> wrote:
> Apart from the typo that Silvan spotted, I have also encountered the issue where
> the .c/.h generator complained about undefined key_state and keymap_format enums
> which are defined in wayland.xml. I'm not sure what the correct way to solve
> this - should they be copied into this protocol?

This is a bug in wayland-scanner, see [1]. Fixing wayland-scanner has been on my
TODO-list for a while, but if you want to do it, go ahead. A (less than ideal)
workaround is to remove the "enum" attribute from the <arg> for now.

[1]: https://lists.freedesktop.org/archives/wayland-devel/2018-April/037960.html
Dorota Czaplejewicz May 21, 2018, 12:26 p.m.
Hello Roderick,

On Fri, 18 May 2018 19:23:15 +0000
<Roderick.Colenbrander@sony.com> wrote:

> Hi Dorota,
> 
> Thanks for sharing your proposal. Ourselves we have interest in this 
> kind of capability as well. Looking at our own use cases, I wonder if 
> this proposal goes far enough.
> 
> We are basically interested in the ability to inject keyboard, mouse, 
> touch (and gamepad). On regular X some of that worked through XTest 
> protocol. From my perspective any virtual keyboard proposal, would make 
> sense to be flexible enough to handle other input devices as well.

I thought about adding mouse or game controller support in the same protocol, but I came to the conclusion that they would be better off as dedicated protocols. We already have an input method protocol, which can be considered an input device, then the separate keyboard protocol already, and using a dedicated one for any other type of device would follow this trend.

The main reason to keep separate, however, is to keep it simple to develop. Personally, I don't have much of an idea about approaching a mouse or gamepad protocol, because they are not in the scope of what I'm doing. That puts a damper on my efforts regarding virtual keyboard. In the same way, I expect protocol designers of the future to only have experience in a subset of devices, and they will probably want to avoid worrying about possible changes to other devices when they update what they care about.
> 
> On the other hand some may bring up, why virtual device support should 
> be in Wayland as input devices can also be spoofed through uinput.
> 
Wayland can still be a good place for these kinds of protocols, due to how it handles seat assignment. I would not be able to ensure that an input method and a virtual keyboard is assigned the same seat if I tried to emulate the keyboard in a lower layer. I suspect that this is not the only possible issue in this area. The effect of a single standard interface is also important – applications don't have to rely on system-specific libraries any more, and make things like nested sessions much more palatable.

> Thanks,
> 
> Roderick Colenbrander
> Sr Manager Hardware & Systems Engineering
> Sony Interactive Entertainment
> 

Have a nice day,
Dorota Czaplejewicz

> On 05/17/2018 09:55 AM, Dorota Czaplejewicz wrote:
> > Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
> > 
> > The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
> > ---
> > 
> > This proposal is another one needed by Purism to support on screen keyboards on a phone screen.
> > 
> > Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.
> > 
> > Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.
> > 
> > Input welcome.
> > 
> > Cheers,
> > Dorota
> > 
> >   Makefile.am                                        |  1 +
> >   .../virtual-keyboard-unstable-v1.xml               | 97 ++++++++++++++++++++++
> >   2 files changed, 98 insertions(+)
> >   create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > 
> > diff --git a/Makefile.am b/Makefile.am
> > index 4b9a901..51a47a2 100644
> > --- a/Makefile.am
> > +++ b/Makefile.am
> > @@ -17,6 +17,7 @@ unstable_protocols =								\
> >   	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
> >   	unstable/xdg-output/xdg-output-unstable-v1.xml				\
> >   	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
> > +	nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
> >   	$(NULL)
> >   
> >   stable_protocols =								\
> > diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > new file mode 100644
> > index 0000000..18130e2
> > --- /dev/null
> > +++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > @@ -0,0 +1,97 @@
> > +<?xml version="1.0" encoding="UTF-8"?>
> > +<protocol name="virtual_keyboard_unstable_v1">
> > +  <copyright>
> > +    Copyright © 2008-2011  Kristian Høgsberg
> > +    Copyright © 2010-2013  Intel Corporation
> > +    Copyright © 2012-2013  Collabora, Ltd.
> > +    Copyright © 2018       Purism SPC
> > +
> > +    Permission is hereby granted, free of charge, to any person obtaining a
> > +    copy of this software and associated documentation files (the "Software"),
> > +    to deal in the Software without restriction, including without limitation
> > +    the rights to use, copy, modify, merge, publish, distribute, sublicense,
> > +    and/or sell copies of the Software, and to permit persons to whom the
> > +    Software is furnished to do so, subject to the following conditions:
> > +
> > +    The above copyright notice and this permission notice (including the next
> > +    paragraph) shall be included in all copies or substantial portions of the
> > +    Software.
> > +
> > +    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> > +    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> > +    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> > +    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> > +    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> > +    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> > +    DEALINGS IN THE SOFTWARE.
> > +  </copyright>
> > +
> > +  <interface name="zwp_virtual_keyboard_v1" version="1">
> > +    <description summary="virtual keyboard">
> > +      The virtual keyboard provides an application with requests which emulate
> > +      the behaviour of a physical keyboard.
> > +
> > +      This interface can be used by clients on its own to provide raw input
> > +      events, or it can accompany the input method protocol.
> > +    </description>
> > +
> > +    <request name="keymap">
> > +      <description summary="keyboard mapping">
> > +        Provide a file descriptor to the compositor which can be
> > +        memory-mapped to provide a keyboard mapping description.
> > +      </description>
> > +      <arg name="format" type="uint" enum="keymap_format" summary="keymap format"/>
> > +      <arg name="fd" type="fd" summary="keymap file descriptor"/>
> > +      <arg name="size" type="uint" summary="keymap size, in bytes"/>
> > +    </request>
> > +
> > +    <request name="key">
> > +      <description summary="key event">
> > +        A key was pressed or released.
> > +        The time argument is a timestamp with millisecond granularity, with an
> > +        undefined base. All requests regarding a single object must share the
> > +        same clock.
> > +      </description>
> > +      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
> > +      <arg name="key" type="uint" summary="key that produced the event"/>
> > +      <arg name="state" type="uint" enum="key_state" summary="physical state of the key"/>
> > +    </request>
> > +
> > +    <request name="modifiers">
> > +      <description summary="modifier and group state">
> > +        Notifies the compositor that the modifier and/or group state has
> > +        changed, and it should update state.
> > +
> > +        The client should use wl_keyboard.modifiers event to synchronize its
> > +        internal state with seat state.
> > +      </description>
> > +      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
> > +      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
> > +      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
> > +      <arg name="group" type="uint" summary="keyboard layout"/>
> > +    </request>
> > +
> > +    <request name="destroy" type="destructor" since="1">
> > +      <description summary="destroy the virtual keyboard keyboard object"/>
> > +    </request>
> > +  </interface>
> > +
> > +  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
> > +    <description summary="virtual keyboard manager">
> > +      A virtual keyboard manager allows an application to provide keyboard
> > +      input events as if they came from a physical keyboard.
> > +    </description>
> > +
> > +    <request name="create_virtual_keyboard">
> > +      <description summary="Create a new virtual keyboard">
> > +        Creates a new virtual keyboard associated to a seat.
> > +
> > +        If the compositor enables a keyboard to perform arbitrary actions, it
> > +        should present an error when an untrusted client requests a new
> > +        keyboard.
> > +      </description>
> > +      <arg name="seat" type="object" interface="wl_seat"/>
> > +      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
> > +    </request>
> > +  </interface>
> > +</protocol>
> >   
>
wl@ongy.net May 21, 2018, 1:09 p.m.
On 2018/5月/18 07:23, Roderick.Colenbrander@sony.com wrote:
> Hi Dorota,
> 
> Thanks for sharing your proposal. Ourselves we have interest in this 
> kind of capability as well. Looking at our own use cases, I wonder if 
> this proposal goes far enough.
> 
> We are basically interested in the ability to inject keyboard, mouse, 
> touch (and gamepad). On regular X some of that worked through XTest 
> protocol. From my perspective any virtual keyboard proposal, would make 
> sense to be flexible enough to handle other input devices as well.
All of these devices are different enough, that they need a dedicated API in 
some way (keys/analog axis/relative movements). This could be implemented as 
different interfaces in the same protocol, or separate protocols.
IMO splitting it up will be nicer, so compositors and clients can update the 
versions they support indipendently, and we aren't forced to bump a version on 
virtual keyboards to add to virtual gamepads.

> 
> On the other hand some may bring up, why virtual device support should 
> be in Wayland as input devices can also be spoofed through uinput.
Doing this as a wayland protocol has the (IMO huge) advantage that it doesn't 
require root permissions, and can be well confined.
Adding an input device into a test session with uinput sounds like a hassle as 
well, since the user session will pick it up as well by default. As wayland 
protocol it can just be spawned into the correct session.

Cheers,
ongy
> 
> Thanks,
> 
> Roderick Colenbrander
> Sr Manager Hardware & Systems Engineering
> Sony Interactive Entertainment
> 
> On 05/17/2018 09:55 AM, Dorota Czaplejewicz wrote:
> > Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
> > 
> > The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
> > ---
> > 
> > This proposal is another one needed by Purism to support on screen keyboards on a phone screen.
> > 
> > Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.
> > 
> > Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.
> > 
> > Input welcome.
> > 
> > Cheers,
> > Dorota
> > 
> >   Makefile.am                                        |  1 +
> >   .../virtual-keyboard-unstable-v1.xml               | 97 ++++++++++++++++++++++
> >   2 files changed, 98 insertions(+)
> >   create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > 
> > diff --git a/Makefile.am b/Makefile.am
> > index 4b9a901..51a47a2 100644
> > --- a/Makefile.am
> > +++ b/Makefile.am
> > @@ -17,6 +17,7 @@ unstable_protocols =								\
> >   	unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
> >   	unstable/xdg-output/xdg-output-unstable-v1.xml				\
> >   	unstable/input-timestamps/input-timestamps-unstable-v1.xml	\
> > +	nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml	\
> >   	$(NULL)
> >   
> >   stable_protocols =								\
> > diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > new file mode 100644
> > index 0000000..18130e2
> > --- /dev/null
> > +++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
> > @@ -0,0 +1,97 @@
> > +<?xml version="1.0" encoding="UTF-8"?>
> > +<protocol name="virtual_keyboard_unstable_v1">
> > +  <copyright>
> > +    Copyright © 2008-2011  Kristian Høgsberg
> > +    Copyright © 2010-2013  Intel Corporation
> > +    Copyright © 2012-2013  Collabora, Ltd.
> > +    Copyright © 2018       Purism SPC
> > +
> > +    Permission is hereby granted, free of charge, to any person obtaining a
> > +    copy of this software and associated documentation files (the "Software"),
> > +    to deal in the Software without restriction, including without limitation
> > +    the rights to use, copy, modify, merge, publish, distribute, sublicense,
> > +    and/or sell copies of the Software, and to permit persons to whom the
> > +    Software is furnished to do so, subject to the following conditions:
> > +
> > +    The above copyright notice and this permission notice (including the next
> > +    paragraph) shall be included in all copies or substantial portions of the
> > +    Software.
> > +
> > +    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> > +    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> > +    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> > +    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> > +    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> > +    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> > +    DEALINGS IN THE SOFTWARE.
> > +  </copyright>
> > +
> > +  <interface name="zwp_virtual_keyboard_v1" version="1">
> > +    <description summary="virtual keyboard">
> > +      The virtual keyboard provides an application with requests which emulate
> > +      the behaviour of a physical keyboard.
> > +
> > +      This interface can be used by clients on its own to provide raw input
> > +      events, or it can accompany the input method protocol.
> > +    </description>
> > +
> > +    <request name="keymap">
> > +      <description summary="keyboard mapping">
> > +        Provide a file descriptor to the compositor which can be
> > +        memory-mapped to provide a keyboard mapping description.
> > +      </description>
> > +      <arg name="format" type="uint" enum="keymap_format" summary="keymap format"/>
> > +      <arg name="fd" type="fd" summary="keymap file descriptor"/>
> > +      <arg name="size" type="uint" summary="keymap size, in bytes"/>
> > +    </request>
> > +
> > +    <request name="key">
> > +      <description summary="key event">
> > +        A key was pressed or released.
> > +        The time argument is a timestamp with millisecond granularity, with an
> > +        undefined base. All requests regarding a single object must share the
> > +        same clock.
> > +      </description>
> > +      <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
> > +      <arg name="key" type="uint" summary="key that produced the event"/>
> > +      <arg name="state" type="uint" enum="key_state" summary="physical state of the key"/>
> > +    </request>
> > +
> > +    <request name="modifiers">
> > +      <description summary="modifier and group state">
> > +        Notifies the compositor that the modifier and/or group state has
> > +        changed, and it should update state.
> > +
> > +        The client should use wl_keyboard.modifiers event to synchronize its
> > +        internal state with seat state.
> > +      </description>
> > +      <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
> > +      <arg name="mods_latched" type="uint" summary="latched modifiers"/>
> > +      <arg name="mods_locked" type="uint" summary="locked modifiers"/>
> > +      <arg name="group" type="uint" summary="keyboard layout"/>
> > +    </request>
> > +
> > +    <request name="destroy" type="destructor" since="1">
> > +      <description summary="destroy the virtual keyboard keyboard object"/>
> > +    </request>
> > +  </interface>
> > +
> > +  <interface name="zwp_virtual_keyboard_manager_v1" version="1">
> > +    <description summary="virtual keyboard manager">
> > +      A virtual keyboard manager allows an application to provide keyboard
> > +      input events as if they came from a physical keyboard.
> > +    </description>
> > +
> > +    <request name="create_virtual_keyboard">
> > +      <description summary="Create a new virtual keyboard">
> > +        Creates a new virtual keyboard associated to a seat.
> > +
> > +        If the compositor enables a keyboard to perform arbitrary actions, it
> > +        should present an error when an untrusted client requests a new
> > +        keyboard.
> > +      </description>
> > +      <arg name="seat" type="object" interface="wl_seat"/>
> > +      <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
> > +    </request>
> > +  </interface>
> > +</protocol>
> > 
> 
> _______________________________________________
> wayland-devel mailing list
> wayland-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/wayland-devel