Keyboard API

This document discusses how we can better handle keyboards (hardware, software, other) whilst better supporting localisation and internationalisation.

Aims

Have a high level API so that app developers do not have to worry about hardware configuration, keyboard mapping or even the type of device. We need to support physical keyboards, virtual keyboards and other - i.e. we cannot assume a keyboard with scancodes or any specific language. Breaking changes should only occur where it cannot be avoided or offers significant benefits worth the pain for users of our API. TypedRune should not change, and hopefully there is not a big impact on TypedKey for handling "control characters".

Expectations

There are many things that we need to take care of, this list will grow as we get more feedback. Some of these elements are not obvious, take time to consider the impact of these combinations.

key	action	result	language	notes
A	tap	rune(a)	en	simply pressing a regular key that matches what is shown on the hardware
Shift+A	hold+tap	en	rune(A)	pressing the regular key with shift held down
Ctrl+C	hold+tap	en	copy	Ctrl+C keyboard shortcut (Windows / Linux)
Cmd+C	hold+tap	en	copy	Cmd+C keyboard shortcut (macOS)
Ctrl+A	hold+tap	fr	selectall	With AZERTY keyboard the A key is where Q is on QWERTY so buttons were re-mapped (fixed - #790)
Ctrl+ц	hold+tap	ru	copy	in Russian the english keyboard shortcuts should work based on where english letters would have been (broken - #1220)

Events

Simply tapping a key can generate many events. What parameters should appear in all of these is not necessarily clear - we will explore here.

Currently we simply:

lang	KeyDown	TypedKey	TypedRune*	KeyUp
en	KeyA	KeyA	a	KeyA
en	KeyReturn	KeyReturn		KeyReturn

• (if visible char)

GLFW

In the GLFW driver we deal with hardware keys, these have a scancode, an ID and a name. To capture the details here is the information we get for the top left alpha key ('Q' in Qwerty) in various scenarios:

layout	key	scancode	name	notes
en-US mac	81 (KeyQ)	12	q
AZERTY mac	81	12	a	(shortcurs mapped to different keys)
Russian	81	12	й	(same shortcuts work)
Dvorak	81	12	'	(shortcuts mapped to different keyskeys)

From this summary it seems like there are two modes of operation - sometimes users expect shortcuts to move (where other english letters are available) and sometimes they should stay (when the language is completely different)?

In the following table we attempt to locate the 'c' key, and then perform the copy shortcutkeykey

layout	key	scancode	name	shortcut	notes
en-US mac	67 (KeyC)	8	c	yes
AZERTY mac	67	8	c	yes
Russian	67	8	c	yes	(could be coincidence)
Dvorak	73 (KeyI)	34	c	yes

Proposal

Currently a Key is defined as: type KeyName string, which provides for an abstraction layer like GLFW does. This is used throughout apps to determine what action to take. However it only works fully with an English keyboard (US, UK etc).

• With GLFW we keys have an ID (the GLFW abstraction), Name (visible), scancode (hardware specific)
• We cannot use keyboard scancodes because a) they are platform specific and b) they are only for physical keyboards.

Work in progess, important factors so far:

• KeyName could refer to the position on a keyboard (does this not break AZERTY where Q is A?)
• func (KeyName) Label() string could return the human readable version of the key. This would be used for displaying shortcut key hints.

If we use internally the keyname but make sure it is never presented, use the visible name instead, we can bind shortcuts to their english equivalent (fixing the russian bug listed above) whilst displaying something meaningful to users. But won't this break AZERTY where the A has moved?

Typing runes is consistently correct, what we call the keys is critical - as KeyA is not always what types 'a'.