With an event filter method, we're logging all key events on any widgets to
check whether a key is pressed or not, to set
WidgetKeyboardEvent::mIsRepeat properly. If iBus and Fcitx work as expected,
they synthesize key events with setting their own modifier state which indicate
the events are synthesized by IME. In this expected case, synthesized key
events are not caught by the filter.
On the other hand, in some environment, they keep handling key events
asynchronously but they or something another module synthesizes key events
without the flag and such events are caught by the filter because the events
are posted into the event queue. Therefore, we decide that such synthesized
events are always generated by auto-repeat (first events which are always
filtered by IME are treated as first press, and then, synthesized events are
treated as repeated events because of no key release events).
This patch makes KeymapWrapper::FilterEvents() ignore coming KeyPress
events if:
- the time is exactly same as previous KeyPress event
- and IMContextWrapper instance is now waiting a GDK_KEY_PRESS event
- and hardware_keycode of waiting GDK_KEY_PRESS event is same as
keyCode of the KeyPress event
Differential Revision: https://phabricator.services.mozilla.com/D15380
--HG--
extra : moz-landing-system : lando
According to the log of bug 1498823, ibus won't set IBUS_IGNORED_MASK to
modifier flags when it synthesizes the event for asynchronous handling in
some environments.
Currently, we assume that iBus and Fcitx set IBUS_IGNORED_MASK or
FcitxKeyState_IgnoredMask. So, we put both real key events and synthesized
key events into the posting event queue and that causes using a lot of
memory until the editor is blurred. Fortunately, timestamp of synthesized
key events are always same as their original events. Therefore, we can look
for original event from the positing event queue.
Although we have gotten no bug reports about this issue of Fcitx, but this
patch adds same hack for Fcitx too because the runtime cost is not
expensive but the symptom is really serious.
Differential Revision: https://phabricator.services.mozilla.com/D15321
--HG--
extra : moz-landing-system : lando
Currently, IMContextWrapper::OnKeyEvent() assumes that IME won't synthesize
keyboard event asynchronously again in some cases. For example, one of the
cases is that user inputs text with a dead key sequence. However, IME may
synthesize key event asynchronously only in a few cases even in a dead key
sequence. Unfortunately, for not losing a chance to dispatch eKeyDown/eKeyUp
event, we need to keep dispatching eKeyDown or eKeyUp event when we receive
original event in dead key sequence. However, according to this bug, we need to
stop dispatching eKeyDown and eKeyUp events when we receive unexpected
async synthesized key event.
If IMContextWrapper::OnKeyEvent() needs to return whether it (has already)
dispatched an eKeyDown or eKeyUp and whether it was consumed, then,
nsWindow can stop dispatching redundant eKeyDown and eKeyUp events.
So, this patch makes IMContextWrapper::OnKeyEvent() return
KeyHandlingState enum class instead of just a bool value to notify the caller
of detail of the event status. And also makes each caller of nsWindow not
dispatch eKeyDown nor eKeyUp event when it returns
KeyHandlingState::eNotHandledButDispatched or
KeyHandlingState::eNotHandledButConsumed.
Differential Revision: https://phabricator.services.mozilla.com/D12517
--HG--
extra : moz-landing-system : lando
IME (e.g., fcitx) may refer selection colors of widget under window which
is associated with IM context to support any colored widget. So, IME
expects good selection colors which have sufficient contrast between
foreground and background, and also selection background color and
widget background color like GtkTextView. However, some desktop themes
set our widget to different selection colors from GtkTextView which may
be unreadable.
nsTextFrame (which paints composition string) expects that composition
string colors coming from IME are sufficiently readable and background
color of composition string and background color of our editor's default
style (coming from LookAndFeel) have sufficient contrast because
nsTextFrame assmes that composition string colors coming from IME are
decided for the default style.
Therefore, this patch creates SelectionStyleProvider which overwrites
selection style of our widget with selection style of GtkTextView so
that IME can refer selection colors of GtkTextView via our widget.
MozReview-Commit-ID: 5vdcSgoEYv0
--HG--
extra : rebase_source : edf375ac393a72d3e44839a76d5c44b6db12dc63
Different from Windows and macOS, we cannot check if active keyboard layout
works as "IME" actually. Therefore, this patch add the telemetry probe
to the dispatcher of eCompositionStart. However, composition string is also
used by some Wester keyboard layouts which have dead keys. So, the meaning
of the result is deferent from the other platforms, but it must be useful
information which IM (e.g., fcitx, ibus) is used by most users.
MozReview-Commit-ID: A7vYuGtcrRw
--HG--
extra : rebase_source : 1f0948b58b999eee4b539d7db05d455b34f4be72
uim is an old IM which uses key snooper to listen to key events rather than
via filter key event API which should be called by applications. It's still
used by Debian 9.x, so, we still need to support this.
Unfortunately, we cannot detect if uim actually uses key snooper because it's
switch by build option of uim. Currently, Debian builds uim as using key
snooper. So, we should assume uim uses key snooper always. On the other
hand, somebody *might* use uim built as not using key snooper, so, let's
decide if uim uses key snooper with new pref,
"intl.ime.hack.uim.using_key_snooper", but its default should be true.
Note that ibus and Fcitx also have the mode to use key snooper (perhaps for
backward compatibility with uim). However, it's not enabled in default
settings and even if it's enabled, Firefox is in whitelist in the default
settings of them for stop using key snooper. Therefore, we don't need to
support key snooper mode for them unless we'll get some requests to
support their key snooping mode.
MozReview-Commit-ID: 6fTsfKrHzvo
--HG--
extra : rebase_source : 8ddf4541db635246e6bb0ddc73b012c9be001c6d
ibus and fcitx usually post key event to other IME process, then, if it causes
some signals to updating composition string, they may not send the posted
key event to us again. Then, IMContextWrapper dispatches neither eKeyDown nor
eKeyUp event since mProcessingKeyEvent becomes non-nullptr only while
OnKeyEvent() is called. So, IMContextWrapper need to store key event if
OnKeyEvent() assumes that given key event is posted to different process.
Then, if IMContextWrapper receives some signals, it should dispatch eKeyDown
and eKeyUp event with stored key event.
Note that we cannot compare the pointer of first event and following event
directly even though usually both events are same address as far as I checked
because according to the source code of ibus, fcitx and GDK, they use
gdk_event_copy() to keep storing original event. According to the document of
the API, it might just increment refcount. However, the actual implementation
of the API always creates another instance and return it. So, it might be
used same address by arena allocation or something accidentally. Anyway,
we shouldn't compare them. Instead, we need to compare each information of
two key events. Unfortunately, we also cannot compare them simply. Both
ibus and fcitx set unused bits of GdkEventKey::state to true when they send
back the event to us. Therefore, we should compare some of or all of the
members by ourselves. I think that matching time must be enough in most
cases since its value of native key events are properly set. However, for
safer code, this patch also checks type, keyval and part of state.
MozReview-Commit-ID: FZSwN61v0Sd
--HG--
extra : rebase_source : e57a654392f476f5ec52d82bdd238eed2eb91e83
ibus and fcitx have asynchronous key event handling mode and it's enabled in
default settings. That is, when they receive a key event from application via
a call of gtk_im_context_filter_keypress(), they may post the key event
information to other IME process, then does nothing but store the copy of the
event with gdk_event_copy() and returns true for the result of
gtk_im_context_filter_keypress(). When the other IME process handles the
event, returns the result to them in our process. Then, they send the stored
key event to us again. Finally, they actually handles the event in our process
actually.
Therefore, we may receive every key event twice. So, this causes dispatching
eKeyDown event and eKeyUp event twice. Preceding key event is always marked
as "processed by IME" since gtk_im_context_filter_keypress() returns true
temporarily and following key event is dispatched as expected. So, we need
to ignore the first event only when gtk_im_context_filter_keypress() returns
true but the event is posted to different process.
Unfortunately, we cannot know if the key event is actually posted to different
process directly. However, we can know if active IM is ibus, fcitx or another
one and if ibus or fcitx is in asynchronous key handling mode.
The former information is provided by gtk_im_multicontext_get_context_id().
It returns a string which is set to the IM multicontext instance by creator.
We'll get "ibus" if IM is ibus, get "fcitx" if IM is fcitx.
The latter information is not provided. However, they consider the mode from
env value. ibus checks IBUS_ENABLE_SYNC_MODE. fcitx checks both
IBUS_ENABLE_SYNC_MODE and FCITX_ENABLE_SYNC_MODE.
Additionally, we can know if received key event has already been posted to
other IME process. They use undefined bit of GdkEventKey::state to store
if the key event has already been posted (1 << 25, they called "ignored" flag).
Although their approach is really hacky but we can refer the information at
least for now.
Finally, when we guess a key event is posted to other IME process, let's
IMContextWrapper::OnKeyEvent() not dispatch eKeyDown nor eKeyUp event.
Note that if it's handled synchronously as unexpected, it may causes
dispatching one or more composition events and/or delete content event.
So, in such case, we dispatch a keyboard event for processing key event
anyway. There is only once case we'll fail to dispatch keyboard event.
If we receive signals to dispatch composition events or delete content
command event when IM receives the result from other IME process but
it doesn't send the key event to us. This will be fixed by the following
patch.
MozReview-Commit-ID: 94PrlnmQ3uJ
--HG--
extra : rebase_source : fc31b0293ff0f0688dd39b0094fdf8f98b6c64d3
uim is an old IM which uses key snooper to listen to key events rather than
via filter key event API which should be called by applications. It's still
used by Debian 9.x, so, we still need to support this.
Unfortunately, we cannot detect if uim actually uses key snooper because it's
switch by build option of uim. Currently, Debian builds uim as using key
snooper. So, we should assume uim uses key snooper always. On the other
hand, somebody *might* use uim built as not using key snooper, so, let's
decide if uim uses key snooper with new pref,
"intl.ime.hack.uim.using_key_snooper", but its default should be true.
Note that ibus and Fcitx also have the mode to use key snooper (perhaps for
backward compatibility with uim). However, it's not enabled in default
settings and even if it's enabled, Firefox is in whitelist in the default
settings of them for stop using key snooper. Therefore, we don't need to
support key snooper mode for them unless we'll get some requests to
support their key snooping mode.
MozReview-Commit-ID: 6fTsfKrHzvo
--HG--
extra : rebase_source : 8ddf4541db635246e6bb0ddc73b012c9be001c6d
ibus and fcitx usually post key event to other IME process, then, if it causes
some signals to updating composition string, they may not send the posted
key event to us again. Then, IMContextWrapper dispatches neither eKeyDown nor
eKeyUp event since mProcessingKeyEvent becomes non-nullptr only while
OnKeyEvent() is called. So, IMContextWrapper need to store key event if
OnKeyEvent() assumes that given key event is posted to different process.
Then, if IMContextWrapper receives some signals, it should dispatch eKeyDown
and eKeyUp event with stored key event.
Note that we cannot compare the pointer of first event and following event
directly even though usually both events are same address as far as I checked
because according to the source code of ibus, fcitx and GDK, they use
gdk_event_copy() to keep storing original event. According to the document of
the API, it might just increment refcount. However, the actual implementation
of the API always creates another instance and return it. So, it might be
used same address by arena allocation or something accidentally. Anyway,
we shouldn't compare them. Instead, we need to compare each information of
two key events. Unfortunately, we also cannot compare them simply. Both
ibus and fcitx set unused bits of GdkEventKey::state to true when they send
back the event to us. Therefore, we should compare some of or all of the
members by ourselves. I think that matching time must be enough in most
cases since its value of native key events are properly set. However, for
safer code, this patch also checks type, keyval and part of state.
MozReview-Commit-ID: FZSwN61v0Sd
--HG--
extra : rebase_source : e54284c27a171f899a6cf87a65935669e2d57021
ibus and fcitx have asynchronous key event handling mode and it's enabled in
default settings. That is, when they receive a key event from application via
a call of gtk_im_context_filter_keypress(), they may post the key event
information to other IME process, then does nothing but store the copy of the
event with gdk_event_copy() and returns true for the result of
gtk_im_context_filter_keypress(). When the other IME process handles the
event, returns the result to them in our process. Then, they send the stored
key event to us again. Finally, they actually handles the event in our process
actually.
Therefore, we may receive every key event twice. So, this causes dispatching
eKeyDown event and eKeyUp event twice. Preceding key event is always marked
as "processed by IME" since gtk_im_context_filter_keypress() returns true
temporarily and following key event is dispatched as expected. So, we need
to ignore the first event only when gtk_im_context_filter_keypress() returns
true but the event is posted to different process.
Unfortunately, we cannot know if the key event is actually posted to different
process directly. However, we can know if active IM is ibus, fcitx or another
one and if ibus or fcitx is in asynchronous key handling mode.
The former information is provided by gtk_im_multicontext_get_context_id().
It returns a string which is set to the IM multicontext instance by creator.
We'll get "ibus" if IM is ibus, get "fcitx" if IM is fcitx.
The latter information is not provided. However, they consider the mode from
env value. ibus checks IBUS_ENABLE_SYNC_MODE. fcitx checks both
IBUS_ENABLE_SYNC_MODE and FCITX_ENABLE_SYNC_MODE.
Additionally, we can know if received key event has already been posted to
other IME process. They use undefined bit of GdkEventKey::state to store
if the key event has already been posted (1 << 25, they called "ignored" flag).
Although their approach is really hacky but we can refer the information at
least for now.
Finally, when we guess a key event is posted to other IME process, let's
IMContextWrapper::OnKeyEvent() not dispatch eKeyDown nor eKeyUp event.
Note that if it's handled synchronously as unexpected, it may causes
dispatching one or more composition events and/or delete content event.
So, in such case, we dispatch a keyboard event for processing key event
anyway. There is only once case we'll fail to dispatch keyboard event.
If we receive signals to dispatch composition events or delete content
command event when IM receives the result from other IME process but
it doesn't send the key event to us. This will be fixed by the following
patch.
MozReview-Commit-ID: 94PrlnmQ3uJ
--HG--
extra : rebase_source : 0bb58ed432bacef8ad13264babd2b21fe950b71c
On Linux, dead key is implemented with "table-based input methods" which are
available even on GtkIMContextSimple (i.e., available even in password fields).
Therefore, IMContextWrapper handles dead key sequence as usual composition of
IME. However, on the other platforms, we dispatch "Dead" eKeyDown and eKeyUp
events for dead key.
We started to mark keyboard events which are handled by IME as "processed by
IME" since bug 1343451, i.e., we started to set mKeyNameIndex to
KEY_NAME_INDEX_Process. However, we should keep previous behavior, i.e., keep
setting it to KEY_NAME_INDEX_Dead. Fortunately, GDK's key event tells us
whether the keyboard event is a dead key event with keysym. So, we can detect
if we're in a dead key sequence simply.
MozReview-Commit-ID: Dv336WptfXN
--HG--
extra : rebase_source : e8a7b5a7eb7c57e1e45de20ebebd56f88457cfc6
Currently, IMContextWrapper doesn't dispatch eKeyDown event and eKeyUp event
if it's handled by IME. However, for conforming to UI Events, it should
not eat given keyboard events completely.
This patch makes IMContextWrapper dispatches eKeyDown event or eKeyUp event
before dispatching first event of composition events or content command
event.
MozReview-Commit-ID: H2jHpViTH5Q
--HG--
extra : rebase_source : a1f4127ba87e03e1ff97690f97fb7bf64b4d4818
Currently, IMContextWrapper doesn't dispatch eKeyDown event and eKeyUp event
if it's handled by IME. However, for conforming to UI Events, it should
not eat given keyboard events completely.
This patch makes IMContextWrapper dispatches eKeyDown event or eKeyUp event
before dispatching first event of composition events or content command
event.
MozReview-Commit-ID: H2jHpViTH5Q
--HG--
extra : rebase_source : 4129620126a34e27af1503e7c4652bb09c7e9bb6
If a keydown event handler moves focus like Ctrl+PageDown handler, IM context
may be changed to DISABLED or something. In such case, native IME would stop
current composition because focus moving in Gecko causes making IME blurred.
However, IMContextWrapper::DispatchCompositionStart() always dispatches
eCompositionStart even in such case.
So, it should stop dispatching eCompositionStart if IME enabled state is
changed during dispatching the preceding keydown event.
Note that this patch moves the setter of mComposingContext from
OnStartCompositionNative() which is a signal listener of "preedit_start" to
DispatchCompositionStart() because if IME starts composition without
"preedit_start" signal, DispatchCompositionStart() will be called but
OnStartCompositionNative() isn't called. However, this fix needs
mComposingContext.
MozReview-Commit-ID: F3F6NuCOrkJ
--HG--
extra : rebase_source : 513528eba0f29eb9b6ce8c5f47e4badbde9cbdb8
When you start new composition during converting with Mozc in e10s mode, the following things occur:
1. Mozc commits previous composition.
2. Gecko dispatches eCompositionCommit event.
3. Mozc sets new composition string (skipping composition start signal).
4. Gecko dispatches eCompositionStart and eCompositionChange event.
5. Selection is changed asynchronously.
6. Gecko sets position of IME windows.
At #4, Gecko stores start of composition as selection start, then, trying to adjust it at #5. However, new selection is caret position in new composition string. Therefore, it's not used for the adjustment. This causes that stored composition start offset is always the start of the previous composition (if the previous patch didn't change EnsureToCacheSelection() behavior). So, IMContextWrapper needs to compute proper composition start offset in this case.
The simplest fix is, modifying selection at #2 as which will be occurred in focused editor. So, this patch makes the selection cache collapsed to the end of committing string.
Note that actual selection may be different if JS changes selection and/or the text in the focused editor. However, it doesn't matter. IMContextWrapper should behave as expected while current composition is active.
MozReview-Commit-ID: 221mDUd8yRP
--HG--
extra : rebase_source : 571b2de85ed6ea1fdadea73b7f95507937cc60e9
IMContextWrapper::EnsureToCacheSelection() always queries actual selection when the caller needs selected string. However, this may be expensive and this is bad behavior for the following patch because it wants to emulate selection range until receiving next selection change notification.
Therefore, this patch makes IMContextWrapper::Selection store selected string instead of just its length like other native IME handlers
Additionally, this patch renames IMContextWrapper::mSelectedString to IMContextWrapper::mSelectedStringRemovedByComposition for making the difference between it and the new string in Selection clearer.
MozReview-Commit-ID: 3bygvW7sKf4
--HG--
extra : rebase_source : b0835b8c1607ecd647444a4d984980943a6fd570
Bug 1343075 - 1a. Add TextEventDispatcherListener::GetIMEUpdatePreference; r=masayuki
Add a GetIMEUpdatePreference method to TextEventDispatcherListener to
optionally control which IME notifications are received by NotifyIME.
This patch also makes nsBaseWidget forward its GetIMEUpdatePreference
call to the widget's native TextEventDispatcherListener.
Bug 1343075 - 1b. Implement GetIMEUpdatePreference for all TextEventDispatcherListener; r=masayuki
This patch implements GetIMEUpdatePreference for all
TextEventDispatcherListener implementations, by moving previous
implementations of nsIWidget::GetIMEUpdatePreference.
Bug 1343075 - 2. Allow setting a PuppetWidget's native TextEventDispatcherListener; r=masayuki
In PuppetWidget, add getter and setter for the widget's native
TextEventDispatcherListener. This allows overriding of PuppetWidget's
default IME handling. For example, on Android, the PuppetWidget's native
TextEventDispatcherListener will communicate directly with Java IME code
in the main process.
Bug 1343075 - 3. Add AIDL interface for main process; r=rbarker
Add AIDL definition and implementation for an interface for the main
process that child processes can access.
Bug 1343075 - 4. Set Gecko thread JNIEnv for child process; r=snorp
Add a JNIEnv* parameter to XRE_SetAndroidChildFds, which is used to set
the Gecko thread JNIEnv for child processes. XRE_SetAndroidChildFds is
the only Android-specific entry point for child processes, so I think
it's the most logical place to initialize JNI.
Bug 1343075 - 5. Support multiple remote GeckoEditableChild; r=esawin
Support remote GeckoEditableChild instances that are created in the
content processes and connect to the parent process GeckoEditableParent
through binders.
Support having multiple GeckoEditableChild instances in GeckoEditable by
keeping track of which child is currently focused, and only allow
calls to/from the focused child by using access tokens.
Bug 1343075 - 6. Add method to get GeckoEditableParent instance; r=esawin
Add IProcessManager.getEditableParent, which a content process can call
to get the GeckoEditableParent instance that corresponds to a given
content process tab, from the main process.
Bug 1343075 - 7. Support GeckoEditableSupport in content processes; r=esawin
Support creating and running GeckoEditableSupport attached to a
PuppetWidget in content processes.
Because we don't know PuppetWidget's lifetime as well as nsWindow's,
when attached to PuppetWidget, we need to attach/detach our native
object on focus/blur, respectively.
Bug 1343075 - 8. Connect GeckoEditableSupport on PuppetWidget creation; r=esawin
Listen to the "tab-child-created" notification and attach our content
process GeckoEditableSupport to the new PuppetWidget.
Bug 1343075 - 9. Update auto-generated bindings; r=me
ibus-pinyin has a bug. When application calls gtk_im_context_reset(), which means selection is changed in application, ibus-pinyin sents a set of composition signals with empty commit string. Therefore, selecting text causes removing it.
For preventing it but not breaking the other IMEs which use surrounding text, we should give up to call gtk_im_context_reset() if IME hasn't retrieved surrounding text after the last selection change. Not having retrieved surrounding text means that the IME doesn't have any cache of contents. Therefore, not calling gtk_im_context_reset() at selection change must be safe for such IMEs.
MozReview-Commit-ID: 5cbIZjpd7zN
--HG--
extra : rebase_source : 6010b3e055d66ebd2ed50f9b3ee8ff2330d3c6ab
Masayuki Nakano
Tooru Fujisawa
Sylvestre Ledru
Dorel Luca
Cosmin Sabou
Jim Chen
Jonathan Watt