Qt WebEngine incompatible with Selenium's "sendKeys" method

Problem

Selenium does not seem able to send key strokes to web engines.  In Qt5.10 (last version we were using) the result was a no-op. Just nothing happened. In 5.15 attempting to send the events actually crashes the process. It looks to me (someone fairly unfamiliar with this code) like this is an issue with how Qt is implementing NativeWebKeyboardEvent.

This unanswered question in the Qt forum appears to be another report of the behavior. 

Investigation

When Selenium sends a key event to the web browser it eventually gets picked up by content::protocol::InputHandler::DispatchKeyEvent (chromium/content/browser/devtools/protocol/input_handler.cc). This DispatchKeyEvent creates a NativeWebKeyboardEvent:

  NativeWebKeyboardEvent event(

      web_event_type,

      GetEventModifiers(modifiers.fromMaybe(blink::WebInputEvent::kNoModifiers),

                        auto_repeat.fromMaybe(false),

                        is_keypad.fromMaybe(false), location.fromMaybe(0), 0),

      GetEventTimeTicks(timestamp));

That event looks to be implemented by each platform chromium is built for. There's also a qt  implementation (src/core/native_web_keyboard_event_qt.cpp). It looks to completely ignore the type that's passed into this ctor:

NativeWebKeyboardEvent::NativeWebKeyboardEvent(blink::WebInputEvent::Type, int, base::TimeTicks)

    : os_event(0)

    , skip_in_browser(false)

{

}

Which means the above event is initialized with type "kUninitialized". The event gets pushed through a few methods before it eventually ends up in RenderWidgetHostImpl::ForwardKeyboardEventWithCommands (hromium/content/browser/renderer_host/render_widget_host_impl.cc). It appears that this is where the disparity in behavior in 5.10 vs 5.15  comes from. In 5.10 there was an early return checking for bad inputs:

  // Double check the type to make sure caller hasn't sent us nonsense that

  // will mess up our key queue.

  if (!WebInputEvent::IsKeyboardEventType(key_event.GetType()))

    return;

The chromium code in 5.15 however just has a debug check for it:

DCHECK(WebInputEvent::IsKeyboardEventType(key_event.GetType()));

In debug it will crash at that assertion, in release it will actually continue on a little bit until you hit a nullptr exception.

How it's crashing leads me to think that this may be a problem with how NativeWebKeyboardEvent is implemented - namely that it doesn't actually set the type of the event to a real value. Might be a naïve interpretation but seems problematic.

Workaround

We're basically getting around this by manually injecting Javascript in to set text. That's really not ideal.

Repro

The repro is set up to run against 5.15.0 with its version of chromedriver. You'd have to find new drivers to run it against 5.10.

The Qt app is essentially a single file which displays an Html page. The Java app is actually one file which just connects to the Qt app, finds  the textarea, and sends it some keys. The sources are attached as well as, in the case of the Java, a packaged Jar.

1. Download, build, and run sample Qt App
2. Download the attached Jar and chromedriver.exe. Co-locate them in a directory.
3. (while Qt app is running) Run the Java app: java -jar <path to jar>

Expected: New text is entered

Actual (5.15): Crash

Actual (5.10): Nothing

(Most of) Stack from our application

> [Inline Frame] Qt5WebEngineCore.dll!mojo::internal::Serialize(blink::WebInputEvent::Type &&) Line 44 C+> [Inline Frame] Qt5WebEngineCore.dll!mojo::internal::Serialize(blink::WebInputEvent::Type &&) Line 44 C  Qt5WebEngineCore.dll!mojo::internal::Serializer<content::mojom::EventDataView,std::unique_ptr<content::InputEvent,std::default_delete<content::InputEvent> > >::Serialize(std::unique_ptr<content::InputEvent,std::default_delete<content::InputEvent> > & input, mojo::internal::Buffer * buffer, content::mojom::internal::Event_Data::BufferWriter * output, mojo::internal::SerializationContext * context) Line 1638 C  [Inline Frame] Qt5WebEngineCore.dll!mojo::internal::Serialize(std::unique_ptr<content::InputEvent,std::default_delete<content::InputEvent> > &) Line 44 C  Qt5WebEngineCore.dll!content::mojom::WidgetInputHandlerProxy::DispatchEvent(std::unique_ptr<content::InputEvent,std::default_delete<content::InputEvent> > in_event, base::OnceCallback<void _cdecl(enum content::InputEventAckSource,ui::LatencyInfo const &,enum content::InputEventAckState,base::Optional<ui::DidOverscrollParams> const &,base::Optional<enum cc::TouchAction> const &)> callback) Line 1607 C  Qt5WebEngineCore.dll!content::InputRouterImpl::FilterAndSendWebInputEvent(const blink::WebInputEvent & input_event, const ui::LatencyInfo & latency_info, base::OnceCallback<void __cdecl(enum content::InputEventAckSource,ui::LatencyInfo const &,enum content::InputEventAckState,base::Optional<ui::DidOverscrollParams> const &,base::Optional<enum cc::TouchAction> const &)> callback) Line 540 C  Qt5WebEngineCore.dll!content::InputRouterImpl::SendKeyboardEvent(const content::EventWithLatencyInfo<content::NativeWebKeyboardEvent> & key_event, base::OnceCallback<void __cdecl(content::EventWithLatencyInfo<content::NativeWebKeyboardEvent> const &,enum content::InputEventAckSource,enum content::InputEventAckState)> event_result_callback) Line 134 C  Qt5WebEngineCore.dll!content::RenderWidgetHostImpl::ForwardKeyboardEventWithCommands(const content::NativeWebKeyboardEvent & key_event, const ui::LatencyInfo & latency, const std::vector<content::EditCommand,std::allocator<content::EditCommand> > * commands, bool * update_event) Line 1431 C  Qt5WebEngineCore.dll!content::RenderWidgetHostImpl::ForwardKeyboardEventWithLatencyInfo(const content::NativeWebKeyboardEvent & key_event, const ui::LatencyInfo & latency) Line 1340 C  Qt5WebEngineCore.dll!content::RenderWidgetHostImpl::ForwardKeyboardEvent(const content::NativeWebKeyboardEvent & key_event) Line 1334 C  Qt5WebEngineCore.dll!content::protocol::InputHandler::InputInjector::InjectKeyboardEvent(const content::NativeWebKeyboardEvent & keyboard_event, std::unique_ptr<content::protocol::Input::Backend::DispatchKeyEventCallback,std::default_delete<content::protocol::Input::Backend::DispatchKeyEventCallback> > callback) Line 398 C  Qt5WebEngineCore.dll!content::protocol::InputHandler::DispatchKeyEvent(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & type, crdtp::glue::detail::ValueMaybe<int> modifiers, crdtp::glue::detail::ValueMaybe<double> timestamp, crdtp::glue::detail::ValueMaybe<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > text, crdtp::glue::detail::ValueMaybe<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > unmodified_text, crdtp::glue::detail::ValueMaybe<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > key_identifier, crdtp::glue::detail::ValueMaybe<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > code, crdtp::glue::detail::ValueMaybe<std::basic_string<char,std::char_traits<char>,std::allocator<char> > > key, crdtp::glue::detail::ValueMaybe<int> windows_virtual_key_code, crdtp::glue::detail::ValueMaybe<int> native_virtual_key_code, crdtp::glue::detail::ValueMaybe<bool> auto_repeat, crdtp::glue::detail::ValueMaybe<bool> is_keypad, crdtp::glue::detail::ValueMaybe<bool> is_system_key, crdtp::glue::detail::ValueMaybe<int> location, std::unique_ptr<content::protocol::Input::Backend::DispatchKeyEventCallback,std::default_delete<content::protocol::Input::Backend::DispatchKeyEventCallback> > callback) Line 625 C  Qt5WebEngineCore.dll!content::protocol::Input::DispatcherImpl::dispatchKeyEvent(int callId, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & method, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & message, std::unique_ptr<content::protocol::DictionaryValue,std::default_delete<content::protocol::DictionaryValue> > requestMessageObject, content::protocol::ErrorSupport * errors) Line 348 C  Qt5WebEngineCore.dll!ui_devtools::protocol::DOM::DispatcherImpl::dispatch(int callId, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & method, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & message, std::unique_ptr<ui_devtools::protocol::DictionaryValue,std::default_delete<ui_devtools::protocol::DictionaryValue> > messageObject) Line 310 C  Qt5WebEngineCore.dll!content::protocol::UberDispatcher::dispatch(int callId, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & in_method, std::unique_ptr<content::protocol::Value,std::default_delete<content::protocol::Value> > parsedMessage, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & rawMessage) Line 1126 C  Qt5WebEngineCore.dll!content::DevToolsSession::HandleCommand(std::unique_ptr<content::protocol::DictionaryValue,std::default_delete<content::protocol::DictionaryValue> > value, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & message) Line 250 C  [Inline Frame] Qt5WebEngineCore.dll!base::internal::FunctorTraits<void (cdecl content::protocol::PageHandler::)(std::unique_ptr<content::protocol::Page::ScreencastFrameMetadata,std::default_delete<content::protocol::Page::ScreencastFrameMetadata> >,content::protocol::Binary const &),void>::Invoke(void(content::protocol::PageHandler::)(std::unique_ptr<content::protocol::Page::ScreencastFrameMetadata,std::default_delete<content::protocol::Page::ScreencastFrameMetadata> >, const content::protocol::Binary &)) Line 498 C  Qt5WebEngineCore.dll!base::internal::InvokeHelper<1,void>::MakeItSo<void (cdecl content::protocol::PageHandler::)(std::unique_ptr<content::protocol::Page::ScreencastFrameMetadata,std::default_delete<content::protocol::Page::ScreencastFrameMetadata> >,content::protocol::Binary const &),base::WeakPtr<content::protocol::PageHandler>,std::unique_ptr<content::protocol::Page::ScreencastFrameMetadata,std::default_delete<content::protocol::Page::ScreencastFrameMetadata> >,content::protocol::Binary const &>(void(content::protocol::PageHandler::)(std::unique_ptr<content::protocol::Page::ScreencastFrameMetadata,std::default_delete<content::protocol::Page::ScreencastFrameMetadata> >, const content::protocol::Binary &) && functor, base::WeakPtr<content::protocol::PageHandler> && weak_ptr, std::unique_ptr<content::protocol::Page::ScreencastFrameMetadata,std::default_delete<content::protocol::Page::ScreencastFrameMetadata> > && <args_0>, const content::protocol::Binary & <args_1>) Line 621 C  [Inline Frame] Qt5WebEngineCore.dll!base::OnceCallback<void __cdecl(std::basic_string<char,std::char_traits<char>,std::allocator<char> > const &)>::Run(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > &) Line 98 C  Qt5WebEngineCore.dll!content::DevToolsManagerDelegate::HandleCommand(content::DevToolsAgentHost * agent_host, content::DevToolsAgentHostClient * client, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & method, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & message, base::OnceCallback<void __cdecl(std::basic_string<char,std::char_traits<char>,std::allocator<char> > const &)> callback) Line 67 C  Qt5WebEngineCore.dll!content::DevToolsSession::DispatchProtocolMessageInternal(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & message, std::unique_ptr<content::protocol::DictionaryValue,std::default_delete<content::protocol::DictionaryValue> > value) Line 228 C  [Inline Frame] Qt5WebEngineCore.dll!std::_String_val<std::_Simple_types<char> >::_Large_string_engaged() Line 1825 C  [Inline Frame] Qt5WebEngineCore.dll!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Tidy_deallocate() Line 3987 C  [Inline Frame] Qt5WebEngineCore.dll!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::{dtor}() Line 2460 C  Qt5WebEngineCore.dll!content::DevToolsSession::DispatchProtocolMessage(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & message) Line 213 C  [Inline Frame] Qt5WebEngineCore.dll!content::DevToolsAgentHostClientImpl::OnMessage(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > &) Line 364 C  Qt5WebEngineCore.dll!content::DevToolsHttpHandler::OnWebSocketMessage(int connection_id, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & data) Line 753 C  [Inline Frame] Qt5WebEngineCore.dll!base::internal::FunctorTraits<void (cdecl viz::ClientGpuMemoryBufferManager::)(gfx::GenericSharedMemoryId,gpu::SyncToken const &),void>::Invoke(void(viz::ClientGpuMemoryBufferManager::)(gfx::GenericSharedMemoryId, const gpu::SyncToken &) method, base::WeakPtr<viz::ClientGpuMemoryBufferManager> &&) Line 498 C  Qt5WebEngineCore.dll!base::internal::InvokeHelper<1,void>::MakeItSo<void (_cdecl viz::ClientGpuMemoryBufferManager::)(gfx::GenericSharedMemoryId,gpu::SyncToken const &),base::WeakPtr<viz::ClientGpuMemoryBufferManager>,gfx::GenericSharedMemoryId,gpu::SyncToken const &>(void(viz::ClientGpuMemoryBufferManager::)(gfx::GenericSharedMemoryId, const gpu::SyncToken &) && functor, base::WeakPtr<viz::ClientGpuMemoryBufferManager> && weak_ptr, gfx::GenericSharedMemoryId && <args_0>, const gpu::SyncToken & <args_1>) Line 621 C  [Inline Frame] Qt5WebEngineCore.dll!base::OnceCallback<void __cdecl(void)>::Run() Line 98 C  Qt5WebEngineCore.dll!base::TaskAnnotator::RunTask(const char * trace_event_name, base::PendingTask * pending_task) Line 142 C  Qt5WebEngineCore.dll!base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl(base::sequence_manager::LazyNow * continuation_lazy_now, bool * ran_task) Line 366 C  Qt5WebEngineCore.dll!base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork() Line 270 C+