Skip to content

[Feature]: Add support for mouse/touch/keyboard UI interactions #498

@dotMorten

Description

@dotMorten

Is your feature request related to a problem? Please describe.

The latest UI Automation features added are great but they are lacking a set of features that involves testing interactions are correctly handled:

  • Keyboard input
  • Mouse input (hover, drag, left/right-click, scroll-wheel, double-click)
  • Touch input (drag, tap, double-tap, pinch, stretch, tap-and-hold)
  • Pen Input (hover, drag, tap)

These are important for testing controls that respond to these events, like UIs where you can drag things around, open context menus on certain locations, zoom in with mouse wheel/touch, test accessibility inputs by using tag keys and ensure things can get focus etc.

Describe the solution you'd like

Some examples of commands:
Tab an image at location (40,50dips) of MyImage control
winapp ui tap MyImage 40,50 -inputtype pen -a myapp

One mouse wheel scroll amount if -120:
winapp ui scroll MyImage -120 -a myapp

Pinch stretch around 40,50 4x:
winapp ui pinch MyImage 40,50 4 -a myapp

Drag from 40,50 to 60,30:
winapp ui drag MyImage 40,50 60,30 -inputtype touch -a myapp

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions