Gestures and Touch Actions

Gestures and Touch Actions in Mobile Testing

With the deprecation of the TouchAction class in newer Appium versions, testers are encouraged to use the W3C Actions API for implementing gestures and touch actions in mobile testing. This API provides a standardized way to simulate user interactions and is part of the WebDriver specification. It supports advanced features such as multi-pointer actions and complex gesture simulations.

What are Gestures?

Gestures are user actions performed on a touchscreen, such as swiping, pinching, scrolling, long-pressing, and tapping. These gestures are essential for testing the functionality and usability of mobile applications.

• Tap: A quick touch on an element, equivalent to a mouse click.
• Swipe: A sliding motion used for navigation or revealing content.
• Long Press: A prolonged touch on an element, often to trigger context menus.
• Scroll: A vertical or horizontal motion to view hidden content.
• Drag and Drop: Moving an element from one location to another.

Key Classes in the W3C Actions API

• PointerInput: Represents a pointer device, such as a finger, for touch actions.
• Sequence: Defines a sequence of actions performed by a pointer device.
• Pause: Introduces a delay between actions.
• PointerMove: Moves a pointer to a specific location.
• PointerDown and PointerUp: Simulate pressing and releasing a touch point.

Example: Implementing Gestures Using the W3C Actions API

Here are examples of how to use the W3C Actions API to implement common gestures:

Tap Example

                  
              import org.openqa.selenium.interactions.PointerInput;
              import org.openqa.selenium.interactions.Sequence;
              
              import java.time.Duration;
              import java.util.Collections;
              
              public class TapExample {
                  public static void main(String[] args) {
                      // Assuming the driver is initialized
                      WebElement element = driver.findElement(By.id("test-element"));
              
                      PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger1");
                      Sequence tap = new Sequence(finger, 0);
              
                      tap.addAction(finger.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(),
                                                             element.getLocation().getX(), element.getLocation().getY()));
                      tap.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
                      tap.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));
              
                      driver.perform(Collections.singletonList(tap));
                  }
              }
                  
                

Swipe Example

                  
              PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger1");
              Sequence swipe = new Sequence(finger, 0);
              
              swipe.addAction(finger.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), 500, 1000));
              swipe.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
              swipe.addAction(finger.createPointerMove(Duration.ofSeconds(1), PointerInput.Origin.viewport(), 500, 500));
              swipe.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));
              
              driver.perform(Collections.singletonList(swipe));
                  
                

Long Press Example

                  
              PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger1");
              Sequence longPress = new Sequence(finger, 0);
              
              longPress.addAction(finger.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), 300, 600));
              longPress.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
              longPress.addAction(new Pause(finger, Duration.ofSeconds(2)));
              longPress.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));
              
              driver.perform(Collections.singletonList(longPress));
                  
                

Best Practices

• Use explicit waits to ensure elements are visible before interacting with them.
• Simulate gestures only when required, to keep scripts maintainable.
• Accurately define touch points using element locations or screen coordinates.

Multi-Touch Gestures

Multi-touch gestures involve using more than one finger to interact with the screen, such as pinch-to-zoom or two-finger swipe. These are especially useful in applications with map views, image galleries, or custom touch interfaces.

Classes Used for Multi-Touch

• PointerInput: Represents individual pointers (fingers).
• Sequence: Groups actions for a pointer into a sequence.
• perform(): Executes multiple sequences simultaneously for multi-touch actions.

Example: Pinch-to-Zoom Gesture

The following example demonstrates how to implement a pinch-to-zoom gesture using the W3C Actions API:

    
import org.openqa.selenium.interactions.PointerInput;
import org.openqa.selenium.interactions.Sequence;

import java.time.Duration;
import java.util.Arrays;

public class MultiTouchExample {
    public static void main(String[] args) {
        // Assuming the driver is initialized
        PointerInput finger1 = new PointerInput(PointerInput.Kind.TOUCH, "finger1");
        PointerInput finger2 = new PointerInput(PointerInput.Kind.TOUCH, "finger2");

        Sequence finger1Sequence = new Sequence(finger1, 0);
        Sequence finger2Sequence = new Sequence(finger2, 0);

        // Start positions for both fingers
        finger1Sequence.addAction(finger1.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), 300, 500));
        finger1Sequence.addAction(finger1.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));

        finger2Sequence.addAction(finger2.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), 300, 600));
        finger2Sequence.addAction(finger2.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));

        // Move fingers apart (zoom out)
        finger1Sequence.addAction(finger1.createPointerMove(Duration.ofSeconds(1), PointerInput.Origin.viewport(), 200, 400));
        finger1Sequence.addAction(finger1.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

        finger2Sequence.addAction(finger2.createPointerMove(Duration.ofSeconds(1), PointerInput.Origin.viewport(), 400, 700));
        finger2Sequence.addAction(finger2.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

        // Perform both sequences simultaneously
        driver.perform(Arrays.asList(finger1Sequence, finger2Sequence));
    }
}
    
  

Example: Two-Finger Swipe

Here is an example of a two-finger swipe gesture using the W3C Actions API:

    
PointerInput finger1 = new PointerInput(PointerInput.Kind.TOUCH, "finger1");
PointerInput finger2 = new PointerInput(PointerInput.Kind.TOUCH, "finger2");

Sequence finger1Swipe = new Sequence(finger1, 0);
Sequence finger2Swipe = new Sequence(finger2, 0);

// Start positions for both fingers
finger1Swipe.addAction(finger1.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), 300, 500));
finger1Swipe.addAction(finger1.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));

finger2Swipe.addAction(finger2.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), 300, 600));
finger2Swipe.addAction(finger2.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));

// Move both fingers together
finger1Swipe.addAction(finger1.createPointerMove(Duration.ofSeconds(1), PointerInput.Origin.viewport(), 300, 400));
finger1Swipe.addAction(finger1.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

finger2Swipe.addAction(finger2.createPointerMove(Duration.ofSeconds(1), PointerInput.Origin.viewport(), 300, 500));
finger2Swipe.addAction(finger2.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

// Perform both sequences simultaneously
driver.perform(Arrays.asList(finger1Swipe, finger2Swipe));
    
  

Best Practices

• Use accurate coordinates to define the touch points for gestures.
• Test multi-touch gestures on real devices, as emulators may not provide accurate results.
• Implement waits or pauses to ensure gestures are executed smoothly.