Skip to main content

​
Overview

ClawScript provides a comprehensive set of functions for automating UI interactions on Android. These functions simulate touch gestures, keyboard input, and screen navigation.
All UI automation functions require Android Accessibility Service to be enabled. See Accessibility Service Setup.

​
Click & Tap Functions

​
simulateClick()

Simulates a tap at specific screen coordinates.
​
x
number
required
X coordinate in pixels from left edge of screen.
​
y
number
required
Y coordinate in pixels from top edge of screen.
// Click at coordinates (100, 200)
Android.simulateClick(100, 200);
Android.delay(1000);

// Click center of screen (assuming 800x1600 display)
Android.simulateClick(400, 800);

​
doubleClick()

Performs a double-tap gesture at specified coordinates.
​
x
number
required
X coordinate in pixels.
​
y
number
required
Y coordinate in pixels.
// Double tap to like
Android.doubleClick(400, 800);
Android.delay(500);

​
longPress()

Performs a long press (press and hold) gesture.
​
x
number
required
X coordinate in pixels.
​
y
number
required
Y coordinate in pixels.
// Long press to open context menu
Android.longPress(400, 600);
Android.delay(1500);

​
Content-Based Clicking

​
clickNodesByContentDescription()

Clicks elements by their accessibility content description.
​
description
string
required
Content description text to match (e.g., “Share”, “Like”, “Next”).
// Click by button label
Android.clickNodesByContentDescription("Next");
Android.delay(1000);

Android.clickNodesByContentDescription("Share");
Android.delay(1000);

Android.clickNodesByContentDescription("Send");
This is more reliable than coordinate-based clicking since UI layouts can vary across devices.

​
Typing & Text Input

​
simulateTypeInFirstEditableField()

Enters text in the first editable field on screen.
​
text
string
required
Text to type into the field.
// Type username
Android.simulateTypeInFirstEditableField("john_doe");
Android.delay(1000);

​
simulateTypeInSecondEditableField()

Enters text in the second editable field on screen.
​
text
string
required
Text to type into the field.
// Type password
Android.simulateTypeInSecondEditableField("myPassword123");
Android.delay(1000);

​
simulateType()

Types text in a field identified by View ID.
​
id
string
required
Android View ID of the target field.
​
text
string
required
Text to type.
// Type in specific field
Android.simulateType("com.example.app:id/username", "john_doe");
Android.delay(1000);

​
simulateTypeByClass()

Types text in a field by its class name.
​
className
string
required
Android class name (e.g., “android.widget.EditText”).
​
text
string
required
Text to type.
Android.simulateTypeByClass("android.widget.EditText", "Hello");

​
pressEnterKey()

Simulates pressing the Enter/Return key. 
// Submit form
Android.simulateTypeInFirstEditableField("search query");
Android.delay(500);
Android.pressEnterKey();
Android.delay(2000);

​
Swipe & Scroll Functions

​
swipeUp()

Simulates an upward swipe gesture (scrolling down the page). 
// Scroll feed
Android.swipeUp();
Android.delay(1000);
Android.swipeUp();
Android.delay(1000);

​
swipeDown()

Simulates a downward swipe gesture (scrolling up the page). 
// Pull to refresh
Android.swipeDown();
Android.delay(2000);

​
swipeLeft()

Simulates a left swipe gesture. 
// Navigate to next story
Android.swipeLeft();
Android.delay(800);

​
swipeRight()

Simulates a right swipe gesture. 
// Navigate to previous story
Android.swipeRight();
Android.delay(800);

​
simulateScrollToBottom()

Scrolls to the bottom of a scrollable view. 
// Scroll to end of page
Android.simulateScrollToBottom();
Android.delay(1500);

​
simulateScrollToTop()

Scrolls to the top of a scrollable view. 
// Return to top
Android.simulateScrollToTop();
Android.delay(1500);

​
View ID Interactions

​
clickElementByViewId()

Clicks an element by its Android View ID.
​
viewId
string
required
Full View ID (e.g., “com.instagram.android:id/profile_tab”).
Android.clickElementByViewId("com.example.app:id/submit_button");
Android.delay(1000);

​
Complete Workflow Examples

function loginToApp(username, password) {
  Android.speakText("Starting login");
  
  // Click username field
  Android.simulateClick(400, 600);
  Android.delay(500);
  
  // Enter username
  Android.simulateTypeInFirstEditableField(username);
  Android.delay(1000);
  
  // Click password field
  Android.simulateClick(400, 800);
  Android.delay(500);
  
  // Enter password
  Android.simulateTypeInSecondEditableField(password);
  Android.delay(1000);
  
  // Submit
  Android.pressEnterKey();
  Android.delay(3000);
  
  Android.speakText("Login complete");
}

loginToApp("john_doe", "myPassword123");

​
Screen Detection

​
isTextPresentOnScreen()

Checks if specific text is visible on screen using AI vision.
​
text
string
required
Text or description to search for.
​
return
boolean
Returns true if text is found, false otherwise.
// Check for error message
if (Android.isTextPresentOnScreen("Error")) {
  Android.speakText("Error detected");
  // Handle error
}

// Check for success
if (Android.isTextPresentOnScreen("Success")) {
  Android.speakText("Operation successful");
}

// Check for specific element
if (Android.isTextPresentOnScreen("Login")) {
  Android.speakText("Need to log in");
  loginToApp("user", "pass");
}
This function uses AI vision analysis and may take 1-2 seconds to complete. Use sparingly for critical checks.

​
Best Practices

UI elements need time to respond. Add 500-1500ms delays after every action.
Android.simulateClick(100, 200);
Android.delay(1000); // Essential!
Content description clicks are more reliable across devices.
// Good
Android.clickNodesByContentDescription("Share");

// Less reliable
Android.simulateClick(650, 1200);
Check screen state before performing actions.
if (Android.isTextPresentOnScreen("Login")) {
  // Perform login
} else {
  Android.speakText("Already logged in");
}
Wrap UI automation in try-catch blocks.
try {
  Android.simulateClick(100, 200);
  Android.delay(1000);
} catch (e) {
  Android.speakText("Click failed");
}

​
Limitations

  • Requires Accessibility Service enabled
  • Coordinates vary by device screen size and resolution
  • Some apps block accessibility interactions
  • Rapid interactions may be rate-limited
  • Background execution may be restricted by Android