Node Operations

Methods for finding, clicking, and interacting with UI elements

Overview

Node operations allow you to find and interact with UI elements (nodes) in the accessibility tree. These methods provide various search strategies to locate elements by content description, view ID, text, class name, and more.

Finding and Clicking Elements

Click by Content Description

Find and click an element by its content description (accessibility label).

fun clickByDesc(description: String)

description · String · required

The content description text to search for (case-insensitive, partial match)

Example:

val service = MyAccessibilityService.instance
service?.clickByDesc("Submit")
service?.clickByDesc("Next")

This method searches recursively through the entire UI tree and clicks the first clickable element whose content description contains the specified text.

---

Click by View ID

Click element(s) by their Android resource ID.

fun clickElementByViewId(viewId: String): Boolean

viewId · String · required

The full resource ID (e.g., “com.example.app:id/button_submit”)

returns · Boolean

true if at least one element was clicked, false otherwise

Example:

val service = MyAccessibilityService.instance
val success = service?.clickElementByViewId("com.zhiliaoapp.musically:id/upload_button")
if (success == true) {
    println("Upload button clicked")
}

By default, this method clicks up to the first 3 elements with the specified view ID, with 120ms delay between clicks. This prevents duplicate clicks on the same element.

Advanced View ID Methods

Click Multiple View IDs (Fallback)

fun clickElementByViewIds(vararg viewIds: String): Boolean

Tries multiple view IDs until one succeeds:

service?.clickElementByViewIds(
    "com.app:id/button_v1",
    "com.app:id/button_v2",
    "com.app:id/button_legacy"
)

Click All Elements with View ID

fun clickAllElementsByViewId(viewId: String): Int

Clicks every element with the specified view ID (returns count of clicked elements).

---

Click by Text Label

Find and click an element by its exact text content.

fun clickButtonWithLabel(label: String)

label · String · required

The exact text label to search for (case-sensitive)

Example:

service?.clickButtonWithLabel("Sign In")
service?.clickButtonWithLabel("Accept")

---

Click by Content Description (Collection)

Click the first element with an exact content description match.

fun clickNodesByContentDescription(targetContentDesc: String)

targetContentDesc · String · required

The exact content description to match

Example:

service?.clickNodesByContentDescription("Profile menu")

---

Advanced Node Finding

Find by Class Name and Index

Find a node by its class name and position in the tree.

fun findNodeByClassNameAndIndex(className: String, index: Int): AccessibilityNodeInfo?

className · String · required

The fully qualified class name (e.g., “android.widget.Button”)

index · Int · required

Zero-based index of the element (in pre-order traversal)

returns · AccessibilityNodeInfo?

The found node, or null if not found

Example:

// Find the third Button in the tree
val button = service?.findNodeByClassNameAndIndex("android.widget.Button", 2)
if (button != null) {
    service?.performNodeClick(button)
}

---

Find by Class, Index, and String

Find a node matching class name, sibling index, and optional text content.

fun findNodeByClassNameAndIndexAndString(
    root: AccessibilityNodeInfo? = rootInActiveWindow,
    className: String,
    targetIndex: Int,
    substring: String?
): AccessibilityNodeInfo?

root · AccessibilityNodeInfo? · default: "rootInActiveWindow"

The root node to start searching from

className · String · required

The fully qualified class name to match

targetIndex · Int · required

The target index among siblings (relative position in parent). Use -1 to ignore index.

substring · String? · required

Optional text/contentDescription that the node should contain. Use null to ignore.

returns · AccessibilityNodeInfo?

The first matching node, or null if not found

Example:

// Find the second RelativeLayout that contains "Settings"
val node = service?.findNodeByClassNameAndIndexAndString(
    className = "android.widget.RelativeLayout",
    targetIndex = 1,
    substring = "Settings"
)

---

Clicking Mechanisms

Perform Node Click

Attempt to click a node by traversing up to find a clickable parent.

fun performNodeClick(node: AccessibilityNodeInfo)

node · AccessibilityNodeInfo · required

The node to click (or its nearest clickable parent)

Example:

val node = service?.findNodeByClassNameAndIndex("android.widget.TextView", 0)
if (node != null) {
    service?.performNodeClick(node)
}

---

Click at Approximate Coordinates

Click any clickable element near specific coordinates.

fun clickAtApproximateCoordinates(x: Float, y: Float, tolerance: Float = 50f): Boolean

x · Float · required

The X coordinate in pixels

y · Float · required

The Y coordinate in pixels

tolerance · Float · default: "50f"

Search radius in pixels

returns · Boolean

true if a clickable element was found and clicked, false otherwise

Example:

// Click near position (500, 800) with 100px tolerance
service?.clickAtApproximateCoordinates(500f, 800f, 100f)

---

Text Input Operations

Type in First Editable Field

Find the first editable text field and enter text.

fun simulateTypeInFirstEditableField(inputText: String)

inputText · String · required

The text to enter into the field

Example:

service?.simulateTypeInFirstEditableField("[e-mail]")

This method uses breadth-first search to find the first EditText that supports ACTION_SET_TEXT. The field is focused before text is entered.

---

Type in Second Editable Field

Find the second editable text field and enter text.

fun simulateTypeInSecondEditableField(inputText: String)

inputText · String · required

The text to enter into the second field

Example:

// Type username in first field, password in second field
service?.simulateTypeInFirstEditableField("username")
service?.simulateTypeInSecondEditableField("password123")

---

Type in Third Editable Field

fun simulateTypeInThirdEditableField(inputText: String)

Same as second field, but targets the third editable field in traversal order.

---

Type by Resource ID

Enter text into a field identified by its resource ID.

fun simulateType(resourceId: String, text: String)

resourceId · String · required

The full resource ID of the input field

text · String · required

The text to enter

Example:

service?.simulateType("com.example.app:id/email_field", "[e-mail]")

---

Type by Class Name

Enter text into all fields matching a class name.

fun simulateTypeByClass(nodeClass: String, inputText: String)

nodeClass · String · required

The class name to match (e.g., “android.widget.EditText”)

inputText · String · required

The text to enter into matching fields

Example:

service?.simulateTypeByClass("android.widget.EditText", "same text for all")

---

Text Extraction

Get All Text from Screen

Extract all visible text from the current screen.

fun getAllTextFromScreen(): String

returns · String

Concatenated text from all text nodes and content descriptions on screen

Example:

val screenText = service?.getAllTextFromScreen()
println("Screen contains: $screenText")

---

Check if Text is Present

Check if specific text exists anywhere on screen.

fun isTextPresentOnScreen(searchText: String): Boolean

searchText · String · required

The text to search for (case-insensitive, partial match)

returns · Boolean

true if the text is found in any node’s text or content description

Example:

if (service?.isTextPresentOnScreen("Success") == true) {
    println("Operation completed successfully")
}

---

Get Content Description

Get the content description of a node containing specific text.

fun getContentDescriptionForNodeContaining(searchString: String): String?

searchString · String · required

The substring to search for in node text or content description

returns · String?

The full content description of the found node, or null if not found

Example:

val desc = service?.getContentDescriptionForNodeContaining("Profile")
println("Found content description: $desc")

---

Keyboard Actions

Press Enter Key

Simulate pressing the Enter/Return key.

fun pressEnterKey()

Example:

service?.simulateTypeInFirstEditableField("search query")
service?.pressEnterKey()

This method tries three strategies in order:

1. Find and click a visible Enter/Done/Send button
2. Send ACTION_IME_ENTER to the focused field (API 33+)
3. Tap at the typical Enter key position as fallback

---

Deletion Operations

Delete by Resource ID

Clear text from a field identified by resource ID.

fun simulateDelete(resourceId: String)

resourceId · String · required

The resource ID of the field to clear

Example:

service?.simulateDelete("com.example.app:id/search_field")

---

Delete by Class Name

Clear text from all fields matching a class name.

fun simulateDeleteByClass(nodeClass: String)

nodeClass · String · required

The class name of fields to clear

Example:

service?.simulateDeleteByClass("android.widget.EditText")

---

Specialized Click Methods

Click Element by Area

Click an element based on its pixel area (width × height).

fun clickElementByArea(targetArea: Int)

targetArea · Int · required

The target area in square pixels

Example:

// Click element with area of 5390 square pixels (77x70)
service?.clickElementByArea(5390)

Helper method for width/height:

fun clickElementByDimensions(width: Int, height: Int)

---

Click Element by Area Range

Click an element whose area falls within a range.

fun clickElementByAreaRange(minArea: Int, maxArea: Int)

minArea · Int · required

Minimum area in square pixels

maxArea · Int · required

Maximum area in square pixels

Example:

service?.clickElementByAreaRange(5000, 6000)

---

App-Specific Methods

These methods are designed for specific apps but demonstrate advanced techniques:

Click First Gallery Item

Clicks the first thumbnail in a TikTok-style gallery view.

fun clickFirstGalleryItem()

Click Video Upload Button

Finds and clicks the video upload button using multiple fallback strategies.

fun clickVideoUploadButton()

Click First Song

Clicks the first item in a RecyclerView (music list).

fun clickFirstSong()

Click Bio Button

Clicks a button whose content description starts with “Bio”.

fun clickBioButton()

Find Toggle Near Text

Finds a toggle element (switch, checkbox) near text containing specific keywords.

fun findToggleNearText(text: String): AccessibilityNodeInfo?

---

Best Practices

Use the most specific search method available (View ID > Content Description > Text)

Always check for null results when using find methods

Add small delays between text input and clicking to allow UI to update

View IDs may change between app versions - test automation regularly

For production automation, prefer content descriptions over text as they’re more stable across localizations