Captcha Solving

Automate captcha solving using PhoneClaw vision capabilities

Overview

PhoneClaw can automate captcha solving using its vision-assisted UI targeting and screen understanding capabilities. Watch the captcha automation demo to see it in action.

How It Works

PhoneClaw uses the magicClicker and magicScraper functions to:

• Identify captcha elements on screen
• Read captcha text or identify images
• Click correct answers or input solutions
• Verify successful completion

Basic Captcha Solving

1. Detect Captcha

   Use magicScraper to identify if a captcha is present.

   const captchaPresent = magicScraper("Is there a captcha verification on screen?")
   
   if (captchaPresent.includes("yes") || captchaPresent.includes("captcha")) {
     speakText("Captcha detected, solving now")
   }
   
2. Read Captcha Content

   Extract the captcha challenge using vision.

   const challenge = magicScraper("What is the captcha asking?")
   speakText(`Challenge: ${challenge}`)
   
3. Solve and Submit

   Click the correct answer or input the solution.

   magicClicker("The correct captcha answer")
   delay(1000)
   magicClicker("Submit button")
   delay(2000)
   

Text-Based Captcha Solving

Solve simple text captchas that require reading and typing:

// Solve text-based captcha
const solveTextCaptcha = () => {
  // Check if captcha is present
  const hasCaptcha = magicScraper("Is there a text captcha on screen?")
  
  if (!hasCaptcha.includes("yes")) {
    speakText("No captcha detected")
    return
  }
  
  // Read the captcha text
  const captchaText = magicScraper("What text is shown in the captcha image?")
  speakText(`Captcha text: ${captchaText}`)
  
  // Click input field
  magicClicker("Captcha text input field")
  delay(1000)
  
  // Type the captcha text (using your input method)
  // typeText(captchaText)
  
  // Submit
  magicClicker("Verify button")
  delay(2000)
  
  // Verify success
  const success = magicScraper("Does the screen show verification successful?")
  
  if (success.includes("yes") || success.includes("success")) {
    speakText("Captcha solved successfully")
    return true
  } else {
    speakText("Captcha failed, retrying")
    return false
  }
}

solveTextCaptcha()

Image Selection Captcha

Solve image-based captchas that require selecting specific images:

// Solve image selection captcha
const solveImageCaptcha = () => {
  // Read the challenge
  const challenge = magicScraper("What objects should be selected in the captcha?")
  speakText(`Looking for: ${challenge}`)
  
  // Find and click matching images
  const imageCount = safeInt(magicScraper("How many images are in the grid?"), 9)
  
  for (let i = 1; i <= imageCount; i++) {
    const hasTarget = magicScraper(`Does image ${i} contain ${challenge}?`)
    
    if (hasTarget.includes("yes")) {
      magicClicker(`Image number ${i} in the captcha grid`)
      delay(500)
      speakText(`Selected image ${i}`)
    }
  }
  
  // Submit selection
  delay(1000)
  magicClicker("Verify button")
  delay(2000)
  
  speakText("Image captcha submitted")
}

solveImageCaptcha()

ReCAPTCHA Automation

Automate Google reCAPTCHA v2 checkboxes:

// Handle reCAPTCHA checkbox
const solveReCaptcha = () => {
  speakText("Solving reCAPTCHA")
  
  // Click the "I'm not a robot" checkbox
  magicClicker("I'm not a robot checkbox")
  delay(3000)
  
  // Check if additional challenge appeared
  const needsChallenge = magicScraper("Is there an image selection challenge visible?")
  
  if (needsChallenge.includes("yes")) {
    speakText("Additional challenge detected")
    
    // Read the challenge type
    const challengeType = magicScraper("Select all images with what object?")
    speakText(`Challenge: ${challengeType}`)
    
    // Solve image challenge
    solveImageCaptcha()
  } else {
    speakText("ReCAPTCHA passed without challenge")
  }
  
  delay(2000)
}

solveReCaptcha()

Retry Logic for Failed Attempts

Implement retry logic for captchas that may fail:

// Captcha solver with retry logic
const solveCaptchaWithRetry = (maxAttempts = 3) => {
  let attempts = 0
  
  while (attempts < maxAttempts) {
    attempts++
    speakText(`Captcha attempt ${attempts} of ${maxAttempts}`)
    
    // Try to solve the captcha
    const success = solveTextCaptcha()
    
    if (success) {
      speakText("Captcha solved successfully")
      return true
    }
    
    delay(2000)
  }
  
  speakText("Failed to solve captcha after maximum attempts")
  
  // Send notification email
  sendAgentEmail(
    "[e-mail]",
    "Captcha Solving Failed",
    `Failed to solve captcha after ${maxAttempts} attempts. Manual intervention may be required.`
  )
  
  return false
}

solveCaptchaWithRetry()

Complete Workflow with Captcha Handling

Integrate captcha solving into a larger automation workflow:

// Complete workflow with captcha handling
const automateWithCaptcha = () => {
  // Navigate to target page
  magicClicker("Browser app")
  delay(2000)
  
  // Fill out form
  magicClicker("Username field")
  delay(1000)
  // Type username...
  
  magicClicker("Password field")
  delay(1000)
  // Type password...
  
  // Submit form
  magicClicker("Submit button")
  delay(3000)
  
  // Check for captcha
  const hasCaptcha = magicScraper("Is there a captcha on screen?")
  
  if (hasCaptcha.includes("yes")) {
    speakText("Captcha detected, solving")
    
    const solved = solveCaptchaWithRetry(3)
    
    if (!solved) {
      speakText("Cannot proceed, captcha failed")
      return
    }
  }
  
  // Continue with workflow after captcha
  delay(2000)
  const loggedIn = magicScraper("Is the user logged in?")
  
  if (loggedIn.includes("yes")) {
    speakText("Successfully logged in")
    // Continue automation...
  }
}

automateWithCaptcha()

Best Practices

• Always verify captcha solution before proceeding
• Implement retry logic for failed attempts
• Use appropriate delays to allow captchas to load
• Test thoroughly as captcha layouts vary by site
• Consider notifying admins when captchas consistently fail
• Respect website terms of service and use automation responsibly

Limitations

• Complex audio captchas may require additional handling
• Some advanced captcha systems may be difficult to automate
• Captcha providers may update their systems, requiring script adjustments
• Rate limiting may apply to automated captcha attempts

See Also

• Account Creation Automation
• ClawScript API Reference