Senior Web Automation Engineer

We’re Hiring: Senior Web Automation Engineer (Adversarial Scraping / RegTech, B2B SaaS)

Location: Singapore/Malaysia (Hybrid)

Employment Type: Full Time (Monday to Friday (40 hrs/week)

Workplace Type: Hybrid

About Us

AsiaVerify is a Singapore-based regulatory technology company building automated compliance solutions across Asia and the UK. Our products power KYB, KYC, AML, and UBO workflows through APIs and data pipelines used by businesses that require fast, accurate, and reliable regulatory data.

Our core challenge is not just data collection — it’s reliably extracting structured data from complex, dynamic, and often protected systems at scale.

The Role

We’re looking for a Senior Web Automation Engineer to design, build, and operate robust data extraction systems targeting government registries, regulatory portals, and public data sources across multiple jurisdictions.

This is not a basic scraping role.

Many of our target systems actively attempt to block automation using:

• Cloudflare and WAF protections
• reCAPTCHA (including v3 / invisible)
• Session tracking and rate limiting
• Dynamic rendering and anti-bot heuristics

Your job is to consistently extract reliable data despite these constraints.

You will own end-to-end automation pipelines — from data acquisition to delivery — and play a key role in improving system resilience, scalability, and maintainability.

Responsibilities

• Design and build resilient web automation and scraping systems targeting government and regulatory data sources across APAC
• Handle adversarial environments, including anti-bot protections, CAPTCHA systems, and session-based restrictions
• Develop strategies for:
• Session persistence and renewal
• Intelligent retry and fallback mechanisms
• Handling IP bans and rate limiting
• Analyze network traffic (XHR/fetch) to identify opportunities to bypass UI scraping via direct API interaction
• Maintain and improve existing pipelines by:
• Diagnosing failures
• Adapting to DOM or behavior changes
• Improving long-term reliability
• Integrate extracted data into downstream systems (APIs, databases, queues)
• Contribute to internal tooling, frameworks, and best practices for automation
• Mentor junior engineers and review technical implementations

Requirements

• 4+ years of hands-on experience in web scraping or browser automation in production environments
• Strong proficiency in Node.js (async patterns, event loop, long-running processes)
• Experience with Puppeteer or Playwright in real-world, complex scenarios (login flows, navigation, state handling)
• Solid understanding of:
• HTML / DOM structure
• HTTP fundamentals (headers, cookies, sessions)
• Proven experience dealing with anti-bot systems, including:
• Cloudflare / WAF protections
• reCAPTCHA (v2 / v3 / invisible)
• Behavioral detection patterns
• Experience with browser fingerprinting and stealth techniques (headful automation, detection avoidance)
• Ability to debug and recover failing automations quickly in unstable environments
• Hands-on experience with proxy infrastructure and IP rotation
• Experience analyzing and replaying network requests to optimize scraping strategies
• Comfortable working with Docker

Nice to have:

• Experience scraping government or regulatory data sources
• Exposure to distributed scraping systems (queues, workers, orchestration)
• Familiarity with data pipelines or ETL systems
• Experience in KYC / KYB / AML / UBO domains

What We’re Looking For

• Someone who has worked on non-trivial scraping problems — not just static sites
• Strong problem-solver who can adapt quickly when systems break
• Comfortable working in environments where:
• There is no API
• Systems change frequently
• Failures are expected and must be handled gracefully

Competitive salary, based on experience.