Automating Web Tasks with the browser_use Library

In the rapidly evolving landscape of automation, simplifying complex tasks is the key to efficiency. Enter the browser_use library—an innovative tool that allows you to automate browser-based activities using plain English instructions. This article delves into how this library works, its impact on automation workflows, and its potential scope for QA automation.

The Power of Simplicity

Traditionally, browser automation required intricate scripting with tools like Selenium or Playwright. These tools demand a strong understanding of locators, selectors, and browser-specific configurations. The browser_use library abstracts these complexities, enabling users to define automation tasks in natural language.

How It Works

Here’s a glimpse of a script that automates Instagram tasks using browser_use and AzureChatOpenAI:

from langchain_openai import AzureChatOpenAI
from browser_use import Agent
from browser_use.browser.browser import Browser, BrowserConfig
import asyncio
from dotenv import load_dotenv
import os
from pydantic import SecretStr


load_dotenv()


# Initialize the model
llm = AzureChatOpenAI(
    model=os.getenv('AZURE_OPENAI_MODEL', ''),
    api_version=os.getenv('AZURE_OPENAI_VERSION', ''),
    azure_endpoint=os.getenv('AZURE_OPENAI_ENDPOINT', ''),
    api_key=SecretStr(os.getenv('AZURE_OPENAI_KEY', '')),
    verbose=True
)


async def main():
    task = f"""
    1. Go to https://chromewebstore.google.com/
    2. Click on extensions
    3. Go to Search tab
    4. Search for react developer tools
    5. Select react developer tools widget
    6. Wait for page to get loaded
    7. Click on Add to chrome
    8. Pop up will show up
    9. Click on add extension
    """
    agent = Agent(
        task=task,
        llm=llm,
        browser=Browser(
            config=BrowserConfig(
                headless=False,
                chrome_instance_path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
            )
        )
    )
    result = await agent.run()
    print(result)

asyncio.run(main())

Breaking Down the Script

1. Natural Language Tasks: The script defines the automation steps as plain English instructions.
2. AI-Powered Interpretation: AzureChatOpenAI interprets these instructions, converting them into actionable commands.
3. Browser Automation: The Agent component interacts with the browser, executing tasks such as logging in, navigating tabs, and sending messages.
4. Asynchronous Execution: The use of asyncio ensures efficient, non-blocking execution.

Impact on Automation Workflows

• Accessibility: Reduces the barrier to entry for non-developers who can define tasks without coding knowledge.
• Efficiency: Speeds up automation script development, making it agile and adaptable.
• Maintainability: Easier to read and modify compared to traditional automation scripts.

Future Scope: QA Automation and Beyond

While this script showcases a simple social media automation, the potential applications are vast:

QA Automation:

• Automate test cases by describing user flows in natural language.
• Reduce time spent on writing and maintaining test scripts.
• Enable cross-browser testing without extra configurations.

RPA (Robotic Process Automation):

• Automate repetitive business processes in finance, HR, and customer support.

Data Scraping:

• Extract data from websites with minimal code, useful for research and analytics.

Custom Workflows:

• Automate workflows like form submissions, report generation, and more.

Conclusion

The browser_use library represents a paradigm shift in automation. By bridging natural language processing with browser automation, it democratizes access to powerful automation capabilities. As AI and automation continue to evolve, tools like this will play a pivotal role in reshaping how we interact with digital systems.

References

• https://docs.browser-use.com/quickstart