We have a new name! Browserbear is now Roborabbit

Getting Started with Browserbear: How to Automate Browsers

This article introduces you to Browserbear, a scalable, cloud-based service that helps you automate the web browser with ease. We'll also go through some automation templates that you can start using immediately.
by Josephine Loo · March 2023

Contents

    In today's digital age, the ability to automate repetitive tasks has become increasingly important. One area where automation has been particularly useful is in web browsing. By automating the web browser, you can complete repetitive tasks without having to do it manually, which enables you to shift time and effort to other critical aspects of your work.

    If you are a developer, you must have heard of automated browser tools and frameworks like Selenium, Puppeteer, and Playwright. These tools are great for browser automation but you will need to write code to give the browser instructions.

    In this article, we will introduce a browser automation tool called Browserbear, a scalable, cloud-based service that requires minimum coding to automate the web browser, making web automation easier than ever for developers.

    What You Can Do with Browser Automation

    Testing

    Testing is a crucial part of web development. Testing websites and web applications ensure that they are working as expected on different browsers and devices. By automating the browser to do testing, you can test more efficiently compared to manual testing and catch issues that might be difficult to find otherwise.

    Web Scraping

    Browser automation can be used to scrape and extract data from websites. This is useful for tasks like market research, competitive analysis, data mining, or any other purposes that require large amounts of data. With browser automation, you can gather data much faster than you could manually.

    Task Automation

    Many user actions can be automated using a browser automation tool. This includes actions like filling out forms, clicking buttons, and navigating to specific pages. By automating these actions, you can automate tasks such as creating user accounts, filling out surveys, or entering data into a web-based application.

    Monitoring

    Automated browsers can also be used to monitor websites and web applications for changes or errors. This can be useful for detecting issues such as broken links, page layout issues, or other problems that could affect user experience. Besides that, businesses can also use browser automation to monitor competitors’ prices or other latest updates.

    What is Browserbear

    Browserbear is a scalable, cloud-based browser automation tool that helps you to automate any browser task. From automated website testing, scraping data, taking scheduled screenshots for archiving, to automating other repetitive browser tasks, you can do it with Browserbear.

    Similar to Bannerbear, the featured automated image and video generation tool from the same company, you can easily create no-code automated workflows by integrating it with other apps like Google Sheets,  Airtable, and more on Zapier. Besides that, you can also use the REST API to trigger tasks to run in the cloud and receive data from the completed tasks. As the tasks run in the cloud, you can create as many tasks as you need without affecting your machine's performance.

    Creating an Browser Automation with Browserbear

    Step 1. Create a Task

    You will need a Browserbear account to create an automation task (create a free trial account ). After logging in to your account, go to Tasks and create a new task.

    create a task in Browserbear

    Enter a name for your task and click the “Save” button.

    save the task

    Step 2. Add Steps

    Now, you can start adding the steps required for your task. Click on “Add Step” to add a new step.

    add new steps to the task

    On the next page, you can select an action from the Action dropdown. Different actions will require you to provide different details for the step.

    select an action for the step

    For example, for the “Go” action, you will need to provide the URL that you want to visit and the Wait Until option so that Browserbear can know when to execute the next step after visiting the URL.

    enter the details for the action

    For some actions like “Assert”, “Click”, “Save Structured Data”, etc., you will need to locate some specific HTML elements to perform the action. For these actions, you can install the Browserbear Helper Chrome extension to get a JSON object that contains the XPath and the HTML snippet of the element, and add it to the Helper textbox.

    an example of the "Click" action that requires the Helper config

    This will help Browserbear to locate the HTML elements on the website. Click the “Save” button and you should see the step added to your task.

    a completed task

    Add as many steps as you need to the task.

    🐻 Bear Tips: Refer to How to Scrape Data from a Website Using Browserbear to learn how to use the Browserbear Helper Chrome extension.

    Step 3. Run The Task

    After you’re done adding all the steps needed, click “Run Task” to execute it.

    run the task

    You should see the status of the run when you scroll down the page.

    the task has finished running

    Click on “Log” to view the result and details of the run.

    the log of the run

    If the action yields a result, you can see the result in the outputs (grouped by actions):

    the result of the task

    Whenever you create a task in Browserbear, you can access and run it via Zapier integration and REST API (making HTTP requests). This enables you to integrate the task with other applications such as Google Sheets, Notion, Airtable, etc., or make a scheduled trigger to create an automated workflow.

    The Browserbear API is asynchronous. If you’re triggering the task using the REST API and want to receive the result automatically, you can use a webhook. The result will be sent to the webhook URL provided when the task has finished running.

    Here’s an example in Python:

    post_url = f"https://api.browserbear.com/v1/tasks/{task_uid}/runs"
    data = {"webhook_url" : "your_webhook_url"}
    
    response = requests.post(url, headers=headers, json=data)
    

    Overwriting the Default Configurations

    Although the automation tasks are created using Browserbear's UI interface, you can still modify the steps in your code by stating a new value for them, like the new URL for the “Go” action below:

    post_url = f"https://api.browserbear.com/v1/tasks/{task_uid}/runs"
    data = {
    	"webhook_url" : "your_webhook_url",
    	"steps": [
    		# overwrite the original URL
    	    {
    	      "uid": "GNqV2ngBmly7O9dPRe", 
    	      "action": "go",
    	      "config": {
    	        "url": "https://twitter.com/apple"
    	      }
    	    }
    	  ]
    }
    
    response = requests.post(url, headers=headers, json=data)
    

    When you make a POST request to the API endpoint, it will go to the new URL instead of the default one that is added from the Browserbear dashboard.

    Sharing Your Browserbear Automation Task

    You can share your Browserbear task by making it public. Then, you will get a shareable link to that task with the steps listed.

    making the task publis

    Any visitor with a Browserbear account will be able to duplicate the task into their account and use it immediately.

    the public page of the task

    Browser Automation Examples/Templates

    For common tasks like filling in a form, testing an HTML element, scraping a web page, and taking a screenshot, you can find ready-made Browserbear tasks in the Task Library.

    Simply select a task and add it to your account to start automating immediately.

    an overview of the tasks available in the task library

    Let’s see what you can do with these tasks:

    1. Welcome to Browserbear - Simple Form Submission

    This task is an introduction to using Browserbear. It is a simple automation that shows you how to enter text into a form, hit submit, and take a screenshot after submitting the form.

    Welcome to Browserbear

    In the outputs, you will see the details of the HTTP request for the “Go” action and the URL of the screenshot:

    ...
    "outputs": {
        "ngJAvaqb8noB6jE531_go": {
         	...
          }
        },
        "gpXvMxmBjqOy35kd8e_take_screenshot": "https://media.browserbear.com/screenshots/74a1lBngYN4m3MLxR5/gpXvMxmBjqOy35kd8e/376010cf2affe8324c8b45ac1eace6a20db8bf52.jpg"
      }
    ...
    

    Screenshot

    the resulting screenshot

    2. Simple Check - Assertions on HTML Elements

    This task uses the “Assert” action to check whether an HTML exists and its visibility on the web page. Besides that, you can also check if its text content matches or contains a particular text.

    Simple Check

    Result:

    ...
    "outputs": {
        "1qXQ0djzOJKBxNKWJ3_go": {
    	...
          }
        },
        "YaD8emgzJR6zE79Qno_assert": true
      }
    ...
    

    3. Extract Job Data - Single Page Data Scraping

    This task goes to an URL (Browserbear’s job board playground), extracts data from selected HTML elements, and saves them as structured data in JSON format. You can use it to extract data from a single web page that has no pagination.

    Extract Job Data

    Result:

    ...
    "outputs": {
        "YaQ4LmEyGlpBj7xZe6_go": {
    	...
          }
        },
        "Xgna6ZmbQQrbjGl5KM_save_structured_data": [
          {
            "title": "Community-Services Assistant",
            "company": "Cricket LLC",
            "country": "Virgin Islands, U.S.",
            "salary": "$14,000 / year",
            "link": "/jobs/55dJFeg_H0s-community-services-assistant/"
          },
          {
            "title": "Farming Associate",
            "company": "Greenlam LLC",
            "country": "Comoros",
            "salary": "$14,000 / year",
            "link": "/jobs/9F5EUh5gAKM-farming-associate/"
          },
    ...
    

    4. Hacker News Articles - Multi-page Data Scraping

    This task works similarly to the previous job data extraction task, but it caters to scenarios where there are multiple pages. Two extra steps ( “Click ” and “Go To Step” ) are added to navigate the browser to the next pages and then repeat the data scraping process.

    Hacker News Article

    The outputs result is similar to the previous task. For data that are scraped from the second page onwards, it will be added to the end of the same data array.

    5. Wikipedia Screenshots - Website Screenshots

    This task takes screenshots of the URL entered in three different sizes:

    • 1024x768 pixels (default)
    • 800x800 pixels
    • 1440x900 pixels

    After taking a screenshot, it will resize the window before taking another one.

    Wikipedia Screenshots

    Result:

    ...
    "outputs": {
        "jWp2wREBNO8z9Gal4q_go": {
    	...
          }
        },
        "AOxY25kzr3XbrDgELn_take_screenshot": "https://media.browserbear.com/screenshots/lgYOK3n34wlQjDE4zb/AOxY25kzr3XbrDgELn/32a6add67365e142c01ed94ffc0ec0db3940457a.jpg",
        "nvpxMrmbqWxBj7Y5V4_take_screenshot": "https://media.browserbear.com/screenshots/lgYOK3n34wlQjDE4zb/nvpxMrmbqWxBj7Y5V4/2224da170cf160665600ff3c996f175e235b369f.jpg",
        "qd26njEyDggbW3gol4_take_screenshot": "https://media.browserbear.com/screenshots/lgYOK3n34wlQjDE4zb/qd26njEyDggbW3gol4/2b4b263961e6eb1e9dd6fd3983ed5079dd545960.jpg"
      }
    ...
    

    Conclusion

    These are some of the ways you can use Browserbear to automate the web browser. While there are several automated browser tools and frameworks available, Browserbear helps you automate the web browser with less code while still allows you to modify/control it using code.

    The importance of automation in today's digital age cannot be overstated. If you want to start automating the web browser, sign up for a Browserbear trial account! It only takes a few minutes to run your first web browser automation.

    About the authorJosephine Loo
    Josephine is an automation enthusiast. She loves automating stuff and helping people to increase productivity with automation.

    Automate & Scale
    Your Web Scraping

    Roborabbit helps you get the data you need to run your business, with our nocode task builder and integrations

    Getting Started with Browserbear: How to Automate Browsers
    Getting Started with Browserbear: How to Automate Browsers