# CLI Benchmark: 4-Way Comparison Source: https://docs.openbrowser.me/cli-comparison Benchmark comparison of openbrowser-ai vs browser-use vs playwright-cli vs agent-browser CLIs # CLI Benchmark: 4-Way Comparison Benchmark date: 2026-03-15 | Claude Sonnet 4.6 on AWS Bedrock | N=3 runs | 6 tasks | Single Bash tool ## Overview Four CLI browser automation tools compared head-to-head. Each tool gets a single generic Bash tool (identical overhead) with an optimized system prompt. The LLM drives each tool autonomously to complete 6 real-world browser tasks. | | openbrowser-ai | browser-use | playwright-cli | agent-browser | | ------------------------ | --------------------------------- | ------------------------ | ------------------------- | ------------------------- | | **Maintainer** | OpenBrowser | browser-use | Playwright | agent-browser | | **Engine** | Raw CDP (direct) | Playwright (CDP) | Playwright (CDP) | Playwright (CDP) | | **Interface** | `openbrowser-ai -c 'code'` | `uvx browser-use ` | `playwright-cli ` | `agent-browser ` | | **Code batching** | Python (multi-operation per call) | No (individual commands) | JS via `run-code` | No (`&&` chaining only) | | **Page state format** | DOM with `[i_N]` indices | DOM with `[N]` indices | A11y tree in `.yml` files | A11y tree with `@eN` refs | | **Page state size** | \~450 chars | \~880 chars | \~1,420 chars | \~590 chars (with `-i`) | | **Variable persistence** | Yes (daemon) | Yes (daemon) | Yes (background process) | Yes (background process) | ## Methodology * **Model**: Claude Sonnet 4.6 on AWS Bedrock (Converse API), us-west-1 * **Tool**: Single generic Bash tool for all 4 approaches (identical tool-definition overhead) * **System prompts**: Per-approach optimized prompts with tool-specific commands and optimization tips * **Fairness**: Both approach order AND task order randomized per run (eliminates OS/DNS caching bias) * **Browser**: Persistent daemon per approach across all 6 tasks, headless mode, browser cleanup between approaches * **Statistics**: N=3 runs, 10,000-sample bootstrap for 95% confidence intervals * **Tasks**: Same 6 tasks as the [MCP benchmark](/comparison) (fact\_lookup, form\_fill, multi\_page\_extract, search\_navigate, deep\_navigation, content\_analysis) against live websites * **Benchmark script**: `benchmarks/e2e_4way_cli_benchmark.py` * **Results data**: `benchmarks/e2e_4way_cli_results.json` ### Tasks | # | Task | Description | Target Site | | :-: | ------------------------ | ----------------------------------------------------------------------------- | ---------------------- | | 1 | **fact\_lookup** | Navigate to a Wikipedia article and extract specific facts (creator and year) | en.wikipedia.org | | 2 | **form\_fill** | Fill out a multi-field form (text input, radio button, checkbox) and submit | httpbin.org/forms/post | | 3 | **multi\_page\_extract** | Extract the titles of the top 5 stories from a dynamic page | news.ycombinator.com | | 4 | **search\_navigate** | Search Wikipedia, click a result, and extract specific information | en.wikipedia.org | | 5 | **deep\_navigation** | Navigate to a GitHub repo and find the latest release version number | github.com | | 6 | **content\_analysis** | Analyze page structure: count headings, links, and paragraphs | example.com | ### Fairness Design Unlike MCP benchmarks where each server defines its own tools (different counts, different schemas, different token overhead), this CLI benchmark uses a **single Bash tool** for all 4 approaches. This eliminates the tool-definition advantage -- the only difference is the system prompt telling the LLM how to use each tool. Additional fairness measures: * **Randomized approach order**: Each run shuffles which CLI goes first, preventing later approaches from benefiting from OS/DNS caching * **Randomized task order**: Each approach sees tasks in a different order per run * **Persistent daemon**: All 4 tools keep a browser session alive across 6 tasks (no cold-start advantage) * **Browser cleanup**: Stale browser processes killed between approaches * **Headless mode**: Eliminates rendering overhead differences ## Results: Overall All 4 tools achieve **100% accuracy** (18/18 task executions across 3 runs). CLI Benchmark: Token Usage vs Duration | Metric | openbrowser-ai | browser-use | playwright-cli | agent-browser | | ------------------------------------- | -------------------: | ----------------: | ----------------: | ----------------: | | **Duration (mean +/- std)** | **84.8 +/- 10.9s** | 106.0 +/- 9.5s | 118.3 +/- 21.4s | 99.0 +/- 6.8s | | **Tool Calls (mean +/- std)** | **15.3 +/- 2.3** | 20.7 +/- 6.4 | 25.7 +/- 8.1 | 25.0 +/- 4.0 | | **Bedrock API Tokens (mean +/- std)** | **36,010 +/- 6,063** | 77,123 +/- 33,354 | 94,130 +/- 35,982 | 90,107 +/- 3,698 | | **Response Chars (mean +/- std)** | **9,452 +/- 472** | 36,241 +/- 12,940 | 84,065 +/- 49,713 | 56,009 +/- 39,733 | | **Token ratio vs openbrowser-ai** | 1x (baseline) | 2.1x more | 2.6x more | 2.5x more | CLI Benchmark: 4-Way Comparison Overview ## Results: Per-Task Token Usage | Task | openbrowser-ai | browser-use | playwright-cli | agent-browser | | -------------------- | -------------: | ----------: | -------------: | ------------: | | fact\_lookup | **2,504** | 4,710 | 16,857 | 9,676 | | form\_fill | **7,887** | 15,811 | 31,757 | 19,226 | | multi\_page\_extract | **2,354** | 2,405 | 8,886 | 8,117 | | search\_navigate | **16,539** | 47,936 | 27,779 | 44,367 | | deep\_navigation | **2,178** | 3,747 | 4,705 | 5,534 | | content\_analysis | 4,548 | **2,515** | 4,147 | 3,189 | openbrowser-ai wins 5 of 6 tasks on tokens. The advantage is largest on complex pages (search\_navigate: 2.9x fewer tokens than browser-use) where code batching avoids repeated page state dumps. browser-use edges ahead on content\_analysis -- a trivial task where all approaches use minimal tokens. CLI Benchmark: Per-Task Token Usage ## Results: Cost Per Benchmark Run (6 Tasks) Based on Bedrock API token usage (input + output tokens at respective rates). | Model | openbrowser-ai | browser-use | playwright-cli | agent-browser | | -------------------------------- | -------------: | ----------: | -------------: | ------------: | | Claude Sonnet 4.6 ($3/$15 per M) | **\$0.12** | \$0.24 | \$0.29 | \$0.27 | | Claude Opus 4.6 ($5/$25 per M) | **\$0.24** | \$0.45 | \$0.56 | \$0.51 | ## Why openbrowser-ai Wins ### 1. Python Code Batching Multiple browser operations in a single `openbrowser-ai -c '...'` call: ```bash theme={null} openbrowser-ai -c ' await navigate("https://en.wikipedia.org/wiki/Python_(programming_language)") info = await evaluate("document.querySelector(\".infobox\")?.innerText") print(info) ' ``` One tool invocation does what competitors need 3-5 tool calls for. Each tool call incurs LLM inference overhead (reading full conversation history), so fewer calls = fewer tokens. ### 2. Compact DOM Representation Page state uses DOM with `[i_N]` indices at \~450 chars: ``` [i_1] [i_2] [i_3] [i_4] [i_5] ``` vs \~880 chars (browser-use DOM), \~590 chars (agent-browser a11y tree with `-i`), or \~1,420 chars (playwright-cli a11y tree in `.yml` file). ### 3. Server-Side Processing The LLM writes Python code that processes data server-side and returns only extracted results via `print()`. Competitors return full page state that the LLM must parse in its context window. ### 4. Variable Persistence The daemon maintains a Python namespace across `-c` calls. Intermediate results (selectors, extracted data, computed values) persist without re-extracting: ```bash theme={null} openbrowser-ai -c 'await navigate("https://example.com"); title = await evaluate("document.title")' openbrowser-ai -c 'print(f"Title was: {title}")' # title still available ``` ## Variance Analysis | CLI Tool | Token std / mean | Duration std / mean | | -------------- | :--------------: | :-----------------: | | openbrowser-ai | 17% | 13% | | browser-use | 43% | 9% | | playwright-cli | 38% | 18% | | agent-browser | 4% | 7% | * **openbrowser-ai**: Moderate variance -- consistent enough for reliable cost estimation * **browser-use**: High token variance (43%) driven by search\_navigate task where the LLM sometimes takes extra exploration turns * **playwright-cli**: High token variance (38%) driven by form\_fill where accessibility tree snapshots vary in size * **agent-browser**: Lowest token variance (4%) but at 2.5x the absolute token cost ## How Each CLI Works ### openbrowser-ai ```bash theme={null} # Python code batching -- multiple operations per call openbrowser-ai -c 'await navigate("url"); data = await evaluate("js"); print(data)' ``` * Persistent daemon over Unix socket * `-c` flag executes async Python in a persistent namespace * All browser functions available: `navigate()`, `click()`, `input_text()`, `evaluate()`, `scroll()`, etc. * Variables persist across calls ### browser-use ```bash theme={null} # Individual CLI commands uvx --from "browser-use[cli]" browser-use open https://example.com uvx --from "browser-use[cli]" browser-use state uvx --from "browser-use[cli]" browser-use input 5 "text" ``` * Individual commands per operation * `input "text"` combines click + type (optimization) * DOM with `[N]` indices * `uvx` isolation due to dependency conflicts ### playwright-cli ```bash theme={null} # JS batching via run-code playwright-cli run-code "async page => { await page.goto('url'); return await page.title(); }" # Snapshots save to .yml files playwright-cli snapshot && cat .playwright-cli/page-*.yml ``` * `run-code` enables JS batching (similar to openbrowser-ai's Python batching) * Snapshots save to `.yml` files, requiring extra `cat` to read * Accessibility tree format (\~1,420 chars per page) ### agent-browser ```bash theme={null} # Individual commands with && chaining agent-browser open https://example.com && agent-browser snapshot -i agent-browser click @e5 agent-browser eval "document.title" ``` * Individual commands, chainable with `&&` * `snapshot -i` flag for compact output (85-95% smaller than full snapshot) * Accessibility tree with `@eN` refs * `eval` for JavaScript execution # Benchmark Comparison Source: https://docs.openbrowser.me/comparison Benchmark comparison of OpenBrowser vs Playwright vs Chrome DevTools vs browser-use vs agent-browser # Browser Automation Comparison Two benchmark suites measuring OpenBrowser against competitors: 1. **[CLI Benchmark](/cli-comparison)** (2026-03-15) -- 4-way comparison of CLI tools driven by an LLM via a single Bash tool 2. **MCP Benchmark** (below) -- 3-way comparison of MCP servers as tool providers ## MCP Server Comparison Benchmark date: 2026-02-21 | OpenBrowser MCP v0.1.26 (CodeAgent, 1 tool) | Playwright MCP (latest) | Chrome DevTools MCP (latest) ### Overview Three approaches to browser automation via MCP, measured on identical tasks. | | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | ---------------- | ------------------------------------ | ------------------------------- | --------------------------------------------------------- | | **Maintainer** | Microsoft | Chrome DevTools team (Google) | OpenBrowser | | **GitHub Stars** | 27,400+ | 26,200+ | -- | | **Engine** | Playwright (Chromium/Firefox/WebKit) | Puppeteer (CDP) | Raw CDP (direct) | | **Tools** | 22 core (34 total) | 26 | 1 (`execute_code`) | | **Approach** | A11y snapshot with every action | A11y snapshot on demand | CodeAgent -- Python code execution with browser namespace | | **Element IDs** | `ref` from snapshot (`[ref=e25]`) | `uid` from snapshot (`uid=1_2`) | Numeric index from state | | **Transport** | stdio | stdio | stdio | | **Language** | TypeScript (Node.js) | TypeScript (Node.js) | Python | ### Token Usage Benchmark #### Methodology All three MCP servers were started as subprocesses and tested via JSON-RPC stdio transport. Same 5-step workflow, same pages, same measurement method. All numbers are real measurements, not estimates. **Workflow**: Navigate to Wikipedia Python page -> get page state -> click link -> go back -> get state again. #### Results: 5-Step Workflow on Wikipedia (Complex Page) | Metric | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | ------------------------------ | -------------: | ------------------: | --------------: | | **Tool calls** | 5 | 5 | 5 | | **Total response chars** | 992,065 | 539,209 | 1,131 | | **Est. response tokens** | **248,016** | **134,802** | **283** | | **Token ratio vs OpenBrowser** | 877x more | 476x more | 1x (baseline) | #### Results: Small Page (httpbin.org/forms/post) | Operation | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | --------------- | -------------: | ------------------: | --------------: | | Navigate | 1,985 chars | 182 chars | 105 chars | | Page state | 1,896 chars | 1,366 chars | 1,488 chars | | Type/fill | 526 chars | 95 chars | 92 chars | | List tabs/pages | 367 chars | 120 chars | 195 chars | #### Per-Operation Token Breakdown (Wikipedia) | Operation | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | -------------------- | ----------------------------: | ----------------------------: | -----------------------: | | Navigate | 495,950 chars (\~124K tokens) | 240 chars (\~60 tokens) | 134 chars (\~34 tokens) | | Snapshot / Get state | 495,437 chars (\~124K tokens) | 538,467 chars (\~135K tokens) | 421 chars (\~105 tokens) | | Click | 341 chars (\~85 tokens) | 218 chars (\~55 tokens) | 80 chars (\~20 tokens) | | Go back | 198 chars (\~50 tokens) | 149 chars (\~37 tokens) | 75 chars (\~19 tokens) | | Get state (2nd) | 139 chars (\~35 tokens) | 135 chars (\~34 tokens) | 421 chars (\~105 tokens) | ### Why the Difference | Design Decision | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | ----------------- | -------------------: | -----------------------------------: | ------------------------------------------: | | Navigate response | Full a11y snapshot | URL confirmation | URL confirmation | | State query | Always full snapshot | Full a11y snapshot (`take_snapshot`) | Compact (105 tokens) or full | | Click response | Updated snapshot | Action confirmation | Action confirmation | | Text extraction | No dedicated tool | No dedicated tool | `evaluate()` + Python processing | | Content search | Dump full snapshot | `evaluate_script` (JS) | `evaluate()` + Python regex/string matching | | Element search | Part of snapshot | `uid` from snapshot | `browser.get_browser_state_summary()` | **Playwright MCP**: Every navigation returns the full page accessibility snapshot (\~124K tokens for Wikipedia). Consistent, but forces the LLM to process the entire page with every action. **Chrome DevTools MCP**: Returns minimal confirmations for navigation. The agent must explicitly call `take_snapshot` to see the page (\~135K tokens for Wikipedia). One snapshot is comparable in size to Playwright's, but actions don't auto-return snapshots. **OpenBrowser MCP**: Actions return minimal confirmations. The agent explicitly requests the level of detail it needs -- from 105 tokens (compact state) to 3,981 tokens (targeted search) to 25,164 tokens (full page text). More tool calls, dramatically fewer tokens. ### What Each Server Returns (Verbatim) All responses below are real output captured from each MCP server on the httpbin.org/forms/post page. #### Navigate **Playwright MCP** -- returns full a11y snapshot with every navigation (\~2,150 chars on httpbin, \~496K chars on Wikipedia): ```yaml theme={null} ### Ran Playwright code await page.goto('https://httpbin.org/forms/post'); ### Page - Page URL: https://httpbin.org/forms/post - Console: 1 errors, 0 warnings ### Snapshot - generic [ref=e2]: - paragraph [ref=e3]: - generic [ref=e4]: - text: "Customer name:" - textbox "Customer name:" [ref=e5] - paragraph [ref=e6]: - generic [ref=e7]: - text: "Telephone:" - textbox "Telephone:" [ref=e8] ... (entire page tree continues) ``` **Chrome DevTools MCP** -- returns URL confirmation only (\~136 chars): ``` # navigate_page response Successfully navigated to https://httpbin.org/forms/post. ## Pages 1: https://httpbin.org/forms/post [selected] ``` **OpenBrowser MCP** -- returns URL confirmation only (\~105 chars): ``` Navigated to: https://httpbin.org/forms/post ``` #### Get Page State / Snapshot **Playwright MCP** `browser_snapshot` -- full a11y tree again (\~1,896 chars on httpbin, \~495K chars on Wikipedia): ```yaml theme={null} - generic [ref=e2]: - paragraph [ref=e3]: - generic [ref=e4]: - text: "Customer name:" - textbox "Customer name:" [ref=e5] ... (entire page tree) ``` **Chrome DevTools MCP** `take_snapshot` -- full a11y tree (\~1,214 chars on httpbin, \~538K chars on Wikipedia): ``` uid=1_0 RootWebArea url="https://httpbin.org/forms/post" uid=1_1 StaticText "Customer name: " uid=1_2 textbox "Customer name: " uid=1_3 StaticText "Telephone: " uid=1_4 textbox "Telephone: " uid=1_5 StaticText "E-mail address: " uid=1_6 textbox "E-mail address: " ... (entire page tree) ``` **OpenBrowser MCP** -- the LLM writes Python code to query exactly what it needs: ```python theme={null} # Get page metadata state = await browser.get_browser_state_summary() print(json.dumps({"url": state.url, "title": state.title})) ``` ```json theme={null} {"url": "https://httpbin.org/forms/post", "title": "httpbin.org/forms/post"} ``` The agent requests only the data it needs via Python code -- page title, specific element attributes, or targeted JS evaluation. No full-page dumps. #### Click / Type / Go Back All three servers return short confirmations for actions: | Operation | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | --------- | ------------------------------ | ------------------------- | --------------------------------- | | Click | `Clicked element` | `Clicked element uid=1_2` | `Clicked element 5` | | Type | Updated snapshot (\~526 chars) | `Filled element uid=1_2` | `Typed 'John Doe' into element 4` | | Go back | \~198 chars | \~149 chars | `Navigated back` | #### Targeted Extraction (OpenBrowser only) No equivalent in Playwright or Chrome DevTools MCP -- both require dumping the full snapshot or running JavaScript from the client side. **Element search** -- the LLM writes Python to find specific elements: ```python theme={null} state = await browser.get_browser_state_summary() for idx, el in state.dom_state.selector_map.items(): if 'submit' in el.get_all_children_text(max_depth=1).lower(): print(f"Found submit button at index {idx}") ``` **Data extraction** -- extract specific data via JS evaluation, not full-page dumps: ```python theme={null} # Extract only what's needed from a 97K-char Wikipedia page data = await evaluate('document.querySelector(".infobox")?.innerText') print(data) # ~900 chars instead of ~97K ``` **Comparison for finding "Guido van Rossum" on Wikipedia:** | Server | Method | Response Size | | ------------------- | ------------------------------------------------------- | ------------: | | Playwright MCP | `browser_snapshot` (dump full tree, search client-side) | \~495K chars | | Chrome DevTools MCP | `take_snapshot` (dump full tree, search client-side) | \~538K chars | | OpenBrowser MCP | `evaluate()` + Python string matching | \~900 chars | ### Cost Comparison MCP tool response costs per 5-step workflow on a complex page (Wikipedia). These tokens are added to the LLM's context window, charged at input token rates. All numbers based on real measurements. | Model | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | ------------------------------- | -------------: | ------------------: | --------------: | | Claude Sonnet 4.6 (\$3/M input) | \$0.744 | \$0.404 | **\$0.001** | | Claude Opus 4.6 (\$5/M input) | \$1.240 | \$0.674 | **\$0.001** | | GPT-5.2 (\$1.75/M input) | \$0.434 | \$0.236 | **\$0.001** | **Per 1,000 workflows:** | Model | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | ----------------- | -------------: | ------------------: | --------------: | | Claude Sonnet 4.6 | \$744 | \$404 | **\$0.85** | | Claude Opus 4.6 | \$1,240 | \$674 | **\$1.42** | | GPT-5.2 | \$434 | \$236 | **\$0.50** | ### E2E LLM Benchmark #### Methodology Six real-world browser tasks run through Claude Sonnet 4.6 on AWS Bedrock (Converse API). Each task uses a single MCP server as tool provider. The LLM decides which tools to call and when the task is complete. All tasks run against live websites. **Tasks**: Wikipedia fact lookup, httpbin form fill, Hacker News data extraction, Wikipedia search + navigation, GitHub release lookup, example.com content analysis. #### Results: Task Success All three servers pass all 6 tasks. | Task | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | -------------------- | :------------: | :-----------------: | :-------------: | | fact\_lookup | PASS | PASS | PASS | | form\_fill | PASS | PASS | PASS | | multi\_page\_extract | PASS | PASS | PASS | | search\_navigate | PASS | PASS | PASS | | deep\_navigation | PASS | PASS | PASS | | content\_analysis | PASS | PASS | PASS | | **Total** | **6/6** | **6/6** | **6/6** | #### Results: Tool Calls and Duration | Metric | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | --------------------------------- | -------------: | ------------------: | --------------: | | **Total tool calls (mean)** | 9.4 | 19.4 | 13.8 | | **Avg tool calls/task** | 1.6 | 3.2 | 2.3 | | **Total duration (mean +/- std)** | 62.7 +/- 4.8s | 103.4 +/- 2.7s | 77.0 +/- 6.7s | Playwright completes most tasks in 1 tool call because every navigation returns the full accessibility snapshot -- the LLM sees the entire page immediately and can answer without follow-up queries. OpenBrowser takes more turns because the LLM writes code to navigate, extract, and verify data step by step. MCP Server Benchmark: Duration vs Token Usage #### Per-Task Duration Per-Task Duration by MCP Server #### Per-Task Tool Calls Tool Calls Per Task by MCP Server #### Results: Token Efficiency Bedrock API token usage measured from the Converse API `usage` field (mean across 5 runs, 10,000-sample bootstrap CIs). | Metric | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | -------------------------------------- | -------------: | ------------------: | ----------------: | | **Bedrock API Tokens (mean)** | 158,787 | 299,486 | **50,195** | | **MCP Response Chars (mean)** | 1,132,173 | 1,147,244 | **7,853** | | **API Token ratio vs OpenBrowser** | **3.2x more** | **6.0x more** | **1x (baseline)** | | **Response char ratio vs OpenBrowser** | **144x more** | **146x more** | **1x (baseline)** | Total Bedrock API Token Usage (Input vs Output) #### Per-Task MCP Response Size | Task | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | -------------------- | -------------: | ------------------: | --------------: | | fact\_lookup | 520,742 chars | 509,058 chars | 3,144 chars | | form\_fill | 4,075 chars | 3,150 chars | 2,305 chars | | multi\_page\_extract | 58,392 chars | 38,880 chars | 294 chars | | search\_navigate | 519,241 chars | 595,590 chars | 2,848 chars | | deep\_navigation | 14,875 chars | 195 chars | 113 chars | | content\_analysis | 485 chars | 501 chars | 499 chars | The pattern: any task involving a complex page (Wikipedia, GitHub releases) produces massive response payloads for Playwright and Chrome DevTools because they dump the full accessibility snapshot. OpenBrowser returns only the data the code explicitly extracts. MCP Response Size: Full Page Dumps vs Server-Side Processing #### Per-Task Input Token Usage Per-Task Input Token Usage by MCP Server #### Results: Cost Per Benchmark Run (6 Tasks) Cost per benchmark run based on Bedrock API token usage (input + output tokens at respective rates). | Model | Playwright MCP | Chrome DevTools MCP | OpenBrowser MCP | | -------------------------------- | -------------: | ------------------: | --------------: | | Claude Sonnet 4.6 ($3/$15 per M) | \$0.50 | \$0.92 | **\$0.18** | | Claude Opus 4.6 ($5/$25 per M) | \$0.83 | \$1.53 | **\$0.30** | Cost Per Benchmark Run by Model #### Why OpenBrowser Uses More Tool Calls but Fewer Tokens Playwright sends the full page with every response, so the LLM gets the answer immediately but pays for \~120K tokens per Wikipedia page load. OpenBrowser returns compact results (\~30-800 chars per call), so the LLM needs more round-trips to navigate and extract but pays far fewer tokens overall. For a single simple task, Playwright's approach is fast. At scale (thousands of workflows, complex pages, multi-step agents), the MCP response size difference is 144x. ### Tool Surface Comparison #### Playwright MCP (22 core tools, 34 total) Navigation, interaction, form filling, file upload, drag and drop, hover, key press, select option, screenshots, snapshots, console messages, dialog handling, network requests, tab management, code execution, PDF export, wait conditions, resize, vision-mode coordinate tools, test assertions. #### Chrome DevTools MCP (26 tools) Input automation (click, drag, fill, fill\_form, hover, press\_key, handle\_dialog, upload\_file), navigation (navigate\_page, new\_page, close\_page, list\_pages, select\_page, wait\_for), emulation (emulate, resize\_page), performance tracing (start/stop/analyze), network debugging (list/get requests), JS execution, console messages, screenshots, DOM snapshots. #### OpenBrowser MCP (1 tool -- CodeAgent) | Tool | Capabilities | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `execute_code` | Python code execution in a persistent namespace with browser automation functions: `navigate()`, `click()`, `input_text()`, `scroll()`, `go_back()`, `select_dropdown()`, `send_keys()`, `upload_file()`, `evaluate()` (JS), `switch()`/`close()` (tabs), `done()` (task completion). Also provides `file_system` for local file operations and pre-imported libraries (json, pandas, re, csv, etc.) | ### Unique to OpenBrowser Features no competitor offers: * **CodeAgent architecture** -- single `execute_code` tool runs Python in a persistent namespace. The LLM writes code to navigate, extract, and process data rather than calling individual tools. Variables and state persist between calls. * **144x smaller MCP responses** -- returns only the data the code explicitly extracts, not full page dumps * **JS evaluation with Python processing** -- `await evaluate("JS expression")` returns Python objects directly (dicts, lists, strings), enabling pandas/regex/json processing in the same code block * **Built-in libraries** -- json, pandas, numpy, matplotlib, csv, re, datetime, requests, BeautifulSoup available in the execution namespace * **File system access** -- `file_system` object for reading/writing local files from browser automation code * **Dropdown support** -- `select_dropdown()` and `dropdown_options()` for native `