Conversation
WebDriverWrapper additions for stealth / anti-bot / advanced scenarios: * set_driver gains experimental_options, extension_paths, enable_bidi * attach_to_existing_browser for sessions started with --remote-debugging-port * execute_cdp_cmd plus convenience shortcuts: set_timezone / locale / device_metrics / user_agent / extra_http_headers / geolocation / network_conditions, block_urls, set_cache_disabled, set_download_directory, add_script_to_evaluate_on_new_document * CDP Fetch interception primitives: enable_fetch_interception, continue_request, fulfill_request, fail_request * W3C BiDi listeners: add_console_listener, add_js_error_listener and their remove counterparts (Selenium 4.16+) * Session persistence: save_cookies / load_cookies / clear_origin_storage * Screenshots / PDF: save_screenshot, save_full_page_screenshot, print_page * Page / window: reload(ignore_cache), bring_to_front, switch_to_window_by_url|title, get_current_url / title / page_source / window_handles / current_window_handle, new_window, close_window Split webdriver_wrapper.py (2340 lines) into themed mixins under webdriver/_wrapper_mixins/ so every file stays under the 750-line limit: _scripting_mixin (script / CDP / BiDi / Fetch), _navigation_mixin (navigation + window), _cookie_mixin, _actions_mixin (ActionChains), _media_mixin (screenshots + print + log). Main file retains lifecycle, element finding, waits, and the module-level _options_dict / _webdriver_dict / _webdriver_manager_dict so existing test patch paths work unchanged. Independent modules: * utils/cdp/event_loop.py — CDPEventListener (background WebSocket loop with sync send/on/context manager; lazy websocket-client import) * utils/cdp/tracing.py — record_trace producing Chrome DevTools- loadable JSON via the event loop * utils/bidi/network.py — cross-browser W3C BiDi network handlers (add_request_handler / add_response_handler / add_auth_handler / clear_network_handlers) wrapping driver.network Executor: register 41 new WR_* aliases for the additions; gate WR_execute_cdp_cmd and WR_add_script_to_evaluate_on_new_document under set_allow_arbitrary_script. MCP server: webrunner_run_actions description lists the advanced commands so MCP clients can discover them. Documentation: English + zh-TW + zh-CN READMEs gain an Advanced WebDriverWrapper section; Sphinx docs (Eng + Zh) extended with 11 new chapters covering launch options, page metadata, CDP shortcuts, Fetch primitives, BiDi listeners, the standalone modules, and the mixin layout. Tests: 109 new mock-based unit tests added across 4 files; all 119 related tests green.
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 345 |
| Duplication | 8 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
* S1192 — extract _FULL_PAGE_SCREENSHOT_LOG / _PRINT_PAGE_LOG constants in _media_mixin.py so the literals are defined once each. * S3776 — split set_driver into _build_driver_options / _apply_experimental_options / _apply_extension_paths helpers; main function cognitive complexity drops from 42 to well under 15. * S7500 — replace the (_ for _ in ()).throw(...) lambda hack in test_cdp_event_loop.py with a named _explode helper. * S1481 — drop unused 'handlers' from test_cdp_tracing.py and unused 'kwargs_or_args' from test_webdriver_wrapper_extensions.py. * S5906 — use assertTrue for the merged["detach"] bool check. * S5443 — rename hard-coded /tmp/ test fixtures to /opt/ paths (mock-only strings, never written) and update the docstring example in set_driver. * S5332 hotspot — Chrome DevTools /json discovery endpoint is HTTP-only by design; add NOSONAR with a comment block explaining why and that the endpoint binds to localhost. All 119 unit tests still pass.
* Bandit B110 (event_loop.py:189) — replace bare pass in CDPEventListener.stop()'s ws.close() except with a debug-log line so the cleanup failure is at least visible in WEBRunner.log. * Bandit B110 (_navigation_mixin.py:161) — same fix for the best-effort window-restore branch in _switch_to_window_by_attr. * Bandit B310 (event_loop.py:72) — annotate the urlopen call with '# nosec B310' plus a comment explaining the scheme is a fixed-literal http:// (only debugger_address host:port varies, no user-controlled scheme so no file:// risk).
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
set_driver(experimental_options=, extension_paths=, enable_bidi=),attach_to_existing_browser,execute_cdp_cmd+ CDP shortcuts (set_timezone/_locale/_device_metrics/_user_agent/_extra_http_headers/_geolocation/_network_conditions/block_urls/set_cache_disabled/set_download_directory/add_script_to_evaluate_on_new_document), Fetch interception primitives (enable_fetch_interception/continue_request/fulfill_request/fail_request), W3C BiDi listeners (add_console_listener/add_js_error_listener, Selenium 4.16+), session persistence (save_cookies/load_cookies/clear_origin_storage),save_full_page_screenshot,print_page,reload(ignore_cache),bring_to_front,switch_to_window_by_url|title, plus page-metadata getters and tab management.webdriver_wrapper.py(2340 lines) reorganised intowebdriver/_wrapper_mixins/—_scripting_mixin,_navigation_mixin,_cookie_mixin,_actions_mixin,_media_mixin— so every file fits under the 750-line project limit. Main file keeps lifecycle, element finding, waits, and the module-level_options_dict/_webdriver_dict/_webdriver_manager_dictso existing testpatch.dictpaths and public imports are unchanged.utils/:cdp/event_loop.py(CDPEventListener— background CDP WebSocket loop with syncsend/on/ context manager; lazywebsocket-clientimport),cdp/tracing.py(record_traceproducing Chrome DevTools-loadable JSON),bidi/network.py(cross-browser W3C BiDi network handlers).WR_*aliases for the additions;WR_execute_cdp_cmdandWR_add_script_to_evaluate_on_new_documentjoin the existingset_allow_arbitrary_scriptgate.webrunner_run_actionsdescription now lists the advanced commands so MCP clients can discover them.Test plan
test_webdriver_wrapper_extensions.py,test_cdp_event_loop.py,test_bidi_network.py,test_cdp_tracing.py); all 119 affected tests green (pytest test/unit_test/test_cdp_commands.py test/unit_test/test_webdriver_wrapper_extensions.py test/unit_test/test_cdp_event_loop.py test/unit_test/test_bidi_network.py test/unit_test/test_cdp_tracing.py test/unit_test/test_executor.py).webdriver_wrapper.pyMRO verified to compose the 5 mixins correctly;execute_cdp_cmd/move_to_element/save_cookies/set_driver/find_elementresolve to the expected modules.build_default_tools+build_browser_tools).from je_web_runner import CDPEventListener, record_trace, bidi_add_request_handler, ....set_driver(enable_bidi=True)→add_console_listener→to_url→ confirm console messages stream to the callback.