Update WebUI for SSL and Session Token Integration#3
Open
jlegrand62 wants to merge 121 commits intodevelopfrom
Open
Update WebUI for SSL and Session Token Integration#3jlegrand62 wants to merge 121 commits intodevelopfrom
jlegrand62 wants to merge 121 commits intodevelopfrom
Conversation
- Added WebSocket connection header mapping - Updated HTTP to HTTPS redirect to avoid trusting user-controlled Host - Enabled HTTP/2 support on port 443 - Improved SSL session caching with timeout settings - Disabled session tickets for better key management - Added security headers for Strict Transport Security, Content Type Options, Frame Options, and Referrer Policy - Updated proxy configuration to keep prefixes for `/p3dx` and `/p3dv` - Removed trailing slash from `proxy_pass` URL in Nginx default configuration - Added WebSocket/SSE requirements with specific timeout settings - Enhanced headers and timeout configurations for all proxied services
- Removed the configuration block for serving static files of the webterm application
- Added new location block for handling `/p3dv/static` requests - Configured proxy settings to forward requests to `P3DV_URL` and `P3DV_PORT` - Set cache control headers with `expires 1h` and `Cache-Control: public, immutable`
- Revised the Content-Security-Policy to allow resources from specific origins and types: - Allowed CSS, font, script sources from self and external CDNs - Added WebSocket connections for Socket.IO and terminal functionality - Added Permissions-Policy header to restrict geolocation, microphone, and camera access
- Added new `api_cert` argument to specify the path to the certificate file for the PlantDB REST API - Reorganized argument groups in the parser for better structure - Added a new SSL checkbox to the configuration modal for enabling HTTPS - Updated callbacks to handle SSL status and display it correctly - Improved error handling during connection tests
- Added `developer/nginx_openssl.md` to the mkdocs configuration.
…structions - Expanded `docker/nginx/README.md` with step-by-step instructions for generating private keys and certificates - Added sections for creating an OpenSSL configuration file, generating the certificate and key, and verifying the certificate
…update `default.conf` - Add a map for WebSocket connection header in `default_full.conf` - Configure SSL, HTTP to HTTPS redirection, security headers, and various service proxies in `default_full.conf` - Enable Gzip compression and adjust Content-Security-Policy in `default.conf` - Update proxy settings and headers across locations for different services
- Updated `pages_folder` and `assets_folder` to use `Path(__file__).parent / "pages"` and `Path(__file__).parent / "assets"`.
- Migrate `localhost` and port constants to environment variables - Introduce `app_config` dictionary for centralizing configuration settings - Update `setup_web_app()` call to use the new configuration setup
- Renamed `url_base_pathname` to `url_prefix` - Ensured `url_prefix` ends with a trailing slash as required by Dash - Updated configuration settings for session cookies: - Added `SESSION_COOKIE_HTTPONLY=True` for preventing JavaScript access - Changed `SESSION_COOKIE_SAMESITE='Lax'` to `SESSION_COOKIE_SAMESITE='Strict'` for CSRF protection
- Updated dependencies list by adding `plantdb.client` and removing `bcrypt` - Added `[tool.setuptools.package-data]` section with assets
- Refactor Dockerfile to use correct user environment variables - Update virtual environment setup in Dockerfile - Enhance cleanup steps in Dockerfile - Modify `entrypoint.sh` to use new environment configuration
- Added section on building the Docker image - Added section on running the Docker container
- Change the title in `app.py` from "Plant Imager" to "Plant Imager WebUI" - Update default port in `wsgi.py` from 8080 to 5000 - Rename environment variable for URL prefix from `WEBUI_PROXY` to `WEBUI_PREFIX` - Refactor SSL context configuration and run settings using `run_config` dictionary
- Added `session-token` output to the login callback. - Updated function signature and return type of the login callback. - Modified parsing logic to include session token in the response. - Refactored logout callback to handle session token. - Added `session-token` store component in app.py.
- Emphasized the term **self-signed** for clarity. - Updated OpenSSL configuration file template to use angle brackets `<>` for parameter placeholders. - Added instructions to replace placeholder parameters within angle brackets `<>`. - Renamed section title from "Build the docker image" to "Build the NGINX docker image" for better clarity.
- Removed redundant SSL certificate copying commands. - Moved default configuration copy location to `/etc/nginx/conf.d`. - Streamlined `envsubst` command for environment variable substitution.
- Added `rest-api-prefix` and `rest-api-ssl` states in scan.py. - Updated `run_scan` function signature and base_url usage. - Added `PyJWT>=2.6.0` dependency in pyproject.toml. - Refactored connection availability check in config.py using `is_server_available`. - Updated modal handling logic for configuration button click.
- Replace all `--api_*` arguments with `--plantdb_*` (host, port, prefix, ssl). - Add new `--plantdb_ssl` boolean flag and store `rest-api-ssl`. - Update default constants to `PLANTDB_HOST`, `PLANTDB_PORT`, `PLANTDB_PREFIX`. - Refactor `setup_web_app` signature and internal `dcc.Store` values. - Update usage examples and documentation strings. - Clean up imports: remove unused `configure_requests_with_certificate` and unused `api_prefix` import.
…ion test
- Import `get_test_dataset` in `test/test_scan_integration.py` and reorder imports for clarity
- Use `os.getenv("PI3_CAMERASERVER_IMAGE")` with a fallback to `get_test_dataset('real_plant') / "images"` for `self.images_path`
- Switch session token setting from `db_client.jwt_token` to `db_client._access_token` in the RPC controller setup
- Minor formatting clean‑up
- Replace `session-token` store IDs with `access-token` in **app.py**, **table.py**, **carousel.py**, **config.py**, **new_user.py**, **scan.py**, and **login.py** - Update function signatures and internal calls from `session_token` to `access_token` across **visu.py**, **utils.py**, **carousel.py**, **table.py**, **config.py**, **new_user.py**, **scan.py**, and **login.py** - Adjust API helper calls (`load_image_from_url`, `get_dataset_dict`, etc.) to pass `access_token` instead of `session_token` - Modify login flow: restore and logout callbacks now validate and clear `access-token` - Add `validate_toml_textarea` callback in **scan.py** to validate TOML input and apply red‑border styling on error - Swap button icons/labels in **scan.py** for clearer UI (configure scanner vs start scan)
…, and adjust UI components - Updated docstrings in `visu.py`, `carousel.py`, and `utils.py` to use **AN access token** wording. - Replaced `access_token` arguments with `session_token` in API calls across `visu.py`, `carousel.py`, `utils.py`, `table.py`, and related helpers. - Modified function signatures: - `load_image_from_url(url, session_token=None)` (formerly `access_token`). - Updated calls to `get_dataset_dict` and `list_task_images_uri` to pass `session_token`. - Added `dcc.Store(id='refresh-token', ...)` in `app.py` for storing refresh tokens. - Swapped login button imports and usages to the new avatar button: - Imported `login_avatar_button` and `login_avatar_button_tooltip` in `nav.py`. - Updated navigation items to use the avatar button. - Renamed creation function to `create_login_avatar_button` in `login.py`. - Replaced `dbc.Textarea` with `dcc.Textarea` for the scan configuration textarea in `scan.py`. - Adjusted login callbacks: - Added `refresh-token` output and updated return signatures. - Refactored error handling and success response to include `refresh_token`. - Updated type hints and docstrings accordingly. - Fixed grammar typo “A access token” → “AN access token” in multiple modules.
- Add `docs/assets/images/webui_dataset_table_screenshot.png` - Add `docs/assets/images/webui_plantdb_screenshot.png` - Add `docs/assets/images/webui_screenshot.png`
…lidation - Changed `dcc.Textarea` to `dbc.Textarea` for `scan-cfg-toml` in `src/webui/plantimager/webui/scan.py` - Updated callback outputs to `valid` and `invalid` properties instead of a style dict - Refactored `validate_toml_textarea` to return a `(bool, bool)` tuple indicating validity, removing custom error styling logic
- Update imports in `src/webui/plantimager/webui/scan.py`: remove unused `set_props` and reorder `config_upload` import. - Style scanner UI buttons: - `config-scan-button` now primary color, full width. - `start-scan-button` label changed to **Start Scanning**, success color, full width. - Add new `cancel-scan-button` (danger color, full width, disabled by default). - Refactor layout: - Add comments separating UI sections. - Adjust row alignment and margins. - Hide `scan-response` alert by default and apply gray color style. - Introduce placeholder cancel callback: - New `@callback` handling `cancel-scan-button` clicks, returns “Cancelling scan…”. - Minor code formatting improvements (line breaks in function signatures, added spacing).
- Updated token assignment in `src/webui/plantimager/webui/scan.py` to call `controller.set_session_token(access_token)` for consistency with the session‑token based authentication API.
…dling - Remove the unused “Available cameras” accordion from `src/webui/plantimager/webui/scan.py` and insert `camera_card` into the main column layout. - Rename token argument from `access_token` to `session_token` in API calls across `src/webui/plantimager/webui/config.py`, `src/webui/plantimager/webui/new_user.py`, and `src/webui/plantimager/webui/carousel.py`. - Pass `as_base64=True` to `list_task_images_uri` in `carousel.py` to request base64‑encoded images. - Update `load_image_from_url` in `src/webui/plantimager/webui/utils.py` to accept `access_token` (still used as the variable) and handle both binary (`encoding == 'binary'`) and base64 (`encoding == 'base64'`) image responses, using `pybase64` for conversion. - Add `import pybase64` to `utils.py`. - Adjust `src/webui/plantimager/webui/pages/table.py` to call `get_dataset_dict` with positional arguments and the renamed token variable. - Update related docstrings to reflect the new token naming.
- Define new `camera_card` list with a `dbc.Card` showing a camera header and an `available-cameras` markdown placeholder. - Replace the scan card header icon class from `bi bi-camera` to `bi bi-upc-scan` in `src/webui/plantimager/webui/scan.py`.
…stry - Import `WeakMethod` and `send_alive_check` in `RPC.py`; use weak references for signal emission and expose alive checks. - Refactor `RPCSignalReceiver` to close its socket in a `finally` block and remove the old `finalize` call. - Set the signal‑receiver thread to non‑daemon and update the client finalizer to close the socket, stop the receiver, and log “Client finalized”. - Extend `RPCServer.__init__` with `alive_timeout` parameter, store it as `self.alive_timeout`, and perform periodic alive checks via `send_alive_check` in `serve_forever`. - Replace per‑instance finalizer with a state‑based `_server_finalizer` that unregisters the device and closes sockets without capturing `self`; update `finalize` call accordingly. - Update registration flow to populate the cleanup state (`uuid`, `registry_addr`) after `register_to_registry`. - Enhance `stop_server` to explicitly unregister the device, clear cleanup state, and log successful unregistration. - Close sockets (`self._socket`, `self._signal_socket`) after the serve loop and log “Server stopped”. - In `deviceregistry.py` add `ALIVE_TIMEOUT`, `RLock`, and `device_health_timeout` to track device health; protect device modifications with the lock. - Implement `_prune_unhealthy_devices` called each loop iteration to remove expired devices. - Add handling for `EventType.CHECK_ALIVE` to refresh health timeout and reply with an ACK. - Rename internal removal methods to `_remove_device_by_uuid` and `_remove_device_by_name`; update all call sites. - Update `_add_device` to set a health timeout entry and use the lock. - Adjust `register_device` and `unregister_device` signatures to return `(name, uuid)` and use context‑manager style socket connections. - Introduce `send_alive_check` helper to query the registry for service liveness.
- Extend `register_to_registry` in `src/commons/plantimager/commons/RPC.py` with a new `overwrite` parameter (default `True`) and pass it to the internal registration call. - Update the example in `src/commons/plantimager/commons/examples/cameraserver.py` to call `register_to_registry(..., overwrite=False)`. - Comment out the verbose debug log in `src/commons/plantimager/commons/deviceregistry.py` to reduce noise. - Add an empty `test/__init__.py` to ensure the test directory is treated as a package.
- Update `set_session_token` abstract method in `controller_device.py` to accept a `tuple[str, str]` instead of a single string - Split `_session_token` into `_access_token` and `_refresh_token` in `scanner.py` and adjust token validation, storage, and propagation to `PlantDBClient` - Adjust `set_session_token` implementation in `scanner.py` to handle the token tuple and validate via `validate_token` - Move `RequestException` import to the top of `scanner.py` and remove duplicate import - Wrap `camera.serve_forever()` in `cameraserver.py` with a `try/except KeyboardInterrupt` block for graceful shutdown - Replace weakref finalizer in `AppBridge.py` with an explicit `finalizer` function that stops the registry, server, joins threads, and terminates the ZMQ context - Rename stop flags to `_stop_flag` in `RPCSignalReceiver` (`RPC.py`) and `DeviceRegistry` (`deviceregistry.py`) and update loop checks accordingly - Set `RPCSignalReceiver.daemon` to `True` (with a FIXME comment) and rename internal stop variable - Add `ZMQBindError` handling when binding sockets in `RPC.py` to log critical errors and exit - Ensure all RPC signals are disconnected on server shutdown and in the device registry finalizer - Update related variable names and references (`_session_token` → `_access_token`, `_refresh_token`) across modified files.
- Import `gc` and call `gc.collect()` in `tearDown` of `test/test_scan_integration.py` for better resource cleanup - Add a print statement when killing subprocesses to aid debugging - Update `rpc_controller.set_session_token` to receive a tuple ``(db_client._access_token, db_client._refresh_token)`` instead of a single token - Comment out `rpc_controller.stop_server()` and manually clear the controller instance with ``rpc_controller._instance = None`` and ``del rpc_controller`` to avoid server shutdown issues in the test environment
- Change the cancel scan button icon to a filled version by updating `html.I` class to `bi bi-x-circle-fill` in `src/webui/plantimager/webui/scan.py`. - Add a comment noting the inactive default and set `color="danger"` on the logout button in `src/webui/plantimager/webui/login.py`.
- Import `traceback` in `src/webui/plantimager/webui/scan.py`. - Wrap `tomllib.load(cfg)` in a `try/except` block catching `tomllib.TOMLDecodeError`. - On parse failure, return `"Failed to parse config file"` along with a concise traceback (`traceback.format_exc(limit=1)`). - Pass the successfully loaded `config_dict` to `controller.set_config`.
ArthurLuciani2
force-pushed
the
feature/p3dv_stack
branch
from
86c1b4d to
cd1f266
Compare
- Remove unused `[ca]` section and `default_days` setting from the OpenSSL config block in `docker/nginx/README.md` - Change `basicConstraints` to `CA:TRUE` and expand `keyUsage` to include `keyCertSign` and `digitalSignature` in the same file - Clean up Markdown table formatting and improve wording for the volume bind instruction - Add a new **Trusting the Self-Signed Certificate** section describing how to obtain the certificate, add it to Ubuntu’s system trust store, and import it into common browsers (Brave, Chrome, Firefox)
- Commented out the `ssl_context` entry in `src/webui/plantimager/webui/wsgi.py` to stop automatically setting an SSL context based on `plantdb_ssl`.
…idation
- Implement `is_instance_of_generic` in `src/commons/plantimager/commons/RPC.py` to validate values against typing generics, unions, tuples, containers, and nested structures
- Extend `RPCSignal` with `arg_types` storage, `validate_args` method, and invoke validation before emitting signals; improve lambda connection to capture `sig_name` correctly
- Add unit tests in `test/test_RPC_isinstance_generic.py` covering simple types, `Any`, list/dict/tuple/set generics, nested generics, unions, and tuple‑of‑specs
- Update `test/test_scan_integration.py` to introduce `db_port` and construct `db_url` dynamically with `f"http://localhost:{db_port}"` and pass the port to the subprocess command.
- Import `logging` and create a `werkzeug` logger in `src/webui/plantimager/webui/wsgi.py`
- Log a warning when `app_config['plantdb_ssl']` is true but the `CERT_PATH` environment variable is unset
- Replace the commented `ssl_context` entry with a dynamic setting: `ssl_context` is `os.getenv('CERT_PATH', None)` only when `app_config['plantdb_ssl']` is enabled
…instructions - Adjust relative image links to `../../docs/assets/images/...` in `src/webui/README.md` - Update licence and Python version badges to reference the Plant‑Imager3 repository - Refine package description and include a screenshot reference - Remove obsolete sections and excess blank lines for cleaner layout - Add an **Environment variable** section documenting `ALLOW_PRIVATE_IP`, `CERT_PATH`, and `VALIDATE_HOST`, and note usage of `dotenv.load_dotenv` with a `.env` file - Reformat the Docker run command indentation for consistency - Remove duplicated security recommendations and unrelated Let’s Encrypt content - Polish markdown formatting and wording throughout the file
- Added `id="navbar-brand"` to the `dbc.NavbarBrand` component in `src/webui/plantimager/webui/nav.py` - Implemented `update_navbar_brand_href` callback that updates the `href` of `navbar-brand` based on `plantdb-prefix` data, defaulting to `/` when no prefix is provided.
- Implemented `coerce_to_generic` in `utils.py` for flexible type coercion supporting generics, unions, mappings, tuples, and nested structures. - Relocated `is_instance_of_generic` from `RPC.py` to `utils.py` and updated imports throughout to maintain functionality. - Modified `RPCSignal.validate_args` to integrate coercion logic, enabling automatic conversion of argument types. - Developed comprehensive unit tests in `test_RPC_isinstance_generic.py` to validate coercion for various cases, including sequences, mappings, tuples, unions, and nested types. - Updated `RPC.py` and related modules to ensure compatibility with the relocated and enhanced utilities.
…ides - Removed obsolete `docs/rest_api.md`. - Added new service documentation: - `docs/usage/services_overview.md` – high‑level architecture overview. - `docs/usage/services_startup.md` – configuration and startup instructions. - `docs/usage/services_usage.md` – usage examples and deployment notes. - Updated `mkdocs.yml` navigation: - Deleted the “REST API” entry. - Introduced a “Usage” section with links to the three new service docs and the existing “Services Startup” page. - Adjusted MkDocs plugin configuration: - Replaced `import` with `inventories` for Python inventories. - Removed the unused `ignore_init_summary` option.
- Delete the `args` block containing `PLANTDB_BRANCH` in `docs/usage/services_startup.md` - Remove the `extra_hosts` entry from the WebUI service definition in the same file
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request focuses on enhancing the WebUI with SSL support and session token handling for improved security and functionality.
Key Changes
SSL Integration:
plantdb_ssl,api_cert) to support secure communications and certificate management.Session Token Management:
session-tokenstate in callbacks for managing session tokens throughout the WebUI.Configuration and Environment Updates:
.envsupport for dynamic environment variable management, enabling easy configuration for WebUI deployment.Code Improvements:
load_image_from_url, server availability checks) for enhanced reuse and better error handling.Frontend Updates:
Documentation and Guides:
README.mdwith Docker setup instructions and SSL implementation notes.developer/NTP_server.mdanddeveloper/nginx_openssl.md) for setting up time synchronization and OpenSSL certificates.Checklist
Notes
This is part of the ongoing effort to modernize WebUI security measures and enhance developer workflow with configuration flexibility.