-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat(framework): Add flwr-connector #7029
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
danielnugraha
wants to merge
36
commits into
main
Choose a base branch
from
add-flwr-connector
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
1f6f7a4
feat(framework): Add flower-agent
danielnugraha ae795a5
Apply suggestion from @Copilot
danielnugraha 34e8ec1
Format
danielnugraha 25a7d92
Merge remote-tracking branch 'refs/remotes/origin/add-flwr-agent' int…
danielnugraha f7557de
Format
danielnugraha b548143
Update framework/py/flwr/supercore/cli/flower_agent.py
danielnugraha 0fd81a9
Rename
danielnugraha 021d667
Format
danielnugraha 5021759
Format
danielnugraha e970ef3
Refactor
danielnugraha 434a5f9
Format
danielnugraha 48ff2ed
Refactor to agentapp
danielnugraha 63a7b34
Refactor telemetry name
danielnugraha f0741de
Refactor to agentapp
danielnugraha 8a1ad14
Pylint fix
danielnugraha 5031b35
feat(framework): Add flwr-model
danielnugraha a785c57
feat(framework): Add flwr-connector
danielnugraha fe6f709
Format
danielnugraha 817fa69
Format
danielnugraha e1b6f26
Format
danielnugraha 51eefc1
Rename dir
danielnugraha 706ac49
Rename dir
danielnugraha f77aa13
Rename dir
danielnugraha bc04644
Rename
danielnugraha 9d69031
Merge branch 'add-flwr-agent' into add-flwr-model
danielnugraha 17371b7
Rename
danielnugraha 242f584
Merge branch 'add-flwr-model' into add-flwr-connector
danielnugraha 278d92d
Rename
danielnugraha 5b6d1b1
Rename
danielnugraha 3e29c84
Merge branch 'add-flwr-agent' into add-flwr-model
danielnugraha 5e210d9
Rename
danielnugraha c16871c
Merge branch 'main' into add-flwr-model
danielnugraha e205c8e
Remove test
danielnugraha 8b8fe88
Merge remote-tracking branch 'refs/remotes/origin/add-flwr-model' int…
danielnugraha ea9f3f7
Merge branch 'add-flwr-model' into add-flwr-connector
danielnugraha 0a4a4d4
Remove test
danielnugraha File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # Copyright 2026 Flower Labs GmbH. All Rights Reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # ============================================================================== | ||
| """`flwr-connector` command.""" | ||
|
|
||
|
|
||
| import argparse | ||
| from logging import DEBUG, INFO | ||
| from queue import Queue | ||
|
|
||
| from flwr.common.args import add_args_flwr_app_common | ||
| from flwr.common.constant import SERVERAPPIO_API_DEFAULT_CLIENT_ADDRESS | ||
| from flwr.common.exit import ExitCode, flwr_exit | ||
| from flwr.common.logger import log, mirror_output_to_queue, restore_output | ||
| from flwr.supercore.executors.run_connector import run_connector | ||
|
|
||
|
|
||
| def flwr_connector() -> None: | ||
| """Run process-isolated Flower ConnectorApp.""" | ||
| args = _parse_args_run_flwr_connector().parse_args() | ||
|
|
||
| if not args.insecure: | ||
| flwr_exit( | ||
| ExitCode.COMMON_TLS_NOT_SUPPORTED, | ||
| "`flwr-connector` does not support TLS yet.", | ||
| ) | ||
|
|
||
| # Capture stdout/stderr | ||
| log_queue: Queue[str | None] = Queue() | ||
| mirror_output_to_queue(log_queue) | ||
|
|
||
| log(INFO, "Start `flwr-connector` process") | ||
| log( | ||
| DEBUG, | ||
| "`flwr-connector` will attempt to connect to SuperLink's " | ||
| "ServerAppIo API at %s", | ||
| args.serverappio_api_address, | ||
| ) | ||
| run_connector( | ||
| serverappio_api_address=args.serverappio_api_address, | ||
| log_queue=log_queue, | ||
| token=args.token, | ||
| certificates=None, | ||
| parent_pid=args.parent_pid, | ||
| runtime_dependency_install=args.runtime_dependency_install, | ||
| ) | ||
|
|
||
| # Restore stdout/stderr | ||
| restore_output() | ||
|
|
||
|
|
||
| def _parse_args_run_flwr_connector() -> argparse.ArgumentParser: | ||
| """Parse `flwr-connector` command line arguments.""" | ||
| parser = argparse.ArgumentParser( | ||
| description="Run a Flower ConnectorApp", | ||
| ) | ||
| parser.add_argument( | ||
| "--serverappio-api-address", | ||
| default=SERVERAPPIO_API_DEFAULT_CLIENT_ADDRESS, | ||
| type=str, | ||
| help="Address of SuperLink's ServerAppIo API (IPv4, IPv6, or a domain name)." | ||
| f"By default, it is set to {SERVERAPPIO_API_DEFAULT_CLIENT_ADDRESS}.", | ||
| ) | ||
| add_args_flwr_app_common(parser=parser) | ||
| return parser | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| # Copyright 2026 Flower Labs GmbH. All Rights Reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # ============================================================================== | ||
| """Tests for ConnectorApp process CLI parsing and wiring.""" | ||
|
|
||
|
|
||
| import importlib | ||
| from types import SimpleNamespace | ||
| from unittest.mock import Mock, patch | ||
|
|
||
| import pytest | ||
|
|
||
| from flwr.common.constant import SERVERAPPIO_API_DEFAULT_CLIENT_ADDRESS | ||
|
|
||
| from .flwr_connector import _parse_args_run_flwr_connector | ||
|
|
||
| flwr_connector_module = importlib.import_module("flwr.supercore.cli.flwr_connector") | ||
|
|
||
|
|
||
| def test_parse_flwr_connector_requires_token() -> None: | ||
| """The ConnectorApp process CLI should require a token.""" | ||
| with pytest.raises(SystemExit): | ||
| _parse_args_run_flwr_connector().parse_args([]) | ||
|
|
||
|
|
||
| def test_parse_flwr_connector_rejects_run_once() -> None: | ||
| """The removed deprecated flag should no longer parse.""" | ||
| with pytest.raises(SystemExit): | ||
| _parse_args_run_flwr_connector().parse_args( | ||
| ["--token", "test-token", "--run-once"] | ||
| ) | ||
|
|
||
|
|
||
| def test_parse_flwr_connector_parses_tokenized_invocation() -> None: | ||
| """The ConnectorApp process CLI should still parse the supported flags.""" | ||
| args = _parse_args_run_flwr_connector().parse_args( | ||
| [ | ||
| "--token", | ||
| "test-token", | ||
| "--insecure", | ||
| "--parent-pid", | ||
| "1234", | ||
| "--allow-runtime-dependency-installation", | ||
| ] | ||
| ) | ||
|
|
||
| assert args.serverappio_api_address == SERVERAPPIO_API_DEFAULT_CLIENT_ADDRESS | ||
| assert args.token == "test-token" | ||
| assert args.insecure is True | ||
| assert args.parent_pid == 1234 | ||
| assert args.runtime_dependency_install is True | ||
|
|
||
|
|
||
| def test_flwr_connector_parses_args_before_mirroring_output() -> None: | ||
| """Argument parsing should happen before stdout/stderr redirection.""" | ||
|
|
||
| class _Parser: | ||
| def parse_args(self) -> SimpleNamespace: | ||
| """Raise a parser error before any side effects happen.""" | ||
| raise SystemExit(2) | ||
|
|
||
| mirror_output_to_queue = Mock() | ||
|
|
||
| with ( | ||
| patch.object(flwr_connector_module, "_parse_args_run_flwr_connector", _Parser), | ||
| patch.object( | ||
| flwr_connector_module, | ||
| "mirror_output_to_queue", | ||
| mirror_output_to_queue, | ||
| ), | ||
| pytest.raises(SystemExit), | ||
| ): | ||
| flwr_connector_module.flwr_connector() | ||
|
|
||
| mirror_output_to_queue.assert_not_called() | ||
|
|
||
|
|
||
| def test_flwr_connector_forwards_cli_args() -> None: | ||
| """The ConnectorApp CLI should forward parsed args to the runtime.""" | ||
| args = SimpleNamespace( | ||
| insecure=True, | ||
| serverappio_api_address="127.0.0.1:9091", | ||
| token="test-token", | ||
| parent_pid=321, | ||
| runtime_dependency_install=True, | ||
| ) | ||
|
|
||
| class _Parser: | ||
| def parse_args(self) -> SimpleNamespace: | ||
| """Return a fixed namespace for CLI forwarding tests.""" | ||
| return args | ||
|
|
||
| mirror_output_to_queue = Mock() | ||
| restore_output = Mock() | ||
| run_connector = Mock() | ||
|
|
||
| with ( | ||
| patch.object(flwr_connector_module, "_parse_args_run_flwr_connector", _Parser), | ||
| patch.object( | ||
| flwr_connector_module, | ||
| "mirror_output_to_queue", | ||
| mirror_output_to_queue, | ||
| ), | ||
| patch.object(flwr_connector_module, "restore_output", restore_output), | ||
| patch.object(flwr_connector_module, "run_connector", run_connector), | ||
| ): | ||
| flwr_connector_module.flwr_connector() | ||
|
|
||
| mirror_output_to_queue.assert_called_once() | ||
| restore_output.assert_called_once_with() | ||
| run_connector.assert_called_once() | ||
| kwargs = run_connector.call_args.kwargs | ||
| assert kwargs["serverappio_api_address"] == "127.0.0.1:9091" | ||
| assert kwargs["log_queue"] is mirror_output_to_queue.call_args.args[0] | ||
| assert kwargs["token"] == "test-token" | ||
| assert kwargs["certificates"] is None | ||
| assert kwargs["parent_pid"] == 321 | ||
| assert kwargs["runtime_dependency_install"] is True |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,75 @@ | ||||||
| # Copyright 2026 Flower Labs GmbH. All Rights Reserved. | ||||||
| # | ||||||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||||||
| # you may not use this file except in compliance with the License. | ||||||
| # You may obtain a copy of the License at | ||||||
| # | ||||||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||||||
| # | ||||||
| # Unless required by applicable law or agreed to in writing, software | ||||||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||||||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||||
| # See the License for the specific language governing permissions and | ||||||
| # limitations under the License. | ||||||
| # ============================================================================== | ||||||
| """`flwr-model` command.""" | ||||||
|
|
||||||
|
|
||||||
| import argparse | ||||||
| from logging import DEBUG, INFO | ||||||
| from queue import Queue | ||||||
|
|
||||||
| from flwr.common.args import add_args_flwr_app_common | ||||||
| from flwr.common.constant import SERVERAPPIO_API_DEFAULT_CLIENT_ADDRESS | ||||||
| from flwr.common.exit import ExitCode, flwr_exit | ||||||
| from flwr.common.logger import log, mirror_output_to_queue, restore_output | ||||||
| from flwr.supercore.executors.run_model import run_model | ||||||
|
|
||||||
|
|
||||||
| def flwr_model() -> None: | ||||||
| """Run process-isolated Flower ModelApp.""" | ||||||
| args = _parse_args_run_flwr_model().parse_args() | ||||||
|
|
||||||
| if not args.insecure: | ||||||
| flwr_exit( | ||||||
| ExitCode.COMMON_TLS_NOT_SUPPORTED, | ||||||
| "`flwr-model` does not support TLS yet.", | ||||||
| ) | ||||||
|
|
||||||
| # Capture stdout/stderr | ||||||
| log_queue: Queue[str | None] = Queue() | ||||||
| mirror_output_to_queue(log_queue) | ||||||
|
|
||||||
| log(INFO, "Start `flwr-model` process") | ||||||
| log( | ||||||
| DEBUG, | ||||||
| "`flwr-model` will attempt to connect to SuperLink's ServerAppIo API at %s", | ||||||
| args.serverappio_api_address, | ||||||
| ) | ||||||
| run_model( | ||||||
| serverappio_api_address=args.serverappio_api_address, | ||||||
| log_queue=log_queue, | ||||||
| token=args.token, | ||||||
| certificates=None, | ||||||
| parent_pid=args.parent_pid, | ||||||
| runtime_dependency_install=args.runtime_dependency_install, | ||||||
| ) | ||||||
|
|
||||||
| # Restore stdout/stderr | ||||||
| restore_output() | ||||||
|
|
||||||
|
|
||||||
| def _parse_args_run_flwr_model() -> argparse.ArgumentParser: | ||||||
| """Parse `flwr-model` command line arguments.""" | ||||||
| parser = argparse.ArgumentParser( | ||||||
| description="Run a Flower ModelApp", | ||||||
| ) | ||||||
| parser.add_argument( | ||||||
| "--serverappio-api-address", | ||||||
| default=SERVERAPPIO_API_DEFAULT_CLIENT_ADDRESS, | ||||||
| type=str, | ||||||
| help="Address of SuperLink's ServerAppIo API (IPv4, IPv6, or a domain name)." | ||||||
|
||||||
| help="Address of SuperLink's ServerAppIo API (IPv4, IPv6, or a domain name)." | |
| help="Address of SuperLink's ServerAppIo API (IPv4, IPv6, or a domain name). " |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
help=string concatenation is missing a separating space/newline between sentences, soargparse --helpwill render as...).By default, .... Consider adding an explicit space at the end of the first literal (or using a single combined literal) for readability.