[ENVIRONMENT] Connect4 Environment #101

GboyeStack-Robotics-ML-Engineer · 2025-10-27T14:12:08Z

Add Connect4 Environment and Supporting Models to OpenEnv

Summary

This pull request introduces a new Connect4 Environment to the OpenEnv framework, expanding its suite of interactive learning environments.
The Connect4 environment follows OpenEnv’s modular design and integrates seamlessly with the existing Environment, Action, and Observation interfaces.

🔑 Key Features

🧠 New Environment
- Implements full Connect4 game logic: move validation, piece placement, win/draw detection.
- Compatible with OpenEnv’s standard reset and step lifecycle.
- Supports unique episode management with uuid.
⚙️ Model Additions
- Connect4Action – Defines valid column selections as agent actions.
- Connect4State – Represents board state, active player, and step count.
- Connect4Observation – Encapsulates board view, legal actions, reward signal, and done status.
🎮 Gameplay Logic
- Detects 4-in-a-row in four directions (vertical, horizontal, diagonal-down, diagonal-up).
- Prevents invalid moves (full columns or out-of-range actions).
- Alternates between players automatically.
- Tracks rewards and termination conditions for RL agents.
🧩 Visualization Support
- Optional rendering enhancements to display step count and reward during play.
- Board updates compatible with existing OpenEnv visualization tools.

Testing out the env
To test out the environment, open two terminals

On terminal 1:
Run the following from root dir (OpenEnv/)

cd src
Then start the server:
- python -m envs.connect4_env.server.app

On the terminal 2:
Run python examples/connect4.py from root directory (OpenEnv/)

This is an example of the output:

meta-cla · 2025-10-27T14:12:14Z

Hi @GboyeStack-Robotics-ML-Engineer!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

meta-cla · 2025-10-27T15:09:00Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

pankit-eng · 2025-10-30T23:08:49Z

examples/atari_simple.py

 import numpy as np
 from envs.atari_env import AtariEnv, AtariAction
-
+# import envs


remove debug code?

pankit-eng · 2025-10-30T23:09:53Z

examples/atari_simple.py

        print("\nResetting environment...")
        result = env.reset()
        print(f"Screen shape: {result.observation.screen_shape}")
+


Copilot

Pull Request Overview

This PR adds a new Connect4 environment integration to the OpenEnv framework, implementing a two-player Connect 4 game that can be accessed via HTTP server. This follows the OpenEnv pattern of server-side environment implementation with client-side HTTP access.

Implements Connect4Environment server with game logic for a 6x7 board
Provides type-safe client wrapper (Connect4Env) for HTTP communication
Adds Docker support and GitHub Actions workflow integration
Includes example script with visualization using matplotlib

Reviewed Changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/envs/connect4_env/server/connect4_environment.py	Implements core Connect4 game logic with win detection and state management
src/envs/connect4_env/server/app.py	Creates FastAPI app wrapper for the Connect4 environment server
src/envs/connect4_env/server/init.py	Exports Connect4Environment for server module
src/envs/connect4_env/server/Dockerfile	Defines Docker image for running Connect4 server
src/envs/connect4_env/models.py	Defines type-safe Action, Observation, and State models
src/envs/connect4_env/client.py	HTTP client for communicating with Connect4 server
src/envs/connect4_env/init.py	Main module exports for Connect4 environment
examples/connect4.py	Example script demonstrating Connect4 environment usage with visualization
examples/atari_simple.py	Modified to use localhost URL instead of Docker image
examples/OpenEnv_Tutorial.ipynb	Updated with execution outputs and kernel version
.github/workflows/docker-build.yml	Adds Connect4 environment to Docker build workflow

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-10-31T00:29:42Z

src/envs/connect4_env/server/Dockerfile

+    CMD curl -f http://localhost:8000/health || exit 1
+
+# Run server
+CMD ["uvicorn", "envs.my_env.server.app:app", "--host", "0.0.0.0", "--port", "8000"]


The CMD directive references 'envs.my_env.server.app:app' but should reference 'envs.connect4_env.server.app:app' to match the actual module path. This will cause the server to fail to start.

Suggested change

CMD ["uvicorn", "envs.my_env.server.app:app", "--host", "0.0.0.0", "--port", "8000"]

CMD ["uvicorn", "envs.connect4_env.server.app:app", "--host", "0.0.0.0", "--port", "8000"]

Copilot · 2025-10-31T00:29:42Z

src/envs/connect4_env/__init__.py

+This module provides OpenEnv integration for Connect4 2600 games via the
+Arcade Learning Environment (ALE).
+


The documentation incorrectly refers to 'Connect4 2600 games via the Arcade Learning Environment (ALE)'. Connect4 is a board game, not an Atari 2600 game. This appears to be copied from the Atari environment template and should be updated to describe Connect4 specifically.

Suggested change

This module provides OpenEnv integration for Connect4 2600 games via the

Arcade Learning Environment (ALE).

This module provides OpenEnv integration for the classic Connect4 board game.

Copilot · 2025-10-31T00:29:42Z

src/envs/connect4_env/client.py

+        >>> print(result.observation.board)
+        >>>
+        >>> # Take an action
+        >>> result = client.step(Connect4Action(action_id=3))


The example uses 'action_id=3' but Connect4Action expects 'column' as the parameter name according to the model definition. This should be 'Connect4Action(column=3)'.

Suggested change

>>> result = client.step(Connect4Action(action_id=3))

>>> result = client.step(Connect4Action(column=3))

Copilot · 2025-10-31T00:29:42Z

examples/connect4.py

+                break
+
+            action_id = int(np.random.choice(result.observation.legal_actions))
+            result = env.step(Connect4Action(action_id))


Connect4Action expects a 'column' parameter, not a positional argument. This should be 'Connect4Action(column=action_id)' to match the dataclass definition.

Suggested change

result = env.step(Connect4Action(action_id))

result = env.step(Connect4Action(column=action_id))

Copilot · 2025-10-31T00:29:43Z

examples/connect4.py

+                return ax.patches
+
+            # Create animation
+            ani = FuncAnimation(fig, animate_frame, frames=len(frames), 


Variable ani is not used.

Copilot · 2025-10-31T00:29:43Z

src/envs/connect4_env/server/connect4_environment.py

+
+    def step(self, action: Connect4Action):
+        col = action.column
+        reward = 0.0


This assignment to 'reward' is unnecessary as it is redefined before this value is used.
This assignment to 'reward' is unnecessary as it is redefined before this value is used.

Suggested change

reward = 0.0

Copilot · 2025-10-31T00:29:44Z

examples/connect4.py

+from envs.connect4_env import Connect4Action, Connect4Env
+
+
+def render_connect4_board(board, ax, player_colors={1: "red", 2: "yellow", -1: "yellow"}, show=True):


Mixing implicit and explicit returns may indicate an error, as implicit returns always return None.

Copilot · 2025-10-31T00:29:44Z

src/envs/connect4_env/server/app.py

@@ -0,0 +1,12 @@
+from core.env_server import create_fastapi_app
+from ..models import Connect4Action, Connect4Observation, Connect4State


Import of 'Connect4State' is not used.

Suggested change

from ..models import Connect4Action, Connect4Observation, Connect4State

from ..models import Connect4Action, Connect4Observation

Copilot · 2025-10-31T00:29:44Z

src/envs/connect4_env/client.py

+from .models import Connect4Action, Connect4Observation, Connect4State
+
+if TYPE_CHECKING:
+    from core.containers.runtime import ContainerProvider


Import of 'ContainerProvider' is not used.

Suggested change

from core.containers.runtime import ContainerProvider

Copilot · 2025-10-31T00:29:44Z

src/envs/connect4_env/server/connect4_environment.py

@@ -0,0 +1,90 @@
+import uuid
+import numpy as np
+from core.env_server import Action, Environment, Observation


Import of 'Action' is not used.
Import of 'Observation' is not used.

Suggested change

from core.env_server import Action, Environment, Observation

from core.env_server import Environment

…neer/OpenEnv

…-ML-Engineer/OpenEnv into connect4_env

Darktex

Looks really good. Just a couple of changes :)

Can you ask your favorite LLM to write tests for the connect4 logic? I love that we don't have to use a library for it, but I'm always worried that we may have some off-by-one errors here and there...
Can you address the minor nits that Copilot found?

That's it!!

GboyeStack-Robotics-ML-Engineer · 2025-10-31T19:00:03Z

@Darktex I already addressed the minor issues that copilot raised. I have commited this. As for the tests I have included a unittest file under the test folder

Darktex · 2025-11-03T23:24:36Z

Thank you so much!!

GboyeStack-Robotics-ML-Engineer added 3 commits October 26, 2025 12:51

Configuring the connect4_env using the atari template

3628bf7

fixing laging behaviour of environment

308e310

Adding rendering logic to the connect 4 environment

964be28

meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 27, 2025

pankit-eng approved these changes Oct 30, 2025

View reviewed changes

Darktex requested a review from Copilot October 31, 2025 00:27

Merge branch 'main' into connect4_env

b57b48d

Copilot AI reviewed Oct 31, 2025

View reviewed changes

GboyeStack-Robotics-ML-Engineer added 5 commits October 31, 2025 15:23

Making changes as requested

00eb136

Merge branch 'main' of https://github.com/GboyeStack-Robotics-ML-Engi…

a7dabb0

…neer/OpenEnv

Merge branch 'main' into connect4_env

ed2dc3a

Merge branch 'connect4_env' of https://github.com/GboyeStack-Robotics…

5652e8a

…-ML-Engineer/OpenEnv into connect4_env

fixing atari example

c0f7833

Darktex requested changes Oct 31, 2025

View reviewed changes

Darktex mentioned this pull request Oct 31, 2025

Added Connect4 Env wrapped on Gym #96

Open

addin unittest for connect4_env

4a6bf6b

Darktex merged commit fca4d4a into meta-pytorch:main Nov 3, 2025
1 check passed

	CMD ["uvicorn", "envs.my_env.server.app:app", "--host", "0.0.0.0", "--port", "8000"]
	CMD ["uvicorn", "envs.connect4_env.server.app:app", "--host", "0.0.0.0", "--port", "8000"]

		This module provides OpenEnv integration for Connect4 2600 games via the
		Arcade Learning Environment (ALE).

	>>> result = client.step(Connect4Action(action_id=3))
	>>> result = client.step(Connect4Action(column=3))

	result = env.step(Connect4Action(action_id))
	result = env.step(Connect4Action(column=action_id))

		from envs.connect4_env import Connect4Action, Connect4Env


		def render_connect4_board(board, ax, player_colors={1: "red", 2: "yellow", -1: "yellow"}, show=True):

		@@ -0,0 +1,12 @@
		from core.env_server import create_fastapi_app
		from ..models import Connect4Action, Connect4Observation, Connect4State

	from core.env_server import Action, Environment, Observation
	from core.env_server import Environment

[ENVIRONMENT] Connect4 Environment #101

[ENVIRONMENT] Connect4 Environment #101

Uh oh!

Conversation

GboyeStack-Robotics-ML-Engineer commented Oct 27, 2025

Summary

🔑 Key Features

Uh oh!

meta-cla bot commented Oct 27, 2025

Action Required

Process

Uh oh!

meta-cla bot commented Oct 27, 2025

Uh oh!

pankit-eng Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

pankit-eng Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Darktex left a comment

Choose a reason for hiding this comment

Uh oh!

GboyeStack-Robotics-ML-Engineer commented Oct 31, 2025

Uh oh!

Darktex commented Nov 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants