Adding support for pydantic extras in models #732

ModdingFox · 2025-07-03T06:01:57Z

This PR adds support for Pydantic models with extra field configuration (allow, forbid, ignore).

Key changes:

Models with extra="allow" now accept and include additional fields during parsing and instantiation
Models with extra="forbid" properly reject extra fields with clear error messages
Models with extra="ignore" accept extra fields during parsing but ignore them during instantiation
Support for both Pydantic v1 and v2 syntax and configuration styles

Behavior:

extra="allow": Extra fields are accepted and included in the final model
extra="forbid": Extra fields cause a NSKeyError during parsing
extra="ignore": Extra fields are accepted during parsing but ignored during model instantiation
Default behavior remains unchanged for models without explicit extra configuration

This resolves the limitation where jsonargparse would always reject extra fields regardless of the Pydantic model's extra configuration.

Did you read the contributing guideline?
Did you update the documentation? (readme and public docstrings)
Did you write unit tests such that there is 100% coverage on related code? (required for bug fixes and new features)
Did you verify that new and existing tests pass locally?
Did you make sure that all changes preserve backward compatibility?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

codecov · 2025-07-06T06:28:20Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (6ad1dd6) to head (0940383).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #732   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        22           
  Lines         6857      6895   +38     
=========================================
+ Hits          6857      6895   +38

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mauvilsa

Thank you for contributing!

Overall looks good. But it seems that the tests don't cover the code. Every line should be covered.

ModdingFox · 2025-07-07T15:26:56Z

Oh wired the full pytest report had it at 100% locally. Will pull the pr back to fresh env and double check ^.^

ModdingFox · 2025-07-07T21:32:25Z

Hopefully this Dockerfile is useful to someone else.

FROM debian:bookworm

# System update and install build tools
RUN apt update
RUN apt upgrade -y
RUN apt install -y build-essential curl git

# Build user
RUN useradd -ms /bin/bash build
USER build
WORKDIR /home/build

# Mise setup
RUN curl https://mise.run | sh
ENV PATH="/home/build/.local/bin:$PATH"

# Python env setup
RUN mise install [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
RUN mise use -g [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
ENV PATH="/home/build/.local/share/mise/shims/:$PATH"

# Init build tool venv
RUN mise exec [email protected] -- python3 -m venv venv
ENV PATH="/home/build/venv/bin:$PATH"

# Clone repo for build and install build tools
RUN git clone https://github.com/ModdingFox/jsonargparse.git
# COPY --chown=build:build . .
# RUN git config --global --add safe.directory /home/build
WORKDIR /home/build/jsonargparse
RUN git checkout pydantic_allow_extra
RUN pip3 install -e ".[dev,all]"

# Setup git pre-commit hook
RUN pre-commit install

# Generate docs
RUN sphinx-build sphinx sphinx/_build sphinx/*.rst

# Run all tests
RUN rm -fr jsonargparse_tests/.coverage jsonargparse_tests/htmlcov
RUN tox -- --cov=../jsonargparse --cov-append
WORKDIR /home/build/jsonargparse/jsonargparse_tests
RUN coverage html

# Setup continer for exec access
WORKDIR /home/build/jsonargparse
ENTRYPOINT ["/bin/bash"]

ModdingFox · 2025-07-15T20:39:54Z

Following up here. Any additional change or comment to this pr?

mauvilsa

Sorry, I have been busy.

Now I have looked in more details and I have several comments. Note that many comments I wrote only once or twice, but the same would apply in other places where the same pattern appears.

mauvilsa · 2025-07-15T13:13:30Z

jsonargparse/_core.py

+                        # Check if this is a Pydantic model with extra configuration
+                        group = self.groups[group_key]
+                        should_raise_error = True
+                        if hasattr(group, "group_class") and group.group_class:


Suggested change

if hasattr(group, "group_class") and group.group_class:

if getattr(group, "group_class", None):

mauvilsa · 2025-07-15T13:20:00Z

jsonargparse/_core.py

+                        should_raise_error = True
+                        if hasattr(group, "group_class") and group.group_class:
+                            from ._optionals import get_pydantic_extra_config
+
+                            extra_config = get_pydantic_extra_config(group.group_class)
+                            if extra_config == "allow":
+                                # Allow extra fields - don't raise an error
+                                should_raise_error = False
+                            elif extra_config == "ignore":
+                                # Ignore extra fields - don't raise an error, Pydantic will ignore during instantiation
+                                should_raise_error = False


All this logic seems to be pydantic-specific. So it might be better to have all of it in a function imported from optionals, e.g. is_allowed_by_pydantic_extra(...). The function get_pydantic_extra_config could still exist just to not have one single big function.

mauvilsa · 2025-07-15T13:26:22Z