feat: optional flag to mount `/tmp` as tmpfs on container creation

[alehostert]

Summary

Mount /tmp as tmpfs by default for all container deployments, with a standard size of 1 GB (configurable per plan, applied at container creation). No user-facing toggle — this becomes a platform-level default, invisible to the end user unless they need to customize the size.

Rationale

Investigated a case where a container's overlay upperdir was accumulating stale files in /tmp (#sql-temptable-*.MAD from MySQL, sess_* from PHP session GC) that were invisible via ls/stat inside the container but present on disk in the layer's diff/ directory.

Root cause: containers run on fuse-overlayfs (rootless podman default), and /tmp is not a separate mountpoint — it's just a regular directory inside the FUSE-backed overlay root. fuse-overlayfs has known directory-cache staleness issues under high create/unlink churn, which is exactly the pattern PHP sessions and MySQL temp tables produce. Result: writes land in the upperdir but become invisible to the process's own directory lookups, so the app never cleans them up — silent, unbounded disk growth over time (170k stale files found in one container in this investigation).

Mounting /tmp as tmpfs sidesteps this at the root:

• /tmp writes go to RAM, never touch the FUSE overlay, so the staleness bug can't manifest there.
• For PHP + MySQL stacks specifically, /tmp is a hot path — PHP session files and MySQL implicit temp tables (sorts, joins without indexes, ORDER BY/GROUP BY spills) are written there constantly. Moving this to tmpfs is a meaningful I/O latency win on top of fixing the leak.
• Data in /tmp is inherently transient by convention, so losing it on container restart is an acceptable tradeoff (and is standard practice for tmpfs /tmp setups outside containers too).

Why default-on (invisible to user)

This is a platform-level reliability fix, not a feature toggle. Most workloads will benefit silently, and there's no reason for the average user to opt out. Key design decisions:

• No UI toggle — the tmpfs mount is added transparently at container creation. The user doesn't see it, doesn't need to think about it.
• Standard size: 1 GB — enough for typical PHP session + MySQL temp table churn.
• Plan-linked sizing — the tmpfs size is defined per plan and applied at container creation time. Changing the plan only affects containers created after the change.

Why not leave it opt-in?

• The fuse-overlayfs staleness bug affects all containers that write to /tmp under churn. Making it opt-in means the people who need it most won't know to enable it until they have a problem.
• Enabling it by default on existing running containers isn't possible without recreation (tmpfs mounts are create-time only, not hot-appliable), so starting fresh with new deployments is the cleanest approach.
• If a container genuinely needs disk-backed /tmp (e.g., writing very large temporary files), the user can open a ticket to discuss alternatives — but that's the exception, not the rule.

Important: tmpfs size is set at creation time and cannot be changed on a running container

Tmpfs is mounted via the mount() syscall at container startup, and Podman does not support resizing a tmpfs mount on a running container — there is no podman update --tmpfs-size or equivalent API. The only way to change the tmpfs size for a given container is to recreate it with the new value.

This has two practical implications:

1. Get the size right at creation. The plan-level default needs to be generous enough for the workload it serves, because there's no hotfix — if tmpfs fills up, the container must be recreated with a larger size.
2. Monitor /tmp usage. Since remediation requires recreation, it's important to track tmpfs utilization per container (via host metrics or guest-side agents) so we can act before it fills up, not after.

Proposed implementation

Container creation

• At container creation time, always add --tmpfs /tmp:rw,size=<plan_size> to the podman run invocation.
• No changes to existing containers — they keep their current overlay-backed /tmp.

Size determination

• Default: 1 GB for all plans initially.
• Future: derive from the plan's compute offering (e.g., a fraction of available RAM, with a floor/ceiling).
• The size must be capped to avoid host OOM if too many containers max out their tmpfs simultaneously. A safe heuristic: min(plan_ram * 0.1, 2 GB).

Plan-level configuration (applies to new containers only)

• Expose an optional tmpfs_size field in the plan configuration. If unset, falls back to the plan default.
• Changing this value does not affect running containers — it only takes effect on the next container creation for that plan.
• This keeps complexity away from the user but still allows tailored sizing for specific plans or application needs.

Verification

• Confirmed today that HostConfig.Tmpfs is empty (map[]) on all currently running containers — this is a net-new capability, not a regression fix.

Example (what the platform does internally)

$ podman run --rm --name os \
    --env PRIMARY_VHOST=myapp.net \
    --env LOG_LEVEL=debug \
    --tmpfs /tmp:rw,size=1G \
    -it docker.io/goinfinite/os:0.3.1

Inside the container:

$ df -h /tmp && mount | grep /tmp
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.0G     0  1.0G   0% /tmp
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,context="system_u:object_r:container_file_t:s0:c202,c629",size=1048576k,uid=1000,gid=1000,inode64)

Caveats / risks

• RAM pressure: 1 GB per container adds up. A host with 20 containers all writing heavily to /tmp could see up to 20 GB of tmpfs consumption. Mitigation: cap size, monitor host memory, and consider setting a podman tmpfs-size limit at the host level (podman info → host.tmpDir).
• OLS / heavy writers: OpenLiteSpeed and similar web servers can write substantial data to /tmp. If tmpfs fills up, writes fail with ENOSPC. Monitor for this and have a playbook to either increase the size (requiring recreation) or fall back to disk-backed /tmp per container.
• No live migration: This only applies to newly created containers. Existing containers keep their current behavior until recreated. Any plan-level tmpfs size change also requires recreation to take effect.