Skip to content

Prevent yml data corruption#862

Draft
TechnicallyCoded wants to merge 1 commit intobooksaw:masterfrom
TechnicallyCoded:feature/data-integrity
Draft

Prevent yml data corruption#862
TechnicallyCoded wants to merge 1 commit intobooksaw:masterfrom
TechnicallyCoded:feature/data-integrity

Conversation

@TechnicallyCoded
Copy link
Copy Markdown

This is not intended to be merged in its current state. I am only opening this PR draft in order to share this solution with anyone who may want/need it.

What this edit does:

  • Changes the behavior of the plugin when saving teams.yml (data file) in order to avoid data loss if your storage device runs out of space (or other filesystem issue). In this order:
    • Copy the existing teams.yml file to teams.yml.bak
    • First create a new file: teams.yml.pre and write the new data
    • Rename teams.yml.pre to teams.yml overwriting the existing teams.yml file with the new data

If during any step, a write is prevented, the list of tasks is aborted and does not proceed. In this scenario, any new data is discarded but the integrity of the file is preserved, preventing a total loss of data. If the file writing process is interrupted in a weird way (during shutdown / force-kill / etc) the .bak is still present and would allow the admins to simply restore this version.

Realistically, a production ready version of this would probably omit the .bak file and preserve the rest.

@booksaw
Copy link
Copy Markdown
Owner

booksaw commented Sep 9, 2025

This looks interesting, is this something you have seen happen on your server before?

@TechnicallyCoded
Copy link
Copy Markdown
Author

Yep, due to storage running out when a plugin misbehaved and wrote gigabytes per second to disk. Any plugin using flatfile storage failed and all data was deleted. Backups were pulled, but obviously, a daily backup is only as up to date as its last save, making us lose data regardless. This method guarantees that storage is available before overwriting by writing first and swapping the file in if it succeeds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants