feat: continue-on-error flag for handling errors while processing multiple files

[iyilmaz24]

• Currently: If multiple files are specified, mark starts processing them but exits on the first error - leaving some files untouched
• Proposed: Adding a continue-on-error flag, that does not change above behavior unless set to true, which in that case will process the entire batch of files & log any encountered errors to console.
• Purpose: This should improve user's experience using mark with batch files, as it makes subsequent runs for error'd batches easier

I would like the maintainer's thoughts on the overall idea / PR but also:

• Formerly on some errors, we only logged the error
• 
• Now in addition, we can also log a helpful string message. I commented out all the changed errors throughout the code and would like to hear thoughts on the new string messages and which ones you dislike or want to replace.

I have also added some tests in a temporary batch-tests directory so that the proposed functionality can be tested.

• This command can be used for testing ./mark --compile-only --continue-on-error -files "**/testdata/batch-tests/*.md"

Note: there is a pattern where we handle the error with fatalErrorHandler and then return nil after. I was not able to directly return fatalErrorHandler(args...) due to the processFile function signature returning *confluence.PageInfo. Any sidesteps / alternatives to this are also welcomed (we are able to return nil since a pointer can be nil).

Possible further enhancements: Outputting a errors.txt file for users containing the names (and maybe also error messages) of all the error'd files for easy retrial

[iyilmaz24]

Hi, I was wondering if this was ready to merge - I was looking into creating some batch tests for this functionality. Would it be better to add them to this PR or make a new PR?

[mrueg]

feel free to include them here, it would be good if they were executable using the regular tests or bats acceptance tests if necessary. Currently the PR needs to be rebased, as there are file conflicts.

[iyilmaz24]

Hi, the batch tests have been added - with tons of help from @richscott (thanks!)

We also made the decision to refactor/relocate some of the code from the main.go file into a util package. It was necessary to import certain functions and structs for the new batch tests which wasn't possible with the previous code organization. We hope this code relocation also helps in the future with extensibility. Excited to hear any thoughts on the changes 🙂

[mrueg]

Thanks for adding tests for this! Can you rebase on the latest master? Otherwise I won't be able to merge the changes.

Making main.go slim was something I had in mind as well as part of #499 so I don't mind the temporary utils package (utils is usually not the best pattern, because it just means that random stuff gets dropped into a single package), I'll work on the reorganization later once urfave/cli/v3 is out (splitting CLI parsing from the rest of the logic).

[iyilmaz24]

Hello, I was able to rebase the PR onto the master branch and it should be good to go. 🙂

[richscott]

I agree completely about the util/ directory - it's definitely an example of the "utils folder kitchen-junk-drawer" anti-pattern. I thought about different package topologies, but didn't feel confident about any that would be durably correct, so out of expediency I made util for now, so Irfan could get the continue-on-error logic into a unit-test.

[richscott]

HI, @mrueg - can you (or anyone else with privileges on this repo) restart these CI checks on this PR? They had the error The job was not started because recent account payments have failed or your spending limit needs to be increased. Please check the 'Billing & plans' section in your settings - I assume it's a transient error, maybe? We'd love to see this PR merged - thanks.

[mrueg]

HI, @mrueg - can you (or anyone else with privileges on this repo) restart these CI checks on this PR? They had the error The job was not started because recent account payments have failed or your spending limit needs to be increased. Please check the 'Billing & plans' section in your settings - I assume it's a transient error, maybe? We'd love to see this PR merged - thanks.

I've reached out to @kovetskiy a week ago, hope he find some time to respond.