Add --regex-rules option (RFC)

[nicklockwood]

This PR adds a new option --regex-rules that allows you to specify simple text replacement rules using regular expression syntax, as follows:

--regex-rules urlMacro/URL\(string: ("[^\\]+")\)!/#URL($1)/

The example above would (mostly) replicate the existing urlMacro rule, making it possible for such custom rules to be implemented without needing to modify the library. This replicates a similar feature already available in SwiftLint.

Since it operates purely at the string level, it arguably doesn't need to be part of SwiftFormat at all, however it is able to advantage of SwiftFormat's file matching, caching and and parallel rule processing, as well as respecting // swiftformat:disable:next ruleName directives in the code, so it does make some sense to include it in the same tool.

I've wanted to add something like this for a while, but I'm not 100% convinced that it will be useful in practice. Much of the benefit of SwiftFormat's token-based parsing is that it can do sophisticated transformations without getting confused by things like arbitrarily-placed whitespace, comments, or recursive structures, which regex can't really cope with. It also can't replicate features like the existing urlMacro rule's ability to conditionally insert an import if the rule is matched.

The SwiftLint version also lets you restrict a rule to a particular token type, e.g. just identifiers, strings or comments, which I've not currently implemented - not because it's especially hard, but because I'm not sure how best to configure it on a per-regex basis given that we aren't using a hierarchical config format like YAML. I'm also not sure that would make a huge difference to the usefulness anyway.

Thoughts @calda? Is this something you'd find useful? Can you think of any improvements to the design?

[codecov]

Codecov Report

❌ Patch coverage is 93.50649% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.29%. Comparing base (b1d6cff) to head (9d06259).

Files with missing lines	Patch %	Lines
Sources/RegexRule.swift	85.71%	4 Missing ⚠️
Sources/SwiftFormat.swift	75.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2280      +/-   ##
===========================================
- Coverage    95.29%   95.29%   -0.01%     
===========================================
  Files          158      159       +1     
  Lines        23767    23835      +68     
===========================================
+ Hits         22649    22713      +64     
- Misses        1118     1122       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[calda]

Nice! This is cool. I think it definitely makes sense to offer.

We have 20-30 SwiftLint regex rules. Most of them are patterns that we just want to disallow. A couple of them are iOS APIs that we have an internal-only replacement for that is mostly API-compatible. Those examples would be nice to autocorrect with a SwiftFormat autocorrect regex instead of a lint-only rule. However like the #URL macro example, they would require additional imports, so aren't totally covered by this. I can think of a few more miscellaneous internal-only things I would be interested in adding (that don't require new imports) now that it would be so easy to try.

Even though there are a lot of limitations, seems worth having.

[calda]

Config could be a bit easier organize / read the config file if each rule had a separate --regex-rule flag, if this is possible:

--regex-rule urlMacro/URL\(string: ("[^\\]+")\)!/#URL($1)
--regex-rule otherAPI/oldAPI/newAPI

[calda]

If the regex rule had the ability to add new imports, it cover more of the use cases we have. Could be something like:

--regex-rule urlMacro/URL\(string: ("[^\\]+")\)!/#URL($1)/import URLFoundation

[nicklockwood]

Config could be a bit easier organize / read the config file if each rule had a separate --regex-rule flag, if this is possible:

As with all the comma-delimited options, this just works out of the box. If you pass the param multiple times then the results are concatenated. (Probably this should be made clearer in the docs somehow):

[nicklockwood]

If the regex rule had the ability to add new imports, it cover more of the use cases we have. Could be something like:

Nice idea, but this syntax really doesn't seem scalable. I feel like maybe it's time to consider moving to yaml or something?

I like that it's possible to provide all the config on the command line though, and I'm a bit reluctant to lose that. I wonder if there's some more readable way to pack arbitrary nested key/value pairs into a single option?

[calda]

Could the option take JSON content? That's more flexible than YAML since it isn't whitespace sensitive. You could put the JSON entirely on one line as needed e.g. over the command line, or if you prefer that organization in your config file.

Could we trivially support both of:

// Multiline for config file
--regex-rule {
  "name": "urlMacro",
  "find": "URL\(string: (\"[^\\]+\")\)!",
  "replace": "#URL($1)",
  "imports": ["URLFoundation"]
}

and:

// Single line if preferred / for command line
--regex-rule {"name": "urlMacro", "find": "URL\(string: (\"[^\\]+\")\)!", "replace": "#URL($1)", "imports": ["URLFoundation"]}

[calda]

Or we could accept both JSON and YAML. JSON feels necessary for the command line use case. For YAML it could look like:

// Multiline for config file
--regex-rule >
  name: urlMacro
  find: "URL\(string: (\"[^\\]+\")\)!"
  replace: "#URL($1)"
  imports:
    - "URLFoundation"

I like the feel of JSON more here though. The balanced parens from the JSON feels nice and is going to be a lot easier to parse as well.

I guess we could wrap the YAML in braces:

// Multiline for config file
--regex-rule {
  name: urlMacro
  find: "URL\(string: (\"[^\\]+\")\)!"
  replace: "#URL($1)"
  imports:
    - "URLFoundation"
}