[11.x] Add the ability to skip migrations within tests

[cosmastech]

It is very difficult to test a single migration. By introducing the ability to explicitly skip migrations, we can prepare our database to match a production environment, run the new migration, and confirm that there are no unintended consequences.

Tasks

• Modify Migrator to allow for skipping migrations
• Add tests

Follow-ups

Add a TestsMigration trait that will simplify the process of testing a migration.

Thoughts

It dawned on me that this works for testing a migration for a while, but eventually the codebase may have other migrations that depend on a skipped one. It would require a developer to pull the test once that happens. (Which kind of makes sense, once you've run the test and deployed your migration, the test is kind of useless).

It may be better to have a syntax like Migrator::noMigrationsAfter('2024_02_02__12_00_00__adduserstable.php'). This could be added a follow up, or I can add it now. Let me know what you prefer.

[github-actions]

Thanks for submitting a PR!

Note that draft PR's are not reviewed. If you would like a review, please mark your pull request as ready for review in the GitHub user interface.

Pull requests that are abandoned in draft may be closed due to inactivity.

[tontonsb]

Ability to skip a migration to test it is a good idea, but I think it should be done for test cases without the RefreshDatabase trait. Your Migrator change already allows me to do a test like this:

public function testSomeMigration(): void
{
    Migrator::withoutMigrations(['some_migration_name']);

    // check stuff in the "old" db state

    Migrator::withoutMigrations([]);

    // check stuff in the "new" db state
}

I think your current implementation would only respect a skip if it happens on the first "refresh" and it would be kept for all other tests.

[cosmastech]

Ability to skip a migration to test it is a good idea, but I think it should be done for test cases without the RefreshDatabase trait. Your Migrator change already allows me to do a test like this:

public function testSomeMigration(): void
{
    Migrator::withoutMigrations(['some_migration_name']);

    // check stuff in the "old" db state

    Migrator::withoutMigrations([]);

    // check stuff in the "new" db state
}

I think your current implementation would only respect a skip if it happens on the first "refresh" and it would be kept for all other tests.

Ah, that's a good point. The database doesn't get refreshed from testMethod to testMethod. Thanks for calling it out.

Maybe I'll drop the attribute change and just add some tests. 🤷