This reproduces an odd issue with CloudFormation upgrading OpenSearch engine versions.
- Deploy the
00-initial.ymltemplate as a new stack. - Try to update the stack with the
02-final.ymltemplate. This fails as you cannot directly upgrade an OpenSearch domain from Elasticsearch 7.10 to OpenSearch 2.19. - Upgrade the domain manually using the OpenSearch console - first to OpenSearch 1.3 and then to OpenSearch 2.19.
- Try again to update the stack with the
02-final.ymltemplate. This fails with an Internal Failure. In CloudTrail we can see anUpgradeDomainevent at this time with anerrorMessageFailed to submit upgrade. Upgrade from OpenSearch_2.19 to OpenSearch_2.19 not supported.- The stack is now in
UPDATE_ROLLBACK_FAILED, because the rollback tried to downgrade the cluster back to Elasticsearch 7.10. Continue the rollback, skipping the OpenSearch domain, to get the stack back to a usable state.
- The stack is now in
- Update the stack with the
01-intermediate.ymltemplate. This should succeed. - Update the stack with the
02-final.ymltemplate. This should also now succeed.