So currently the status check will refire with every action that is performed on the PR. The same for if you add a commit or update the title for example.
It would be nice if you could set in the metadata of each task what trigger you want this task to respond to.
For example if you have an NPM test status check, you could want that to trigger on every commit, but you probably wouldn't want to re run your tests if you just updated the title of the PR or added a label.