Cleans up the config check task so it's slightly easier to maintain.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>