More thorough changes detection (#7472)

* More thorough changes detection

This basically trades false-negatives ("dumb" last commit not touching
Rust on a Rust-touching branch) for false-positives (all builds against
`stable` and `beta` will run all tests almost always).
This seems to be a reasonable tradeoff.

Closes #6507

* We don't have Appveyor anymore

* before_script sections are not merged

* Limiting commit depth breaks branching detection