Ensure that we fetch another collation if the first collation was invalid (#3362)

* Ensure that we fetch another collation if the first collation was invalid

* Feedback
11 jobs for master in 7 minutes and 18 seconds (queued for 3 seconds)