Use a merge queue

[mbrobbel]

Maybe a merge queue would make sense?

Originally posted by @kylebarron in #8290 (comment)

[alamb]

@blaginin has taken the lead doing this for the datafusion repos, so maybe he has some tips / context to share

[blaginin]

Hey! Here is a datafusion issue apache/datafusion#6880 with the plan I used to do the transition

I'm finishing it hopefully this or early next week, and then don't mind doing it for Arrow

[blaginin]

In general, the most important bits are:

• you'll need to make steps you care about as required
• you need to ask ASF infra team to manually enable it for you, as .asf.yaml doesn't support this currently
• You also need to disable merging several PRs that are already in the queue because of ASF legal policies