Inconsiderate use of inline_always can result in code bloat (which, if taken too far, leads to L1i cache misses) and increased compilation times.
The inlining heuristics of rustc normally aren't too bad, so it might be worthwhile to investigate how many of these annotations can be replaced with plain inline or removed entirely without incurring a significant performance cost.