Added outlier analysis, public-facing dataset extraction tools, and out-of-tree pass

[khoing0810]

• opt_analysis_tools.py contains tools that download bitcode modules from HuggingFace dataset, extract transformation and analysis pass time, and export as csv and json format (recommended csv for now since it works better with pandas and other data analysis library). Modular enough to expand to accommodate specific needs (random sampling for downloading bitcode modules, plot more figures with matplotlib, etc.)
• outlier_analysis.py contains tools to extract and analyze outliers. Ray framework is required to run some of the functions to extract outliers.
• RemoveFunctionBody pass is built out-of-tree to be compatible with local build of LLVM. To build the pass, go to llvm-ir-dataset-utils-internal/llvm_ir_dataset_utils/tools/RemoveFunctionBodyPass and run ./install command. Syntax to run with opt: opt -load build/libRemoveFunctionBody.so -load-pass-plugin=build/libRemoveFunctionBody.so -passes='remove-fn-body' -index <index in integer here> <path to bitcode module file> (this is inside RemoveFunctionBodyPass directory, if somewhere else, please modify the path to -load and -load-pass-plugin). Using both -load and -load-pass-plugin to make the pass work with argument.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
llvm-ir-dataset-utils	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 20, 2024 6:44pm

[boomanaiden154]

Can you split this into a couple different PRs? Maybe one for the out-of-tree pass, one for the opt tooling, and one for the actual outlier analysis?

You'll want to stack the PRs: https://blog.logrocket.com/using-stacked-pull-requests-in-github/

[khoing0810]

I think I messed this workflow up since I forced push into upstream main branch, clearing up all previous commits, where they are in 3 separate PRs now. (#31 opt analysis tools, #29 outlier analysis, #30 out-of-tree function body removal pass)