RFC: Create high level CMake targets

[larryliu0820]

🚀 The feature, motivation and pitch

Currently it's hard to let a C++ project to depend on ExecuTorch and we should fix it by providing fixed number of user facing targets.

Context

Take this Zephyr application for example, it is hard to figure out what CMake targets to use, because there are too many of them.

Some context on the initial design: we started off ExecuTorch aiming to be customizable and reduce binary size of the core runtime as much as possible. This results in a fragmentation of libraries, we expect applications to choose and pick these libraries to build their final binary. As we are getting close to GA, we want to hide more implementation details, and group these libraries into user facing categories to avoid confusion.

The full list of the libraries that belong to EXECUTORCH_LIBRARIES can be found here.

Proposal

We create these user facing libraries:

• executorch_core
  • Existing target, provides definition of the core runtime Method, Program, flatbuffer lib etc. Make sure we have the basic model deserialization and execution functionality.
• kernels
  • Depends on user provided CMake options, provide a (dtype) selective build powered kernel library. By default this library contains optimized kernels and fallback to portable kernels.
• backends
  • An aggregation of all the backends user specified, through a CMake option.
• extensions
  • A combination of libextension_modules, libextension_data_loader, libextension_tensor etc. This is also configurable by checking CMake options.

For all of the existing targets, we want to add a _ prefix to the target name, to indicate that they are internal only and probably shouldn't be linked directly.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

[JacobSzwejbka]

So executorch_core is unchanged and then the proposal is to add wrappers for kernels/backends/extensions that will be populated based on what flags are set?

[GregoryComer]

Sounds great. We've seen a number of GH issues asking about this, so I think it's a great problem to solve. What are your thoughts on also having a top-level target that includes all configured backends and extensions? There was some discussion about this a while back in the context of the build changes.

[larryliu0820]

So executorch_core is unchanged and then the proposal is to add wrappers for kernels/backends/extensions that will be populated based on what flags are set?

Yep

[larryliu0820]

Sounds great. We've seen a number of GH issues asking about this, so I think it's a great problem to solve. What are your thoughts on also having a top-level target that includes all configured backends and extensions? There was some discussion about this a while back in the context of the build changes.

Are you saying something like executorch that contains everything? I'm ok with that

[larryliu0820]

#11761 this goes along with this RFC

[JacobSzwejbka]

top-level target that includes all configured backends and extensions?

Is this possible? Do some backends not conflict or require specific toolchains that might not always be present?

[larryliu0820]

top-level target that includes all configured backends and extensions?

Is this possible? Do some backends not conflict or require specific toolchains that might not always be present?

It's orthogonal. If a user turn on EXECUTORCH_BUILD_MEDIATEK they will have to provide the toolchain/SDK anyways.

[JacobSzwejbka]

It's orthogonal. If a user turn on EXECUTORCH_BUILD_MEDIATEK they will have to provide the toolchain/SDK anyways.

Im confused. Whats the value of a top level target that includes all backends if its just going to error out on missing toolchains.

[larryliu0820]

So that user doesn't have to link specific backend targets. Like right now they will have to link both libxnnpack_backend, libXNNPACK, libxnnpack-microkernels-prod, libcoremldelegate explicitly

[larryliu0820]

Once we have this, we can start to do CMake EXPORT so that we can cleanup executorch-config.cmake. See this comment: #12320 (comment)

Also tokenizers repo is already doing it: https://github.com/pytorch-labs/tokenizers/blob/main/CMakeLists.txt#L163C5-L163C11

[swolchok]

Once we have this, we can start to do CMake EXPORT

I don't think this is necessary for CMake EXPORT; I have a draft PR for EXPORT locally that I hope to send out soon, after I finish debugging some of the obvious issues with #12425

[swolchok]

draft PR for EXPORT

#8954 has been updated and stacked on top of PRs that seem to make it work OK locally.