Skip to content

Commit 3901267

Browse files
committed
Update CDS extractor README.md with TODOs
1 parent cb8518b commit 3901267

File tree

1 file changed

+63
-2
lines changed

1 file changed

+63
-2
lines changed

extractors/cds/tools/README.md

Lines changed: 63 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,64 @@ The extractor uses a sophisticated compilation approach:
170170
- **Parallel Processing**: Where possible, processes independent projects concurrently
171171
- **Resource Management**: Cleans up temporary files and cached dependencies
172172

173-
## Integration with CodeQL
173+
## Integration with `cds` CLI
174+
175+
### Installation of CDS (Node) Dependencies
176+
177+
#### Installation of `@sap/cds` and `@sap/cds-dk`
178+
179+
The CDS extractor attempts to optimize performance for most projects by caching the installation of the unique combinations of resolved CDS dependencies across all projects under a given source root.
180+
181+
The "unique combinations of resolved CDS dependencies" means that we resolve the **latest** available version **within the semantic version range** for each `@sap/cds` and `@sap/cds-dk` dependency specified in the `package.json` file for a given CAP project.
182+
183+
In practice, this means that if "project-a" requires `@sap/cds@^6.0.0` and "project-b" requires `@sap/cds@^7.0.0` while the latest available version is `@sap/[email protected]` (as a trivial example), the extractor will install `@sap/[email protected]` once and reuse it for both projects.
184+
185+
This is much faster than installing all dependencies for every project individually, especially for large projects with many CDS files. However, this approach has some limitations and trade-offs:
186+
187+
- This latest-first approach is more likely to choose the same version for multiple projects, which can reduce analysis time and can improve consistency in analysis between projects.
188+
- This approach does not read (or respect) the `package-lock.json` file, which means that we are more likely to use a `cds` version that is different from the one most recently tested/used by the project developers.
189+
- We are more likely to encounter incompatibility issues where a particular project hasn't been tested with the latest version of `@sap/cds` or `@sap/cds-dk`.
190+
191+
We can mitigate some of these issues through a (to be implemented) compilation retry mechanism for projects where some CDS compilation task(s) fail to produce the expected `.cds.json` output file(s).
192+
The proposed retry mechanism would install the full set of dependencies for the affected project(s) while respecting the `package-lock.json` file, and then re-run the compilation for the affected project(s).
193+
194+
```text
195+
TODO: retry mechanism expected before next release of the CDS extractor
196+
```
197+
198+
#### Installation of Additional Project-Specific Dependencies
199+
200+
```text
201+
TODO: implement installation of dependencies required for compilation to succeed for a given project
202+
```
203+
204+
### Integration with `cds compile` command
205+
206+
The CDS extractor uses the `cds compile` command to compile `.cds` files into `.cds.json` files, which are then processed by CodeQL's JavaScript extractor.
207+
208+
Where possible, a single `model.cds.json` file is generated for each project, containing all the compiled definitions from the project's `.cds` files. This results in a faster extraction process overall with minimal duplication of CDS code elements (e.g., annotations, entities, services, etc.) within the CodeQL database created from the extraction process.
209+
210+
Where project-level compilation is not possible (e.g., due to project structure), the extractor generates individual `.cds.json` files for each `.cds` file in the project. The main downside to this approach is that if one `.cds` file imports another `.cds` file, the imported definitions will be duplicated in the CodeQL database, which can lead to false positives in queries that expect unique definitions.
211+
212+
```text
213+
TODO: use the unique (session) ID of the CDS extractor run to as the `<session>` part of `<basename>.<session>.cds.json` and set JS extractor env vars to only extractor `.<session>.cds.json` files
214+
```
215+
216+
### Integration with `cds env` command
217+
218+
The current version of the CDS extractor expects CAP projects to follow the [default project structure][CAP-project-structure], particularly regarding the names of the (`app`, `db`, & `srv`) subdirectories in which the extractor will look for `.cds` files to process (in addition to the root directory of the project).
219+
220+
The proposed solution will use the `cds env` command to discover configurations that affect the structure of the project and/or the expected "compilation tasks" for the project, such as any user customization of environment configurations such as:
221+
222+
- `cds.folders.app`
223+
- `cds.folders.db`
224+
- `cds.folders.srv`
225+
226+
```text
227+
TODO : add support for integration with `cds env` CLI command as a means of consistently getting configurations for CAP projects
228+
```
229+
230+
## Integration with `codeql` CLI
174231

175232
### File Processing
176233

@@ -199,7 +256,7 @@ The extractor processes both:
199256
The extractor provides comprehensive logging:
200257

201258
- **Performance Tracking**: Times for each extraction phase
202-
- **Memory Usage**: Memory consumption at key milestones
259+
- **Memory Usage**: Memory consumption at key milestones
203260
- **Error Reporting**: Detailed error messages with context
204261
- **Project Discovery**: Information about detected CDS projects
205262

@@ -212,10 +269,14 @@ The extractor provides comprehensive logging:
212269
## References
213270

214271
- [SAP Cloud Application Programming Model][CAP]
272+
- [Default Structure of a CAP Project][CAP-project-structure]
215273
- [Core Data Services (CDS)][CDS]
274+
- [Project-Specific Configurations][CDS-ENV-project-configs]
216275
- [Conceptual Definition Language (CDL)][CDL]
217276
- [CodeQL Documentation](https://codeql.github.com/docs/)
218277

219278
[CAP]: https://cap.cloud.sap/docs/about/
279+
[CAP-project-structure]: https://cap.cloud.sap/docs/get-started/#project-structure
220280
[CDS]: https://cap.cloud.sap/docs/cds/
281+
[CDS-ENV-project-configs]: https://cap.cloud.sap/docs/node.js/cds-env#project-specific-configurations
221282
[CDL]: https://cap.cloud.sap/docs/cds/cdl

0 commit comments

Comments
 (0)