You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: extractors/cds/tools/README.md
+63-2Lines changed: 63 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -170,7 +170,64 @@ The extractor uses a sophisticated compilation approach:
170
170
-**Parallel Processing**: Where possible, processes independent projects concurrently
171
171
-**Resource Management**: Cleans up temporary files and cached dependencies
172
172
173
-
## Integration with CodeQL
173
+
## Integration with `cds` CLI
174
+
175
+
### Installation of CDS (Node) Dependencies
176
+
177
+
#### Installation of `@sap/cds` and `@sap/cds-dk`
178
+
179
+
The CDS extractor attempts to optimize performance for most projects by caching the installation of the unique combinations of resolved CDS dependencies across all projects under a given source root.
180
+
181
+
The "unique combinations of resolved CDS dependencies" means that we resolve the **latest** available version **within the semantic version range** for each `@sap/cds` and `@sap/cds-dk` dependency specified in the `package.json` file for a given CAP project.
182
+
183
+
In practice, this means that if "project-a" requires `@sap/cds@^6.0.0` and "project-b" requires `@sap/cds@^7.0.0` while the latest available version is `@sap/[email protected]` (as a trivial example), the extractor will install `@sap/[email protected]` once and reuse it for both projects.
184
+
185
+
This is much faster than installing all dependencies for every project individually, especially for large projects with many CDS files. However, this approach has some limitations and trade-offs:
186
+
187
+
- This latest-first approach is more likely to choose the same version for multiple projects, which can reduce analysis time and can improve consistency in analysis between projects.
188
+
- This approach does not read (or respect) the `package-lock.json` file, which means that we are more likely to use a `cds` version that is different from the one most recently tested/used by the project developers.
189
+
- We are more likely to encounter incompatibility issues where a particular project hasn't been tested with the latest version of `@sap/cds` or `@sap/cds-dk`.
190
+
191
+
We can mitigate some of these issues through a (to be implemented) compilation retry mechanism for projects where some CDS compilation task(s) fail to produce the expected `.cds.json` output file(s).
192
+
The proposed retry mechanism would install the full set of dependencies for the affected project(s) while respecting the `package-lock.json` file, and then re-run the compilation for the affected project(s).
193
+
194
+
```text
195
+
TODO: retry mechanism expected before next release of the CDS extractor
196
+
```
197
+
198
+
#### Installation of Additional Project-Specific Dependencies
199
+
200
+
```text
201
+
TODO: implement installation of dependencies required for compilation to succeed for a given project
202
+
```
203
+
204
+
### Integration with `cds compile` command
205
+
206
+
The CDS extractor uses the `cds compile` command to compile `.cds` files into `.cds.json` files, which are then processed by CodeQL's JavaScript extractor.
207
+
208
+
Where possible, a single `model.cds.json` file is generated for each project, containing all the compiled definitions from the project's `.cds` files. This results in a faster extraction process overall with minimal duplication of CDS code elements (e.g., annotations, entities, services, etc.) within the CodeQL database created from the extraction process.
209
+
210
+
Where project-level compilation is not possible (e.g., due to project structure), the extractor generates individual `.cds.json` files for each `.cds` file in the project. The main downside to this approach is that if one `.cds` file imports another `.cds` file, the imported definitions will be duplicated in the CodeQL database, which can lead to false positives in queries that expect unique definitions.
211
+
212
+
```text
213
+
TODO: use the unique (session) ID of the CDS extractor run to as the `<session>` part of `<basename>.<session>.cds.json` and set JS extractor env vars to only extractor `.<session>.cds.json` files
214
+
```
215
+
216
+
### Integration with `cds env` command
217
+
218
+
The current version of the CDS extractor expects CAP projects to follow the [default project structure][CAP-project-structure], particularly regarding the names of the (`app`, `db`, & `srv`) subdirectories in which the extractor will look for `.cds` files to process (in addition to the root directory of the project).
219
+
220
+
The proposed solution will use the `cds env` command to discover configurations that affect the structure of the project and/or the expected "compilation tasks" for the project, such as any user customization of environment configurations such as:
221
+
222
+
-`cds.folders.app`
223
+
-`cds.folders.db`
224
+
-`cds.folders.srv`
225
+
226
+
```text
227
+
TODO : add support for integration with `cds env` CLI command as a means of consistently getting configurations for CAP projects
228
+
```
229
+
230
+
## Integration with `codeql` CLI
174
231
175
232
### File Processing
176
233
@@ -199,7 +256,7 @@ The extractor processes both:
199
256
The extractor provides comprehensive logging:
200
257
201
258
-**Performance Tracking**: Times for each extraction phase
202
-
-**Memory Usage**: Memory consumption at key milestones
259
+
-**Memory Usage**: Memory consumption at key milestones
203
260
-**Error Reporting**: Detailed error messages with context
204
261
-**Project Discovery**: Information about detected CDS projects
205
262
@@ -212,10 +269,14 @@ The extractor provides comprehensive logging:
212
269
## References
213
270
214
271
-[SAP Cloud Application Programming Model][CAP]
272
+
-[Default Structure of a CAP Project][CAP-project-structure]
0 commit comments