Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
bb29561
- add rst parser
juiwenchen Nov 1, 2025
f69f089
update parser
juiwenchen Nov 3, 2025
6f581fe
update grammar
juiwenchen Nov 3, 2025
cd04d0f
update
juiwenchen Nov 3, 2025
8136363
update docs
juiwenchen Nov 3, 2025
b9f11e5
updated
juiwenchen Nov 4, 2025
84cd871
only allow inline argument
juiwenchen Nov 4, 2025
d750567
update
juiwenchen Nov 4, 2025
0f52aaa
update
juiwenchen Nov 5, 2025
e723e05
update
juiwenchen Nov 5, 2025
d2fd124
update test
juiwenchen Nov 5, 2025
494a6dd
add preprocess
juiwenchen Nov 5, 2025
68e2218
adapt src-trace
juiwenchen Nov 5, 2025
4d73856
updated snapshot
juiwenchen Nov 5, 2025
8a77bc6
updated snapshot
juiwenchen Nov 5, 2025
cb7e3e3
update
juiwenchen Nov 5, 2025
4d2b262
update docs
juiwenchen Nov 5, 2025
bed1667
make mypy happy
juiwenchen Nov 10, 2025
ec221a4
add TCs
juiwenchen Nov 10, 2025
7788ea0
update parser
juiwenchen Nov 11, 2025
b92872a
fix row in rst source map
juiwenchen Nov 11, 2025
a7b24d8
fix rst text positions
juiwenchen Nov 11, 2025
6a2c776
update snapshot
juiwenchen Nov 11, 2025
e5f2e1d
fix column position
juiwenchen Nov 11, 2025
1097776
update TC
juiwenchen Nov 11, 2025
cd70f44
improve warning handling and docs string
juiwenchen Nov 28, 2025
9f26d21
fixed TC path
juiwenchen Dec 2, 2025
dc0edd9
removed local specific name
juiwenchen Dec 5, 2025
303de47
rm not necessary extra fields from src-trace
juiwenchen Dec 10, 2025
5bfc031
add tc
juiwenchen Dec 10, 2025
7857f8d
adapt comments
juiwenchen Dec 12, 2025
555de94
update TC
juiwenchen Dec 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 71 additions & 8 deletions docs/source/components/analyse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Use simplified comment patterns to define **Sphinx-Needs** items without complex
Marked RST Blocks
~~~~~~~~~~~~~~~~~

Embed complete reStructuredText content within source code comments for rich documentation that can be extracted and processed.
Embed complete reStructuredText directives which is extracted and parsed as the grammar of **Sphinx-Needs** definition blocks.

Limitations
-----------
Expand All @@ -55,6 +55,8 @@ Extraction Examples

The following examples are configured with :ref:`the analyse configuration <analyse_config>`,

.. _`analyse_need_id_refs`:

Sphinx-Needs ID References
~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -118,6 +120,8 @@ Below is an example of a C++ source file containing need ID references and the c
- ``marker`` - The marker string used for identification
- ``type`` - Type of extraction ("need-id-refs")

.. _`analyse_rst`:

Marked RST Blocks
~~~~~~~~~~~~~~~~~

Expand All @@ -126,6 +130,7 @@ This example demonstrates how the analyse extracts RST blocks from comments.
.. tabs::

.. code-tab:: cpp
:linenos:

#include <iostream>

Expand Down Expand Up @@ -190,6 +195,8 @@ The module supports both multi-line and single-line RST blocks:
- **Multi-line blocks**: Use ``@rst`` and ``@endrst`` on separate lines
- **Single-line blocks**: Use ``@rst content @endrst`` on the same line

.. _`analyse_oneline`:

One-line Needs
--------------

Expand All @@ -199,14 +206,70 @@ For comprehensive information about one-line needs configuration and usage, see

**Basic Example:**

.. code-block:: c

// @Function Implementation, IMPL_001, impl, [REQ_001, REQ_002]
.. tabs::

.. code-tab:: c
:linenos:

// @Function Foo, IMPL_1
void foo() {}

// @Function Bar, IMPL_2
void bar() {}

This single comment line creates a complete **Sphinx-Needs** item equivalent to:
// @Function Baz\, as I want it, IMPL_3
void baz() {}

.. code-block:: rst
.. code-tab:: json

.. impl:: Function Implementation
:id: IMPL_001
:links: REQ_001, REQ_002
[
{
"filepath": "./sphinx-codelinks/tests/data/oneline_comment_default/default_oneliners.c",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/oneline_comment_default/default_oneliners.c#L1",
"source_map": {
"start": { "row": 0, "column": 4 },
"end": { "row": 0, "column": 24 }
},
"tagged_scope": "void foo() {}",
"need": {
"title": "Function Foo",
"id": "IMPL_1",
"type": "impl",
"links": []
},
"type": "need"
},
{
"filepath": "./sphinx-codelinks/tests/data/oneline_comment_default/default_oneliners.c",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/oneline_comment_default/default_oneliners.c#L4",
"source_map": {
"start": { "row": 3, "column": 4 },
"end": { "row": 3, "column": 24 }
},
"tagged_scope": "void bar() {}",
"need": {
"title": "Function Bar",
"id": "IMPL_2",
"type": "impl",
"links": []
},
"type": "need"
},
{
"filepath": "./sphinx-codelinks/tests/data/oneline_comment_default/default_oneliners.c",
"remote_url": "https://github.com/useblocks/sphinx-codelinks/blob/951e40e7845f06d5cfc4ca20ebb984308fdaf985/tests/data/oneline_comment_default/default_oneliners.c#L7",
"source_map": {
"start": { "row": 6, "column": 4 },
"end": { "row": 6, "column": 39 }
},
"tagged_scope": "void baz() {}",
"need": {
"title": "Function Baz, as I want it",
"id": "IMPL_3",
"type": "impl",
"links": []
},
"type": "need"
}
]
11 changes: 10 additions & 1 deletion docs/source/components/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -382,14 +382,15 @@ Configures how **Sphinx-CodeLinks** analyse source files to extract markers from
[codelinks.projects.my_project.analyse.marked_rst]
start_sequence = "@rst"
end_sequence = "@endrst"
link_options = ["links"]

get_need_id_refs
^^^^^^^^^^^^^^^^

Enables the extraction of need IDs from source code comments. When enabled, **SourceAnalyse** will parse comments for specific markers that indicate need IDs, allowing them to be extracted for further usages.

**Type:** ``bool``
**Default:** ``False``
**Default:** ``True``

.. code-block:: toml

Expand Down Expand Up @@ -510,6 +511,8 @@ Configuration for Sphinx-Needs ID reference extraction.

- ``markers`` (``list[str]``) - List of marker strings that identify need ID references

.. _analyse_marked_rst:

analyse.marked_rst
^^^^^^^^^^^^^^^^^^

Expand All @@ -523,8 +526,14 @@ Configuration for marked RST block extraction.
[codelinks.projects.my_project.analyse.marked_rst]
start_sequence = "@rst"
end_sequence = "@endrst"
strip_leading_sequences = ["*"]
indented_spaces = 3
link_options = ["links"]

**Configuration fields:**

- ``start_sequence`` (``str``) - Marker that begins an RST block
- ``end_sequence`` (``str``) - Marker that ends an RST block
- ``strip_leading_sequences`` (``list[str]``) - List of leading sequences to strip from each line of the RST text between the markers
- ``indented_spaces`` (``int``) - Number of leading spaces to consider as indentation in the RST text
- ``link_options`` (``list[str]``) - List of option names whose values should be treated as Sphinx-Needs link fields
24 changes: 23 additions & 1 deletion docs/source/components/directive.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,29 @@
Directive
=========

.. attention:: ``src-trace`` directive currently only supports :ref:`one-line need definition <oneline>`.
.. attention:: ``src-trace`` directive does NOT support :ref:`Sphinx-Needs ID Refs <analyse_need_id_refs>`.

``src-trace`` Directive generates Sphinx-Needs items from source code comments. There are two ways to define need items in source code:

1. **One-line need definition**: Define needs in a single line comment.

Example in C++:

.. code-block:: cpp

// @ title, id_123, implementation, [link1, link2]

2. **RST block need definition**: Define needs in a RST block comment.

Example in C++:

.. code-block:: cpp

/*
.. implementation:: title
:id: id_123
:links: link1, link2
*/

``CodeLinks`` provides ``src-trace`` directive and it can be used in the following ways:

Expand Down
23 changes: 23 additions & 0 deletions docs/source/components/rst_parser.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
RST directive Parser
====================

The :ref:`analyse <analyse>` module provides a simplified parser for reStructuredText (reST) directives using the ``Lark`` parsing library.
It is designed to only parse the RST text extracted by :ref:`RST markers <analyse_rst>`, focusing on specific directive types and their associated options and content.
By doing so, the parser avoids the complexity of a full reST parser while still capturing the essential structure needed for Sphinx-Needs integration from the source code.

The parser doesn't have the Sphinx-Needs directive validation logic. It only checks the syntax of the RST directives and extracts:
- directive type
- argument (title)
- options
- content

These extracted data will be furthered used with :external+needs:ref:`add_need <api>` function to create Sphinx-Needs items in ``src-trace`` directive.

**Limitations**

Since the parser does not implement the full reST specification, it has some limitations:

- Comments in the RST text are not supported.
- The parser expects consistent indentation for options and content blocks.
- It only takes an inline directive argument/title (no multi-line arguments/titles).
- It only takes inline option values (no multi-line option values).
30 changes: 30 additions & 0 deletions docs/source/development/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,36 @@
Changelog
=========

.. _`unreleased`:

Unreleased
-----------

New and Improved
................

- ✨ Added RST directive parser for ``analyse`` module to resolve the marked RST into Sphinx-Needs related data.

Marked RST blocks can be resolved from the marked RST text in source files using the simplified RST directive parser.
The resolved RST blocks will be dumped into the JSON output along with other extracted markers.
To make the parser more stable, 3 new configuration options are added to control the parsing behavior:

- ``strip_leading_sequences``: List of leading character sequences to strip from each line.

This option allows users to specify a list of leading character sequences (e.g., ``*``, ``-``) that should be stripped
from each line of the marked RST block before parsing.
This is useful for handling comment styles that include leading characters in multi-line comments.

- ``indented_spaces``: Number of spaces that indicate an indented block.
- ``link_options``: List of option names whose values should be converted to lists when parsed.

- ✨ Enhanced the ``src-trace`` directive to support creating need items from marked RST blocks in addition to one-line need definitions.

The ``src-trace`` directive can now create Sphinx-Needs need items from both one-line need definitions and marked RST blocks.
This provides more flexibility for defining needs in source code comments.

:Released: 02.10.2025

.. _`release:1.1.0`:

1.1.0
Expand Down
3 changes: 1 addition & 2 deletions docs/source/development/roadmap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ Configuration Files
Source Code Parsing
-------------------

- Introduce a configurable option to strip leading characters (e.g., ``*``) from commented RST blocks.
- Enrich tagged scopes with additional metadata.
- Extend language support by adding parsers for more comment styles, including but not limited to:

Expand All @@ -28,5 +27,5 @@ Source Code Parsing
Defining Needs in Source Code
-----------------------------

- Introduce flexible ways to define ``Sphinx-Needs`` items in source code, such as using raw RST text and multi-line comments.
- Introduce flexible ways to define ``Sphinx-Needs`` items in source code, such as custom multi-line need comments.
- Implement a feature to export needs defined in source code to a ``needs.json`` file, improving CI workflows and portability.
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Contents
components/cli
components/configuration
components/directive
components/rst_parser
components/oneline
components/analyse
components/discover
Expand Down
1 change: 1 addition & 0 deletions docs/src_trace.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ src_dir = "../tests/data/dcdc" # Relative path from this TOML config to the sour
[codelinks.projects.dcdc.analyse]
get_need_id_refs = false
get_oneline_needs = true
get_rst = true

[codelinks.projects.dcdc.analyse.oneline_comment_style]
# Configuration for oneline comment style
Expand Down
2 changes: 1 addition & 1 deletion docs/ubproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
ignore = ["block.title_line"]

[needs]
id_required = true
id_required = false

[[needs.types]]
directive = "feature"
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ dependencies = [
"tree-sitter-c-sharp>=0.23.1",
"tree-sitter-yaml>=0.7.1",
"tree-sitter-rust>=0.23.0",
"lark>=1.3.1",
]

[build-system]
Expand Down
Loading