Skip to content

Commit 9368498

Browse files
nariyanibhargavnariyanibhargav
authored
Merge pull request #212 from splunk/ACD-4534-pytest-splunk-addon-Doc-Update
Updating documentation
2 parents 8ba901b + 3136783 commit 9368498

File tree

8 files changed

+234
-61
lines changed

8 files changed

+234
-61
lines changed

‎docs/conf.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@
4343
'sphinx.ext.coverage',
4444
'sphinx.ext.viewcode',
4545
'sphinx.ext.napoleon',
46+
'sphinx_panels'
4647
]
4748

4849
# Add any paths that contain templates here, relative to this directory.

‎docs/how_to_use.rst‎

Lines changed: 45 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,19 @@ How To Use
66

77
Create a test file in the tests folder
88

9-
.. code:: python3
9+
.. dropdown:: Example Test File
1010

11-
from pytest_splunk_addon.standard_lib.addon_basic import Basic
12-
class Test_App(Basic):
13-
def empty_method():
14-
pass
11+
.. code:: python3
12+
13+
from pytest_splunk_addon.standard_lib.addon_basic import Basic
14+
class Test_App(Basic):
15+
def empty_method():
16+
pass
1517
1618
1719
.. _test_execution:
1820

19-
There are two ways to execute the tests:
21+
There are three ways to execute the tests:
2022

2123
**1. Running tests with an external Splunk instance**
2224

@@ -28,39 +30,56 @@ There are two ways to execute the tests:
2830

2931
.. code:: bash
3032
31-
pytest --splunk-type=external --splunk-app=<path-to-addon-package> --splunk-host=<hostname> --splunk-port=<splunk-management-port> --splunk-user=<username> --splunk-password=<password> --splunk-hec-token=<splunk_hec_token>
33+
pytest --splunk-type=external --splunk-app=<path-to-addon-package> --splunk-data-generator=<path to pytest-splunk-addon-data.conf file> --splunk-host=<hostname> --splunk-port=<splunk-management-port> --splunk-user=<username> --splunk-password=<password> --splunk-hec-token=<splunk_hec_token>
3234
3335
3436
**2. Running tests with docker splunk**
3537

3638
.. code:: bash
3739
38-
pip install pytest-splunk-addon[docker]
40+
git clone [email protected]:splunk/pytest-splunk-addon.git
41+
cd pytest-splunk-addon
42+
pip install poetry
43+
poetry install
44+
45+
Create a Dockerfile-splunk file
46+
47+
.. dropdown:: Example Dockerfile
3948

40-
Create a Dockerfile-splunk file
49+
.. code:: Dockerfile
4150
42-
.. literalinclude:: ../Dockerfile.splunk
43-
:language: Dockerfile
51+
ARG SPLUNK_VERSION=latest
52+
FROM splunk/splunk:$SPLUNK_VERSION
53+
ARG SPLUNK_VERSION=latest
54+
ARG SPLUNK_APP_ID=TA_UNKNOWN
55+
ARG SPLUNK_APP_PACKAGE=$SPLUNK_APP_PACKAGE
56+
RUN echo Splunk VERSION=$SPLUNK_VERSION
57+
COPY deps/apps /opt/splunk/etc/apps/
58+
COPY $SPLUNK_APP_PACKAGE /opt/splunk/etc/apps/$SPLUNK_APP_ID
4459
4560
Create docker-compose.yml
4661

47-
.. literalinclude:: ../docker-compose.yml
48-
:language: YAML
49-
:lines: 9-
62+
.. dropdown:: Example docker-compose file
63+
64+
.. literalinclude:: ../docker-compose.yml
65+
:language: YAML
66+
:lines: 9-
5067

5168
.. _conftest_file:
5269

5370
Create conftest.py in the test folder along with :ref:`the test file <test_file>`
5471

55-
.. literalinclude:: ../tests/conftest.py
56-
:language: python
57-
:lines: 2,12-
72+
.. dropdown:: Example conftest file
73+
74+
.. literalinclude:: ../tests/conftest.py
75+
:language: python
76+
:lines: 1-2,12-
5877

5978
Run pytest with the add-on, using the following command:
6079

6180
.. code:: bash
6281
63-
pytest --splunk-type=docker --splunk-password=Changed@11
82+
pytest --splunk-type=docker --splunk-data-generator=<path to pytest-splunk-addon-data.conf file>
6483
6584
The tool assumes the Splunk Add-on is located in a folder "package" in the project root.
6685

@@ -84,8 +103,8 @@ The tool assumes the Splunk Add-on is located in a folder "package" in the proje
84103

85104
.. code:: bash
86105
87-
pytest --splunk-type=external
88-
--splunk-app=<path-to-addon-package>
106+
pytest --splunk-type=external # Whether you want to run the addon with docker or an external Splunk instance
107+
--splunk-app=<path-to-addon-package> # Path to Splunk app package. The package should have the configuration files in the default folder.
89108
--splunk-host=<hostname> # Receiver Splunk instance where events are searchable.
90109
--splunk-port=<splunk_management_port> # default 8089
91110
--splunk-user=<username> # default admin
@@ -247,9 +266,11 @@ Extending pytest-splunk-addon
247266

248267
The following snippet shows an example in which the setup fixture is used to enable a saved search.
249268

250-
.. literalinclude:: ../tests/enable_saved_search_conftest.py
251-
:language: python
252-
:lines: 2,31-
269+
.. dropdown:: enable_saved_search_conftest.py
270+
271+
.. literalinclude:: ../tests/enable_saved_search_conftest.py
272+
:language: python
273+
:lines: 2,31-
253274

254275

255276
**4. Check mapping of an add-on with custom data models**
@@ -266,4 +287,4 @@ Extending pytest-splunk-addon
266287

267288
<hr width=100%>
268289

269-
.. [#] xfail indicates that you expect a test to fail for some reason. A common example is a test for a feature not yet implemented, or a bug not yet fixed. When a test passes despite being expected to fail, it's an xpass and will be reported in the test summary.
290+
.. [#] xfail indicates that you expect a test to fail for some reason. A common example is a test for a feature not yet implemented, or a bug not yet fixed. When a test passes despite being expected to fail, it's an xpass and will be reported in the test summary.

‎docs/overview.rst‎

Lines changed: 3 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -23,40 +23,7 @@ Features
2323
Release notes
2424
-------------
2525

26-
1.3.0
27-
""""""""""""""""""""""""""
28-
29-
**New Features:**
30-
31-
* Removed dependency on SA-Eventgen to generate data for testing.
32-
* pytest-splunk-addon now generates data independently, which is required for testing.
33-
* pytest-splunk-addon generates tests for testing index-time properties in Splunk Technology Add-ons.
34-
35-
**Bugfixes:**
36-
37-
* Enhanced requirements for the following CIM data models:
38-
39-
+-----------------------+-----------------------------------------------------------+
40-
| CIM Data Model | Field Name |
41-
+=======================+===========================================================+
42-
| IDS | src, dest, src_port, dest_port, user |
43-
+-----------------------+-----------------------------------------------------------+
44-
| Network Resolution | src, dest |
45-
+-----------------------+-----------------------------------------------------------+
46-
| Network Traffic | bytes, bytes_in, bytes_out, icmp_code |
47-
| | |
48-
| | packets, packets_in, packets_out |
49-
| | |
50-
| | src, src_translated_port, src_port |
51-
| | |
52-
| | dest, dest_translated_port, dest_port |
53-
+-----------------------+-----------------------------------------------------------+
54-
| Web | app, uri_path, url_length |
55-
+-----------------------+-----------------------------------------------------------+
56-
57-
**Known Issues:**
58-
59-
* Fields for modular regular expressions are not extracted in the plugin.
26+
Find details about all the releases here: :ref:`Release History Page<release_history>`
6027

6128
Installation
6229
------------
@@ -72,4 +39,5 @@ Or, in any case, if pip is unavailable:
7239
7340
1. git clone https://github.com/splunk/pytest-splunk-addon.git
7441
2. cd pytest-splunk-addon
75-
3. python3 setup.py install
42+
3. pip install poetry
43+
4. poetry install

‎docs/release_history.rst‎

Lines changed: 157 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,165 @@
1+
.. _release_history:
2+
3+
=================
14
Release History
25
=================
36

47
**GitHub**
58

69
The best way to track the development of pytest-splunk-addon is through `the GitHub Repo <https://github.com/splunk/pytest-splunk-addon/>`_.
710

11+
1.3.12 (2020-11-09)
12+
""""""""""""""""""""""""""
13+
**Changes:**
14+
15+
* Added ``--ignore-addon-errors`` pytest param to suppress Splunk Addon internal errors.
16+
* Updated ``--no-splunk-cleanup`` pytest param to ``--splunk-cleanup`` which is used to clean the data on the Splunk instance before testing.
17+
18+
**Known Issues:**
19+
20+
* Event ingestion through SC4S via UDP port
21+
* Fields for modular regular expressions are not extracted in the plugin.
22+
23+
1.3.11 (2020-10-27)
24+
""""""""""""""""""""""""""
25+
**Changes:**
26+
27+
* Fixed string literal causing SyntaxError within helmut lib.
28+
* Added ``--no-splunk-cleanup`` CLI param, which disables the cleanup of Splunk environment before the tests execute.
29+
* Added ``--splunk-web-scheme`` pytest argument which can be used to set the web scheme (http/https) of the Splunk instance.
30+
* Updated pytest-splunk-addon package to remove static fixtures that are now installed as part of the environment.
31+
32+
**Known Issues:**
33+
34+
* Event ingestion through SC4S via UDP port
35+
* Fields for modular regular expressions are not extracted in the plugin.
36+
37+
1.3.9 (2020-10-15)
38+
""""""""""""""""""""""""""
39+
**Changes:**
40+
41+
* Updated build process which uses python's poetry to install dependencies.
42+
* Added support in pytest-splunk-addon to test with on-prem forwarder configured to a standalone or SH of cloud stack by providing SH in --splunk-host and forwarder in --splunk-forwarder-host and other appropriate params.
43+
44+
**Known Issues:**
45+
46+
* Event ingestion through SC4S via UDP port
47+
* Fields for modular regular expressions are not extracted in the plugin.
48+
49+
1.3.6 (2020-9-25)
50+
""""""""""""""""""""""""""
51+
**Changes:**
52+
53+
* Added support for ingestion of data via pytest-splunk-addon with a user-defined index ``index = <index_name>``.
54+
55+
**Known Issues:**
56+
57+
* Event ingestion through SC4S via UDP port
58+
* Fields for modular regular expressions are not extracted in the plugin.
59+
60+
1.3.5 (2020-9-14)
61+
""""""""""""""""""""""""""
62+
**Changes:**
63+
64+
* Updated the host pattern from using ``_`` to using ``-``.
65+
* Updated host generation logic to fix an issue for unique IP based hosts from being duplicated due to a limit. Now hosts are getting generated uniquely.
66+
67+
**Known Issues:**
68+
69+
* Event ingestion through SC4S via UDP port
70+
* Fields for modular regular expressions are not extracted in the plugin.
71+
72+
73+
1.3.4 (2020-9-11)
74+
""""""""""""""""""""""""""
75+
**Changes:**
76+
77+
* Removed threading mechanism while sending data using SC4S as SC4S expects sequential ingestion of data rather than parallel ingestion.
78+
79+
**Known Issues:**
80+
81+
* Event ingestion through SC4S via UDP port
82+
* Fields for modular regular expressions are not extracted in the plugin.
83+
84+
1.3.3 (2020-9-09)
85+
""""""""""""""""""""""""""
86+
**Changes:**
87+
88+
* Added log messages to our tests to help debug issues.
89+
90+
**Known Issues:**
91+
92+
* Event ingestion through SC4S via UDP port
93+
* Fields for modular regular expressions are not extracted in the plugin.
94+
95+
1.3.2 (2020-8-26)
96+
""""""""""""""""""""""""""
97+
**Changes:**
98+
99+
* Enhanced requirements for the following CIM data models:
100+
101+
+-----------------------+-----------------------------------------------------------+
102+
| CIM Data Model | Field Name |
103+
+=======================+===========================================================+
104+
| Change | action, object_category, object_id, object_path, |
105+
| | object_attrs |
106+
+-----------------------+-----------------------------------------------------------+
107+
108+
* Now, the tokenised events can be stored in json files in the *.tokenized_events* folder. If these files are not required, use the ``--discard-eventlogs`` option when executing the tests.
109+
110+
**Known Issues:**
111+
112+
* Event ingestion through SC4S via UDP port
113+
* Fields for modular regular expressions are not extracted in the plugin.
114+
115+
1.3.1 (2020-8-24)
116+
""""""""""""""""""""""""""
117+
**Changes:**
118+
119+
* Now handles situations where TRANSFORMS REGEX uses _VAL in transforms.conf.
120+
* pytest-splunk-addon now handles eval functions using NULL more efficiently.
121+
122+
**Known Issues:**
123+
124+
* Event ingestion through SC4S via UDP port
125+
* Fields for modular regular expressions are not extracted in the plugin.
126+
127+
1.3.0 (2020-8-21)
128+
""""""""""""""""""""""""""
129+
**Features:**
130+
131+
* pytest-splunk-addon now generates data with it's own data generator feature which replaces SA-Eventgen for accuracy. This feature can ingest data using HEC event, HEC Raw and SC4S (TCP).
132+
* pytest-splunk-addon now generates Index Time test cases for your Splunk Technology Add-ons.
133+
* Added a utility to create a new pytest-splunk-addon-data.conf file from existing eventgen.conf file.
134+
* Backward compatibility for search time tests using existing eventgen.conf.
135+
136+
**Bugfixes:**
137+
138+
* Enhanced requirements for the following CIM data models:
139+
140+
+-----------------------+-----------------------------------------------------------+
141+
| CIM Data Model | Field Name |
142+
+=======================+===========================================================+
143+
| IDS | src, dest, src_port, dest_port, user |
144+
+-----------------------+-----------------------------------------------------------+
145+
| Network Resolution | src, dest |
146+
+-----------------------+-----------------------------------------------------------+
147+
| Network Traffic | bytes, bytes_in, bytes_out, icmp_code |
148+
| | |
149+
| | packets, packets_in, packets_out |
150+
| | |
151+
| | src, src_translated_port, src_port |
152+
| | |
153+
| | dest, dest_translated_port, dest_port |
154+
+-----------------------+-----------------------------------------------------------+
155+
| Web | app, uri_path, url_length |
156+
+-----------------------+-----------------------------------------------------------+
157+
158+
**Known Issues:**
159+
160+
* Event ingestion through SC4S via UDP port
161+
* Fields for modular regular expressions are not extracted in the plugin.
162+
8163
1.2.0 (2020-06-04)
9164
""""""""""""""""""""""""""
10165
**Features:**
@@ -17,6 +172,7 @@ The best way to track the development of pytest-splunk-addon is through `the Git
17172
**Bugfixes:**
18173

19174
* Invalid search query generation for Malware Data Model is now fixed.
175+
* Invalid search query for clustered fields in CIM testing.
20176

21177
**Known Issues:**
22178

@@ -27,7 +183,7 @@ The best way to track the development of pytest-splunk-addon is through `the Git
27183

28184
**Features:**
29185

30-
* The codebase was reformatted to an object-oriented approach to increase the readability, scalability and the reusability of the plugin.
186+
* The codebase was reformatted to an object-oriented approach to increase the readability, scalability, and the reusability of the plugin.
31187
* pytest-splunk-addon now generates tests for checking CIM compatibility in your Splunk Technology Add-ons.
32188

33189
**Bugfixes:**

‎docs/sample_generator.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ pytest-splunk-addon-data.conf.spec
2828
latest = now
2929
timezone = 0000
3030
breaker = {{regex}}
31+
host_prefix = {{host_prefix}}
3132

3233
[<sample file name>]
3334
* The stanza can contain the sample File Name or Regex to match multiple sample files.
@@ -85,6 +86,9 @@ breaker = <regex>
8586
* This parameter is optional. If it is not provided by the user, the events will be ingested into Splunk,
8687
as per the *input_type* provided.
8788

89+
host_prefix = <host_prefix>
90+
* This param is used as an identification for the **host** field, for the events which are ingested using SC4S.
91+
8892
Token replacement settings
8993
-----------------------------
9094
The following replacementType -> replacement values are supported

‎docs/troubleshoot.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
Troubleshoot
1+
Troubleshooting
22
===================
33

44
**1. Test Case takes forever to run when using splunk-type=external**

0 commit comments

Comments
 (0)