You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _search-plugins/ubi/index.md
+56-23Lines changed: 56 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ redirect_from:
11
11
**Introduced 2.15**
12
12
{: .label .label-purple }
13
13
14
-
**References UBI Specification 1.0.0**
14
+
**References UBI Specification 1.3.0**
15
15
{: .label .label-purple }
16
16
17
17
User Behavior Insights (UBI) is a schema for capturing user search behavior. Search behavior consists of the queries that the user submits, the results that are presented to them, and the actions they take on those results. The UBI schema links all user interactions (events) to the search result they were performed on. That is, it not only captures the chronological sequence of events but also captures the causal links between events. Analysis of this behavior is used for improving the quality of search results.
@@ -20,33 +20,66 @@ Client applications such as web pages or apps capture user behavior and send UBI
20
20
21
21
In principle, queries sent to the server and results returned by the server can be sent to the UBI endpoint from the client. But as an optimization, they can instead be sent directly to the UBI endpoint from the server, without incurring a round-trip to the client. That is the function of the UBI plugin and is not a requirement to adopt UBI.
22
22
23
-
UBI includes the following elements:
24
-
* A machine-readable [schema](https://github.com/o19s/ubi) that faciliates interoperablity of the UBI specification.
25
-
* A client-side JavaScript [example reference implementation]({{site.url}}{{site.baseurl}}/search-plugins/ubi/data-structures/) that shows how to capture events and send them to the OpenSearch UBI plugin.
26
-
* An OpenSearch [plugin](https://github.com/opensearch-project/user-behavior-insights) that captures server-side behavior.
27
-
28
-
<!-- vale off -->
29
23
30
-
The UBI documentation is organized into two categories: *Explanation and reference*and *Tutorials and how-to guides*:
24
+
> "how our users are using our product, whether search results were useful for them and whether they clicked on top-n results we gave and all related stuff" -- Data scientist working on search.
31
25
32
-
*Explanation and reference*
33
-
34
-
| Link | Description |
35
-
| :--------- | :------- |
36
-
|[UBI Request/Response Specification](https://github.com/o19s/ubi/)| The industry-standard schema for UBI requests and responses. The current version references UBI Specification 1.0.0. |
37
-
|[UBI index schema]({{site.url}}{{site.baseurl}}/search-plugins/ubi/schemas/)| Documentation on the individual OpenSearch query and event stores. |
26
+
UBI includes the following elements:
27
+
* A machine-readable [schema](https://github.com/o19s/ubi) that facilitates interoperability of the UBI specification.
28
+
*[ubi.js](https://github.com/opensearch-project/user-behavior-insights/tree/main/ubi-javascript-collector/ubi.js): An (optional) client-side JavaScript library for capturing searches and events.
29
+
* An (optional) OpenSearch [plugin](https://github.com/opensearch-project/user-behavior-insights) that streamlines the recording of query data.
38
30
31
+
Advanced features in OpenSearch, such as the [Search Relevance Workbench]({{site.url}}{{site.baseurl}}/search-plugins/search-relevance/using-search-relevance-workbench/) and the [Hybrid Search Optimizer]({{site.url}}{{site.baseurl}}/search-plugins/search-relevance/optimize-hybrid-search/), build on the data collected according to the UBI specification.
39
32
40
-
*Tutorials and how-to guides*
33
+
<!-- vale off -->
41
34
42
-
| Link | Description |
43
-
| :--------- | :------- |
44
-
|[UBI plugin](https://github.com/opensearch-project/user-behavior-insights)| How to install and use the UBI plugin. |
45
-
|[UBI client data structures]({{site.url}}{{site.baseurl}}/search-plugins/ubi/data-structures/)| Sample JavaScript structures for populating the event store. |
46
-
|[Example UBI query DSL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/dsl-queries/)| How to write queries for UBI data in OpenSearch query DSL. |
47
-
|[Example UBI SQL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/sql-queries/)| How to write analytic queries for UBI data in SQL. |
48
-
|[UBI dashboard tutorial]({{site.url}}{{site.baseurl}}/search-plugins/ubi/ubi-dashboard-tutorial/)| How to build a dashboard containing UBI data. |
49
-
|[Chorus Opensearch Edition](https://github.com/o19s/chorus-opensearch-edition/?tab=readme-ov-file#structured-learning-using-chorus-opensearch-edition) katas | A series of structured tutorials that teach you how to use UBI with OpenSearch through a demo e-commerce store. |
35
+
<table>
36
+
<trstyle="vertical-align: top;">
37
+
<td>
38
+
<h2>Tutorials</h2>
39
+
<ul>
40
+
<li><a href="{{site.url}}{{site.baseurl}}/search-plugins/ubi/ubi-dashboard-tutorial/">Learn to create custom dashboards</a> for visualizing UBI data.</li>
41
+
<li> Based on <a href="https://github.com/o19s/chorus-opensearch-edition">Chorus for OpenSearch</a> demo:
42
+
<ul>
43
+
<li><a href="https://github.com/o19s/chorus-opensearch-edition/blob/main/katas/002_derive_interaction_data.md">Derive Interaction Data from User Clicks.</a></li>
44
+
<li><a href="https://github.com/o19s/chorus-opensearch-edition/blob/main/katas/006_protecting_sensitive_information.md">Protecting sensistive information when using UBI.</a></li>
45
+
<li><a href="https://github.com/o19s/chorus-opensearch-edition/blob/main/katas/007_configure_AB_with_TDI.md">Configuring an AB test with Team Draft Interleaving</a></li>
46
+
</ul>
47
+
</li>
48
+
</ul>
49
+
</td>
50
+
<td>
51
+
<h2>How To Guides</h2>
52
+
<ul>
53
+
<li>How to <a href="https://github.com/opensearch-project/user-behavior-insights?tab=readme-ov-file#user-quick-start">install and use the UBI plugin</a> in OpenSearch.</li>
54
+
<li>How to use <a href="{{site.url}}{{site.baseurl}}/search-plugins/ubi/ubi-javascript-collector/">ubi.js</a>, a client-side JavaScript library for capturing events.</li>
55
+
<li>How to <a href="{{site.url}}{{site.baseurl}}/search-plugins/ubi/dsl-queries/">write queries for UBI data using OpenSearch query DSL.</a></li>
56
+
<li>How to <a href="{{site.url}}{{site.baseurl}}/search-plugins/ubi/sql-queries/">write analytic queries for UBI data using SQL.</a></li>
<li>Learn more about this standard via <a href="https://www.UBISearch.dev">https://www.UBISearch.dev</a>, the community clearinghouse.</li>
66
+
<li>Watch <a href="https://youtu.be/0chun264PRQ">Leveraging UBI to enhance Search Relevance</a> talk to understand how to use this data to improve search quality.</li>
67
+
<li>Go deeper with UBI. Watch <a href="https://www.youtube.com/watch?v=xi261oUamXc">You’ve Deployed User Behavior Insights. Now What?</a> to see what else you can do.</li>
68
+
</ul>
69
+
</td>
70
+
<td>
71
+
<h2>Reference</h2>
72
+
<ul>
73
+
<li><a href="https://github.com/opensearch-project/user-behavior-insights">UBI Plugin for OpenSearch</a></li>
74
+
<li><a href="{{site.url}}{{site.baseurl}}/search-plugins/ubi/schemas/">UBI Schema in OpenSearch</a></li>
75
+
<li>Repository for the <a href="https://github.com/o19s/ubi">UBI Schema</a>.</li>
Copy file name to clipboardExpand all lines: _search-plugins/ubi/schemas.md
+12-18Lines changed: 12 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,8 +16,8 @@ The User Behavior Insights (UBI) data collection process involves tracking and r
16
16
17
17
For UBI to function properly, the connections between the following fields must be consistently maintained within an application that has UBI enabled:
18
18
19
-
-[`object_id`](#object_id) represents an ID for whatever object the user receives in response to a query. For example, if you search for books, it might be an ISBN code of a book, such as `978-3-16-148410-0`.
20
-
-[`query_id`](#query_id) is a unique ID for the raw query language executed and the `object_id`values of the _hits_ returned by the user's query.
19
+
-[`object_id`](#object_id) represents an ID for whatever object the user receives in response to a query. For example, if you search for books, it might be an ISBN for a book, such as `978-3-16-148410-0`.
20
+
-[`query_id`](#query_id) is a unique ID for the raw query executed, while the `object_id`maps to the primary identifier of the _hits_ returned by the user's query.
21
21
-[`client_id`](#client_id) represents a unique query source. This is typically a web browser used by a unique user.
22
22
-[`object_id_field`](#object_id_field) specifies the name of the field in your index that provides the `object_id`. For example, if you search for books, the value might be `isbn_code`.
23
23
-[`action_name`](#action_name), though not technically an ID, specifies the exact user action (such as `click`, `add_to_cart`, `watch`, `view`, or `purchase`) that was taken (or not taken) for an object with a given `object_id`.
@@ -138,11 +138,11 @@ All underlying query information and results (`object_ids`) are stored in the `u
138
138
139
139
The `ubi_queries` index [schema](https://github.com/OpenSearch-project/user-behavior-insights/tree/main/src/main/resources/queries-mapping.json) includes the following fields:
140
140
141
-
-`timestamp` (events and queries): A UNIX timestamp indicating when the query was received.
141
+
-`timestamp` (events and queries): An ISO 8601--formatted timestamp indicating when the query was received.
142
142
143
-
-`query_id` (events and queries): The unique ID of the query provided by the client or generated automatically. Different queries with the same text generate different `query_id` values.
144
-
145
-
-`client_id` (events and queries): A user/client ID provided by the client application.
143
+
-`query_id` (events and queries): The unique ID of the query provided by the client or generated by the search engine. Different queries with the same text generate different `query_id` values.
144
+
145
+
-`client_id` (events and queries): A client ID provided by the client application.
146
146
147
147
-`query_response_objects_ids` (queries): An array of object IDs. An ID can have the same value as the `_id`, but it is meant to be the externally valid ID of a document, item, or product.
148
148
@@ -169,14 +169,14 @@ The following are the predefined, minimal fields in the `ubi_events` index:
169
169
<pid="query_id"> </p>
170
170
171
171
-`query_id` (size 100): The unique identifier of a query, which is typically a UUID but can be any string.
172
-
The `query_id` is either provided by the client or generated at index time by the UBI plugin. The `query_id` values in both the **UBI queries** and **UBI events** indexes must be consistent.
172
+
The `query_id` is either provided by the client or generated at query time by the UBI plugin. The `query_id` values in both the **UBI queries** and **UBI events** indexes must be consistent.
173
173
174
174
<pid="client_id"> </p>
175
175
176
176
-`client_id`: The client that issues the query. This is typically a web browser used by a unique user.
177
177
The `client_id` in both the **UBI queries** and **UBI events** indexes must be consistent.
178
178
179
-
-`timestamp`: When the event occurred, either in UNIX format or formatted as `2018-11-13T20:20:39+00:00`.
179
+
-`timestamp`: The time at which the event occurredin ISO 8601 format, such as `2018-11-13T20:20:39+00:00Z`.
180
180
181
181
-`message_type` (size 100): A logical bin for grouping actions (each with an `action_name`). For example, `QUERY` or `CONVERSION`.
182
182
@@ -193,18 +193,12 @@ The following are the predefined, minimal fields in the `ubi_events` index:
193
193
194
194
-`event_attributes.position.ordinal`: Tracks the list position that a user can select (for example, selecting the third element can be described as `event{onClick, results[4]}`).
195
195
196
-
-`event_attributes.position.{x,y}`: Tracks x and y values defined by the client.
197
-
198
-
-`event_attributes.position.page_depth`: Tracks the page depth of the results.
199
-
200
-
-`event_attributes.position.scroll_depth`: Tracks the scroll depth of the page results.
201
-
202
-
-`event_attributes.position.trail`: A text field that tracks the path/trail that a user took to get to this location.
203
-
196
+
-`event_attributes.position.xy.{x,y}`: Tracks x and y values defined by the client.
197
+
204
198
-`event_attributes.object`: Contains identifying information about the object returned by the query (for example, a book, product, or post).
205
199
The `object` structure can refer to the object by internal ID or object ID. The `object_id` is the ID that links prior queries to this object. This field comprises the following subfields:
206
200
207
-
-`event_attributes.object.internal_id`: A unique ID that OpenSearch can use to internally index the object, for example, the `_id` field in the indexes.
201
+
-`event_attributes.object.internal_id`: The unique ID that OpenSearch uses to internally index the object, for example, the `_id` field in the indexes.
208
202
209
203
<pid="object_id">
210
204
@@ -214,7 +208,7 @@ The following are the predefined, minimal fields in the `ubi_events` index:
214
208
215
209
<pid="object_id_field">
216
210
217
-
-`event_attributes.object.object_id_field`: Indicates the type/class of the object and the name of the search index field that contains the `object_id`.
211
+
-`event_attributes.object.object_id_field`: Indicates the type/class of the object and the name of the search index field that contains the `object_id`, such as `ssn`, `isbn`, or `ean`.
218
212
219
213
-`event_attributes.object.description`: An optional description of the object.
0 commit comments