Skip to content

Commit 2449143

Browse files
authored
Merge branch 'main' into get-collections-free-text
2 parents f6aecf3 + 8dd200e commit 2449143

File tree

5 files changed

+225
-38
lines changed

5 files changed

+225
-38
lines changed

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
1515
- GET `/collections` collection search fields extension ex. `/collections?fields=id,title`. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
1616
- Improved error messages for sorting on unsortable fields in collection search, including guidance on how to make fields sortable. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
1717
- Added field alias for `temporal` to enable easier sorting by temporal extent, alongside `extent.temporal.interval`. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
18+
- Added `ENABLE_COLLECTIONS_SEARCH` environment variable to make collection search extensions optional (defaults to enabled). [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
1819

1920
### Changed
2021

README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,8 @@ SFEOS implements extended capabilities for the `/collections` endpoint, allowing
133133

134134
These extensions make it easier to build user interfaces that display and navigate through collections efficiently.
135135

136+
> **Configuration**: Collection search extensions can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.
137+
136138
> **Note**: Sorting is only available on fields that are indexed for sorting in Elasticsearch/OpenSearch. With the default mappings, you can sort on:
137139
> - `id` (keyword field)
138140
> - `extent.temporal.interval` (date field)
@@ -272,6 +274,7 @@ You can customize additional settings in your `.env` file:
272274
| `ENABLE_DIRECT_RESPONSE` | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false` | Optional |
273275
| `RAISE_ON_BULK_ERROR` | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` | Optional |
274276
| `DATABASE_REFRESH` | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
277+
| `ENABLE_COLLECTIONS_SEARCH` | Enable collection search extensions (sort, fields). | `true` | Optional |
275278
| `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. If set to `false`, the POST `/collections` route and related transaction endpoints (including bulk transaction operations) will be unavailable in the API. This is useful for deployments where mutating the catalog via the API should be prevented. | `true` | Optional |
276279
| `STAC_ITEM_LIMIT` | Sets the environment variable for result limiting to SFEOS for the number of returned items and STAC collections. | `10` | Optional |
277280
| `STAC_INDEX_ASSETS` | Controls if Assets are indexed when added to Elasticsearch/Opensearch. This allows asset fields to be included in search queries. | `false` | Optional |
@@ -418,6 +421,10 @@ The system uses a precise naming convention:
418421
- **Root Path Configuration**: The application root path is the base URL by default.
419422
- For AWS Lambda with Gateway API: Set `STAC_FASTAPI_ROOT_PATH` to match the Gateway API stage name (e.g., `/v1`)
420423

424+
- **Feature Configuration**: Control which features are enabled:
425+
- `ENABLE_COLLECTIONS_SEARCH`: Set to `true` (default) to enable collection search extensions (sort, fields). Set to `false` to disable.
426+
- `ENABLE_TRANSACTIONS_EXTENSIONS`: Set to `true` (default) to enable transaction extensions. Set to `false` to disable.
427+
421428

422429
## Collection Pagination
423430

stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py

Lines changed: 25 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,9 @@
5656
logger = logging.getLogger(__name__)
5757

5858
TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
59+
ENABLE_COLLECTIONS_SEARCH = get_bool_env("ENABLE_COLLECTIONS_SEARCH", default=True)
5960
logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)
61+
logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH)
6062

6163
settings = ElasticsearchSettings()
6264
session = Session.create_from_settings(settings)
@@ -114,25 +116,26 @@
114116

115117
extensions = [aggregation_extension] + search_extensions
116118

117-
# Create collection search extensions
118-
# Only sort extension is enabled for now
119-
collection_search_extensions = [
120-
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
121-
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
122-
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
123-
# CollectionSearchFilterExtension(
124-
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
125-
# ),
126-
FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
127-
]
128-
129-
# Initialize collection search with its extensions
130-
collection_search_ext = CollectionSearchExtension.from_extensions(
131-
collection_search_extensions
132-
)
133-
collections_get_request_model = collection_search_ext.GET
119+
# Create collection search extensions if enabled
120+
if ENABLE_COLLECTIONS_SEARCH:
121+
# Create collection search extensions
122+
collection_search_extensions = [
123+
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
124+
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
125+
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
126+
# CollectionSearchFilterExtension(
127+
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
128+
# ),
129+
FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
130+
]
131+
132+
# Initialize collection search with its extensions
133+
collection_search_ext = CollectionSearchExtension.from_extensions(
134+
collection_search_extensions
135+
)
136+
collections_get_request_model = collection_search_ext.GET
134137

135-
extensions.append(collection_search_ext)
138+
extensions.append(collection_search_ext)
136139

137140
database_logic.extensions = [type(ext).__name__ for ext in extensions]
138141

@@ -169,10 +172,13 @@
169172
"search_get_request_model": create_get_request_model(search_extensions),
170173
"search_post_request_model": post_request_model,
171174
"items_get_request_model": items_get_request_model,
172-
"collections_get_request_model": collections_get_request_model,
173175
"route_dependencies": get_route_dependencies(),
174176
}
175177

178+
# Add collections_get_request_model if collection search is enabled
179+
if ENABLE_COLLECTIONS_SEARCH:
180+
app_config["collections_get_request_model"] = collections_get_request_model
181+
176182
api = StacApi(**app_config)
177183

178184

stac_fastapi/opensearch/stac_fastapi/opensearch/app.py

Lines changed: 25 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,9 @@
5656
logger = logging.getLogger(__name__)
5757

5858
TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
59+
ENABLE_COLLECTIONS_SEARCH = get_bool_env("ENABLE_COLLECTIONS_SEARCH", default=True)
5960
logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)
61+
logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH)
6062

6163
settings = OpensearchSettings()
6264
session = Session.create_from_settings(settings)
@@ -114,25 +116,26 @@
114116

115117
extensions = [aggregation_extension] + search_extensions
116118

117-
# Create collection search extensions
118-
# Only sort extension is enabled for now
119-
collection_search_extensions = [
120-
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
121-
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
122-
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
123-
# CollectionSearchFilterExtension(
124-
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
125-
# ),
126-
FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
127-
]
128-
129-
# Initialize collection search with its extensions
130-
collection_search_ext = CollectionSearchExtension.from_extensions(
131-
collection_search_extensions
132-
)
133-
collections_get_request_model = collection_search_ext.GET
119+
# Create collection search extensions if enabled
120+
if ENABLE_COLLECTIONS_SEARCH:
121+
# Create collection search extensions
122+
collection_search_extensions = [
123+
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
124+
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
125+
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
126+
# CollectionSearchFilterExtension(
127+
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
128+
# ),
129+
FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
130+
]
131+
132+
# Initialize collection search with its extensions
133+
collection_search_ext = CollectionSearchExtension.from_extensions(
134+
collection_search_extensions
135+
)
136+
collections_get_request_model = collection_search_ext.GET
134137

135-
extensions.append(collection_search_ext)
138+
extensions.append(collection_search_ext)
136139

137140
database_logic.extensions = [type(ext).__name__ for ext in extensions]
138141

@@ -166,13 +169,16 @@
166169
post_request_model=post_request_model,
167170
landing_page_id=os.getenv("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"),
168171
),
169-
"collections_get_request_model": collections_get_request_model,
170172
"search_get_request_model": create_get_request_model(search_extensions),
171173
"search_post_request_model": post_request_model,
172174
"items_get_request_model": items_get_request_model,
173175
"route_dependencies": get_route_dependencies(),
174176
}
175177

178+
# Add collections_get_request_model if collection search is enabled
179+
if ENABLE_COLLECTIONS_SEARCH:
180+
app_config["collections_get_request_model"] = collections_get_request_model
181+
176182
api = StacApi(**app_config)
177183

178184

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
"""Test the ENABLE_COLLECTIONS_SEARCH environment variable."""
2+
3+
import os
4+
import uuid
5+
from unittest import mock
6+
7+
import pytest
8+
9+
from ..conftest import create_collection, refresh_indices
10+
11+
12+
@pytest.mark.asyncio
13+
@mock.patch.dict(os.environ, {"ENABLE_COLLECTIONS_SEARCH": "false"})
14+
async def test_collections_search_disabled(app_client, txn_client, load_test_data):
15+
"""Test that collection search extensions are disabled when ENABLE_COLLECTIONS_SEARCH=false."""
16+
# Create multiple collections with different ids to test sorting
17+
base_collection = load_test_data("test_collection.json")
18+
19+
# Use unique prefixes to avoid conflicts between tests
20+
test_prefix = f"disabled-{uuid.uuid4().hex[:8]}"
21+
collection_ids = [f"{test_prefix}-c", f"{test_prefix}-a", f"{test_prefix}-b"]
22+
23+
for i, coll_id in enumerate(collection_ids):
24+
test_collection = base_collection.copy()
25+
test_collection["id"] = coll_id
26+
test_collection["title"] = f"Test Collection {i}"
27+
await create_collection(txn_client, test_collection)
28+
29+
# Refresh indices to ensure collections are searchable
30+
await refresh_indices(txn_client)
31+
32+
# When collection search is disabled, sortby parameter should be ignored
33+
resp = await app_client.get(
34+
"/collections",
35+
params=[("sortby", "+id")],
36+
)
37+
assert resp.status_code == 200
38+
39+
# Verify that results are NOT sorted by id (should be in insertion order or default order)
40+
resp_json = resp.json()
41+
collections = [
42+
c for c in resp_json["collections"] if c["id"].startswith(test_prefix)
43+
]
44+
45+
# Extract the ids in the order they were returned
46+
returned_ids = [c["id"] for c in collections]
47+
48+
# If sorting was working, they would be in alphabetical order: a, b, c
49+
# But since sorting is disabled, they should be in a different order
50+
# We can't guarantee the exact order, but we can check they're not in alphabetical order
51+
sorted_ids = sorted(returned_ids)
52+
assert (
53+
returned_ids != sorted_ids or len(collections) < 2
54+
), "Collections appear to be sorted despite ENABLE_COLLECTIONS_SEARCH=false"
55+
56+
# Fields parameter should also be ignored
57+
resp = await app_client.get(
58+
"/collections",
59+
params=[("fields", "id")], # Request only id field
60+
)
61+
assert resp.status_code == 200
62+
63+
# Verify that all fields are still returned, not just id
64+
resp_json = resp.json()
65+
for collection in resp_json["collections"]:
66+
if collection["id"].startswith(test_prefix):
67+
# If fields filtering was working, only id would be present
68+
# Since it's disabled, other fields like title should still be present
69+
assert (
70+
"title" in collection
71+
), "Fields filtering appears to be working despite ENABLE_COLLECTIONS_SEARCH=false"
72+
73+
74+
@pytest.mark.asyncio
75+
@mock.patch.dict(os.environ, {"ENABLE_COLLECTIONS_SEARCH": "true"})
76+
async def test_collections_search_enabled(app_client, txn_client, load_test_data):
77+
"""Test that collection search extensions work when ENABLE_COLLECTIONS_SEARCH=true."""
78+
# Create multiple collections with different ids to test sorting
79+
base_collection = load_test_data("test_collection.json")
80+
81+
# Use unique prefixes to avoid conflicts between tests
82+
test_prefix = f"enabled-{uuid.uuid4().hex[:8]}"
83+
collection_ids = [f"{test_prefix}-c", f"{test_prefix}-a", f"{test_prefix}-b"]
84+
85+
for i, coll_id in enumerate(collection_ids):
86+
test_collection = base_collection.copy()
87+
test_collection["id"] = coll_id
88+
test_collection["title"] = f"Test Collection {i}"
89+
await create_collection(txn_client, test_collection)
90+
91+
# Refresh indices to ensure collections are searchable
92+
await refresh_indices(txn_client)
93+
94+
# Test that sortby parameter works - sort by id ascending
95+
resp = await app_client.get(
96+
"/collections",
97+
params=[("sortby", "+id")],
98+
)
99+
assert resp.status_code == 200
100+
101+
# Verify that results are sorted by id in ascending order
102+
resp_json = resp.json()
103+
collections = [
104+
c for c in resp_json["collections"] if c["id"].startswith(test_prefix)
105+
]
106+
107+
# Extract the ids in the order they were returned
108+
returned_ids = [c["id"] for c in collections]
109+
110+
# Verify they're in ascending order
111+
assert returned_ids == sorted(
112+
returned_ids
113+
), "Collections are not sorted by id ascending"
114+
115+
# Test that sortby parameter works - sort by id descending
116+
resp = await app_client.get(
117+
"/collections",
118+
params=[("sortby", "-id")],
119+
)
120+
assert resp.status_code == 200
121+
122+
# Verify that results are sorted by id in descending order
123+
resp_json = resp.json()
124+
collections = [
125+
c for c in resp_json["collections"] if c["id"].startswith(test_prefix)
126+
]
127+
128+
# Extract the ids in the order they were returned
129+
returned_ids = [c["id"] for c in collections]
130+
131+
# Verify they're in descending order
132+
assert returned_ids == sorted(
133+
returned_ids, reverse=True
134+
), "Collections are not sorted by id descending"
135+
136+
# Test that fields parameter works - request only id field
137+
resp = await app_client.get(
138+
"/collections",
139+
params=[("fields", "id")],
140+
)
141+
assert resp.status_code == 200
142+
resp_json = resp.json()
143+
144+
# When fields=id is specified, collections should only have id field
145+
for collection in resp_json["collections"]:
146+
if collection["id"].startswith(test_prefix):
147+
assert "id" in collection, "id field is missing"
148+
assert (
149+
"title" not in collection
150+
), "title field should be excluded when fields=id"
151+
152+
# Test that fields parameter works - request multiple fields
153+
resp = await app_client.get(
154+
"/collections",
155+
params=[("fields", "id,title")],
156+
)
157+
assert resp.status_code == 200
158+
resp_json = resp.json()
159+
160+
# When fields=id,title is specified, collections should have both fields but not others
161+
for collection in resp_json["collections"]:
162+
if collection["id"].startswith(test_prefix):
163+
assert "id" in collection, "id field is missing"
164+
assert "title" in collection, "title field is missing"
165+
assert (
166+
"description" not in collection
167+
), "description field should be excluded when fields=id,title"

0 commit comments

Comments
 (0)