Skip to content

Commit affecb0

Browse files
committed
fix: IndexError on whitespace-only text with overlapping (#18)
1 parent 7b99d44 commit affecb0

File tree

3 files changed

+8
-3
lines changed

3 files changed

+8
-3
lines changed

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
## Changelog 🔄
22
All notable changes to `semchunk` will be documented here. This project adheres to [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
33

4+
## [3.2.2] - 2025-06-09
5+
### Fixed
6+
- Fixed `IndexError` being raised when chunking whitespace only texts with overlapping enabled ([#18](https://github.com/isaacus-dev/semchunk/issues/18)).
7+
48
## [3.2.1] - 2025-03-27
59
### Fixed
610
- Fixed minor typos in the README and docstrings.
@@ -149,6 +153,7 @@ All notable changes to `semchunk` will be documented here. This project adheres
149153
### Added
150154
- Added the `chunk()` function, which splits text into semantically meaningful chunks of a specified size as determined by a provided token counter.
151155

156+
[3.2.2]: https://github.com/isaacus-dev/semchunk/compare/v3.2.1...v3.2.2
152157
[3.2.1]: https://github.com/isaacus-dev/semchunk/compare/v3.2.0...v3.2.1
153158
[3.2.0]: https://github.com/isaacus-dev/semchunk/compare/v3.1.3...v3.2.0
154159
[3.1.3]: https://github.com/isaacus-dev/semchunk/compare/v3.1.2...v3.1.3

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
44

55
[project]
66
name = "semchunk"
7-
version = "3.2.1"
7+
version = "3.2.2"
88
authors = [
99
{name="Isaacus", email="[email protected]"},
1010
{name="Umar Butler", email="[email protected]"},

src/semchunk/semchunk.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -274,8 +274,8 @@ def chunk(
274274
else:
275275
chunks, offsets = [], []
276276

277-
# Overlap chunks if desired.
278-
if overlap:
277+
# Overlap chunks if desired and there are chunks to overlap.
278+
if overlap and chunks:
279279
# Rename variables for clarity.
280280
subchunk_size = local_chunk_size
281281
subchunks = chunks

0 commit comments

Comments
 (0)