Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
89fd009
Update main ver to 6.1.0-SNAPSHOT
leerho Jan 28, 2025
b014dca
Merge pull request #251 from apache/main_to_6.1.0-SNAPSHOT
leerho Jan 28, 2025
adb0f2e
fixed operator precedence bug in setPosition() calls
pawel-wiejacha Jan 30, 2025
af83326
Merge pull request #253 from pawel-wiejacha/fix_set_position
leerho Jan 30, 2025
b7b7437
Fixed all the tests which lacked the ability to test this bug.
leerho Jan 31, 2025
b55af70
Merge pull request #254 from apache/fix_setPosition
leerho Feb 1, 2025
4c73fb4
a two letter typo.
leerho Feb 2, 2025
cabf9ca
More comprehensive tests.
leerho Feb 3, 2025
7f40e8b
Add missing license
leerho Feb 3, 2025
e0faea2
Better test coverage AND actual checking for the @pawel-weijacha bug for
leerho Feb 3, 2025
f6a7c58
Merge pull request #256 from apache/typo_in_main
leerho Feb 3, 2025
22c7908
Merge pull request #257 from apache/better_test_coverage
leerho Feb 3, 2025
0722943
Redo var names
leerho Feb 4, 2025
1b9e702
Merge pull request #259 from apache/Redo_var_names
leerho Feb 4, 2025
05b76b0
Investigate Hash bug
leerho Feb 9, 2025
38df8a7
Only change required to fix the Hash Bug.
leerho Feb 12, 2025
1902d17
Added additional MurmurHash3 tests, especially to catch the Hash bug.
leerho Feb 12, 2025
e9f998a
Merge pull request #260 from apache/hash_bug
leerho Feb 14, 2025
9d29ac9
Update how Memory Request Server works.
leerho Feb 16, 2025
b529f86
Revised isSameResource()
leerho Feb 17, 2025
8ef26d7
Merge branch 'main' into update_memReqSvr
leerho Feb 17, 2025
9a3b29b
Improve javadoc for 'isSameResource(...)'
leerho Feb 17, 2025
6f8adea
improve error comment.
leerho Feb 17, 2025
602eac2
Allow getMemorySegment as read-only.
leerho Feb 17, 2025
c0b1979
Improve javadocs
leerho Feb 18, 2025
52ac28c
Move MurmurHash3 out of Memory into ds-java.
leerho Feb 18, 2025
2ffc5db
Merge pull request #262 from apache/update_memReqSvr
leerho Feb 18, 2025
36b980a
Small change to POM.
leerho Feb 18, 2025
0a4d369
Merge branch 'main' into target_is_6.0.X
leerho Feb 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,6 @@ under the License.

<modelVersion>4.0.0</modelVersion>

<!--
<parent>
<groupId>org.apache</groupId>
<artifactId>apache</artifactId>
<version>32</version>
</parent> -->

<groupId>org.apache.datasketches</groupId>
<artifactId>datasketches-memory</artifactId>
<version>6.0.0</version>
Expand Down Expand Up @@ -85,7 +78,7 @@ under the License.

<properties>
<!-- Test -->
<testng.version>7.10.2</testng.version>
<testng.version>7.11.0</testng.version>

<!-- System-wide properties -->
<maven.version>3.6.3</maven.version>
Expand Down Expand Up @@ -189,7 +182,7 @@ under the License.
<configuration>
<rules>
<requireJavaVersion>
<version>${java.version}</version>
<version>${java.version},</version>
</requireJavaVersion>
<requireMavenVersion>
<version>[${maven.version},4.0.0)</version>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,39 +27,121 @@
* manage continuous requests for larger or smaller memory.
* This capability is only available for writable, non-file-memory-mapping resources.
*
* <p>The operation of this implementation is controlled by three conditions:</p>
* <ul>
* <li><b><i>origOffHeap:</i></b> If <i>true</i>, the originally allocated WritableMemory is off-heap.</li>
*
* <li><b><i>oneArena:</i></b> If <i>true</i>, all subsequent off-heap allocations will use the same Arena
* obtained from the original off-heap WritableMemory. Otherwise, subsequent off-heap allocations will
* use a new confined Arena created by this implementation.</li>
*
* <li><b><i>offHeap:</i></b> If <i>true</i>, all subsequent allocations will be off-heap.
* If the originally allocated WritableMemory is on-heap, this variable is ignored.</li>
* </ul>
*
* <p>These three variables work together as follows:</p>
*
* <ul>
* <li>If the original WritableMemory is on-heap, all subsequent allocations will also be on-heap.</li>
*
* <li>If <i>origOffHeap</i> = <i>true</i>, <i>oneArena</i> = <i>true</i>, and <i>offHeap</i> = <i>true</i>,
* all subsequent allocations will also be off-heap and associated with the original Arena.
* It is the responsibility of the user to close the original Arena using a Try-With-Resource block, or directly.</li>
*
* <li>If the original WritableMemory is off-heap, <i>oneArena</i> is true, and <i>offHeap</i> is false,
* all subsequent allocations will be on-heap.
* It is the responsibility of the user to close the original Arena using a Try-With-Resource block, or directly.</li>
*
* <li>If the original WritableMemory is off-heap, <i>oneArena</i> is false, and <i>offHeap</i> is true,
* all subsequent allocations will also be off-heap and associated with a new confined Arena assigned by this implementation.
* It is the responsibility of the user to close the original Arena using a Try-With-Resource block, or directly,
* and close the last returned new WritableMemory directly.</li>
* </ul>
*
* <p>In summary:</p>
*
* <table> <caption><b>Configuration Options</b></caption>
* <tr><th>Original Off-Heap</th> <th>OneArena</th> <th>OffHeap</th> <th>Subsequent Allocations</th></tr>
* <tr><td>false</td> <td>N/A</td> <td>N/A</td> <td>All on-heap</td></tr>
* <tr><td>true</td> <td>N/A</td> <td>false</td> <td>All on-heap</td></tr>
* <tr><td>true</td> <td>true</td> <td>true</td> <td>All off-heap in original Arena</td></tr>
* <tr><td>true</td> <td>false</td> <td>true</td> <td>All off-heap in separate Arenas</td></tr>
* </table>
*
* @author Lee Rhodes
*/
@SuppressWarnings("resource") //can't use TWRs here
public final class DefaultMemoryRequestServer implements MemoryRequestServer {
private final long alignmentBytes;
private final ByteOrder byteOrder;
private final boolean oneArena;
private final boolean offHeap;

/**
* Default constructor.
*/
public DefaultMemoryRequestServer() { }
public DefaultMemoryRequestServer() {
alignmentBytes = 8;
byteOrder = ByteOrder.nativeOrder();
oneArena = false;
offHeap = false;
}

@Override
public WritableMemory request(
final long newCapacityBytes,
/**
* Optional constructor 1.
* @param oneArena if true, the original arena will be used for all requested allocations.
* @param offHeap if true, new allocations will be off-heap.
*/
public DefaultMemoryRequestServer(
final boolean oneArena,
final boolean offHeap) {
this.alignmentBytes = 8;
this.byteOrder = ByteOrder.nativeOrder();
this.oneArena = oneArena;
this.offHeap = offHeap;
}

/**
* Optional constructor 2.
* @param alignmentBytes requested segment alignment for all allocations. Typically 1, 2, 4 or 8.
* @param byteOrder the given <i>ByteOrder</i>. It must be non-null.
* @param oneArena if true, the same arena will be used for all requested allocations.
* @param offHeap if true, new allocations will be off-heap.
*/
public DefaultMemoryRequestServer(
final long alignmentBytes,
final ByteOrder byteOrder,
final Arena arena) {
final WritableMemory newWmem;
final boolean oneArena,
final boolean offHeap) {
this.alignmentBytes = alignmentBytes;
this.byteOrder = byteOrder;
this.oneArena = oneArena;
this.offHeap = offHeap;
}

if (arena != null) {
newWmem = WritableMemory.allocateDirect(newCapacityBytes, alignmentBytes, byteOrder, this, arena);
}
else { //On-heap
@Override
public WritableMemory request(
final WritableMemory oldWmem,
final long newCapacityBytes) {

//On-heap
if (oldWmem.getArena() == null || !offHeap) {
if (newCapacityBytes > Integer.MAX_VALUE) {
throw new IllegalArgumentException("Requested capacity exceeds Integer.MAX_VALUE.");
}
newWmem = WritableMemory.allocate((int)newCapacityBytes, byteOrder, this);
return WritableMemory.allocate((int)newCapacityBytes, byteOrder, this);
}

return newWmem;
//Acquire Arena
final Arena arena = (oneArena) ? oldWmem.getArena() : Arena.ofConfined();
return WritableMemory.allocateDirect(newCapacityBytes, alignmentBytes, byteOrder, this, arena);
}

@Override
public void requestClose(final Arena arena) {
if (arena.scope().isAlive()) { arena.close(); }
public void requestClose(final WritableMemory wmemToClose) {
final Arena arena = wmemToClose.getArena();
if (oneArena || arena == null || !arena.scope().isAlive()) { return; } //can't close
arena.close();
}

}
14 changes: 7 additions & 7 deletions src/main/java/org/apache/datasketches/memory/Memory.java
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ static Memory wrap(
int lengthBytes,
ByteOrder byteOrder) {
final MemorySegment slice = MemorySegment.ofArray(array).asSlice(offsetBytes, lengthBytes).asReadOnly();
return WritableMemoryImpl.wrapSegmentAsArray(slice, byteOrder, null);
return WritableMemoryImpl.wrapSegment(slice, byteOrder);
}

//intentionally removed wrap(boolean[])
Expand All @@ -234,7 +234,7 @@ static Memory wrap(
*/
static Memory wrap(char[] array) {
final MemorySegment seg = MemorySegment.ofArray(array).asReadOnly();
return WritableMemoryImpl.wrapSegmentAsArray(seg, ByteOrder.nativeOrder(), null);
return WritableMemoryImpl.wrapSegment(seg, ByteOrder.nativeOrder());
}

/**
Expand All @@ -244,7 +244,7 @@ static Memory wrap(char[] array) {
*/
static Memory wrap(short[] array) {
final MemorySegment seg = MemorySegment.ofArray(array).asReadOnly();
return WritableMemoryImpl.wrapSegmentAsArray(seg, ByteOrder.nativeOrder(), null);
return WritableMemoryImpl.wrapSegment(seg, ByteOrder.nativeOrder());
}

/**
Expand All @@ -254,7 +254,7 @@ static Memory wrap(short[] array) {
*/
static Memory wrap(int[] array) {
final MemorySegment seg = MemorySegment.ofArray(array).asReadOnly();
return WritableMemoryImpl.wrapSegmentAsArray(seg, ByteOrder.nativeOrder(), null);
return WritableMemoryImpl.wrapSegment(seg, ByteOrder.nativeOrder());
}

/**
Expand All @@ -264,7 +264,7 @@ static Memory wrap(int[] array) {
*/
static Memory wrap(long[] array) {
final MemorySegment seg = MemorySegment.ofArray(array).asReadOnly();
return WritableMemoryImpl.wrapSegmentAsArray(seg, ByteOrder.nativeOrder(), null);
return WritableMemoryImpl.wrapSegment(seg, ByteOrder.nativeOrder());
}

/**
Expand All @@ -274,7 +274,7 @@ static Memory wrap(long[] array) {
*/
static Memory wrap(float[] array) {
final MemorySegment seg = MemorySegment.ofArray(array).asReadOnly();
return WritableMemoryImpl.wrapSegmentAsArray(seg, ByteOrder.nativeOrder(), null);
return WritableMemoryImpl.wrapSegment(seg, ByteOrder.nativeOrder());
}

/**
Expand All @@ -284,7 +284,7 @@ static Memory wrap(float[] array) {
*/
static Memory wrap(double[] array) {
final MemorySegment seg = MemorySegment.ofArray(array).asReadOnly();
return WritableMemoryImpl.wrapSegmentAsArray(seg, ByteOrder.nativeOrder(), null);
return WritableMemoryImpl.wrapSegment(seg, ByteOrder.nativeOrder());
}
//END OF CONSTRUCTOR-TYPE METHODS

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@

package org.apache.datasketches.memory;

import java.lang.foreign.Arena;
import java.nio.ByteOrder;

/**
* The MemoryRequestServer is a callback interface to provide a means to request more or less memory
* for heap and off-heap WritableMemory resources that are not file-memory-mapped backed resources.
Expand All @@ -33,26 +30,21 @@
public interface MemoryRequestServer {

/**
* Request new WritableMemory with the given newCapacityBytes.
* Request a new WritableMemory with the given newCapacityBytes.
* @param oldWmem the previous WritableMemory to be possibly closed and which provides an associated Arena
* that may be used for allocating the new WritableMemory.
* If the arena is null, the requested WritableMemory will be on-heap.
* @param newCapacityBytes The capacity being requested.
* @param alignmentBytes requested segment alignment. Typically 1, 2, 4 or 8.
* @param byteOrder the given <i>ByteOrder</i>. It must be non-null.
* @param arena the given arena to manage the new off-heap WritableMemory.
* If arena is null, the requested WritableMemory will be on-heap.
* Warning: This class is not thread-safe. Specifying an Arena that allows multiple threads is not recommended.
*
* @return new WritableMemory with the requested capacity.
*/
WritableMemory request(
long newCapacityBytes,
long alignmentBytes,
ByteOrder byteOrder,
Arena arena);
WritableMemory request(WritableMemory oldWmem, long newCapacityBytes);

/**
* Request to close the area managing all the related resources, if applicable.
* Be careful when you request to close the given Arena, you may be closing other resources as well.
* @param arena the given arena to use to close all its managed resources.
* Request to close the given WritableMemory. If applicable, it will be closed by its associated Arena.
* Be careful. Closing the associated Arena may be closing other resources as well.
* @param wmemToClose the given WritableMemory to close.
*/
void requestClose( Arena arena);
void requestClose(WritableMemory wmemToClose);

}
Loading
Loading