Streamly.Internal.Data.MutArray

An unboxed mutable array. An array is created with a given length and capacity. Length is the number of valid elements in the array. Capacity is the maximum number of elements that the array can be expanded to without having to reallocate the memory.

The elements in the array can be mutated in-place without changing the reference (constructor). However, the length of the array cannot be mutated in-place. A new array reference is generated when the length changes. When the length is increased (upto the maximum reserved capacity of the array), the array is not reallocated and the new reference uses the same underlying memory as the old one.

Several routines in this module allow the programmer to control the capacity of the array. The programmer can control the trade-off between memory usage and performance impact due to reallocations when growing or shrinking the array.

Return a copy of the array in pinned memory if unpinned, else return the original array.

Return a copy of the array in unpinned memory if pinned, else return the original array.

Return True if the array is allocated in pinned memory.

Cast an array having elements of type a into an array having elements of type b. The length of the array should be a multiple of the size of the target element otherwise Nothing is returned.

Cast an array having elements of type a into an array having elements of type b. The array size must be a multiple of the size of type b otherwise accessing the last element of the array may result into a crash or a random value.

Pre-release

Cast an MutArray a into an MutArray Word8.

unsafeAsPtr arr f, f is a function used as f ptr len where ptr is a pointer to the beginning of array and len is the byte-length of the array.

Unsafe WARNING:

1. The array must be pinned, otherwise it will lead to memory corruption.
2. The user must not use the pointer beyond the supplied length.

Pre-release

Create an empty array.

Allocates an unpinned array of zero length but growable to the specified capacity without reallocation.

emptyWithAligned allocator alignment count allocates a new array of zero length and with a capacity to hold count elements, using allocator size alignment as the memory allocator function.

Alignment must be greater than or equal to machine word size and a power of 2.

Alignment is ignored if the allocator allocates unpinned memory.

Pre-release

Allocates a pinned array of zero length but growable to the specified capacity without reallocation.

O(1) Slice an array in constant time.

Unsafe: The bounds of the slice are not checked.

Unsafe

Pre-release

O(1) Get a reference to a slice from a mutable array. Throws an error if the slice extends out of the array bounds.

The capacity of the slice is the same as its length i.e. it does not have any unused or reserved space at the end.

The slice shares the same underlying mutable array when created. However, if the slice or the original array is reallocated by growing or shrinking then it will be copied to new memory and they will no longer share the same memory.

Pre-release

Like breakAt but does not check whether the index is valid.

>>> unsafeBreakAt i arr = (MutArray.unsafeSliceOffLen 0 i arr, MutArray.unsafeSliceOffLen i (MutArray.length arr - i) arr)

Create two slices of an array without copying the original array. The specified index i is the first index of the second slice.

Drops the separator byte

>>> arr <- MutArray.fromList "hello world"
>>> (a,b) <- MutArray.breakEndBy (== ' ') arr
>>> MutArray.toList a
"hello "
>>> MutArray.toList b
"world"

Break the array into two slices when the predicate succeeds. The array element matching the predicate is dropped. If the predicate never succeeds the second array is empty.

>>> arr <- MutArray.fromList "hello world"
>>> (a,b) <- MutArray.breakEndBy_ (== ' ') arr
>>> MutArray.toList a
"hello"
>>> MutArray.toList b
"world"

Pre-release

>>> arr <- MutArray.fromList "hello world"
>>> (a,b) <- MutArray.revBreakEndBy (== ' ') arr
>>> MutArray.toList a
"hello"
>>> MutArray.toList b
" world"

>>> arr <- MutArray.fromList "hello world"
>>> (a,b) <- MutArray.revBreakEndBy_ (== ' ') arr
>>> MutArray.toList a
"hello"
>>> MutArray.toList b
"world"

Strip elements which match the predicate, from both ends.

>>> arr <- MutArray.fromList "   hello world    "
>>> a <- MutArray.dropAround (== ' ') arr
>>> MutArray.toList a
"hello world"

Pre-release

Strip elements which match the predicate, from the start of the array.

>>> arr <- MutArray.fromList "    hello world"
>>> a <- MutArray.dropWhile (== ' ') arr
>>> MutArray.toList a
"hello world"

Pre-release

Strip elements which match the predicate, from the end of the array.

>>> arr <- MutArray.fromList "hello world    "
>>> a <- MutArray.revDropWhile (== ' ') arr
>>> MutArray.toList a
"hello world"

Pre-release

Like unsafeCreateOf but takes a new array allocator alloc size function as argument.

>>> unsafeCreateOfWith alloc n = MutArray.unsafeAppendN (alloc n) n

Pre-release

createWithOf alloc n folds a maximum of n elements into an array allocated using the alloc function.

The array capacity is guranteed to be at least n.

>>> createWithOf alloc n = Fold.take n (MutArray.unsafeCreateOfWith alloc n)
>>> createWithOf alloc n = MutArray.appendN (alloc n) n

Like createOf but does not check the array bounds when writing. The fold driver must not call the step function more than n times otherwise it will corrupt the memory and crash. This function exists mainly because any conditional in the step function blocks fusion causing 10x performance slowdown.

>>> unsafeCreateOf = MutArray.unsafeCreateOfWith MutArray.emptyOf

createOf n folds a maximum of n elements from the input stream to an MutArray.

The array capacity is guranteed to be at least n.

>>> createOf = MutArray.createWithOf MutArray.emptyOf
>>> createOf n = Fold.take n (MutArray.unsafeCreateOf n)
>>> createOf n = MutArray.appendMax n MutArray.empty

createMinOf count folds the whole input to a single array. The array starts at a size big enough to hold minCount elements, the size is doubled every time the array needs to be grown.

The array capacity is guaranteed to be at least count.

Caution! Do not use this on infinite streams.

>>> f n = MutArray.appendWith (* 2) (MutArray.emptyOf n)
>>> createWith n = Fold.rmapM MutArray.rightSize (f n)
>>> createWith n = Fold.rmapM MutArray.fromChunksK (MutArray.buildChunks n)

Pre-release

Fold the whole input to a single array.

Same as 'createMinOf using an initial array size of arrayChunkBytes bytes rounded up to the element size. If the array is expected to be smaller than arrayChunkBytes then use createMinOf to avoid wasting memory.

Caution! Do not use this on infinite streams.

Like createOf but writes the array in reverse order.

Pre-release

Like unsafeCreateOf but creates a pinned array.

Like createOf but creates a pinned array.

Like create but creates a pinned array.

Clone the elements of a MutArray. Does not clone the reserve capacity.

To clone a slice of MutArray you can create a slice with "unsafeSliceOffLen" and then use "clone".

The new MutArray is unpinned in nature. Use "clone'" to clone the MutArray in pinned memory.

Create a MutArray from the first N elements of a list. The array is allocated to size N, if the list terminates before N elements then the array may hold less than N elements.

Like fromListN but creates a pinned array.

Create a MutArray from a list. The list must be of finite size.

Like fromList but creates a pinned array.

Like fromListN but writes the array in reverse order.

Pre-release

Like fromList but writes the contents of the list in reverse order.

Create a MutArray of given size from a stream.

>>> fromStreamN n = Stream.fold (MutArray.createOf n)

Create an Array from a stream. This is useful when we want to create a single array from a stream of unknown size. createOf is at least twice as efficient when the size is already known.

Note that if the input stream is too large memory allocation for the array may fail. When the stream size is not known, chunksOf followed by processing of indvidual arrays in the resulting stream should be preferred.

Pre-release

Convert a pure stream in Identity monad to a mutable array.

Convert a pure stream in Identity monad to a mutable array.

fromCString# addr copies a C string consisting of bytes and terminated by a null byte, into a Word8 array. The null byte is not copied.

>>> MutArray.fromCString# "hello"#

Unsafe:

The caller has to ensure that:

1. the addr is pinned and alive during the call.
2. the pointer passed is valid up to the point where null byte is found.

fromW16CString# addr copies a C string consisting of 16-bit wide chars and terminated by a 16-bit null char, into a Word16 array. The null character is not copied.

Useful for copying UTF16 strings on Windows.

Unsafe:

The caller has to ensure that:

1. the addr is pinned and alive during the call.
2. the pointer passed is valid up to the point where null Word16 is found.

fromPtrN len addr copies len bytes from addr into an array.

Unsafe:

The caller has to ensure that:

1. the pointer is pinned and alive during the call.
2. the pointer passed is valid up to the given length.

Convert an array stream to an array. Note that this requires peak memory that is double the size of the array stream.

Also see fromChunksRealloced.

Also see fromChunksK.

unsafeCreateWithPtr' capacity populator creates a pinned array of capacity bytes and invokes the populator function to populate it. populator ptr len gets the pointer to the array and MUST return the amount of the capacity populated in bytes.

Unsafe because the populator is allowed to use the pointer only up to specified length. In other words, bytes populated MUST be less than or equal to the total capacity.

O(1) Write the given element at the given index in the array. Performs in-place mutation of the array.

>>> putIndex ix arr val = MutArray.modifyIndex ix arr (const (val, ()))
>>> f = MutArray.putIndices
>>> putIndex ix arr val = Stream.fold (f arr) (Stream.fromPure (ix, val))

Write the given element to the given index of the array. Does not check if the index is out of bounds of the array.

Pre-release

Write an input stream of (index, value) pairs to an array. Throws an error if any index is out of bounds.

Pre-release

Modify a given index of an array using a modifier function.

Unsafe because it does not check the bounds of the array.

Pre-release

Modify a given index of an array using a modifier function.

Pre-release

Modify the array indices generated by the supplied stream.

Pre-release

Modify each element of an array using the supplied modifier function.

This is an in-place equivalent of an immutable map operation.

Pre-release

Swap the elements at two indices.

Pre-release

Swap the elements at two indices without validating the indices.

Unsafe: This could result in memory corruption if indices are not valid.

Pre-release

O(1) Lookup the element at the given index. Index starts from 0.

Return the element at the specified index without checking the bounds.

Unsafe because it does not check the bounds of the array.

O(1) Lookup the element at the given index from the end of the array. Index starts from 0.

Slightly faster than computing the forward index and using getIndex.

Given an unfold that generates array indices, read the elements on those indices from the supplied MutArray. An error is thrown if an index is out of bounds.

Pre-release

Convert a MutArray into a stream.

>>> read = Stream.unfold MutArray.reader

Convert a MutArray into a stream in reverse order.

>>> readRev = Stream.unfold MutArray.readerRev

Convert a MutArray into a list.

Resumable unfold of an array.

Unfold an array into a stream.

Unfold an array into a stream in reverse order.

O(1) Get the used length of the array i.e. the number of elements in the array.

Note that byteLength is less expensive than this operation, as length involves a costly division operation.

O(1) Get the byte length of the array.

Get the total capacity of an array. An array may have space reserved beyond the current used length of the array.

Pre-release

The remaining capacity in the array for appending more elements without reallocation.

Pre-release

The page or block size used by the GHC allocator. Allocator allocates at least a block and then allocates smaller allocations from within a block.

The default chunk size by which the array creation routines increase the size of the array when the array is grown linearly.

Given an Unboxed type (unused first arg) and real allocation size (including overhead), return how many elements of that type will completely fit in it, returns at least 1.

realloc newCapacity array reallocates the array to the specified capacity in bytes.

If the new size is less than the original array the array gets truncated. If the new size is not a multiple of array element size then it is rounded down to multiples of array size. If the new size is more than largeObjectThreshold then it is rounded up to the block size (4K).

If the original array is pinned, the newly allocated array is also pinned.

reallocBytesWith label capSizer minIncrBytes array. The label is used in error messages and the capSizer is used to determine the capacity of the new array in bytes given the current byte length of the array.

growTo newCapacity array changes the total capacity of the array so that it is enough to hold the specified number of elements. Nothing is done if the specified capacity is less than the length of the array.

If the capacity is more than largeObjectThreshold then it is rounded up to the block size (4K).

Nothing is done if the requested capacity is <= 0.

Pre-release

Like growTo but specifies the required reserve (unused) capacity rather than the total capacity. Increases the reserve capacity, if required, to at least the given amount.

Nothing is done if the requested capacity is <= 0.

Like growTo but if the requested byte capacity is more than largeObjectThreshold then it is rounded up to the closest power of 2.

Nothing is done if the requested capacity is <= 0.

Pre-release

Resize the allocated memory to drop any reserved free space at the end of the array and reallocate it to reduce wastage.

Up to 25% wastage is allowed to avoid reallocations. If the capacity is more than largeObjectThreshold then free space up to the blockSize is retained.

Pre-release

Reset the array end position to start, thus truncating the array to 0 length, making it empty. The capacity of the array remains unchanged. The array refers to the same memory as before.

Strict left fold of an array.

Right fold of an array.

Fold an array using a Fold.

For example:

>>> findIndex eq = MutArray.fold (Fold.findIndex eq)

Pre-release

Fold an arary starting from end up to beginning.

For example:

>>> findIndexRev eq = MutArray.foldRev (Fold.findIndex eq)

Byte compare two arrays. Compare the length of the arrays. If the length is equal, compare the lexicographical ordering of two underlying byte arrays otherwise return the result of length comparison.

Unsafe: Note that the Unbox instance of sum types with constructors of different sizes may leave some memory uninitialized which can make byte comparison unreliable.

Pre-release

Byte equality of two arrays.

>>> byteEq arr1 arr2 = (==) EQ <$> MutArray.byteCmp arr1 arr2

Unsafe: See byteCmp.

You may not need to reverse an array because you can consume it in reverse using readerRev. To reverse large arrays you can read in reverse and write to another array. However, in-place reverse can be useful to take adavantage of cache locality and when you do not want to allocate additional memory.

Generate the next permutation of the sequence, returns False if this is the last permutation.

Unimplemented

Partition an array into two halves using a partitioning predicate. The first half retains values where the predicate is False and the second half retains values where the predicate is True.

Pre-release

Shuffle corresponding elements from two arrays using a shuffle function. If the shuffle function returns False then do nothing otherwise swap the elements. This can be used in a bottom up fold to shuffle or reorder the elements.

Unimplemented

divideBy level partition array performs a top down hierarchical recursive partitioning fold of items in the container using the given function as the partition function. Level indicates the level in the tree where the fold would stop.

This performs a quick sort if the partition function is 'partitionBy (< pivot)'.

Unimplemented

mergeBy level merge array performs a pairwise bottom up fold recursively merging the pairs using the supplied merge function. Level indicates the level in the tree where the fold would stop.

This performs a random shuffle if the merge function is random. If we stop at level 0 and repeatedly apply the function then we can do a bubble sort.

Unimplemented

Given an array sorted in ascending order except the last element being out of order, use bubble sort to place the last element at the right place such that the array remains sorted in ascending order.

Pre-release

Find the minimum and maximum elements in the array using the provided comparison function.

snocWith sizer arr elem mutates arr to append elem. The used length of the array increases by 1.

If there is no reserved space available in arr it is reallocated to a size in bytes determined by the sizer oldSizeBytes function, where oldSizeBytes is the original size of the array in bytes. The sizer function should return a capacity more than or equal to the current used size. If the capacity returned is less than or equal to the current used size, the array is still grown by one element.

If the new array size is more than largeObjectThreshold then it is rounded up to blockSize.

Note that the returned array may be a mutated version of the original array.

Pre-release

The array is mutated to append an additional element to it. If there is no reserved space available in the array then it is reallocated to double the original size and aligned to a power of 2.

This is useful to reduce allocations when appending unknown number of elements.

Note that the returned array may be a mutated version of the original array.

Performs only O(n * log n) copies to grow, but is liberal with memory allocation compared to snocGrowBy.

The array is mutated to append an additional element to it.

If there is no reserved space available in the array then it is reallocated to grow it by adding space for the requested number of elements, the new size is rounded up to blockSize when the size becomes more than largeObjectThreshold. If the size specified is <= 0 then the array is grown by one element.

Note that the returned array may be a mutated version of the original array.

Performs O(n^2) copies to grow but is thrifty on memory compared to snoc.

Pre-release

Like snoc but does not reallocate when pre-allocated array capacity becomes full.

Internal

Really really unsafe, appends the element into the first array, may cause silent data corruption or if you are lucky a segfault if the first array does not have enough space to append the element.

Internal

appendWith sizer action mutates the array generated by action to append the input stream. If there is no reserved space available in the array it is reallocated to a size in bytes determined by sizer oldSize, where oldSize is the current size of the array in bytes. If the sizer returns less than or equal to the current size then the size is incremented by one element.

Note that the returned array may be a mutated version of original array.

>>> appendWith sizer = Fold.foldlM' (MutArray.snocWith sizer)

Pre-release

unsafeAppendMax n arr appends up to n input items to the supplied array.

Unsafe: Do not drive the fold beyond n elements, it will lead to memory corruption or segfault.

Internal

Allocates space for n additional elements. The fold terminates after appending n elements. If less than n elements are supplied then the space for the remaining elements is guaranteed to be reserved.

>>> appendMax n arr = Fold.take n (MutArray.unsafeAppendMax n arr)

Fold append2 arr mutates the array arr to append the input stream. If there is no reserved space available in the array it is reallocated to double the size and aligned to power of 2.

Note that the returned array may be a mutated version of original array.

>>> append2 arr = Fold.foldlM' MutArray.snoc (pure arr)

appendGrowBy arr mutates the array arr to append the input stream. If there is no reserved space available in the array it is reallocated to add space for the min number of elements supplied and align to block size if the array becomes larger than largeObjectThreshold.

Note that the returned array may be a mutated version of original array.

>>> appendGrowBy n arr = Fold.foldlM' (MutArray.snocGrowBy n) (pure arr)

Append specified number of bytes from a given pointer to the MutArray.

Unsafe:

The caller has to ensure that:

1. the MutArray is valid up to the given length.
2. the source pointer is pinned and alive during the call.
3. the pointer passed is valid up to the given length.

The array is grown only by the required amount of space.

>>> appendStream arr = Stream.fold (MutArray.append (pure arr))

>>> appendStreamN n arr = Stream.fold (MutArray.appendMax n arr)

Copy two arrays into a newly allocated array. If the first array is pinned the spliced array is also pinned.

Note: If you freeze and splice it will create a new array.

The first array is extended in-place to append the second array. If there is no reserved space available in the first array then a new allocation of exact required size is done.

Note that the returned array may be an extended version of first array, referring to the same memory as the original array.

>>> splice = MutArray.spliceWith (+)

If the original array is pinned the spliced array is also pinned.

Pre-release

spliceWith sizer dst src mutates dst to append src. If there is no reserved space available in dst it is reallocated to a size determined by the sizer dstBytes srcBytes function, where dstBytes is the size of the first array and srcBytes is the size of the second array, in bytes.

Note that the returned array may be a mutated version of first array.

Pre-release

Like append but the growth of the array is exponential. Whenever a new allocation is required the previous array size is at least doubled.

This is useful to reduce allocations when folding many arrays together.

Note that the returned array may be a mutated version of first array.

>>> spliceExp = MutArray.spliceWith (\l1 l2 -> max (l1 * 2) (l1 + l2))

Pre-release

Really really unsafe, appends the second array into the first array. If the first array does not have enough space it may cause silent data corruption or if you are lucky a segfault.

Unbox a Haskell type and append the resulting bytes to a mutable byte array. The array is grown exponentially when more space is needed.

Like snoc except that the value is unboxed to the byte array.

Note: If you are serializing a large number of small fields, and the types are statically known, then it may be more efficient to declare a record of those fields and derive an Unbox instance of the entire record.

Like poke but does not grow the array when pre-allocated array capacity becomes full.

Internal

Skip the specified number of bytes in the array. The data in the skipped region remains uninitialzed.

Create a Haskell value from its unboxed representation from the head of a byte array, return the value and the remaining array.

Like uncons except that the value is deserialized from the byte array.

Note: If you are deserializing a large number of small fields, and the types are statically known, then it may be more efficient to declare a record of those fields and derive an Unbox instance of the entire record.

Really really unsafe, create a Haskell value from an unboxed byte array, does not check if the array is big enough, may return garbage or if you are lucky may cause a segfault.

Internal

Discard the specified number of bytes at the beginning of the array.

chunksOf n stream groups the elements in the input stream into arrays of n elements each.

Same as the following but may be more efficient:

>>> chunksOf n = Stream.foldMany (MutArray.createOf n)

Pre-release

Like chunksOf but creates pinned arrays.

Buffer a stream into a stream of arrays.

>>> buildChunks n = Fold.many (MutArray.createOf n) Fold.toStreamK

Breaking an array into an array stream can be useful to consume a large array sequentially such that memory of the array is released incrementatlly.

See also: arrayStreamKFromStreamD.

Unimplemented

Create arrays from the input stream using a predicate to find the end of the chunk. When the predicate matches, the chunk ends, the matching element is included in the chunk.

Definition:

>>> chunksEndBy p = Stream.foldMany (Fold.takeEndBy p MutArray.create)

Like chunksEndBy but creates pinned arrays.

Create chunks using newline as the separator, including it.

Like chunksEndByLn but creates pinned arrays.

Generate a stream of array slices using a predicate. The array element matching the predicate is dropped.

Pre-release

Generate a stream of array slices using a predicate. The array element matching the predicate is included.

Pre-release

Use the "reader" unfold instead.

concat = unfoldMany reader

We can try this if there are any fusion issues in the unfold.

Use the "readerRev" unfold instead.

concat = unfoldMany readerRev

We can try this if there are any fusion issues in the unfold.

This mutates the first array (if it has space) to append values from the second one. This would work for immutable arrays as well because an immutable array never has additional space so a new array is allocated instead of mutating it.

Parser createCompactMax maxElems coalesces adjacent arrays in the input stream only if the combined size would be less than or equal to maxElems elements. Note that it won't split an array if the original array is already larger than maxElems.

maxElems must be greater than 0.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Note that a fold compacting to less than or equal to a given size is not possible, as folds cannot backtrack.

Internal

Pinned version of createCompactMax.

Fold createCompactMin minElems coalesces adjacent arrays in the input stream until the size becomes greater than or equal to minElems.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Pinned version of createCompactMin.

compactMin n stream coalesces adjacent arrays in the stream until the compacted array size becomes greater than or equal to n.

>>> compactMin n = Stream.foldMany (MutArray.createCompactMin n)

'compactExact n' coalesces adajacent arrays in the input stream to arrays of exact size n.

Unimplemented

Like compactMin but a scan.

Like compactMin' but a scan.

O(1) Slice an array in constant time.

Unsafe: The bounds of the slice are not checked.

Unsafe

Pre-release

O(1) Get a reference to a slice from a mutable array. Throws an error if the slice extends out of the array bounds.

The capacity of the slice is the same as its length i.e. it does not have any unused or reserved space at the end.

The slice shares the same underlying mutable array when created. However, if the slice or the original array is reallocated by growing or shrinking then it will be copied to new memory and they will no longer share the same memory.

Pre-release

Generate a stream of array slices using a predicate. The array element matching the predicate is dropped.

Pre-release

Strip elements which match the predicate, from both ends.

>>> arr <- MutArray.fromList "   hello world    "
>>> a <- MutArray.dropAround (== ' ') arr
>>> MutArray.toList a
"hello world"

Pre-release

Strip elements which match the predicate, from the start of the array.

>>> arr <- MutArray.fromList "    hello world"
>>> a <- MutArray.dropWhile (== ' ') arr
>>> MutArray.toList a
"hello world"

Pre-release

Strip elements which match the predicate, from the end of the array.

>>> arr <- MutArray.fromList "hello world    "
>>> a <- MutArray.revDropWhile (== ' ') arr
>>> MutArray.toList a
"hello world"

Pre-release

Drops the separator byte

Create two slices of an array without copying the original array. The specified index i is the first index of the second slice.

Like breakAt but does not check whether the index is valid.

>>> unsafeBreakAt i arr = (MutArray.unsafeSliceOffLen 0 i arr, MutArray.unsafeSliceOffLen i (MutArray.length arr - i) arr)

realloc newCapacity array reallocates the array to the specified capacity in bytes.

If the new size is less than the original array the array gets truncated. If the new size is not a multiple of array element size then it is rounded down to multiples of array size. If the new size is more than largeObjectThreshold then it is rounded up to the block size (4K).

If the original array is pinned, the newly allocated array is also pinned.

createWithOf alloc n folds a maximum of n elements into an array allocated using the alloc function.

The array capacity is guranteed to be at least n.

>>> createWithOf alloc n = Fold.take n (MutArray.unsafeCreateOfWith alloc n)
>>> createWithOf alloc n = MutArray.appendN (alloc n) n

Create a Haskell value from its unboxed representation from the head of a byte array, return the value and the remaining array.

Like uncons except that the value is deserialized from the byte array.

Note: If you are deserializing a large number of small fields, and the types are statically known, then it may be more efficient to declare a record of those fields and derive an Unbox instance of the entire record.

Really really unsafe, create a Haskell value from an unboxed byte array, does not check if the array is big enough, may return garbage or if you are lucky may cause a segfault.

Internal

Unbox a Haskell type and append the resulting bytes to a mutable byte array. The array is grown exponentially when more space is needed.

Like snoc except that the value is unboxed to the byte array.

Note: If you are serializing a large number of small fields, and the types are statically known, then it may be more efficient to declare a record of those fields and derive an Unbox instance of the entire record.

Like poke but does not grow the array when pre-allocated array capacity becomes full.

Internal

Cast an array having elements of type a into an array having elements of type b. The array size must be a multiple of the size of type b otherwise accessing the last element of the array may result into a crash or a random value.

Pre-release

emptyWithAligned allocator alignment count allocates a new array of zero length and with a capacity to hold count elements, using allocator size alignment as the memory allocator function.

Alignment must be greater than or equal to machine word size and a power of 2.

Alignment is ignored if the allocator allocates unpinned memory.

Pre-release

O(1) Slice an array in constant time.

Unsafe: The bounds of the slice are not checked.

Unsafe

Pre-release

Write the given element to the given index of the array. Does not check if the index is out of bounds of the array.

Pre-release

Modify a given index of an array using a modifier function.

Unsafe because it does not check the bounds of the array.

Pre-release

Return the element at the specified index without checking the bounds.

Unsafe because it does not check the bounds of the array.

Really really unsafe, appends the element into the first array, may cause silent data corruption or if you are lucky a segfault if the first array does not have enough space to append the element.

Internal

Really really unsafe, appends the second array into the first array. If the first array does not have enough space it may cause silent data corruption or if you are lucky a segfault.

Skip the specified number of bytes in the array. The data in the skipped region remains uninitialzed.

Discard the specified number of bytes at the beginning of the array.

We could take the approach of doubling the memory allocation on each overflow. This would result in more or less the same amount of copying as in the chunking approach. However, if we have to shrink in the end then it may result in an extra copy of the entire data.

>>> fromStreamD = StreamD.fold MutArray.create

Allocates a pinned empty array that with a reserved capacity of bytes. The memory of the array is uninitialized and the allocation is aligned as per the Unboxed instance of the type.

pinnedNewBytes = (unsafeCast :: Array Word8 -> a) . emptyOf'

Pre-release

pinnedWriteNAligned align n folds a maximum of n elements from the input stream to a MutArray aligned to the given size.

Pre-release

Parser createCompactMax maxElems coalesces adjacent arrays in the input stream only if the combined size would be less than or equal to maxElems elements. Note that it won't split an array if the original array is already larger than maxElems.

maxElems must be greater than 0.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Note that a fold compacting to less than or equal to a given size is not possible, as folds cannot backtrack.

Internal

Pinned version of createCompactMax.

Fold createCompactMin minElems coalesces adjacent arrays in the input stream until the size becomes greater than or equal to minElems.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Pinned version of createCompactMin.

Pinned version of lCompactGE.

Like compactGE but for transforming folds instead of stream.

> lCompactGE n = Fold.many (MutArray.fCompactGE n)

Generates unpinned arrays irrespective of the pinning status of input arrays.

compactMin n stream coalesces adjacent arrays in the stream until the compacted array size becomes greater than or equal to n.

>>> compactMin n = Stream.foldMany (MutArray.createCompactMin n)

Allocates a pinned array of zero length but growable to the specified capacity without reallocation.

Like chunksOf but creates pinned arrays.

Like createOf but creates a pinned array.

Like create but creates a pinned array.

Like fromListN but creates a pinned array.

Like fromList but creates a pinned array.

Like unsafeCreateOf but creates a pinned array.

Generate a stream of array slices using a predicate. The array element matching the predicate is dropped.

Pre-release

Like emptyWithAligned but using an allocator is a pinned memory allocator and the alignment is dictated by the Unboxed instance of the type.

Internal

NOTE: this is deprecated because it can lead to accidental problems if the user tries to use it to mutate the array because it does not return the new array after pinning.

growTo newCapacity array changes the total capacity of the array so that it is enough to hold the specified number of elements. Nothing is done if the specified capacity is less than the length of the array.

If the capacity is more than largeObjectThreshold then it is rounded up to the block size (4K).

Nothing is done if the requested capacity is <= 0.

Pre-release

createMinOf count folds the whole input to a single array. The array starts at a size big enough to hold minCount elements, the size is doubled every time the array needs to be grown.

The array capacity is guaranteed to be at least count.

Caution! Do not use this on infinite streams.

>>> f n = MutArray.appendWith (* 2) (MutArray.emptyOf n)
>>> createWith n = Fold.rmapM MutArray.rightSize (f n)
>>> createWith n = Fold.rmapM MutArray.fromChunksK (MutArray.buildChunks n)

Pre-release

The array is mutated to append an additional element to it. If there is no reserved space available in the array then it is reallocated to grow it by arrayChunkBytes rounded up to blockSize when the size becomes more than largeObjectThreshold.

Note that the returned array may be a mutated version of the original array.

Performs O(n^2) copies to grow but is thrifty on memory.

Pre-release

unsafeAppendN n arr appends up to n input items to the supplied array.

Unsafe: Do not drive the fold beyond n elements, it will lead to memory corruption or segfault.

Any free space left in the array after appending n elements is lost.

Internal

Append n elements to an existing array. Any free space left in the array after appending n elements is lost.

>>> appendN n initial = Fold.take n (MutArray.unsafeAppendN n initial)

append action mutates the array generated by action to append the input stream. If there is no reserved space available in the array it is reallocated to double the size and aligned to power of 2.

Note that the returned array may be a mutated version of original array.

>>> append = Fold.foldlM' MutArray.snoc

Generate a stream of array slice descriptors ((index, len)) of specified length from an array, starting from the supplied array index. The last slice may be shorter than the requested length depending on the array length.

Pre-release

Generate a stream of slices of specified length from an array, starting from the supplied array index. The last slice may be shorter than the requested length depending on the array length.

Pre-release

compactLE maxElems coalesces adjacent arrays in the input stream only if the combined size would be less than or equal to maxElems elements. Note that it won't split an array if the original array is already larger than maxElems.

maxElems must be greater than 0.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Like compactBySizeLE but generates pinned arrays.

Split a stream of arrays on a given separator byte, dropping the separator and coalescing all the arrays between two separators into a single array.

Split a stream of arrays on a given separator byte, dropping the separator and coalescing all the arrays between two separators into a single array.

Compact byte arrays on newline character, dropping the newline char.

createOfLast n folds a maximum of n elements from the end of the input stream to an MutArray.

Serialize the supplied Haskell value at the end of the mutable array, growing the array size. If there is no reserve capacity left in the array the array is reallocated to double the current size.

Like snoc except that the value is serialized to the byte array.

Note: If you are serializing a large number of small fields, and the types are statically known, then it may be more efficient to declare a record of those fields and derive an Serialize instance of the entire record.

Unstable API

Deserialize a Haskell value from the beginning of a mutable array. The deserialized value is removed from the array and the remaining array is returned.

Like uncons except that the value is deserialized from the byte array.

Note: If you are deserializing a large number of small fields, and the types are statically known, then it may be more efficient to declare a record of those fields and derive Serialize instance of the entire record.

Unstable API

Serializes a (Ptr, len) pair in the same way as an array. The serialized value can be de-serialized as an array or consumed as a pointer using deserializePtrN.

The Ptr must be pinned or the existence of the Ptr must be ensured by the user of this API.

Unimplemented

Consume a serialized array or (Ptr, length) from the MutArray using an IO action that consumes the pointer directly.

WARNING! The array must be a pinned array.

Unimplemented

Generate a stream of slices of specified length from an array, starting from the supplied array index. The last slice may be shorter than the requested length depending on the array length.

Pre-release

Generate a stream of array slice descriptors ((index, len)) of specified length from an array, starting from the supplied array index. The last slice may be shorter than the requested length depending on the array length.

Pre-release

compactLE maxElems coalesces adjacent arrays in the input stream only if the combined size would be less than or equal to maxElems elements. Note that it won't split an array if the original array is already larger than maxElems.

maxElems must be greater than 0.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Like compactBySizeLE but generates pinned arrays.

Split a stream of arrays on a given separator byte, dropping the separator and coalescing all the arrays between two separators into a single array.

Split a stream of arrays on a given separator byte, dropping the separator and coalescing all the arrays between two separators into a single array.