Streamly.Internal.Data.Array

Makes an immutable array using the underlying memory of the mutable array.

Please make sure that there are no other references to the mutable array lying around, so that it is never used after freezing it using unsafeFreeze. If the underlying array is mutated, the immutable promise is lost.

Pre-release

Similar to unsafeFreeze but uses rightSize on the mutable array first.

Makes a mutable array using the underlying memory of the immutable array.

Please make sure that there are no other references to the immutable array lying around, so that it is never used after thawing it using unsafeThaw. If the resulting array is mutated, any references to the older immutable array are mutated as well.

Pre-release

Return a copy of the Array in pinned memory if unpinned, else return the original array.

Return a copy of the Array in unpinned memory if pinned, else return the original array.

Return True if the array is allocated in pinned memory.

Use an Array a as Ptr a.

See unsafePinnedAsPtr in the Mutable array module for more details.

Unsafe

1. The accessor must not access the array beyond the specified length.
2. The accessor must not mutate the array.

Pre-release

Use an Array a as ForeignPtr a.

Unsafe because of direct pointer operations. The user must ensure that they are writing within the legal bounds of the array.

Pre-release

O(1) Slice an array in constant time.

Caution: The bounds of the slice are not checked.

Unsafe

Pre-release

O(1) Lookup the element at the given index. Index starts from 0.

Like getIndex but indexes the array in reverse from the end.

Pre-release

Like breakAt but does not check whether the index is valid.

Create two slices of an array without copying the original array. The specified index i is the first index of the second slice.

Strip elements which match the predicate, from both ends.

Pre-release

Strip elements which match the predicate, from the start of the array.

Pre-release

Strip elements which match the predicate, from the end of the array.

Pre-release

Fold "step" has a dependency on "initial", and each step is dependent on the previous invocation of step due to state passing, finally extract depends on the result of step, therefore, as long as the fold is driven in the correct order the operations would be correctly ordered. We need to ensure that we strictly evaluate the previous step completely before the next step.

To not share the same array we need to make sure that the result of "initial" is not shared. Existential type ensures that it does not get shared across different folds. However, if we invoke "initial" multiple times for the same fold, there is a possiblity of sharing the two because the compiler would consider it as a pure value. One such example is the chunksOf combinator, or using an array creation fold with foldMany combinator. Is there a proper way in GHC to tell it to not share a pure expression in a particular case?

For this reason array creation folds have a MonadIO constraint. Pure folds could be unsafe and dangerous. This is dangerous especially when used with foldMany like operations.

>>> unsafePureWrite = Array.unsafeMakePure Array.create

createOf n folds a maximum of n elements from the input stream to an Array.

Like createOf but creates a pinned array.

Like createOf but does not check the array bounds when writing. The fold driver must not call the step function more than n times otherwise it will corrupt the memory and crash. This function exists mainly because any conditional in the step function blocks fusion causing 10x performance slowdown.

Fold the whole input to a single array.

Caution! Do not use this on infinite streams.

Like create but creates a pinned array.

A version of "create" that let's you pass in the initial capacity of the array in terms of the number of elements.

Semantically createWith 10 and createWith 100 will behave in the same way. createWith 100 will be more performant though.

create = createWith elementCount

Pre-release

Create an Array from the first N elements of a list. The array is allocated to size N, if the list terminates before N elements then the array may hold less than N elements.

Like fromListN but creates a pinned array.

Create an Array from a list. The list must be of finite size.

Like fromList but creates a pinned array.

Create an Array from the first N elements of a list in reverse order. The array is allocated to size N, if the list terminates before N elements then the array may hold less than N elements.

Pre-release

Create an Array from a list in reverse order. The list must be of finite size.

Pre-release

Create an Array from the first N elements of a stream. The array is allocated to size N, if the stream terminates before N elements then the array may hold less than N elements.

>>> fromStreamN n = Stream.fold (Array.createOf n)

Pre-release

Create an Array from a stream. This is useful when we want to create a single array from a stream of unknown size. writeN is at least twice as efficient when the size is already known.

>>> fromStream = Stream.fold Array.create

Note that if the input stream is too large memory allocation for the array may fail. When the stream size is not known, chunksOf followed by processing of indvidual arrays in the resulting stream should be preferred.

Pre-release

Convert a pure stream in Identity monad to an immutable array.

Same as the following but with better performance:

>>> fromPureStream = Array.fromList . runIdentity . Stream.toList

Copy a null terminated immutable Addr# Word8 sequence into an array.

Unsafe: The caller is responsible for safe addressing.

Note that this is completely safe when reading from Haskell string literals because they are guaranteed to be NULL terminated:

Note, you can use lazy unsafePerformIO _only if_ the pointer is immutable.

>>> Array.toList $ unsafePerformIO $ Array.fromCString# "\1\2\3\0"#
[1,2,3]

Copy a C string consisting of 16-bit wide chars and terminated by a 16-bit null char, into a Word16 array. The null character is not copied.

Useful for copying UTF16 strings on Windows.

Copy a C string consisting of 16-bit wide chars and terminated by a 16-bit null char, into a Word16 array. The null character is not copied.

Useful for copying UTF16 strings on Windows.

fromPtrN len addr copies len bytes from addr into an array. The memory pointed by addr must be pinned or static.

Unsafe: The caller is responsible to ensure that the pointer passed is valid up to the given length.

Given a stream of arrays, splice them all together to generate a single array. The stream must be finite.

Convert an array stream to an array. Note that this requires peak memory that is double the size of the array stream.

unsafeFromForeignPtr fptr len converts the ForeignPtr to an Array.

Return element at the specified index without checking the bounds.

Unsafe because it does not check the bounds of the array.

Return element at the specified index without checking the bounds.

Convert an Array into a stream.

Pre-release

Convert an Array into a stream in reverse order.

Pre-release

Convert an Array into a list.

Unfold an array into a stream, does not check the end of the array, the user is responsible for terminating the stream within the array bounds. For high performance application where the end condition can be determined by a terminating fold.

Written in the hope that it may be faster than "read", however, in the case for which this was written, "read" proves to be faster even though the core generated with unsafeRead looks simpler.

Pre-release

Unfold an array into a stream.

Unfold an array into a stream in reverse order.

>>> null arr = Array.byteLength arr == 0

Note that this may be faster than checking Array.length as length calculation involves a division operation.

Pre-release

O(1) Get the length of the array i.e. the number of elements in the array.

O(1) Get the byte length of the array.

Byte compare two arrays. Compare the length of the arrays. If the length is equal, compare the lexicographical ordering of two underlying byte arrays otherwise return the result of length comparison.

Unsafe: Note that the Unbox instance of sum types with constructors of different sizes may leave some memory uninitialized which can make byte comparison unreliable.

Pre-release

Byte equality of two arrays.

>>> byteEq arr1 arr2 = (==) EQ $ Array.byteCmp arr1 arr2

Unsafe: See byteCmp.

Compare an array with a list.

Check equality of an array with a list.

Copy two immutable arrays into a new array. If you want to splice more than two arrays then this operation would be highly inefficient because it would make a copy on every splice operation, instead use the fromChunksK operation to combine n immutable arrays.

chunksOf n stream groups the elements in the input stream into arrays of n elements each.

Same as the following but may be more efficient:

>>> chunksOf n = Stream.foldMany (Array.createOf n)

Pre-release

Like chunksOf but creates pinned arrays.

Create arrays from the input stream using a predicate to find the end of the chunk. When the predicate matches, the chunk ends, the matching element is included in the chunk.

Definition:

>>> chunksEndBy p = Stream.foldMany (Fold.takeEndBy p Array.create)

Like chunksEndBy but creates pinned arrays.

Create chunks using newline as the separator, including it.

Like chunksEndByLn but creates pinned arrays.

Split the array into a stream of slices using a predicate. The element matching the predicate is dropped.

Pre-release

Convert a stream of arrays into a stream of their elements.

>>> concat = Stream.unfoldEach Array.reader

Convert a stream of arrays into a stream of their elements reversing the contents of each array before flattening.

>>> concatRev = Stream.unfoldEach Array.readerRev

Fold createCompactMin n coalesces adjacent arrays in the input stream until the size becomes greater than or equal to n.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Pinned version of createCompactMin.

compactBySize n stream coalesces adjacent arrays in the stream until the size becomes greater than or equal to n.

>>> compactBySize n = Stream.foldMany (Array.createCompactMin n)

Generates unpinned arrays irrespective of the pinning status of input arrays.

O(1) Slice an array in constant time.

Caution: The bounds of the slice are not checked.

Unsafe

Pre-release

Strip elements which match the predicate, from both ends.

Pre-release

Strip elements which match the predicate, from the start of the array.

Pre-release

Strip elements which match the predicate, from the end of the array.

Pre-release

Create two slices of an array without copying the original array. The specified index i is the first index of the second slice.

Like breakAt but does not check whether the index is valid.

Same as read

Same as readRev

pinnedWriteNAligned alignment n folds a maximum of n elements from the input stream to an Array aligned to the given size.

Pre-release

Fold createCompactMin n coalesces adjacent arrays in the input stream until the size becomes greater than or equal to n.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Pinned version of createCompactMin.

Like compactBySizeGE but for transforming folds instead of stream.

>>> lCompactBySizeGE n = Fold.many (Array.createCompactMin n)

Generates unpinned arrays irrespective of the pinning status of input arrays.

Pinned version of lCompactGE.

compactBySize n stream coalesces adjacent arrays in the stream until the size becomes greater than or equal to n.

>>> compactBySize n = Stream.foldMany (Array.createCompactMin n)

Generates unpinned arrays irrespective of the pinning status of input arrays.

Like createOf but creates a pinned array.

Like create but creates a pinned array.

Like fromListN but creates a pinned array.

Like fromList but creates a pinned array.

Like chunksOf but creates pinned arrays.

Return element at the specified index without checking the bounds.

Unsafe because it does not check the bounds of the array.

Return element at the specified index without checking the bounds.

Unfold an array into a stream, does not check the end of the array, the user is responsible for terminating the stream within the array bounds. For high performance application where the end condition can be determined by a terminating fold.

Written in the hope that it may be faster than "read", however, in the case for which this was written, "read" proves to be faster even though the core generated with unsafeRead looks simpler.

Pre-release

createOfLast n folds a maximum of n elements from the end of the input stream to an Array.

Given a stream of array indices, read the elements on those indices from the supplied Array. An exception is thrown if an index is out of bounds.

This is the most general operation. We can implement other operations in terms of this:

read =
     let u = lmap (arr -> (0, length arr - 1)) Unfold.enumerateFromTo
      in Unfold.lmap f (indexReader arr)

readRev =
     let i = length arr - 1
      in Unfold.lmap f (indexReaderFromThenTo i (i - 1) 0)

Pre-release

Unfolds (from, then, to, array) generating a finite stream whose first element is the array value from the index from and the successive elements are from the indices in increments of then up to to. Index enumeration can occur downwards or upwards depending on whether then comes before or after from.

getIndicesFromThenTo =
    let f (from, next, to, arr) =
            (Stream.enumerateFromThenTo from next to, arr)
     in Unfold.lmap f getIndices

Unimplemented

Given a sorted array, perform a binary search to find the given element. Returns the index of the element if found.

Unimplemented

Unimplemented

Perform a linear search to find all the indices where a given element is present in an array.

Unimplemented

Cast an array having elements of type a into an array having elements of type b. The length of the array should be a multiple of the size of the target element otherwise Nothing is returned.

Cast an Array a into an Array Word8.

Cast an array having elements of type a into an array having elements of type b. The array size must be a multiple of the size of type b otherwise accessing the last element of the array may result into a crash or a random value.

Pre-release

Convert an array of any type into a null terminated CString Ptr. If the array is unpinned it is first converted to a pinned array which requires a copy.

Unsafe

O(n) Time: (creates a copy of the array)

Pre-release

Generate a stream of slices of specified length from an array, starting from the supplied array index. The last slice may be shorter than the requested length.

Pre-release/

Transform an array into another array using a stream transformation operation.

Pre-release

Fold an array using a stream fold operation.

Pre-release

Fold an array using a Fold.

Pre-release

Insert the given element between arrays and flatten.

>>> concatSepBy x = Stream.unfoldEachSepBy x Array.reader

Insert the given element after each array and flatten. This is similar to unlines.

>>> concatEndBy x = Stream.unfoldEachEndBy x Array.reader

Insert the given array after each array and flatten.

>>> concatEndBySeq x = Stream.unfoldEachEndBySeq x Array.reader

compactMax n coalesces adjacent arrays in the input stream only if the combined size would be less than or equal to n.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Like compactMax but generates pinned arrays.

Split a stream of byte arrays on a given separator byte, dropping the separator and coalescing all the arrays between two separators into a single array.

Like compactSepByByte_, but considers the separator in suffix position instead of infix position.

Compact byte arrays on newline character, dropping the newline char.

Fold a stream of arrays using a Fold. This is equivalent to the following:

>>> foldChunks f = Stream.fold f . Stream.unfoldEach Array.reader

Fold a stream of arrays using a Fold and return the remaining stream.

The following alternative to this function allows composing the fold using the parser Monad:

foldBreakStreamK f s =
      fmap (first (fromRight undefined))
    $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s

We can compare perf and remove this one or define it in terms of that.

Parse an array stream using the supplied Parser. Returns the parse result and the unconsumed stream. Throws ParseError if the parse fails.

The following alternative to this function allows composing the parser using the parser Monad:

>>> parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)

We can compare perf and remove this one or define it in terms of that.

Internal

Properties: 1. Identity: deserialize . serialize == id 2. Encoded equivalence: serialize a == serialize a

Serialize a Haskell type to a pinned byte array. The array is allocated using pinned memory so that it can be used directly in OS APIs for writing to file or sending over the network.

Properties:

1. Identity: deserialize . serialize' == id
2. Encoded equivalence: serialize' a == serialize' a

Decode a Haskell type from a byte array containing its serialized representation.

Generate a stream of slices of specified length from an array, starting from the supplied array index. The last slice may be shorter than the requested length.

Pre-release/

Cast an array having elements of type a into an array having elements of type b. The array size must be a multiple of the size of type b otherwise accessing the last element of the array may result into a crash or a random value.

Pre-release

Serialize a Haskell type to a pinned byte array. The array is allocated using pinned memory so that it can be used directly in OS APIs for writing to file or sending over the network.

Properties:

1. Identity: deserialize . serialize' == id
2. Encoded equivalence: serialize' a == serialize' a

Insert the given element between arrays and flatten.

>>> concatSepBy x = Stream.unfoldEachSepBy x Array.reader

Insert the given element after each array and flatten. This is similar to unlines.

>>> concatEndBy x = Stream.unfoldEachEndBy x Array.reader

Insert the given array after each array and flatten.

>>> concatEndBySeq x = Stream.unfoldEachEndBySeq x Array.reader

compactMax n coalesces adjacent arrays in the input stream only if the combined size would be less than or equal to n.

Generates unpinned arrays irrespective of the pinning status of input arrays.

Like compactMax but generates pinned arrays.

Split a stream of byte arrays on a given separator byte, dropping the separator and coalescing all the arrays between two separators into a single array.

Like compactSepByByte_, but considers the separator in suffix position instead of infix position.

Fold an array using a Fold.

Pre-release