Build Guide
Building
Compiler (GHC) Versions
GHC 8.6 and above are recommended. For best performance use GHC 8.8 or
8.10 along with fusion-plugin
(see below). Benchmarks show that GHC
8.8 has significantly better performance than GHC 8.6 in many cases.
GHC 9.0 and GHC 9.2.1 have some performance issues, please see this issue for details. GHC 9.2.2 is expected to have the fixes that will bring the performance on par with previous versions.
Distributions
Tested with stackage lts-18.27
and nix 21.05
.
Memory requirements
Building streamly itself may require upto 4GB memory. Depending on the size of the application you may require 1-16GB memory to build. For most applications up to 8GB of memory should be sufficient.
To reduce the memory footprint you may want to break big modules into
smaller ones and reduce unnecessary inlining on large functions. You can
also use the -Rghc-timing
GHC option to report the memory usage during
compilation.
See the “Build times and space considerations” section below for more details.
Compilation Options
Recommended Options
Add fusion-plugin
to the build-depends
section of your program in
the cabal file and use the following GHC options:
-O2
-fdicts-strict
-fmax-worker-args=16
-fspec-constr-recursive=16
-fplugin Fusion.Plugin
Important Notes:
- fusion-plugin can
improve performance significantly by better stream fusion, many
cases. If the perform regresses due to fusion-plugin please open
an issue. You may remove the
-fplugin
option for regular builds but it is recommended for deployment builds and performance benchmarking. Note, for GHC 8.4 or lower fusion-plugin cannot be used. - In certain cases it is possible that GHC takes too long to compile
with
-fspec-constr-recursive=16
, if that happens please reduce the value or remove that option. - At the very least
-O -fdicts-strict
compilation options are absolutely required to avoid issues in some cases. For example, the programmain = S.drain $ S.concatMap S.fromList $ S.repeat []
may hog memory without these options.
See Explanation for details about these flags.
Explanation
-fdicts-strict
is needed to avoid a GHC issue leading to memory leak in some cases.-fspec-constr-recursive
is needed for better stream fusion by enabling theSpecConstr
optimization in more cases. Large values used with this flag may lead to huge compilation times and code bloat, if that happens please avoid it or use a lower value (e.g. 3 or 4).-fmax-worker-args
is needed for better stream fusion by enabling theSpecConstr
optimization in some important cases.-fplugin=Fusion.Plugin
enables predictable stream fusion optimization in certain cases by helping the compiler inline internal bindings and therefore enabling case-of-case optimization. In some cases, especially in some file IO benchmarks, it can make a difference of 5-10x better performance.
Multi-core Parallelism
Concurrency without a threaded runtime may be a bit more efficient. Do not use threaded runtime unless you really need multi-core parallelism. To get multi-core parallelism use the following GHC options:
-threaded -with-rtsopts "-N"
Platform Specific Features
Streamly supports Linux, macOS and Windows operating systems. Some modules and functionality may depend on specific OS kernel features. Features/modules may get disabled if the kernel/OS does not support it.
Linux
File system events notification module is supported only for kernel versions 2.6.36 onwards.
macOS
File system events notification module supports macOS 10.7+ . You must
have the Cocoa
framework installed which is supplied by the macOS
SDK. If Cocoa
is not installed, you may see an error like this:
error: ld: framework not found Cocoa
Native build
Usually, if you have a working GHC you would already have the SDK
installed. See the documentation of Xcode
or xcode-select
tool for
more details.
Nix build
Please note that cabal2nix may not always be able to generate a complete nix
expression on macOS
. See this
issue.
You may need to add nixpkgs.darwin.apple_sdk.frameworks.Cocoa
to
your buildInputs
or executableFrameworkDepends
. Something like
this:
executableFrameworkDepends =
if builtins.currentSystem == "x86_64-darwin"
then [nixpkgs.darwin.apple_sdk.frameworks.Cocoa]
else [];