Don't default to `OPENBLAS_NUM_THREADS=1` in notebooks #233

ajinkya-k · 2025-01-20T14:28:12Z

Hello!

engine: julia produces different output than REPL and also jupyter: julia-1.11. Also tagging @dmbates (who is the dev for MixedModels.jl) here who also had the same issue on his machine.

See example with screenshot below.

MWE:

---
title: "Evaluation with `engine:julia`"
format: html
engine: julia # or `jupyter: julia-1.11`
---

```{julia}
using DataFrames, MixedModels
dat = DataFrame(MixedModels.dataset("InstEval"));
```

```{julia}
m1 = fit(
  MixedModel,
  @formula(y ~ 1 + service + (1 | s) + (1 | d) + (1 + service | dept)),
  dat,
  progress=false    # suppress the display of a progress bar
)
print(m1)
```

REPL output:

Linear mixed model fit by maximum likelihood
 y ~ 1 + service + (1 | s) + (1 | d) + (1 + service | dept)
    logLik     -2 logLik       AIC         AICc          BIC     
 -118823.5292  237647.0584  237663.0584  237663.0603  237736.6901

Variance components:
            Column    Variance  Std.Dev.   Corr.
s        (Intercept)  0.1052731 0.3244581
d        (Intercept)  0.2624317 0.5122809
dept     (Intercept)  0.0029301 0.0541304
         service: Y   0.0259415 0.1610636 -0.57
Residual              1.3849984 1.1768596
 Number of obs: 73421; levels of grouping factors: 2972, 1128, 14

  Fixed-effects parameters:
─────────────────────────────────────────────────────
                  Coef.  Std. Error       z  Pr(>|z|)
─────────────────────────────────────────────────────
(Intercept)   3.2785      0.0240519  136.31    <1e-99
service: Y   -0.0496954   0.0460234   -1.08    0.2802
─────────────────────────────────────────────────────

jupyter: julia-1.11 and engine: jupyter output:

The text was updated successfully, but these errors were encountered:

jkrumbiegel · 2025-01-20T14:43:13Z

If you copy the cell like five times, do all results in jupyter not have a NaN and all in julia engine have it? Does setting a random seed make a difference?

MichaelHatherly · 2025-01-20T14:56:25Z

I'm seeing pretty consistent results between QNR and the REPL for that code. No NaNs showing up so far.

@ajinkya-k Can we confirm that the Manifest.toml's are identical and that the exact same version of julia is being used in all cases?

ajinkya-k · 2025-01-20T15:28:34Z

@jkrumbiegel There are no RNG calls in MixedModels fitting so that should not matter, but I did try setting seed just before running fit, yet the problem persists. Also, copying the chunk multiple times produces the same output as before.

dmbates · 2025-01-20T15:30:50Z

Fitting this model is a delicate optimization problem. Even slight differences in the floating point evaluations or order of operations in the numerical linear algebra could result in convergence to different local optima.

We are most interested in being able to configure QNR so that we reproduce the environment of the Jupyter evaluation. We have

engine: julia
julia: 
  exeflags: 
    - -tauto
    - --project

in the header of the document from which this example was taken. For me, the Jupyter kernel configuration julia-1.11 is

{
  "display_name": "julia 1.11.2",
  "argv": [
    "/Users/dmbates/.julia/juliaup/julia-1.11.2+0.aarch64.apple.darwin14/bin/julia",
    "-i",
    "--color=yes",
    "-tauto",
    "--project",
    "/Users/dmbates/.julia/packages/IJulia/dR0lE/src/kernel.jl",
    "{connection_file}"
  ],
  "language": "julia",
  "env": {},
  "interrupt_mode": "signal"
}

I would hope that the repetition of the -tauto and --project flags would result in a consistent environment. Can you think of anything else we could do to exert control over the Julia environment? The local Project.toml file just lists MixedModels in the [deps] section.

ajinkya-k · 2025-01-20T15:35:27Z

I can confirm that the same packages are being used:

and here is the Manifest.toml:

`Manifest.toml`

# This file is machine-generated - editing it directly is not advised

julia_version = "1.11.2"
manifest_format = "2.0"
project_hash = "ad6b4c59c244f9fb905367d224b1084c64023323"

[[deps.Adapt]]
deps = ["LinearAlgebra", "Requires"]
git-tree-sha1 = "50c3c56a52972d78e8be9fd135bfb91c9574c140"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "4.1.1"
weakdeps = ["StaticArrays"]

    [deps.Adapt.extensions]
    AdaptStaticArraysExt = "StaticArrays"

[[deps.AliasTables]]
deps = ["PtrArrays", "Random"]
git-tree-sha1 = "9876e1e164b144ca45e9e3198d0b689cadfed9ff"
uuid = "66dad0bd-aa9a-41b7-9441-69ab47430ed8"
version = "1.1.3"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
version = "1.1.2"

[[deps.ArrayLayouts]]
deps = ["FillArrays", "LinearAlgebra"]
git-tree-sha1 = "2bf6e01f453284cb61c312836b4680331ddfc44b"
uuid = "4c555306-a7a7-4459-81d9-ec55ddd5c99a"
version = "1.11.0"
weakdeps = ["SparseArrays"]

    [deps.ArrayLayouts.extensions]
    ArrayLayoutsSparseArraysExt = "SparseArrays"

[[deps.Arrow]]
deps = ["ArrowTypes", "BitIntegers", "CodecLz4", "CodecZstd", "ConcurrentUtilities", "DataAPI", "Dates", "EnumX", "Mmap", "PooledArrays", "SentinelArrays", "StringViews", "Tables", "TimeZones", "TranscodingStreams", "UUIDs"]
git-tree-sha1 = "00f0b3f05bc33cc5b68db6cc22e4a7b16b65e505"
uuid = "69666777-d1a9-59fb-9406-91d4454c9d45"
version = "2.8.0"

[[deps.ArrowTypes]]
deps = ["Sockets", "UUIDs"]
git-tree-sha1 = "404265cd8128a2515a81d5eae16de90fdef05101"
uuid = "31f734f8-188a-4ce0-8406-c8a06bd891cd"
version = "2.3.0"

[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"
version = "1.11.0"

[[deps.BSplineKit]]
deps = ["ArrayLayouts", "BandedMatrices", "FastGaussQuadrature", "ForwardDiff", "LinearAlgebra", "PrecompileTools", "Random", "Reexport", "SparseArrays", "Static", "StaticArrays", "StaticArraysCore"]
git-tree-sha1 = "15ab25b14c48783b1b73f80b14883fce7050daea"
uuid = "093aae92-e908-43d7-9660-e50ee39d5a0a"
version = "0.17.7"

[[deps.BandedMatrices]]
deps = ["ArrayLayouts", "FillArrays", "LinearAlgebra", "PrecompileTools"]
git-tree-sha1 = "fc8d8197de6c69ad6fd01c255b6b386ca8199331"
uuid = "aae01518-5342-5314-be14-df237901396f"
version = "1.9.0"
weakdeps = ["SparseArrays"]

    [deps.BandedMatrices.extensions]
    BandedMatricesSparseArraysExt = "SparseArrays"

[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
version = "1.11.0"

[[deps.BenchmarkTools]]
deps = ["Compat", "JSON", "Logging", "Printf", "Profile", "Statistics", "UUIDs"]
git-tree-sha1 = "e38fbc49a620f5d0b660d7f543db1009fe0f8336"
uuid = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
version = "1.6.0"

[[deps.BitIntegers]]
deps = ["Random"]
git-tree-sha1 = "6158239ac409f960abbc232a9b24c00f5cce3108"
uuid = "c3b6d118-76ef-56ca-8cc7-ebb389d030a1"
version = "0.3.2"

[[deps.CEnum]]
git-tree-sha1 = "389ad5c84de1ae7cf0e28e381131c98ea87d54fc"
uuid = "fa961155-64e5-5f13-b03f-caf6b980ea82"
version = "0.5.0"

[[deps.Chairmarks]]
deps = ["Printf", "Random"]
git-tree-sha1 = "9a49491e67e7a4d6f885c43d00bb101e6e5a434b"
uuid = "0ca39b1e-fe0b-4e98-acfc-b1656634c4de"
version = "1.3.1"
weakdeps = ["Statistics"]

    [deps.Chairmarks.extensions]
    StatisticsChairmarksExt = ["Statistics"]

[[deps.CodecLz4]]
deps = ["Lz4_jll", "TranscodingStreams"]
git-tree-sha1 = "0db0c70ca94c0a79cadad269497f25ca88b9fa91"
uuid = "5ba52731-8f18-5e0d-9241-30f10d1ec561"
version = "0.4.5"

[[deps.CodecZstd]]
deps = ["TranscodingStreams", "Zstd_jll"]
git-tree-sha1 = "d0073f473757f0d39ac9707f1eb03b431573cbd8"
uuid = "6b39b394-51ab-5f42-8807-6242bab2b4c2"
version = "0.8.6"

[[deps.CommonSubexpressions]]
deps = ["MacroTools"]
git-tree-sha1 = "cda2cfaebb4be89c9084adaca7dd7333369715c5"
uuid = "bbf7d656-a473-5ed7-a52c-81e309532950"
version = "0.3.1"

[[deps.CommonWorldInvalidations]]
git-tree-sha1 = "ae52d1c52048455e85a387fbee9be553ec2b68d0"
uuid = "f70d9fcc-98c5-4d4a-abd7-e4cdeebd8ca8"
version = "1.0.0"

[[deps.Compat]]
deps = ["TOML", "UUIDs"]
git-tree-sha1 = "8ae8d32e09f0dcf42a36b90d4e17f5dd2e4c4215"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "4.16.0"
weakdeps = ["Dates", "LinearAlgebra"]

    [deps.Compat.extensions]
    CompatLinearAlgebraExt = "LinearAlgebra"

[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"
version = "1.1.1+0"

[[deps.ConcurrentUtilities]]
deps = ["Serialization", "Sockets"]
git-tree-sha1 = "f36e5e8fdffcb5646ea5da81495a5a7566005127"
uuid = "f0e56b4a-5159-44fe-b623-3e5288b988bb"
version = "2.4.3"

[[deps.Crayons]]
git-tree-sha1 = "249fe38abf76d48563e2f4556bebd215aa317e15"
uuid = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f"
version = "4.1.1"

[[deps.DataAPI]]
git-tree-sha1 = "abe83f3a2f1b857aac70ef8b269080af17764bbe"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.16.0"

[[deps.DataFrames]]
deps = ["Compat", "DataAPI", "DataStructures", "Future", "InlineStrings", "InvertedIndices", "IteratorInterfaceExtensions", "LinearAlgebra", "Markdown", "Missings", "PooledArrays", "PrecompileTools", "PrettyTables", "Printf", "Random", "Reexport", "SentinelArrays", "SortingAlgorithms", "Statistics", "TableTraits", "Tables", "Unicode"]
git-tree-sha1 = "fb61b4812c49343d7ef0b533ba982c46021938a6"
uuid = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
version = "1.7.0"

[[deps.DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "1d0a14036acb104d9e89698bd408f63ab58cdc82"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.20"

[[deps.DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"
version = "1.11.0"

[[deps.Dictionaries]]
deps = ["Indexing", "Random", "Serialization"]
git-tree-sha1 = "61ab242274c0d44412d8eab38942a49aa46de9d0"
uuid = "85a47980-9c8c-11e8-2b9f-f7ca1fa99fb4"
version = "0.4.3"

[[deps.DiffResults]]
deps = ["StaticArraysCore"]
git-tree-sha1 = "782dd5f4561f5d267313f23853baaaa4c52ea621"
uuid = "163ba53b-c6d8-5494-b064-1a9d43ac40c5"
version = "1.1.0"

[[deps.DiffRules]]
deps = ["IrrationalConstants", "LogExpFunctions", "NaNMath", "Random", "SpecialFunctions"]
git-tree-sha1 = "23163d55f885173722d1e4cf0f6110cdbaf7e272"
uuid = "b552c78f-8df3-52c6-915a-8e097449b14b"
version = "1.15.1"

[[deps.Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"
version = "1.11.0"

[[deps.Distributions]]
deps = ["AliasTables", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns"]
git-tree-sha1 = "03aa5d44647eaec98e1920635cdfed5d5560a8b9"
uuid = "31c24e10-a181-5473-b8eb-7969acd0382f"
version = "0.25.117"

    [deps.Distributions.extensions]
    DistributionsChainRulesCoreExt = "ChainRulesCore"
    DistributionsDensityInterfaceExt = "DensityInterface"
    DistributionsTestExt = "Test"

    [deps.Distributions.weakdeps]
    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
    DensityInterface = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
    Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[deps.DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "2fb1e02f2b635d0845df5d7c167fec4dd739b00d"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.9.3"

[[deps.Downloads]]
deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
version = "1.6.0"

[[deps.EnumX]]
git-tree-sha1 = "bdb1942cd4c45e3c678fd11569d5cccd80976237"
uuid = "4e289a0a-7415-4d19-859d-a7e5c4648b56"
version = "1.0.4"

[[deps.ExprTools]]
git-tree-sha1 = "27415f162e6028e81c72b82ef756bf321213b6ec"
uuid = "e2ba6199-217a-4e67-a87a-7c52f15ade04"
version = "0.1.10"

[[deps.FastGaussQuadrature]]
deps = ["LinearAlgebra", "SpecialFunctions", "StaticArrays"]
git-tree-sha1 = "fd923962364b645f3719855c88f7074413a6ad92"
uuid = "442a2c76-b920-505d-bb47-c5924d526838"
version = "1.0.2"

[[deps.FileWatching]]
uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee"
version = "1.11.0"

[[deps.FillArrays]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "6a70198746448456524cb442b8af316927ff3e1a"
uuid = "1a297f60-69ca-5386-bcde-b61e274b549b"
version = "1.13.0"
weakdeps = ["PDMats", "SparseArrays", "Statistics"]

    [deps.FillArrays.extensions]
    FillArraysPDMatsExt = "PDMats"
    FillArraysSparseArraysExt = "SparseArrays"
    FillArraysStatisticsExt = "Statistics"

[[deps.ForwardDiff]]
deps = ["CommonSubexpressions", "DiffResults", "DiffRules", "LinearAlgebra", "LogExpFunctions", "NaNMath", "Preferences", "Printf", "Random", "SpecialFunctions"]
git-tree-sha1 = "a2df1b776752e3f344e5116c06d75a10436ab853"
uuid = "f6369f11-7733-5829-9624-2563aa707210"
version = "0.10.38"
weakdeps = ["StaticArrays"]

    [deps.ForwardDiff.extensions]
    ForwardDiffStaticArraysExt = "StaticArrays"

[[deps.Future]]
deps = ["Random"]
uuid = "9fa8497b-333b-5362-9e8d-4d0656e87820"
version = "1.11.0"

[[deps.GLM]]
deps = ["Distributions", "LinearAlgebra", "Printf", "Reexport", "SparseArrays", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns", "StatsModels"]
git-tree-sha1 = "273bd1cd30768a2fddfa3fd63bbc746ed7249e5f"
uuid = "38e38edf-8417-5370-95a0-9cbb8c7f171a"
version = "1.9.0"

[[deps.HypergeometricFunctions]]
deps = ["LinearAlgebra", "OpenLibm_jll", "SpecialFunctions"]
git-tree-sha1 = "b1c2585431c382e3fe5805874bda6aea90a95de9"
uuid = "34004b35-14d8-5ef3-9330-4cdb6864b03a"
version = "0.3.25"

[[deps.IfElse]]
git-tree-sha1 = "debdd00ffef04665ccbb3e150747a77560e8fad1"
uuid = "615f187c-cbe4-4ef1-ba3b-2fcf58d6d173"
version = "0.1.1"

[[deps.Indexing]]
git-tree-sha1 = "ce1566720fd6b19ff3411404d4b977acd4814f9f"
uuid = "313cdc1a-70c2-5d6a-ae34-0150d3930a38"
version = "1.1.1"

[[deps.InlineStrings]]
git-tree-sha1 = "45521d31238e87ee9f9732561bfee12d4eebd52d"
uuid = "842dd82b-1e85-43dc-bf29-5d0ee9dffc48"
version = "1.4.2"
weakdeps = ["ArrowTypes", "Parsers"]

    [deps.InlineStrings.extensions]
    ArrowTypesExt = "ArrowTypes"
    ParsersExt = "Parsers"

[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
version = "1.11.0"

[[deps.InvertedIndices]]
git-tree-sha1 = "6da3c4316095de0f5ee2ebd875df8721e7e0bdbe"
uuid = "41ab1584-1d38-5bbf-9106-f11c6c58b48f"
version = "1.3.1"

[[deps.IrrationalConstants]]
git-tree-sha1 = "630b497eafcc20001bba38a4651b327dcfc491d2"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.2.2"

[[deps.IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[deps.JLLWrappers]]
deps = ["Artifacts", "Preferences"]
git-tree-sha1 = "a007feb38b422fbdab534406aeca1b86823cb4d6"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.7.0"

[[deps.JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "31e996f0a15c7b280ba9f76636b3ff9e2ae58c9a"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.4"

[[deps.JSON3]]
deps = ["Dates", "Mmap", "Parsers", "PrecompileTools", "StructTypes", "UUIDs"]
git-tree-sha1 = "1d322381ef7b087548321d3f878cb4c9bd8f8f9b"
uuid = "0f8b85d8-7281-11e9-16c2-39a750bddbf1"
version = "1.14.1"
weakdeps = ["ArrowTypes"]

    [deps.JSON3.extensions]
    JSON3ArrowExt = ["ArrowTypes"]

[[deps.LaTeXStrings]]
git-tree-sha1 = "dda21b8cbd6a6c40d9d02a73230f9d70fed6918c"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.4.0"

[[deps.LazyArtifacts]]
deps = ["Artifacts", "Pkg"]
uuid = "4af54fe1-eca0-43a8-85a7-787d91b784e3"
version = "1.11.0"

[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
version = "0.6.4"

[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
version = "8.6.0+0"

[[deps.LibGit2]]
deps = ["Base64", "LibGit2_jll", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"
version = "1.11.0"

[[deps.LibGit2_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll"]
uuid = "e37daf67-58a4-590a-8e99-b0245dd2ffc5"
version = "1.7.2+0"

[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
version = "1.11.0+1"

[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
version = "1.11.0"

[[deps.LinearAlgebra]]
deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
version = "1.11.0"

[[deps.LogExpFunctions]]
deps = ["DocStringExtensions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "13ca9e2586b89836fd20cccf56e57e2b9ae7f38f"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.29"

    [deps.LogExpFunctions.extensions]
    LogExpFunctionsChainRulesCoreExt = "ChainRulesCore"
    LogExpFunctionsChangesOfVariablesExt = "ChangesOfVariables"
    LogExpFunctionsInverseFunctionsExt = "InverseFunctions"

    [deps.LogExpFunctions.weakdeps]
    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
    ChangesOfVariables = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
    InverseFunctions = "3587e190-3f89-42d0-90ee-14403ec27112"

[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"
version = "1.11.0"

[[deps.Lz4_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl"]
git-tree-sha1 = "191686b1ac1ea9c89fc52e996ad15d1d241d1e33"
uuid = "5ced341a-0733-55b8-9ab6-a4889d929147"
version = "1.10.1+0"

[[deps.MacroTools]]
git-tree-sha1 = "72aebe0b5051e5143a079a4685a46da330a40472"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.15"

[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"
version = "1.11.0"

[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
version = "2.28.6+0"

[[deps.Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "ec4f7fbeab05d7747bdf98eb74d130a2a2ed298d"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.2.0"

[[deps.MixedModels]]
deps = ["Arrow", "BSplineKit", "Compat", "DataAPI", "Distributions", "GLM", "JSON3", "LinearAlgebra", "Markdown", "MixedModelsDatasets", "NLopt", "PooledArrays", "PrecompileTools", "ProgressMeter", "Random", "SparseArrays", "StaticArrays", "Statistics", "StatsAPI", "StatsBase", "StatsFuns", "StatsModels", "StructTypes", "Tables", "TypedTables"]
git-tree-sha1 = "5af40242899244ee1a73f48b640425ca93d3ad4f"
uuid = "ff71e718-51f3-5ec2-a782-8ffcbfa3c316"
version = "4.28.0"

[[deps.MixedModelsDatasets]]
deps = ["Arrow", "Artifacts", "LazyArtifacts"]
git-tree-sha1 = "5f508af97ecf39645febed8ba2fabf5cfdc682e0"
uuid = "7e9fb7ac-9f67-43bf-b2c8-96ba0796cbb6"
version = "0.1.1"

[[deps.Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"
version = "1.11.0"

[[deps.Mocking]]
deps = ["Compat", "ExprTools"]
git-tree-sha1 = "2c140d60d7cb82badf06d8783800d0bcd1a7daa2"
uuid = "78c3b35d-d492-501b-9361-3d52fe80e533"
version = "0.8.1"

[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
version = "2023.12.12"

[[deps.NLopt]]
deps = ["CEnum", "NLopt_jll"]
git-tree-sha1 = "dd79cb2a2d779209e42d76795edfde1c778eb5e0"
uuid = "76087f3c-5699-56af-9a33-bf431cd00edd"
version = "1.1.2"

    [deps.NLopt.extensions]
    NLoptMathOptInterfaceExt = ["MathOptInterface"]

    [deps.NLopt.weakdeps]
    MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"

[[deps.NLopt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl"]
git-tree-sha1 = "dd645a206cc1e8bc4ff0dcfe3da08e39b60dd662"
uuid = "079eb43e-fd8e-5478-9966-2cf3e3edb778"
version = "2.9.0+0"

[[deps.NaNMath]]
deps = ["OpenLibm_jll"]
git-tree-sha1 = "030ea22804ef91648f29b7ad3fc15fa49d0e6e71"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "1.0.3"

[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
version = "1.2.0"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"
version = "0.3.27+1"

[[deps.OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"
version = "0.8.1+2"

[[deps.OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl"]
git-tree-sha1 = "1346c9208249809840c91b26703912dff463d335"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.6+0"

[[deps.OrderedCollections]]
git-tree-sha1 = "12f1439c4f986bb868acda6ea33ebc78e19b95ad"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.7.0"

[[deps.PDMats]]
deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"]
git-tree-sha1 = "966b85253e959ea89c53a9abebbf2e964fbf593b"
uuid = "90014a1f-27ba-587c-ab20-58faa44d9150"
version = "0.11.32"

[[deps.Parsers]]
deps = ["Dates", "PrecompileTools", "UUIDs"]
git-tree-sha1 = "8489905bcdbcfac64d1daa51ca07c0d8f0283821"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.8.1"

[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "Random", "SHA", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
version = "1.11.0"
weakdeps = ["REPL"]

    [deps.Pkg.extensions]
    REPLExt = "REPL"

[[deps.PooledArrays]]
deps = ["DataAPI", "Future"]
git-tree-sha1 = "36d8b4b899628fb92c2749eb488d884a926614d3"
uuid = "2dfb63ee-cc39-5dd5-95bd-886bf059d720"
version = "1.4.3"

[[deps.PrecompileTools]]
deps = ["Preferences"]
git-tree-sha1 = "5aa36f7049a63a1528fe8f7c3f2113413ffd4e1f"
uuid = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
version = "1.2.1"

[[deps.Preferences]]
deps = ["TOML"]
git-tree-sha1 = "9306f6085165d270f7e3db02af26a400d580f5c6"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.4.3"

[[deps.PrettyChairmarks]]
deps = ["Chairmarks", "Printf", "Statistics"]
git-tree-sha1 = "6ced454f88c64b178f7222de91ebcb253c7a4f0f"
uuid = "aafa11c5-44f9-44a1-b829-427e6ce1ffc2"
version = "1.0.0"

[[deps.PrettyTables]]
deps = ["Crayons", "LaTeXStrings", "Markdown", "PrecompileTools", "Printf", "Reexport", "StringManipulation", "Tables"]
git-tree-sha1 = "1101cd475833706e4d0e7b122218257178f48f34"
uuid = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d"
version = "2.4.0"

[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"
version = "1.11.0"

[[deps.Profile]]
uuid = "9abbd945-dff8-562f-b5e8-e1ebf5ef1b79"
version = "1.11.0"

[[deps.ProgressMeter]]
deps = ["Distributed", "Printf"]
git-tree-sha1 = "8f6bc219586aef8baf0ff9a5fe16ee9c70cb65e4"
uuid = "92933f4c-e287-5a05-a399-4b506db050ca"
version = "1.10.2"

[[deps.PtrArrays]]
git-tree-sha1 = "1d36ef11a9aaf1e8b74dacc6a731dd1de8fd493d"
uuid = "43287f4e-b6f4-7ad1-bb20-aadabca52c3d"
version = "1.3.0"

[[deps.QuadGK]]
deps = ["DataStructures", "LinearAlgebra"]
git-tree-sha1 = "cda3b045cf9ef07a08ad46731f5a3165e56cf3da"
uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc"
version = "2.11.1"

    [deps.QuadGK.extensions]
    QuadGKEnzymeExt = "Enzyme"

    [deps.QuadGK.weakdeps]
    Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9"

[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "StyledStrings", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
version = "1.11.0"

[[deps.Random]]
deps = ["SHA"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
version = "1.11.0"

[[deps.Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[deps.Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[deps.Rmath]]
deps = ["Random", "Rmath_jll"]
git-tree-sha1 = "852bd0f55565a9e973fcfee83a84413270224dc4"
uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa"
version = "0.8.0"

[[deps.Rmath_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl"]
git-tree-sha1 = "58cdd8fb2201a6267e1db87ff148dd6c1dbd8ad8"
uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f"
version = "0.5.1+0"

[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
version = "0.7.0"

[[deps.Scratch]]
deps = ["Dates"]
git-tree-sha1 = "3bac05bc7e74a75fd9cba4295cde4045d9fe2386"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.2.1"

[[deps.SentinelArrays]]
deps = ["Dates", "Random"]
git-tree-sha1 = "712fb0231ee6f9120e005ccd56297abbc053e7e0"
uuid = "91c51154-3ec4-41a3-a24f-3f23e20d615c"
version = "1.4.8"

[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
version = "1.11.0"

[[deps.ShiftedArrays]]
git-tree-sha1 = "503688b59397b3307443af35cd953a13e8005c16"
uuid = "1277b4bf-5013-50f5-be3d-901d8477a67a"
version = "2.0.0"

[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"
version = "1.11.0"

[[deps.SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "66e0a8e672a0bdfca2c3f5937efb8538b9ddc085"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.2.1"

[[deps.SparseArrays]]
deps = ["Libdl", "LinearAlgebra", "Random", "Serialization", "SuiteSparse_jll"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
version = "1.11.0"

[[deps.SpecialFunctions]]
deps = ["IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
git-tree-sha1 = "64cca0c26b4f31ba18f13f6c12af7c85f478cfde"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "2.5.0"

    [deps.SpecialFunctions.extensions]
    SpecialFunctionsChainRulesCoreExt = "ChainRulesCore"

    [deps.SpecialFunctions.weakdeps]
    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"

[[deps.SplitApplyCombine]]
deps = ["Dictionaries", "Indexing"]
git-tree-sha1 = "c06d695d51cfb2187e6848e98d6252df9101c588"
uuid = "03a91e81-4c3e-53e1-a0a4-9c0c8f19dd66"
version = "1.2.3"

[[deps.Static]]
deps = ["CommonWorldInvalidations", "IfElse", "PrecompileTools"]
git-tree-sha1 = "87d51a3ee9a4b0d2fe054bdd3fc2436258db2603"
uuid = "aedffcd0-7271-4cad-89d0-dc628f76c6d3"
version = "1.1.1"

[[deps.StaticArrays]]
deps = ["LinearAlgebra", "PrecompileTools", "Random", "StaticArraysCore"]
git-tree-sha1 = "47091a0340a675c738b1304b58161f3b0839d454"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.9.10"

    [deps.StaticArrays.extensions]
    StaticArraysChainRulesCoreExt = "ChainRulesCore"
    StaticArraysStatisticsExt = "Statistics"

    [deps.StaticArrays.weakdeps]
    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
    Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[deps.StaticArraysCore]]
git-tree-sha1 = "192954ef1208c7019899fbf8049e717f92959682"
uuid = "1e83bf80-4336-4d27-bf5d-d5a4f845583c"
version = "1.4.3"

[[deps.Statistics]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "ae3bb1eb3bba077cd276bc5cfc337cc65c3075c0"
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
version = "1.11.1"
weakdeps = ["SparseArrays"]

    [deps.Statistics.extensions]
    SparseArraysExt = ["SparseArrays"]

[[deps.StatsAPI]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "1ff449ad350c9c4cbc756624d6f8a8c3ef56d3ed"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.7.0"

[[deps.StatsBase]]
deps = ["AliasTables", "DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "29321314c920c26684834965ec2ce0dacc9cf8e5"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.34.4"

[[deps.StatsFuns]]
deps = ["HypergeometricFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"]
git-tree-sha1 = "b423576adc27097764a90e163157bcfc9acf0f46"
uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c"
version = "1.3.2"

    [deps.StatsFuns.extensions]
    StatsFunsChainRulesCoreExt = "ChainRulesCore"
    StatsFunsInverseFunctionsExt = "InverseFunctions"

    [deps.StatsFuns.weakdeps]
    ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
    InverseFunctions = "3587e190-3f89-42d0-90ee-14403ec27112"

[[deps.StatsModels]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "Printf", "REPL", "ShiftedArrays", "SparseArrays", "StatsAPI", "StatsBase", "StatsFuns", "Tables"]
git-tree-sha1 = "9022bcaa2fc1d484f1326eaa4db8db543ca8c66d"
uuid = "3eaba693-59b7-5ba5-a881-562e759f1c8d"
version = "0.7.4"

[[deps.StringManipulation]]
deps = ["PrecompileTools"]
git-tree-sha1 = "a6b1675a536c5ad1a60e5a5153e1fee12eb146e3"
uuid = "892a3eda-7b42-436c-8928-eab12a02cf0e"
version = "0.4.0"

[[deps.StringViews]]
git-tree-sha1 = "ec4bf39f7d25db401bcab2f11d2929798c0578e5"
uuid = "354b36f9-a18e-4713-926e-db85100087ba"
version = "1.3.4"

[[deps.StructTypes]]
deps = ["Dates", "UUIDs"]
git-tree-sha1 = "159331b30e94d7b11379037feeb9b690950cace8"
uuid = "856f2bd8-1eba-4b0a-8007-ebc267875bd4"
version = "1.11.0"

[[deps.StyledStrings]]
uuid = "f489334b-da3d-4c2e-b8f0-e476e12c162b"
version = "1.11.0"

[[deps.SuiteSparse]]
deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"]
uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9"

[[deps.SuiteSparse_jll]]
deps = ["Artifacts", "Libdl", "libblastrampoline_jll"]
uuid = "bea87d4a-7f5b-5778-9afe-8cc45184846c"
version = "7.7.0+0"

[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
version = "1.0.3"

[[deps.TZJData]]
deps = ["Artifacts"]
git-tree-sha1 = "006a327222dda856e2304959e566ff0104ac8594"
uuid = "dc5dba14-91b3-4cab-a142-028a31da12f7"
version = "1.3.1+2024b"

[[deps.TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[deps.Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "OrderedCollections", "TableTraits"]
git-tree-sha1 = "598cd7c1f68d1e205689b1c2fe65a9f85846f297"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.12.0"

[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
version = "1.10.0"

[[deps.TimeZones]]
deps = ["Artifacts", "Dates", "Downloads", "InlineStrings", "Mocking", "Printf", "Scratch", "TZJData", "Unicode", "p7zip_jll"]
git-tree-sha1 = "fcbcffdc11524d08523e92ae52214b29d90b50bb"
uuid = "f269a46b-ccf7-5d73-abea-4c690281aa53"
version = "1.20.0"

    [deps.TimeZones.extensions]
    TimeZonesRecipesBaseExt = "RecipesBase"

    [deps.TimeZones.weakdeps]
    RecipesBase = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"

[[deps.TranscodingStreams]]
git-tree-sha1 = "0c45878dcfdcfa8480052b6ab162cdd138781742"
uuid = "3bb67fe8-82b1-5028-8e26-92a6c54297fa"
version = "0.11.3"

[[deps.TypedTables]]
deps = ["Adapt", "Dictionaries", "Indexing", "SplitApplyCombine", "Tables", "Unicode"]
git-tree-sha1 = "84fd7dadde577e01eb4323b7e7b9cb51c62c60d4"
uuid = "9d95f2ec-7b3d-5a63-8d20-e2491e220bb9"
version = "1.4.6"

[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
version = "1.11.0"

[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
version = "1.11.0"

[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
version = "1.2.13+1"

[[deps.Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl"]
git-tree-sha1 = "622cf78670d067c738667aaa96c553430b65e269"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.7+0"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
version = "5.11.0+0"

[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
version = "1.59.0+0"

[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
version = "17.4.0+2"

jkrumbiegel · 2025-01-20T15:59:14Z

As you are on an ARM mac, can you confirm you're running the exact same Julia each time and not once arm and once x64 versions maybe? Just as a sanity check.

Other than that, maybe optimization settings could make a difference? You could compare Base.julia_cmd() for both and see if anything jumps out at you.

ajinkya-k · 2025-01-20T16:06:55Z

I do have an arm macos device, and can confirm that both are using the same binary, the only difference is one of them uses startup-file=no but I dont have any startup.jl file so it should not matter

dmbates · 2025-01-20T16:32:26Z

I am also on an arm macos device and I do have a startup.jl file that loads AppleAccelerate.jl which could cause a change.

I will look into Base.julia_cmd()

MichaelHatherly · 2025-01-20T16:37:35Z

Malt is used to run the workers, and it does https://github.com/JuliaPluto/Malt.jl/blob/124c471c574d0958fd349858fd3686b798068ff3/src/Malt.jl#L255. Do we reckon that setting "OPENBLAS_NUM_THREADS=1" might have anything to do with this?

dmbates · 2025-01-20T16:56:16Z

Malt is used to run the workers, and it does https://github.com/JuliaPluto/Malt.jl/blob/124c471c574d0958fd349858fd3686b798068ff3/src/Malt.jl#L255. Do we reckon that setting "OPENBLAS_NUM_THREADS=1" might have anything to do with this?

Yes, that's the difference

% OPENBLAS_NUM_THREADS=1 julia -tauto --project --startup=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.2 (2024-12-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using MixedModels

julia> m1 = fit(MixedModel, @formula(y ~ 1 + service + (1|s) + (1|d) + (1 + service|dept)), MixedModels.dataset(:InstEval))
Minimizing 219    Time: 0:00:04 (19.19 ms/it)
Linear mixed model fit by maximum likelihood
 y ~ 1 + service + (1 | s) + (1 | d) + (1 + service | dept)
    logLik     -2 logLik       AIC         AICc          BIC     
 -118825.3984  237650.7968  237666.7968  237666.7987  237740.4285

Variance components:
            Column   Variance Std.Dev.   Corr.
s        (Intercept)  0.105160 0.324284
d        (Intercept)  0.264545 0.514340
dept     (Intercept)  0.000000 0.000000
         service: Y   0.022730 0.150766  +NaN
Residual              1.385073 1.176891
 Number of obs: 73421; levels of grouping factors: 2972, 1128, 14

  Fixed-effects parameters:
─────────────────────────────────────────────────────
                  Coef.  Std. Error       z  Pr(>|z|)
─────────────────────────────────────────────────────
(Intercept)   3.27728     0.0188109  174.22    <1e-99
service: Y   -0.0503874   0.0433777   -1.16    0.2454
─────────────────────────────────────────────────────

Thank you for looking into this. I think that in some ways this just shows that the example is too fragile if changes like this in the environment cause convergence to another optimum. Do you have any suggestions of how we could force loading of AppleAccelerate.jl without explicitly doing so in the document? In other words, can we defeat the --startup=no in the call in Malt.jl?

MichaelHatherly · 2025-01-20T17:20:00Z

In other words, can we defeat the --startup=no in the call in Malt.jl?

Adding a --startup-file=yes into the julia.exeflags part of the notebook frontmatter would probably do the trick since it would override the one set by Malt.

ajinkya-k · 2025-01-20T17:39:58Z

could the hardcoded OPENBLAS_NUM_THREADS=1 be changed to get the input from the user and with 1 as the default fallback?

jkrumbiegel · 2025-01-20T17:41:46Z

I think the frontmatter would count as "explicitly in the document".

There's an env variable QUARTONOTEBOOKRUNNER_EXEFLAGS you can use to set more flags. Be careful though not to make your documents unreproducible this way.

ajinkya-k · 2025-01-20T17:58:19Z

Ah okay. I misunderstood the code

jkrumbiegel · 2025-01-20T19:37:24Z

could the hardcoded OPENBLAS_NUM_THREADS=1 be changed

yeah maybe we should remove that env var in our vendored Malt version, I'm not sure what it's original purpose was, though. I guess if you set your own env variable in the frontmatter you could override it.

ajinkya-k · 2025-01-20T19:45:00Z

could the hardcoded OPENBLAS_NUM_THREADS=1 be changed

yeah maybe we should remove that env var in our vendored Malt version, I'm not sure what it's original purpose was, though. I guess if you set your own env variable in the frontmatter you could override it.

would the override have to be inside the qmd file? Because if its already set i feel that if the ENV variable is set before invoking quarto the link in Malt.jl will override any initial ENV

palday · 2025-01-21T04:37:49Z

It's possible to call

using LinearAlgebra
BLAS.set_number_threads(4)

to set the number of BLAS threads manually in an interactive session.

If I toggle back and forth between 1 and 2 threads, I see the changed model fit.

MichaelHatherly · 2025-01-21T09:24:09Z

There's an env variable QUARTONOTEBOOKRUNNER_EXEFLAGS you can use to set more flags. Be careful though not to make your documents unreproducible this way.

Avoid QUARTONOTEBOOKRUNNER_EXEFLAGS please. That is not a public API at present.

would the override have to be inside the qmd file? Because if its already set i feel that if the ENV variable is set before invoking quarto the link in Malt.jl will override any initial ENV

Yes, anything set outside of quarto will get overridden. Use the notebook frontmatter to set it to something else if you need to. Or set_num_threads in a cell as noted above.

julia:
    env: ["OPENBLAS_NUM_THREADS=2"]

There's some further context in the Malt PR that added this JuliaPluto/Malt.jl#59.

dmbates · 2025-01-21T15:10:57Z

I think this issue can be closed as we know the cause of the discrepancy.

jkrumbiegel · 2025-01-21T16:10:43Z

I've read through the threads in Malt that mentioned the openblas settings. Looks like this was inherited from Distributed which set it to avoid OOM errors on windows. I'm not sure if we should also have it, it seems unexpected for users that openblas will behave differently in quarto notebooks than in normal repls.

ajinkya-k · 2025-01-21T16:24:50Z

Using the following is not working for me:

julia:
    env: ["OPENBLAS_NUM_THREADS=2"]

See screenshot:

MichaelHatherly · 2025-01-21T16:39:51Z

Does it change if you remove the spaces?

ajinkya-k · 2025-01-21T16:43:07Z

ah of course! That fixed it. Sorry for the trouble

MichaelHatherly · 2025-01-21T17:09:02Z

No worries.

MichaelHatherly · 2025-01-22T10:19:29Z

I've read through the threads in Malt that mentioned the openblas settings. Looks like this was inherited from Distributed which set it to avoid OOM errors on windows. I'm not sure if we should also have it, it seems unexpected for users that openblas will behave differently in quarto notebooks than in normal repls.

We can only remove that if we are certain that it's not going to cause trouble for any users of this package. Given that it's relatively simple to override the behaviour with an env var it doesn't seem high priority to me.

jkrumbiegel · 2025-01-22T14:36:59Z

Is it not going to cause all kinds of numerical code to be slower than necessary? I'm not sure but I assumed that was why it's a bit problematic

MichaelHatherly · 2025-01-22T14:48:22Z

Is it not going to cause all kinds of numerical code to be slower than necessary?

Yes.

Like was said in #233 (comment), if it can be shown that removing the current default don't result in OOM on Windows then we can certainly do it. I don't have easy access to a Windows system to thoroughly test this, so it'll have to rely on someone with access to do so.

dmbates · 2025-01-22T14:55:48Z

You can easily check the effect of multi-threaded custom BLAS with peakflops. It is nearly a 10-fold difference on an arm64 MacBook.

% OPENBLAS_NUM_THREADS=1 julia --startup=no   
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.3 (2025-01-21)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using LinearAlgebra

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libopenblas64_.dylib

julia> peakflops()
4.96176686992832e10

julia> using AppleAccelerate

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
├ [ LP64] Accelerate
└ [ILP64] Accelerate

julia> peakflops()
4.16963396263107e11

julia> versioninfo()
Julia Version 1.11.3
Commit d63adeda50d (2025-01-21 19:42 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 8 × Apple M1 Pro
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 6 virtual cores)

jkrumbiegel · 2025-01-22T15:15:31Z

Like was said in #233 (comment), if it can be shown that removing the current default don't result in OOM on Windows then we can certainly do it.

Even if it still causes OOM issues, is the responsibility to avoid that on us or on the users with those systems? We're not doing anything much more special than spinning up a vanilla julia process with a bit of code loaded in, so I find it a bit strange having to handle BLAS issues on some systems in QNR.

MichaelHatherly · 2025-01-22T15:27:57Z

is the responsibility to avoid that on us or on the users with those systems?

It is our responsibility to provide understandable error messages in the event of a notebook OOM. I've not reproduced the OOM so don't know what it would look like to a user.

If we can successfully do that such that they are guided to adding OPENBLAS_NUM_THREADS=1 to their own notebooks that need it then of course we can change the default. If we can't and it's just a cryptic stacktrace then we should leave the default as is.

Someone with a Windows system is welcome to check this and propose an alternative behaviour that works for everyone.

MichaelHatherly · 2025-01-22T15:30:02Z

I am reopening this as an up-for-grabs issue. We can change the behaviour if it can be confirmed that it won't result in confusing errors for users.

ajinkya-k · 2025-01-23T15:13:01Z

I removed the default OPENBLAS env setting here:

QuartoNotebookRunner.jl/src/Malt.jl

Line 374 in f736668

"OPENBLAS_NUM_THREADS" => "1",

yet the worker still uses one by default. I have no idea where that is coming from. but I did see "OPENBLAS_DEFAULT_NUM_THREADS" => "1" and "OPENBLAS_MAIN_FREE" => "1" when i printed all the environment variables inside the quarto notebook using for evv in ENV println(evv) end . Note that running the same in REPL did not produce these env vars. Maybe quarto-cli is setting them somewhere?

MichaelHatherly · 2025-01-23T15:49:26Z

Note that running the same in REPL did not produce these env vars. Maybe quarto-cli is setting them somewhere?

In a REPL I am seeing those env vars:

❯ julia --project=debug --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.8 (2025-01-22)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> ENV["OPENBLAS_DEFAULT_NUM_THREADS"]
"1"

julia> ENV["OPENBLAS_MAIN_FREE"]
"1"

appear to be set by https://github.com/JuliaLang/julia/blob/88c71dd2164030ef9563a57ef3f1072581d8efca/stdlib/OpenBLAS_jll/src/OpenBLAS_jll.jl#L46-L74.

ajinkya-k · 2025-01-23T16:30:34Z

Sorry I think there was some issue in my session.

MichaelHatherly closed this as not planned Won't fix, can't repro, duplicate, stale Jan 21, 2025

MichaelHatherly changed the title ~~Code returns different output when using engine: julia~~ Don't default to OPENBLAS_NUM_THREADS=1 in notebooks Jan 22, 2025

MichaelHatherly added the up for grabs label Jan 22, 2025

MichaelHatherly reopened this Jan 22, 2025

ajinkya-k added a commit to ajinkya-k/QuartoNotebookRunner.jl that referenced this issue Mar 1, 2025

removed OPENBLAS_NUM_THREADS default from Malt.jl, fixes PumasAI#233

d5523ae

ajinkya-k mentioned this issue Mar 1, 2025

removed OPENBLAS_NUM_THREADS default from Malt.jl, fixes #233 #272

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't default to `OPENBLAS_NUM_THREADS=1` in notebooks #233

Don't default to `OPENBLAS_NUM_THREADS=1` in notebooks #233

ajinkya-k commented Jan 20, 2025

jkrumbiegel commented Jan 20, 2025

MichaelHatherly commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

dmbates commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025 •

edited

Loading

jkrumbiegel commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

dmbates commented Jan 20, 2025

MichaelHatherly commented Jan 20, 2025

dmbates commented Jan 20, 2025

MichaelHatherly commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

jkrumbiegel commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

jkrumbiegel commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025 •

edited

Loading

palday commented Jan 21, 2025

MichaelHatherly commented Jan 21, 2025

dmbates commented Jan 21, 2025

jkrumbiegel commented Jan 21, 2025

ajinkya-k commented Jan 21, 2025

MichaelHatherly commented Jan 21, 2025

ajinkya-k commented Jan 21, 2025

MichaelHatherly commented Jan 21, 2025

MichaelHatherly commented Jan 22, 2025

jkrumbiegel commented Jan 22, 2025

MichaelHatherly commented Jan 22, 2025

dmbates commented Jan 22, 2025

jkrumbiegel commented Jan 22, 2025

MichaelHatherly commented Jan 22, 2025

MichaelHatherly commented Jan 22, 2025

ajinkya-k commented Jan 23, 2025 •

edited

Loading

MichaelHatherly commented Jan 23, 2025

ajinkya-k commented Jan 23, 2025

Don't default to OPENBLAS_NUM_THREADS=1 in notebooks #233

Don't default to OPENBLAS_NUM_THREADS=1 in notebooks #233

Comments

ajinkya-k commented Jan 20, 2025

jkrumbiegel commented Jan 20, 2025

MichaelHatherly commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

dmbates commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025 • edited Loading

jkrumbiegel commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

dmbates commented Jan 20, 2025

MichaelHatherly commented Jan 20, 2025

dmbates commented Jan 20, 2025

MichaelHatherly commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

jkrumbiegel commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025

jkrumbiegel commented Jan 20, 2025

ajinkya-k commented Jan 20, 2025 • edited Loading

palday commented Jan 21, 2025

MichaelHatherly commented Jan 21, 2025

dmbates commented Jan 21, 2025

jkrumbiegel commented Jan 21, 2025

ajinkya-k commented Jan 21, 2025

MichaelHatherly commented Jan 21, 2025

ajinkya-k commented Jan 21, 2025

MichaelHatherly commented Jan 21, 2025

MichaelHatherly commented Jan 22, 2025

jkrumbiegel commented Jan 22, 2025

MichaelHatherly commented Jan 22, 2025

dmbates commented Jan 22, 2025

jkrumbiegel commented Jan 22, 2025

MichaelHatherly commented Jan 22, 2025

MichaelHatherly commented Jan 22, 2025

ajinkya-k commented Jan 23, 2025 • edited Loading

MichaelHatherly commented Jan 23, 2025

ajinkya-k commented Jan 23, 2025

Don't default to `OPENBLAS_NUM_THREADS=1` in notebooks #233

Don't default to `OPENBLAS_NUM_THREADS=1` in notebooks #233

ajinkya-k commented Jan 20, 2025 •

edited

Loading

ajinkya-k commented Jan 20, 2025 •

edited

Loading

ajinkya-k commented Jan 23, 2025 •

edited

Loading