Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot use env vars to point to other Conda env with running workers #278

Open
visr opened this issue Mar 7, 2025 · 7 comments
Open

Cannot use env vars to point to other Conda env with running workers #278

visr opened this issue Mar 7, 2025 · 7 comments

Comments

@visr
Copy link

visr commented Mar 7, 2025

In trying to debug #276 what confused me for a while was that I got errors like this:

ERROR: Julia server returned error after receiving "run" command:
Failed to run notebook: C:\ProgramData\DevDrives\repo\ribasim\Ribasim\docs\guide\delwaq.qmd
ERROR: EvaluationError: Encountered 13 errors during evaluation

Error 1 of 13
@ C:\ProgramData\DevDrives\repo\ribasim\Ribasim\docs\guide\delwaq.qmd:37
Python: ModuleNotFoundError: No module named 'ribasim'
Python stacktrace:
 [1] <module>
   @ C:\ProgramData\DevDrives\repo\ribasim\Ribasim\docs\guide\delwaq.qmd:38

While I was running quarto from environment B I noticed that QuartoNoteBookRunner still had A as the active environment. I ran from A earlier. But I didn't realize that QuartoNotebookRunner kicks of a server that stays alive, and needs to be manually killed such that it will start a new server for environment B.

I'm not sure what can be done about this, but ideally the server gets restarted in such a case if possible.

@MichaelHatherly
Copy link
Collaborator

There is some work that @jkrumbiegel is doing in that general direction in quarto-dev/quarto-cli#11803 that may help this situation a bit.

@MichaelHatherly
Copy link
Collaborator

While I was running quarto from environment B I noticed that QuartoNoteBookRunner still had A as the active environment.

How had you set the environment? Via --project in the julia.exeflags frontmatter, or some other way?

@visr
Copy link
Author

visr commented Mar 7, 2025

I ran quarto from a pixi environment. This starts up the right server, but if you then try to run quarto from another pixi environment the server will not be configured correctly.

https://github.com/visr/qnr-plot-mwe/blob/2ab6a65ec72d89f757177d6bdc49d070da48b55f/pixi.toml

@jkrumbiegel
Copy link
Collaborator

There's a difference between the environment that the main server process runs with and the environment that a given worker process runs with. I'm not sure I understood what you meant by

While I was running quarto from environment B I noticed that QuartoNoteBookRunner still had A as the active environment.

And in general the main server should almost never need to be killed, it's generally the workers that you'd want to restart if something goes wrong with them. But we don't have an interface via quarto for that, yet. The PR Michael linked is a start.

@visr
Copy link
Author

visr commented Mar 7, 2025

Ah I see, so probably the worker environments are the issue. And specifically the environment variables that Conda uses to point to the right environment.

Specifically what I observed was that all of these were still pointing to Conda environment A when running QuartoNotebookRunner from environment B, and only after killing julia processes this was resolved since it spun up new workers with different environment variables.

@jkrumbiegel
Copy link
Collaborator

jkrumbiegel commented Mar 7, 2025

Ah yes env variables are a special case because I think they're inherited from the server process, but that spins up only once. But there's also a way to set env vars via frontmatter if I don't misremember

@visr
Copy link
Author

visr commented Mar 7, 2025

That would make sense. There are some examples of env vars via frontmatter in #217.

@visr visr changed the title Using an existing Server for the wrong environment Cannot use env vars to point to other Conda env with running workers Mar 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants