You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: doc/Scripting.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
# Scripting (Python)
2
2
3
3
```{note}
4
-
These pages document user-facing scripting workflows. Looking for developer information about python? See [Python](system_components/Python).
4
+
These pages document user-facing scripting workflows. Looking for developer information about python or how Python is used in the IBEX backend? See [Python](system_components/Python).
For troubleshooting, see [ibex_bluesky_core troubleshooting guide](https://isiscomputinggroup.github.io/ibex_bluesky_core/dev/troubleshooting.html#).
15
+
For troubleshooting, see {external+ibex_bluesky_core:doc}`ibex_bluesky_core troubleshooting guide <dev/troubleshooting>`.
16
16
17
-
Please add troubleshooting information to that guide (via a pull-request on `ibex_bluesky_core`) rather than this page.
17
+
Please add troubleshooting information to that guide via a pull-request onto the [ibex_bluesky_core repository](https://github.com/ISISComputingGroup/ibex_bluesky_core/), rather than this page.
We allow plotting in IBEX via `matplotlib`. These plots will appear in the IBEX client.
8
+
We allow plotting in IBEX via {external+matplotlib:doc}`matplotlib <index>`. These plots will appear in the IBEX client.
8
9
9
-
As of [ticket 6719](https://github.com/ISISComputingGroup/IBEX/issues/6719), we are using a custom matplotlib backend. This backend comprises two main components:
10
-
- The websocket backend in `genie_python\matplotlib_backend\ibex_websocket_backend.py`, which is based on matplotlib's `WebAgg` backend. This serves plots over a websocket connection.
11
-
- The front-end code in `/uk.ac.stfc.isis.ibex.ui.graphing` in the GUI. This reads data from the websockets published by the backend. It is effectively a Java reimplementation of `WebAgg`'s front-end code, which is in `python\Lib\site-packages\matplotlib\backends\web_backend`.
10
+
As of [ticket 6719](https://github.com/ISISComputingGroup/IBEX/issues/6719), we are using a custom {external+matplotlib:doc}`matplotlib backend <users/explain/figure/backends>`. This backend comprises two main components:
11
+
- The websocket backend in `genie_python\matplotlib_backend\ibex_websocket_backend.py`, which is based on matplotlib's {external+matplotlib:py:obj}`WebAgg <matplotlib.backends.backend_webagg>` backend. This serves plots over a websocket connection.
12
+
- The front-end code in the `uk.ac.stfc.isis.ibex.ui.graphing`plugin in the GUI. This reads data from the websockets published by the backend. It is effectively a Java reimplementation of {external+matplotlib:py:obj}`WebAgg <matplotlib.backends.backend_webagg>`'s front-end code.
12
13
13
14
## Python (backend) component
14
15
15
-
Our custom backend is defined in `genie_python\matplotlib_backend\ibex_websocket_backend.py`. It uses/builds on `WebAggCore` under the hood.
16
+
Our custom backend is defined in `genie_python\matplotlib_backend\ibex_websocket_backend.py`. It uses/builds on {external+matplotlib:py:obj}`WebAggCore <matplotlib.backends.backend_webagg_core>` under the hood.
16
17
17
18
Fundamentally, it is a relatively simple backend which publishes a websocket. This websocket can send or receive:
18
19
- Binary messages, send only. These are PNG-encoded frames which should be published by the frontend.
19
20
- String messages, send & receive. These are encoded using JSON. They can be used, for example:
20
21
* To notify the front-end that a new frame is available
21
22
* To allow the front-end to request a new frame
22
23
* To allow the front-end to resize a plot
23
-
* To allow the front-end to pan/zoom the plot, via mouse handlers. Note the front end simply sends mouse events to the backend, `WebAgg` then interprets these and redraws an appropriate plot.
24
+
* To allow the front-end to pan/zoom the plot, via mouse handlers. Note the front end simply sends mouse events to the backend, {external+matplotlib:py:obj}`WebAgg <matplotlib.backends.backend_webagg>` then interprets these and redraws an appropriate plot.
24
25
25
26
We have modified the backend to:
26
27
- Be non-blocking, so that scientists can update their plot and still continue running scripts in the background
27
-
- To notify the IBEX client on calling `plt.show()`, so that IBEX can show the plot windows.
28
+
- To notify the IBEX client on calling {external+matplotlib:py:obj}`matplotlib.pyplot.show`, so that IBEX can show the plot windows.
28
29
29
30
30
31
## GUI (java) component
31
32
32
-
`WebAgg` is published by matplotlib as a browser-based backend. IBEX previously rendered plots in an embedded web browser view within an OPI, but this had significant reliability problems.
33
+
{external+matplotlib:py:obj}`WebAgg <matplotlib.backends.backend_webagg>` is published by matplotlib as a browser-based backend. IBEX previously rendered plots in an embedded web browser view within an OPI, but this had significant reliability problems.
33
34
34
-
We have since moved to reimplementing the javascript based frontend code in java, as it is very simple. When matplotlib is updated, we should verify that plots still work and update the protocol in the GUI if the underlying protocol in `WebAgg` has changed or been extended. However this should be relatively rare as WebAgg is designed to be embedded in other applications (per [mpl docs](https://matplotlib.org/stable/gallery/user_interfaces/embedding_webagg_sgskip.html)).
35
+
We have since moved to reimplementing the javascript based frontend code in java, as it is very simple. When matplotlib is updated, we should verify that plots still work and update the protocol in the GUI if the underlying protocol in {external+matplotlib:py:obj}`WebAgg <matplotlib.backends.backend_webagg>` has changed or been extended. However this should be relatively rare as WebAgg is designed to be embedded in other applications (per {external+matplotlib:doc}`gallery/user_interfaces/embedding_webagg_sgskip`).
35
36
36
37
The front-end receives binary messages from the backend, and displays the binary message content as a PNG in the user interface. The user interface also occasionally forces plots to be redrawn, so that any missed updates will not cause a plot to completely freeze/fail to update.
The time stamp on these are for the first `get_pv` call or equivalent after a disconnect.
18
+
The time stamp on these are for the first {external+genie_python:py:obj}`get_pv <genie.get_pv>` call or equivalent after a disconnect.
19
19
20
20
## Command problems
21
21
22
-
### Can not set or get a block reports disconnected
22
+
### Can not set or get a block; reports disconnected
23
23
24
24
There can be multiple problems, check:
25
25
26
26
1. Block exists
27
27
1. Block is spelt correctly, use `b.` and autocomplete
28
-
1. Try getting the underlying PV `g.get_pv("IN:<instrument>:CS:SB:<Block name?")`
29
-
1. Restart the GUI genie_python console
28
+
1. Try getting the underlying PV `g.get_pv("IN:<instrument>:CS:SB:<Block name>")`
29
+
1. Restart the GUI Python console
30
30
31
-
## Import problems
31
+
## Problems finding a python interpreter
32
32
33
-
### Can't find Python 3
33
+
### Can't find the Uktena Python distribution
34
34
35
35
If you get an error message similar to
36
36
37
37
```
38
38
*** Cannot find GENIE-PYTHON 3 - some things are not likely to work ***
39
39
```
40
-
on running `config_env.bat`, you need to ensure you have Python 3 available on your system.
40
+
on running `config_env.bat`, you need to ensure you have the {doc}`/system_components/Python` installed on your system.
41
+
You may need to follow the {doc}`/system_components/python/Building-and-installing-uktena` instructions.
41
42
42
-
## Other Issues
43
+
```{note}
44
+
As of October 2025, many processes in the IBEX backend depend on the Uktena python distribution. However, we are
45
+
gradually migrating these to use {doc}`python virtual environments </system_components/python/Python-venvs>`; the eventual goal is that Uktena will not be required
46
+
for the IBEX server backend.
47
+
```
48
+
49
+
### No appropriate `venv` has been created for a backend process
43
50
44
-
### Can not set change users
51
+
We are gradually migrating our server-side processes to use {doc}`uv python environments </system_components/python/Python-venvs>`.
45
52
46
-
Users seems not to get set properly using g.change_users, see ticket [5812](https://github.com/ISISComputingGroup/IBEX/issues/5812). Look into this it is more than a one off.
53
+
These virtual environments are created on instruments [by the IBEX deployment script](https://github.com/ISISComputingGroup/ibex_utils/blob/b5998462ddd4d5aa4123e30104166043151cefea/installation_and_upgrade/ibex_install_utils/tasks/system_tasks.py#L146). If that step has previously been missed or failed during a deployment, it will need to be re-run.
54
+
55
+
If you need to do this manually for one specific module, you may run:
56
+
```
57
+
rmdir /s /q .venv
58
+
uv venv .venv
59
+
uv pip sync requirements-frozen.txt
60
+
```
61
+
at the top-level of the relevant module. Modules using this approach will have a `requirements-frozen.txt`.
62
+
63
+
On a developer machine, the virtual environments are created by the [developer update script](https://github.com/ISISComputingGroup/ibex_utils/blob/master/installation_and_upgrade/developer_update.bat); if you are missing virtual environments on your developer machine, you will need to re-run that script.
64
+
65
+
## Other Issues
47
66
48
67
### Repeated error messages in console while waiting
49
68
@@ -55,9 +74,13 @@ If you get repeated errors of the form:
55
74
```
56
75
You may need to restart the genie_python session. The root cause of this issue is currently unknown. See ticket [5893](https://github.com/ISISComputingGroup/IBEX/issues/5893) for details, including a script which can scan all instruments for occurrences of this issue. If this issue is seen again, please create a new ticket to investigate further and also link it here.
57
76
58
-
### can read local PVs from instrument but not e.g. `CS:INSTLIST` or accelerator ones like beam current
77
+
### Can read local PVs but not central PVs
59
78
60
-
In one case this was due to the firewall rule for `A:\python3\python.exe` had been disabled - a process firewall exception is needed to allow it to receive the UDP name query reply
79
+
Central PVs could include:
80
+
-`CS:INSTLIST` (the instrument list)
81
+
- Accelerator PVs, for example beam current
82
+
83
+
In one case, this was due to the firewall rule for `A:\python3\python.exe` which had been disabled - a process firewall exception is needed to allow it to receive the UDP name query reply.
61
84
62
85
### Can't import channel-access modules
63
86
@@ -89,7 +112,7 @@ When trying to import channel access libraries (`CaChannel`, `pcaspy`, `aioca`,
89
112
90
113
It's also technically possible to replace `import pip` with:
91
114
92
-
```
115
+
```python
93
116
import _distutils_hack
94
117
_distutils_hack.remove_shim()
95
118
import pip
@@ -101,4 +124,4 @@ But this is not recommended for obvious reasons.
101
124
102
125
Pyright keeps a cache directory in `c:\Users\<user>\.cache\pyright-python`, this can get corrupted, if it does get corrupted pyright will entirely fail to execute. This cache directory can be deleted (at the cost of the next script-check operation being much slower).
103
126
104
-
Error from `g.load_script` will be a `json.decoder.JSONDecodeError` as pyright does not return JSON in this case (but rather, returns some non-JSON error message).
127
+
Error from {external+genie_python:py:obj}`g.load_script <genie.load_script>` will be a `json.decoder.JSONDecodeError` as pyright does not return JSON in this case (but rather, returns some non-JSON error message).
On `g.load_script` genie_python now runs pylint on the scripts. This help the user see errors before the script is run. However sometimes this causes its own issues. There is a {external+ibex_user_manual:doc}`page in the user manual <scripting/Error-Checking-Troubleshooting>` describing what the user should do.
3
+
On {external+genie_python:py:obj}`g.load_script <genie.load_script>` genie_python now runs pylint on the scripts. This help the user see errors before the script is run. However sometimes this causes its own issues. There is a {external+ibex_user_manual:doc}`page in the user manual <scripting/Error-Checking-Troubleshooting>` describing what the user should do.
4
4
5
-
Here are some more tips that we don't necessarily want to encourage.
5
+
If a warning cannot be fixed, and you are sure the code is otherwise correct, warnings can be ignored from a line by adding a comment:
6
6
7
-
Warnings can be removed from a line by adding a comment
e.g. `from IMAT_library import * # pylint: disable=wildcard-import, unused-wildcard-import`
16
+
If the error is a pyright (typing) error, as opposed to a pylint error, then the comment syntax is:
12
17
18
+
```python
19
+
some_code() # pyright: ignore
20
+
```
13
21
14
22
### Linting dynamically defined variables
23
+
15
24
Python allows programmers to set attributes dynamically using expressions like `locals()['foo'] = 1`, which creates a local variable called `foo` with value `1`. Pylint doesn't support dynamic assignment, so if `foo` was subsequently referenced in the code it would be counted as an undefined variable.
16
25
17
-
In cases where we need to lint scripts containing dynamic assignments, we can write a Pylint [transform plugin](http://pylint.pycqa.org/en/latest/how_tos/transform_plugins.html) to let Pylint know that the dynamically assigned variables are OK and should not be counted as undefined. This has been done [here](https://github.com/ISISComputingGroup/genie_python/blob/0a5f5093486e85e550b8168810e3d5cd762e34ff/Lib/site-packages/genie_python/scanning_instrument_pylint_plugin.py) to support dynamic assignments in the [SCANS library](https://github.com/ISISComputingGroup/IBEX/issues/5214), where some scripts were dynamically adding all methods of classes derived from `ScanningInstrument` to the local module.
26
+
In cases where we need to lint scripts containing dynamic assignments, we can write a Pylint [transform plugin](http://pylint.pycqa.org/en/latest/how_tos/transform_plugins.html) to let Pylint know that the dynamically assigned variables are OK and should not be counted as undefined. This has been done [here](https://github.com/ISISComputingGroup/genie/blob/19cdfeedaf5ec9107f0328a69204b621858c859a/src/genie_python/scanning_instrument_pylint_plugin.py#L14) to support dynamic assignments in the [SCANS library](https://github.com/ISISComputingGroup/IBEX/issues/5214), where some scripts were dynamically adding all methods of classes derived from `ScanningInstrument` to the local module.
The general instrument scripts are in this [repository](https://github.com/ISISNeutronMuon/InstrumentScripts) and include scripts like `do_sans` and `do_trans`, as well as the (old) scans library.
3
+
The general instrument scripts are in this [repository](https://github.com/ISISNeutronMuon/InstrumentScripts). These scripts include:
4
+
- The ['old' scans library](https://pygenie-scans.readthedocs.io/en/latest/index.html); as of October 2025, this is used by SANS instruments and reflectometers, but is being gradually replaced by {doc}`Bluesky-based scans <Bluesky-scanning>`
5
+
- A shared SANS scripting framework. Contains definitions of `do_sans` and `do_trans`
6
+
- Shared utilities for Muon beamlines, for example including:
7
+
* Background plot configuration (using {doc}`/specific_iocs/other/Background-Script-IOC` and {doc}`Matplotlib`)
8
+
* {doc}`Zero-field system </specific_iocs/magnets/Zero-field-controller>` calibration routines
9
+
* {doc}`DAE pre & post commands </specific_iocs/dae/DAE-Pre-and-Post-commands>`
0 commit comments