docs: Add basic logging documentation #926

dan-fernandes · 2025-04-22T12:33:24Z

No description provided.

codecov · 2025-04-22T12:34:52Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.47%. Comparing base (24a3cfd) to head (481fb2b).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #926   +/-   ##
=======================================
  Coverage   94.47%   94.47%           
=======================================
  Files          41       41           
  Lines        2551     2551           
=======================================
  Hits         2410     2410           
  Misses        141      141

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

DominicOram

Minor comments. Some additional questions:

So this isn't enabled by default? Why not just make a bluesky stream in graylog and log into that by default? I see no reason anyone would not want it on?
If we do just enable it as default I think the info about fluentd is too much detail. Instead you just need to say where it's going by default and how to change it if you need to
A link to our graylog instance would be good, even better if it's directly to the default stream
What fields are added by default? Can the user change fields if they want to? This is probably a more "Advanced" feature

docs/how-to/configure-logging.md

dan-fernandes · 2025-05-01T13:26:17Z

@DominicOram

So this isn't enabled by default?

If it's enabled by default you would log to the Graylog instance by just checking out / spinning up the repo while on a Diamond computer. Would a better compromise be having Graylog enabled by default, but to localhost?

The rest I agree with and will implement before requesting another review

DominicOram · 2025-05-01T17:13:05Z

Hmm... Does blueAPI have no general way of knowing if you're in dev or prod? Is there any plan for this? I appreciate it's outside the scope of the issue so happy for it not to be on as default for now

DiamondJoseph · 2025-05-07T07:51:09Z

Hmm... Does blueAPI have no general way of knowing if you're in dev or prod? Is there any plan for this? I appreciate it's outside the scope of the issue so happy for it not to be on as default for now

I don't think there's any plans for that- development can mean so many things and a single setting can't encapsulate them all:
Does dev mean development of the blueapi service or of plans and devices?
Is any deployment in kubernetes "production"?

What does dev mean in terms of behaviour?

Doesn't connect/connects mock mode/connects to particular [sim?] devices?
Doesn't connect to a message bus/connects to a local/specific message bus
Doesn't send to graylog? Sends to local development graylog? Logs at DEBUG?
Doesn't use an identity provider/uses a mock/mocks the impl?
Doesn't talk to numTracker/talks to a dummy instance/mocks the impl?

All of these are things you sometimes need to do while developing blueapi, and half of them are things you might need to do while developing plans.

I think the config should just expose as much as is reasonable, and provide convenient defaults that work without any external infrastructure, both when starting the service locally/in a container and when installing the Helm chart to a fresh cluster.

callumforrester

Looks good, a few suggestions

docs/how-to/configure-logging.md

Co-authored-by: Callum Forrester <[email protected]>

tpoliaw · 2025-06-16T10:08:01Z

docs/how-to/configure-logging.md

+
+# BlueAPI Cofiguration
+
+:::{seealso}


Which version of markdown supports the :::{seealso} tags?

It's a sphinx directive.

This builds into BlueAPI's GitHub Pages entry.

docs/how-to/configure-logging.md

Happy to park discussions of enabling by default in favour of getting documentation in.

Co-authored-by: Peter Holloway <[email protected]>

DiamondJoseph

Approving with potential rewording because if I had my way I would rewrite all of the docs myself and I don't want to do that

DiamondJoseph · 2025-07-07T15:32:20Z

docs/how-to/configure-logging.md

+
+# Configure Logging
+
+By default BlueAPI will log to stdout at the [INFO level](https://docs.python.org/3/library/logging.html#logging-levels), but can be reconfigured to log at any level, and to output to Graylog.


Suggested change

By default BlueAPI will log to stdout at the [INFO level](https://docs.python.org/3/library/logging.html#logging-levels), but can be reconfigured to log at any level, and to output to Graylog.

By default BlueAPI will log to stdout at the [INFO level](https://docs.python.org/3/library/logging.html#logging-levels), but can be configured to log at any level, and to output simultaneously at the same level to [Graylog](https://graylog.org).

DiamondJoseph · 2025-07-07T15:33:10Z

docs/how-to/configure-logging.md

+
+When logging to [Graylog](https://graylog.org) is enabled, BlueAPI will also continue to log to stdout.
+


Suggested change

When logging to [Graylog](https://graylog.org) is enabled, BlueAPI will also continue to log to stdout.

DiamondJoseph · 2025-07-07T15:33:46Z

docs/how-to/configure-logging.md

+    graylog:
+      enabled: True
+      host: "graylog-log-target.diamond.ac.uk"
+      port: 12232


Suggested change

port: 12232

port: 12231

We should adjust the defaults to hit the right graylog too

DiamondJoseph · 2025-07-07T15:35:24Z

docs/how-to/configure-logging.md

+To instrument to a custom module in BlueAPI, instantiate a logger from the [standard library logging package, then use any of its log methods](https://docs.python.org/3/library/logging.html#logger-objects).
+
+BlueAPI is configured to handle logging from any python code it executes.


Suggested change

To instrument to a custom module in BlueAPI, instantiate a logger from the [standard library logging package, then use any of its log methods](https://docs.python.org/3/library/logging.html#logger-objects).

BlueAPI is configured to handle logging from any python code it executes.

Logs from plans run by BlueAPI will use inherit the configured logging, simply instantiate a logger from the [standard library logging package, then use any of its log methods](https://docs.python.org/3/library/logging.html#logger-objects).

Does this mean plans are able to reconfigure the central logging system as well?

DiamondJoseph · 2025-07-07T15:36:07Z

docs/how-to/configure-logging.md

+import logging
+logger = logging.getLogger(__name__)
+logger.info("FOO")


Suggested change

import logging

logger = logging.getLogger(__name__)

logger.info("FOO")

import logging

logger = logging.getLogger(__name__)

def my_plan() -> MsgGenerator:

logger.info("Running my_plan")

logger.debug("Loaded module with my_plan")

DiamondJoseph · 2025-07-07T15:37:25Z

docs/how-to/configure-logging.md

+Services hosted on the DLS clusters automatically have their stdout forwarded to Graylog via a service called fluentd. Due to this, BlueAPI services hosted on the cluster will always log to Graylog.
+
+When BlueAPI's native Graylog support is enabled it forwards structured data rather than plaintext.
+
+When BlueAPI's native Graylog is enabled fluentd will be automatically disabled to avoid log duplication.


Suggested change

Services hosted on the DLS clusters automatically have their stdout forwarded to Graylog via a service called fluentd. Due to this, BlueAPI services hosted on the cluster will always log to Graylog.

When BlueAPI's native Graylog support is enabled it forwards structured data rather than plaintext.

When BlueAPI's native Graylog is enabled fluentd will be automatically disabled to avoid log duplication.

Services hosted on the DLS clusters automatically have their stdout forwarded to Graylog via a service called fluentd. Therefore BlueAPI hosted on the cluster will always have logs available from Graylog, however BlueAPI's native Graylog forwards structured data rather than plaintext.

When native Graylog is enabled fluentd will be automatically disabled to avoid log duplication.

stale

Add basic logging documentation

7e552e6

dan-fernandes changed the title ~~Add basic logging documentation~~ docs: Add basic logging documentation Apr 22, 2025

dan-fernandes added 3 commits April 22, 2025 12:39

Remove elipses

4141a8a

Add everything that happens to come to mind

0c901bc

Rewording, and ass see-also

a058f03

dan-fernandes requested review from DominicOram and callumforrester April 22, 2025 13:01

DominicOram previously requested changes Apr 23, 2025

View reviewed changes

docs/how-to/configure-logging.md Outdated Show resolved Hide resolved

docs/how-to/configure-logging.md Outdated Show resolved Hide resolved

dan-fernandes added 6 commits May 6, 2025 14:19

Remove references to propagation

7a49365

Seperate docs into new Instrumenting Libraries section

0ea61df

Separate docs into new Running BlueAPI on the Cluster section

f815e5f

Add logging example

af46c00

Add Where to Find Logs section

6e2fa96

Add Log Message Structure

12ca1bd

callumforrester previously requested changes May 29, 2025

View reviewed changes

dan-fernandes and others added 4 commits May 30, 2025 09:44

Update docs/how-to/configure-logging.md

bb2946f

Co-authored-by: Callum Forrester <[email protected]>

Update docs/how-to/configure-logging.md

abe07d9

Co-authored-by: Callum Forrester <[email protected]>

Update docs/how-to/configure-logging.md

d332a04

Co-authored-by: Callum Forrester <[email protected]>

Apply suggestions from code review

9c2156b

Co-authored-by: Callum Forrester <[email protected]>

DiamondJoseph requested a review from a team as a code owner June 13, 2025 13:56

tpoliaw reviewed Jun 16, 2025

View reviewed changes

dan-fernandes and others added 4 commits July 1, 2025 08:15

Add yaml styling to code snippet

8d61305

Co-authored-by: Peter Holloway <[email protected]>

Reword configuration explanation

146aaf9

Reword incorrect explanation

285a563

Co-authored-by: Peter Holloway <[email protected]>

Add sentence explainig fluentd auto disable

19f6337

dan-fernandes and others added 3 commits July 1, 2025 07:36

Reword bad sentence

e0a51d5

Change sentence to explicitly state Diamond cluster

8b49625

Co-authored-by: Peter Holloway <[email protected]>

Remove incorrect info

4e80594

dan-fernandes requested a review from tpoliaw July 1, 2025 07:47

DiamondJoseph approved these changes Jul 7, 2025

View reviewed changes

callumforrester approved these changes Jul 8, 2025

View reviewed changes

Merge branch 'main' into add-logging-documentation

481fb2b

dan-fernandes merged commit bec4e11 into main Jul 8, 2025
17 checks passed

dan-fernandes deleted the add-logging-documentation branch July 8, 2025 07:34


		# Configure Logging

		By default BlueAPI will log to stdout at the [INFO level](https://docs.python.org/3/library/logging.html#logging-levels), but can be reconfigured to log at any level, and to output to Graylog.


		When logging to [Graylog](https://graylog.org) is enabled, BlueAPI will also continue to log to stdout.

		To instrument to a custom module in BlueAPI, instantiate a logger from the [standard library logging package, then use any of its log methods](https://docs.python.org/3/library/logging.html#logger-objects).

		BlueAPI is configured to handle logging from any python code it executes.

docs: Add basic logging documentation #926

docs: Add basic logging documentation #926

Uh oh!

Conversation

dan-fernandes commented Apr 22, 2025

Uh oh!

codecov bot commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

DominicOram left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dan-fernandes commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DominicOram commented May 1, 2025

Uh oh!

DiamondJoseph commented May 7, 2025

Uh oh!

callumforrester left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DiamondJoseph left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Apr 22, 2025 •

edited

Loading

dan-fernandes commented May 1, 2025 •

edited

Loading