Online DB UI discussion

[lmoureaux]

Brief summary of issue

A browser-based UI should be provided for the features of gemonlinedb. This issue is to discuss it.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Proposed structure

Here is a proposal for the structure of the UI:

• Monitoring page
  • Whether the configuration is loaded from the DB or XML files
  • Whether the DB connection is up/the XML files are found
  • Whether a valid configuration can be built (linked)
• Expert page: A table describing the current system topology and providing access to the configuration of every element.

Mockups

System topology table:

Configuration viewer:

[jsturdy]

What do you see as the entrypoint here?
I was thinking of a couple of things (that I'd seen in other projects):

• select the TStore view to load
• select a tag of a view, loaded from a list of known tags (may need a query, not sure how this is implemented)
• (maybe?) import an XML file from a list of files loaded into some object store
• (maybe?) import an arbitrary XML file loaded from the filesystem
• (maybe?) import an arbitrary XML file from the operators computer
• (maybe?) import a configuration from connected hardware (would probably be an expert action)

From this point, the monitoring (mentioned above) gets populated, but before that, based on the first two points, there could be a dropdown to select a view and tag.
In this model, one doesn't actually necessarily load a complete system configuration, rather an arbitrary subset of the configurations.

For a full system view, I'd probably organize things based on the HW level:

• At the shelf list level, one has columns for each AMC13 with each row each of the possible settings
• Selecting a specific shelf, then at the AMC list level, one has columns for each AMC
• Selecting a specific AMC, one then has columns for each OptoHybrid
• Selecting an OptoHybrid, one would have columns for each VFAT/GBT (probably split these into two tabs or something similar, or a selection for OH-GBT or OH-VFAT)

For the expert tools, I haven't yet settled upon how a verification/validation of a loaded config should be done, i.e., whether it should be done in this web UI, or in the HW manager classes, to isolate HW access from other operations, but would be useful to have the monitoring be able to compare loaded vs configuration values, highlighting differences.

Probably a few other features that would be nice/needed

[lmoureaux]

What do you see as the entrypoint here?

There doesn't need to be only one entry point. One could have:

• The current system configuration (what would be written to hardware if "configure" was pressed)
• Sets of XML files for operation without a DB
• A DB browser from where one can select content from the DB
• The actual system configuration (reading back from hardware)

The options above define the sources from which configuration objects could be pulled. The set of actions that one will be able to perform also needs to be set. So far I can foresee the following:

• View configurations
• Compare two or more configurations
• (?) Editing capabilities
• (Import XML)
• (Export XML)

Since good UIs are designed with users in mind, maybe we should settle on an "ideal" set of use cases.

select the TStore view to load

I don't understand how this would work. As far as I understand, TStore views are just an abstraction layer around SQL queries and can return any result, maybe not even related to GEM. What I have in mind for the "DB browser" is a set of predefined "views" that will allow users to find configurations based on different types of input (run number, part id, date and time, ...) Each of them will likely need a separate implementation, but I'll share as much code as possible.

select a tag of a view, loaded from a list of known tags (may need a query, not sure how this is implemented)

The DB field I could find that comes close to a "tag" is COMMENT_DESCRIPTION in the condition data sets and runs tables.

In this model, one doesn't actually necessarily load a complete system configuration, rather an arbitrary subset of the configurations.

• Is this subset allowed to contain two entries for the same hardware object?
• Is the user notified about missing configuration objects that would prevent , and how?

For a full system view, I'd probably organize things based on the HW level

Looks good, I'll update the mockups.

I haven't yet settled upon how a verification/validation of a loaded config should be done, i.e., whether it should be done in this web UI, or in the HW manager classes

Basic consistency checks will be done by the DB module when loading XML files and "linking" a full system configuration. Bounds checking for registers smaller than 32 bits can be added. Other than that, I'd suggest that any "smart" check is done by another module (even if the results are reported in the DB application).

[jsturdy]

• The actual system configuration (reading back from hardware)

I haven't yet settled upon how a verification/validation of a loaded config should be done, i.e., whether it should be done in this web UI, or in the HW manager classes

Here is what I meant by this point: I haven't settled in my mind whether I think the web UI of the GEM online DB library should have direct access to hardware, or whether this should be populated via some mechanism from the HW manager class itself. We'll want to minimize the number of things that are actually doing HW access. However, at the same time, I'm envisioning this web UI as something that could (but may not necessarily need to) run standalone from the "standard" supervisor+managers mode.

In this model, one doesn't actually necessarily load a complete system configuration, rather an arbitrary subset of the configurations.

I don't understand how this would work. As far as I understand, TStore views are just an abstraction layer around SQL queries and can return any result, maybe not even related to GEM. What I have in mind for the "DB browser" is a set of predefined "views" that will allow users to find configurations based on different types of input (run number, part id, date and time, ...) Each of them will likely need a separate implementation, but I'll share as much code as possible.

I have been envisioning having the TStore views (yes, just wrappers around specific queries) be the things that this UI is built around. Then everything in the UI that relies on information in the DB, comes by selecting the appropriate view(s) to load.

Additionally, we'll have to decide how much overlap we want to have with the OMS views (existing or in preparation); I'd lean towards minimal/none, and have a lot of the stuff that could be implemented here, just implemented in the OMS tools. This would include stuff like (again, depending on the use case for this UI):

• Compare two or more configurations
• A DB browser from where one can select content from the DB

The primary reason is that I want to have a minimally interaction with the DB in our side, not defining all the possible queries one could do, not necessarily querying the DB for arbitrary parameters, specific run ranges, etc., if this distinction makes sense.

Then on to use cases (for this UI):

The options above define the sources from which configuration objects could be pulled. The set of actions that one will be able to perform also needs to be set. So far I can foresee the following:

• View configurations
• Compare two or more configurations (more?)
• (?) Editing capabilities
• (Import XML)
• (Export XML)

I remove "Import XML" because I view that as one of the entrypoints above. Otherwise, I agree those are the primary use cases for what this UI should provide.

• Configuration is loaded into the c++ objects, search for a part to be modified (possibly), modify the value, export the updated settings, this is uploaded to the DB.
• Similar to above, but read-only comparison of the DB loaded config with an imported XML.
• Similar to above, but read-only comparison two imported XMLs.
• Similar to above, but read-only comparison of the DB loaded/XML config with current HW settings (depends on the above point about HW access)
• One that maybe we haven't discussed yet is the loading/importing of updated configuration parameters after a scan of some kind. That data could either be formatted in the scan analysis tool for direct upload, or could be loaded (from a single, arbitrary format) into this tool and subsequently exported and then uploaded

@mexanick, @bdorney, assuming a functional OMS for GEM DB, do you see additional missing components from this UI tool? I will posit it's unlikely that we'll see all of them until we have it in beta testing.

[lmoureaux]

Could you provide some examples of views one would be able to select? I need some clarification here because of how broad the "view" concept is.

I haven't settled in my mind whether I think the web UI of the GEM online DB library should have direct access to hardware, or whether this should be populated via some mechanism from the HW manager class itself. [...] However, at the same time, I'm envisioning this web UI as something that could (but may not necessarily need to) run standalone from the "standard" supervisor+managers mode.

I like the second option better, for I'd like to keep the DB module as ignorant of the actual hardware as possible. The HW manager could implement some API to expose the current parameters. The relevant parts of the UI would be enabled only if the manager is running.

Some remarks:

1. We want to support small stands without a DB backend. One should be able to perform most operations using only XML files.
2. I can't see a good use case for storing incomplete configurations to the DB (eg only few VFATs). On the other hand, it should be supported for XML.
3. I'm guessing the role of DB structures based on the schema and column labels. It may result in a suboptimal/uncommon usage pattern. Do you know if there are some guidelines?

[jsturdy]

Could you provide some examples of views one would be able to select? I need some clarification here because of how broad the "view" concept is.

Broad strokes, the TStore view objects will return the content of the "views" that we've requested to be defined by the DB guys, i.e., one view for VFAT config info, one for OptoHybrid, one for AMC, and one for AMC13, so the underlying query would be something along the lines of:

SELECT * FROM CMS_GEM_MUON_VIEW.V_CMS_GEM_AMC13S_VIEW

Currently, these views are almost sufficient (there are two views for VFAT3s, the chip and channel views, and I think a merged view would be potentially better, but might feed into your point 3 above), I think we yet need to refine our requirements with the DB group, as they are currently missing the link to the construction DB and the component location information.
We could add to these queries some optional flags, such as (just to ensure that we get "installed" components, rather than simply "tracked" components, N.B. the status flag doesn't currently exist)

SELECT * FROM CMS_GEM_MUON_VIEW.V_CMS_GEM_AMC13S_VIEW WHERE status LIKE %active%

Additionally, I had previously asked if we could request a tag attribute that would update the view whenever the data changed, but a pro tag that we moved to a specific set of configuration data, but this definitely needs to be followed up on

1. We want to support small stands without a DB backend. One should be able to perform most operations using only XML files.
2. I can't see a good use case for storing incomplete configurations to the DB (eg only few VFATs). On the other hand, it should be supported for XML.

Agreed, however, depending on how this information does end up getting tracked in the construction DB, the TStore view (an editable XML file) could be modified to changed to something like:

SELECT * FROM CMS_GEM_MUON_VIEW.V_CMS_GEM_AMC13S_VIEW WHERE location LIKE %teststandX%

3. I'm guessing the role of DB structures based on the schema and column labels. It may result in a suboptimal/uncommon usage pattern. Do you know if there are some guidelines?

Can you expand this a little? I'm not sure I'm following exactly.

[lmoureaux]

3. I'm guessing the role of DB structures based on the schema and column labels. It may result in a suboptimal/uncommon usage pattern. Do you know if there are some guidelines?
   Can you expand this a little? I'm not sure I'm following exactly.

This point was related to what we discussed today about the possibility to save only what was changed or start a new configuration run.