Define anchors/sections from config regex

[ahoereth]

Is your feature request related to a problem? Please describe.

Navigating large log files which have clear sections over time.

Describe the solution you'd like

:goto #400 allows one to go to a specific anchor or section (? uses both terms).

Thing is, there appears to be no way to define sections or anchors using the configuration.

Describe alternatives you've considered

Tried using partitions which can be defined in the configuration, but was unable to find a way for jumping to specific partitions.

[tstack]

Thing is, there appears to be no way to define sections or anchors using the configuration.

You can set the partition name for a log message using the SQL interface. Here's an example:

https://github.com/tstack/lnav/blob/master/src/scripts/partition-by-boot.lnav

but was unable to find a way for jumping to specific partitions.

You should be able to use :goto #the-partition-name and if you type :goto # and then press TAB, it should tab complete the available partition names.

[tstack]

I can define partitions from the log config, is that correct?

Ah, I totally forgot about that.

You need to add another object level, here is an example:

lnav/src/formats/vmw_log.json

Lines 244 to 254 in e47c367

	"partitions": {
	"test-partition": {
	"description": "Partition for gtest sections",
	"paths": [
	{
	"glob": "*/test.log"
	}
	],
	"pattern": "^\\[ RUN \\] ([^\\n]+)"
	}
	},

Here's the schema, for reference:

https://lnav.org/schemas/format-v1.schema.json

[ahoereth]

Wow figured it out, user error.

For future reference and example partition configuration:

    "partitions": {
      "cycle": {
        "pattern": "CYCLE START #([0-9]+)"
      }
    },

[tstack]

"CYCLE START #([0-9]+)"

Note that this regex will be applied to all lines, so it can affect performance. If the scale you're working at isn't too big and you don't notice, then it's probably fine. Otherwise, let me know.

[ahoereth]

Thanks for the hint. Scale is a tens of gb easily per file, <100gb. Is there a good way to measure "time to interactable"? Then I will try to give some values.

[ahoereth]

Alternative for me would be to use a column: Each row has one column which contains the cycle number, thousand or so lines will have the same value in it and the value always ever only remains the same or increases. Using this as the opid-field atm. Can I somehow jump to a specific opid value? Would solve the same problem for me at least.

[tstack]

Can I somehow jump to a specific opid value?

The OPID should show up in the breadcrumb bar up top. You can navigate to the crumb (press backquote to focus and then right arrow over to it; or click on it with the mouse) and enter the desired OPID in there.

The opid is indexed, so it should have reasonable performance. I've made some additional improvements to opid scanning on the top of tree.

[ahoereth]

Hm that generally works, thank! But navigate the ui feels less quick than typing a command, although that could probably also become muscle memory-- I will probably stick with :goto for now but will see if I run into performance issues.

[tstack]

But navigate the ui feels less quick than typing a command

You can also write a script to go to a particular OPID. Put something like the following into a script named "goto-opid.lnav":

;SELECT log_line FROM all_logs WHERE log_opid = $1
:goto $log_line

Then, you can call it with |goto-opid 5.