Newline character as list delimiter?

[supergeoff]

Cant figure out how to parse a line by line file (CSV, or other list...)

from pyleri import Choice, Grammar, Keyword, List, Optional, Ref, Regex, Sequence, Token
import os


class MyGrammar(Grammar):
	element = Regex(r"\w+")
	START = List(element, "\n", opt=True)


grammar = MyGrammar()
result = grammar.parse("word\nword\n").as_str()

print(result)

error at position 5, expecting:

Tried with combination of \n, \n, os.linesep...

No luck

Any idea?

[joente]

Maybe your use-case can be solved with Repeat instead of a List:

from pyleri import Repeat, Grammar, Regex

class MyGrammar(Grammar):
	element = Regex(r"\w+")
	START = Repeat(element)

grammar = MyGrammar()
print(grammar.parse("word\nword\n").as_str())  # parsed successfully

However, if it get's more complicated you might run into to problems since pyleri handles all white-space (including line breaks) equally. This could be solved rather easy though, for example we could allow a user to define the white-space used in a grammar by RE_WHITESPACE, like we do with RE_KEYWORDS.

[supergeoff]

As I understand list does work for delimiter that are anything non whitespace (including line breaks)?
Allowing the definition of white-space with someting like RE_KEYWORDS would be a nice feature indeed.

Thank you for your quick reply.

Edit: I can see a lot of example where I have lines with white space that are only separated by linebreaks...

[wqp89324]

Would the RE_WHITESPACE feature be available soon? Thanks!

[wqp89324]

@joente you mentioned "pyleri handles all white-space (including line breaks) equally", what does it mean exactly? Are all white-space (including line breaks) equally ignored?

[joente]

@wqp89324, pyleri is currently using strip without any arguments for handling white-space between nodes.