Problems getting Grammar to Parse input from file

[grogancolin]

My grammar will parse from a string defined inside D, however if I try to read the string in from a file, it falls over.

My Grammar is this:

Properties:
    File        <- Line+ LastLine
    Line        <- Word :Equals TypeDef :';' :endOfLine
    LastLine    <- Word :Equals TypeDef :';' :eoi
    Word        <~ (:endOfLine)* (:S)* (!Equals .)+ (:S)*
    TypeDef     <- (:S)* 
                    ( 'Int'
                      /'Float'
                      /'String'
                    ) (:S)* (Constraints)?

    Constraints <- :'(' (Constraint :';')+ :')' 
    Constraint  <-  (:S)*
                    ('Random'
                     /'Unique'
                    )
                    (:S)*
    Equals  <- '='

    S       <- (' ' / '/t')

And the D code that works is:

    string input = "Name   =               Int;
    Other=Float;
    One More = String(Random;);";
    auto propsFile = Properties(input).children[0];

However, if I read the exact same string from a file using readText (or File.byLine), it dies with the error message:

+-Properties.LastLine (failure)
    +-Properties.Word (failure)
       +-any failure at line 3, col 0, after "Random;);
" expected any char, but got ""

Any ideas what I'm doing wrong?

[grogancolin]

My guess is it cant find :eoi on LastLine rule and thus never ends, but why is this? And how do I fix it?

[PhilippeSigaud]

Right now, it seems that it finds an empty line at the very end. Your input
file has three lines (from 0 to 2), so line 3 col 0 is not really part of
your input.

I guess that readText or File.byLine consumes the end of file char? Could
you test it by adding a '\n' char at the end of your input (not in your
file, but after reading it).

Or maybe your editor automatically add an end of line char for the very
last line of the file?

[grogancolin]

Im using Vim, so I dont think it's appending anything weird.

Adding the '\n' char didnt work, but got me thinking to look at the raw bytes of the input.
I printed out the last 4 bytes from each input.

From D string)  [111, 109, 59, 41]
Read from File) [109, 59, 41, 10]

That extra '10' on the end is a Line Feed char, if I cut this char off, it works, so this issue can close I guess. Might be good to document however? I'm sure it's been found out before...

Thanks!

[PhilippeSigaud]

Im using Vim, so I dont think it's appending anything weird.

OK.

Adding the '\n' char didnt work, but got me thinking to look at the raw
bytes of the input.
I printed out the last 4 bytes from each input.

From D string) [111, 109, 59, 41]
Read from File) [109, 59, 41, 10]

That extra '10' on the end is a Line Feed char, if I cut this char off, it
works

Good! So it's indeed the reader itself which adds it.

I'm not following the D forums, due to lack of time, but did you ask on
D.Learn why reading a file add a '\n' at the end?

[grogancolin]

Good idea, will ask now. Would be good to find out why it is doing it and if theres a way of making it not do it. The documentation doesn't point to anything obvious.

Thread here:
http://forum.dlang.org/post/[email protected]

[PhilippeSigaud]

Hmm, and I see that Vim adds a newline at the end of a file.

What if you just use:

Line        <- Word :Equals TypeDef :';' :endOfLine
LastLine    <- :eoi

Does that work?

(Sorry, I don't have access to a D compiler right now, else I'd test it myself)

[grogancolin]

Sorry, only getting to this now.

Yeah, it does indeed.
I had to change the logic to handle reading the data back from the parser, but thats fine.
Thanks for the help philippe!

[PhilippeSigaud]

You're welcome!