A3P3 Identifiers and Errors

[A. Venkateswaran]

Hi,

What if I have invalid characters interspersed with what looks like an
identifier? For example:

bo$b

Which is the correct output?
a) Invalid character at line 1
b) <ID bo>
Invalid character at line 1

As I have it now, since the lexer chooses the regular expression which
matches the longest string, the regexp matching "bo" would be selected
producing an <ID bob> token, then input would begin again starting with "$",
which would match nothing, throwing the error message. So if output (a)
from above is correct, does this mean we need to explicitly check for the
following pattern: <letter><any number of valid identifier characters><one or more invalid identifier characters><any number of valid
identifier characters> ? so that we can catch this error?

Along the same lines, here's another:

bo!b

Which is correct?
a) <ID bo><NOT><ID b>
b) <ID bo>
Invalid character at line 1
c) Invalid character at line 1

Thank in advance,

Anandh

[Ali Navrozally]

Hello,
Just a follow-up to Anandh's question:

How would the input 5-2 be tokenized? Is <int 5><int -2> acceptable?
Or is it neccessary to have <int 5><ADDOP -><int 2> ?

Thanks,

Ali

[James Schofield]

From the SL spec:

"If a negative value is needed, the unary minus operator may be used in
combination with a literal."

My interpretation of this is that the tokenizer should only return
positive <INT> tokens. The minus operator would cause the value to be
negated at a later stage of the compilation.

For example:
-5 => <ADDOP -><INT 5>
5-2 => <INT 5><ADDOP -><INT 2>

James

Post by Ali Navrozally
Hello,
How would the input 5-2 be tokenized? Is <int 5><int -2> acceptable?
Or is it neccessary to have <int 5><ADDOP -><int 2> ?
Thanks,
Ali

[Troy Mark Gonsalves]

keep the addop

Post by Ali Navrozally
How would the input 5-2 be tokenized? Is <int 5><int -2> acceptable?
Or is it neccessary to have <int 5><ADDOP -><int 2> ?

[Troy Mark Gonsalves]

bo$b
is
<ID bo>
Invalid character at line 1

bo!b
is <ID bo><NOT><ID b>

Troy

[Puneet P Khanduri]

What about 123FOOBAR?
Should it be tokenized as <INT 123><ID FOOBAR>
or
Invalid Identifier => Illegal character at line NN.\n
I have changed my code to consider this as an error and
do the System.exit sequence for such cases. Is that OK?

[Troy Mark Gonsalves]

Post by Puneet P Khanduri
What about 123FOOBAR?
Should it be tokenized as <INT 123><ID FOOBAR>

Yes.