Ggtcf

Does a druid starting with a bow start with no arrows?

Is it canonical bit space?

Why is consensus so controversial in Britain?

Theorems that impeded progress

Why was the shrinking from 8″ made only to 5.25″ and not smaller (4″ or less)?

CEO ridiculed me with gay jokes and grabbed me and wouldn't let go - now getting pushed out of company

Doing something right before you need it - expression for this?

What is going on with Captain Marvel's blood colour?

How to prevent "they're falling in love" trope

Modeling an IP Address

Facing a paradox: Earnshaw's theorem in one dimension

Stopping power of mountain vs road bike

How do conventional missiles fly?

What's the difference between 'rename' and 'mv'?

Why can't we play rap on piano?

What reasons are there for a Capitalist to oppose a 100% inheritance tax?

Why "Having chlorophyll without photosynthesis is actually very dangerous" and "like living with a bomb"?

Can one be a co-translator of a book, if he does not know the language that the book is translated into?

How can I fix/modify my tub/shower combo so the water comes out of the showerhead?

Took a trip to a parallel universe, need help deciphering

What mechanic is there to disable a threat instead of killing it?

Assassin's bullet with mercury

What is the intuition behind short exact sequences of groups; in particular, what is the intuition behind group extensions?

Were any external disk drives stacked vertically?

Python lex - TypeError: Unknown text

Calling an external command in PythonWhat are metaclasses in Python?Is there a way to run Python on Android?Finding the index of an item given a list containing it in PythonDifference between append vs. extend list methods in PythonHow can I safely create a nested directory in Python?Does Python have a ternary conditional operator?How to get the current time in PythonHow can I make a time delay in Python?Does Python have a string 'contains' substring method?

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;

0

I'm trying to write a simple lex parser. The cope is currently:

from ply import lex

tokens = (
 'COMMENT',
 'OTHER'
 )

 t_COMMENT = r'^#.*n'

 t_OTHER = r'^[^#].*n'

 def t_error(t):
 raise TypeError("Unknown text '%s'" % (t.value,))

 lex.lex()

lex.input(yaml)
 for tok in iter(lex.token, None):
 print repr(tok.type), repr(tok.value)

But is fails to parse simple input file:

 # This is a real comment
 #And this one also

 #/*
 # *
 # *Variable de feeu
 # */
 ma_var: True

 It is done, over, kaput 

With the following output:

l
'COMMENT' '# This is a real commentn'
Traceback (most recent call last):
 File "parser_adoc.py", line 62, in <module>
 main2()
 File "parser_adoc.py", line 57, in main2
 for tok in iter(lex.token, None):
 File "/usr/lib/python2.7/site-packages/ply/lex.py", line 384, in token
 newtok = self.lexerrorf(tok)
 File "parser_adoc.py", line 44, in t_error
 raise TypeError("Unknown text '%s'" % (t.value,))
TypeError: Unknown text '#And this one also

#/*
# *
# *Variable de feeu
# */
ma_var: True

this is done
'

So in summary, I defined 2 regex:

• One for line beginning with #




• One for lines beginning not with #

But it's not working.
I don't understand what's wrong with my regex.

Could you help?

Simon

python lex

share|improve this question

edited Mar 8 at 23:58

Raoul Debaze

asked Mar 8 at 23:53

Raoul DebazeRaoul Debaze

56112

add a comment | 

0

I'm trying to write a simple lex parser. The cope is currently:

from ply import lex

tokens = (
 'COMMENT',
 'OTHER'
 )

 t_COMMENT = r'^#.*n'

 t_OTHER = r'^[^#].*n'

 def t_error(t):
 raise TypeError("Unknown text '%s'" % (t.value,))

 lex.lex()

lex.input(yaml)
 for tok in iter(lex.token, None):
 print repr(tok.type), repr(tok.value)

But is fails to parse simple input file:

 # This is a real comment
 #And this one also

 #/*
 # *
 # *Variable de feeu
 # */
 ma_var: True

 It is done, over, kaput 

With the following output:

l
'COMMENT' '# This is a real commentn'
Traceback (most recent call last):
 File "parser_adoc.py", line 62, in <module>
 main2()
 File "parser_adoc.py", line 57, in main2
 for tok in iter(lex.token, None):
 File "/usr/lib/python2.7/site-packages/ply/lex.py", line 384, in token
 newtok = self.lexerrorf(tok)
 File "parser_adoc.py", line 44, in t_error
 raise TypeError("Unknown text '%s'" % (t.value,))
TypeError: Unknown text '#And this one also

#/*
# *
# *Variable de feeu
# */
ma_var: True

this is done
'

So in summary, I defined 2 regex:

• One for line beginning with #




• One for lines beginning not with #

But it's not working.
I don't understand what's wrong with my regex.

Could you help?

Simon

python lex

share|improve this question

edited Mar 8 at 23:58

Raoul Debaze

asked Mar 8 at 23:53

Raoul DebazeRaoul Debaze

56112

add a comment | 

I'm trying to write a simple lex parser. The cope is currently:

from ply import lex

tokens = (
 'COMMENT',
 'OTHER'
 )

 t_COMMENT = r'^#.*n'

 t_OTHER = r'^[^#].*n'

 def t_error(t):
 raise TypeError("Unknown text '%s'" % (t.value,))

 lex.lex()

lex.input(yaml)
 for tok in iter(lex.token, None):
 print repr(tok.type), repr(tok.value)

But is fails to parse simple input file:

 # This is a real comment
 #And this one also

 #/*
 # *
 # *Variable de feeu
 # */
 ma_var: True

 It is done, over, kaput 

With the following output:

l
'COMMENT' '# This is a real commentn'
Traceback (most recent call last):
 File "parser_adoc.py", line 62, in <module>
 main2()
 File "parser_adoc.py", line 57, in main2
 for tok in iter(lex.token, None):
 File "/usr/lib/python2.7/site-packages/ply/lex.py", line 384, in token
 newtok = self.lexerrorf(tok)
 File "parser_adoc.py", line 44, in t_error
 raise TypeError("Unknown text '%s'" % (t.value,))
TypeError: Unknown text '#And this one also

#/*
# *
# *Variable de feeu
# */
ma_var: True

this is done
'

So in summary, I defined 2 regex:

• One for line beginning with #




• One for lines beginning not with #

But it's not working.
I don't understand what's wrong with my regex.

Could you help?

Simon

python lex

share|improve this question

edited Mar 8 at 23:58

Raoul Debaze

asked Mar 8 at 23:53

Raoul DebazeRaoul Debaze

56112

from ply import lex

tokens = (
 'COMMENT',
 'OTHER'
 )

 t_COMMENT = r'^#.*n'

 t_OTHER = r'^[^#].*n'

 def t_error(t):
 raise TypeError("Unknown text '%s'" % (t.value,))

 lex.lex()

lex.input(yaml)
 for tok in iter(lex.token, None):
 print repr(tok.type), repr(tok.value)

 # This is a real comment
 #And this one also

 #/*
 # *
 # *Variable de feeu
 # */
 ma_var: True

 It is done, over, kaput 

l
'COMMENT' '# This is a real commentn'
Traceback (most recent call last):
 File "parser_adoc.py", line 62, in <module>
 main2()
 File "parser_adoc.py", line 57, in main2
 for tok in iter(lex.token, None):
 File "/usr/lib/python2.7/site-packages/ply/lex.py", line 384, in token
 newtok = self.lexerrorf(tok)
 File "parser_adoc.py", line 44, in t_error
 raise TypeError("Unknown text '%s'" % (t.value,))
TypeError: Unknown text '#And this one also

#/*
# *
# *Variable de feeu
# */
ma_var: True

this is done
'

share|improve this question

edited Mar 8 at 23:58

Raoul Debaze

asked Mar 8 at 23:53

Raoul DebazeRaoul Debaze

56112

share|improve this question

edited Mar 8 at 23:58

Raoul Debaze

asked Mar 8 at 23:53

Raoul DebazeRaoul Debaze

56112

asked Mar 8 at 23:53

Raoul DebazeRaoul Debaze

56112

asked Mar 8 at 23:53

Raoul DebazeRaoul Debaze

56112

1 Answer
1

active

oldest

votes

0

In python regexes (which PLY uses), ^ refers to the beginning of the string, not the beginning of the line, unless multi-line mode has been set. So since both of your rules start with ^, they can only match on the first line.

You could fix this by wrapping your regexes in (?m:...), which enables multi-line mode, but that's not even necessary here. Instead you can just remove the ^ from the beginning of your rules and it will work as you intend. Since both of your rules always match the entire line, the next token will always start at the beginning of the line - no need to anchor them.

share|improve this answer

answered Mar 9 at 11:10

sepp2ksepp2k

300k39601617

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55072576%2fpython-lex-typeerror-unknown-text%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

In python regexes (which PLY uses), ^ refers to the beginning of the string, not the beginning of the line, unless multi-line mode has been set. So since both of your rules start with ^, they can only match on the first line.

You could fix this by wrapping your regexes in (?m:...), which enables multi-line mode, but that's not even necessary here. Instead you can just remove the ^ from the beginning of your rules and it will work as you intend. Since both of your rules always match the entire line, the next token will always start at the beginning of the line - no need to anchor them.

share|improve this answer

answered Mar 9 at 11:10

sepp2ksepp2k

300k39601617

add a comment | 

0

In python regexes (which PLY uses), ^ refers to the beginning of the string, not the beginning of the line, unless multi-line mode has been set. So since both of your rules start with ^, they can only match on the first line.

You could fix this by wrapping your regexes in (?m:...), which enables multi-line mode, but that's not even necessary here. Instead you can just remove the ^ from the beginning of your rules and it will work as you intend. Since both of your rules always match the entire line, the next token will always start at the beginning of the line - no need to anchor them.

share|improve this answer

answered Mar 9 at 11:10

sepp2ksepp2k

300k39601617

add a comment | 

In python regexes (which PLY uses), ^ refers to the beginning of the string, not the beginning of the line, unless multi-line mode has been set. So since both of your rules start with ^, they can only match on the first line.

You could fix this by wrapping your regexes in (?m:...), which enables multi-line mode, but that's not even necessary here. Instead you can just remove the ^ from the beginning of your rules and it will work as you intend. Since both of your rules always match the entire line, the next token will always start at the beginning of the line - no need to anchor them.

share|improve this answer

answered Mar 9 at 11:10

sepp2ksepp2k

300k39601617

share|improve this answer

answered Mar 9 at 11:10

sepp2ksepp2k

300k39601617

answered Mar 9 at 11:10

sepp2ksepp2k

300k39601617

answered Mar 9 at 11:10

sepp2ksepp2k

300k39601617