Ggtcf

Small nick on power cord from an electric alarm clock, and copper wiring exposed but intact

Expectation in a stochastic differential equation

Is it professional to write unrelated content in an almost-empty email?

what's the use of '% to gdp' type of variables?

Physiological effects of huge anime eyes

Which one is the true statement?

Expressing the idea of having a very busy time

Easy to read palindrome checker

Is there a reasonable and studied concept of reduction between regular languages?

How to Implement Deterministic Encryption Safely in .NET

Why is the US ranked as #45 in Press Freedom ratings, despite its extremely permissive free speech laws?

Do scriptures give a method to recognize a truly self-realized person/jivanmukta?

Yu-Gi-Oh cards in Python 3

Computationally populating tables with probability data

Is it ever safe to open a suspicious HTML file (e.g. email attachment)?

Is fine stranded wire ok for main supply line?

Audio Conversion With ADS1243

Is there an equivalent of cd - for cp or mv

Traduction de « Life is a roller coaster »

Is it ok to trim down a tube patch?

What happened in Rome, when the western empire "fell"?

Is there a way to save my career from absolute disaster?

0-rank tensor vs vector in 1D

My ex-girlfriend uses my Apple ID to login to her iPad, do I have to give her my Apple ID password to reset it?

PySpark error “SyntaxError: invalid syntax around” while trying to run a python file from command line

-1

I am trying to run a python file with PySpark 2.3.2 (Python 3.6.5) through command line and this is the error I'm getting:

>>> spark-submit test.py
File "<stdin>", line 1
spark-submit test.py
 ^
SyntaxError: invalid syntax

If I do initialize pyspark and try counting some .txt file's characters it works fine.

PS.: Before initializing PySpark I did navigate to the directory where "test.py" is located.

PS.2: I have PySpark configured in IntelliJ and it works just fine if I run the same test.py file from it.

Thanks in advance.

python pyspark syntax-error spark-submit

share|improve this question

asked Mar 8 at 17:23

Spencer ChedidSpencer Chedid

42

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  did you try python test.py?
  
  – mauve
  Mar 8 at 17:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Possible duplicate of How to run a script in PySpark
  
  – mad_
  Mar 8 at 17:28
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You're trying to run the script inside a python shell... exit() the python shell and then run it.
  
  – roganjosh
  Mar 8 at 17:31
  
  

  
  
  
  

add a comment | 

-1

I am trying to run a python file with PySpark 2.3.2 (Python 3.6.5) through command line and this is the error I'm getting:

>>> spark-submit test.py
File "<stdin>", line 1
spark-submit test.py
 ^
SyntaxError: invalid syntax

If I do initialize pyspark and try counting some .txt file's characters it works fine.

PS.: Before initializing PySpark I did navigate to the directory where "test.py" is located.

PS.2: I have PySpark configured in IntelliJ and it works just fine if I run the same test.py file from it.

Thanks in advance.

python pyspark syntax-error spark-submit

share|improve this question

asked Mar 8 at 17:23

Spencer ChedidSpencer Chedid

42

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  did you try python test.py?
  
  – mauve
  Mar 8 at 17:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Possible duplicate of How to run a script in PySpark
  
  – mad_
  Mar 8 at 17:28
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You're trying to run the script inside a python shell... exit() the python shell and then run it.
  
  – roganjosh
  Mar 8 at 17:31
  
  

  
  
  
  

add a comment | 

I am trying to run a python file with PySpark 2.3.2 (Python 3.6.5) through command line and this is the error I'm getting:

>>> spark-submit test.py
File "<stdin>", line 1
spark-submit test.py
 ^
SyntaxError: invalid syntax

If I do initialize pyspark and try counting some .txt file's characters it works fine.

PS.: Before initializing PySpark I did navigate to the directory where "test.py" is located.

PS.2: I have PySpark configured in IntelliJ and it works just fine if I run the same test.py file from it.

Thanks in advance.

python pyspark syntax-error spark-submit

share|improve this question

asked Mar 8 at 17:23

Spencer ChedidSpencer Chedid

42

>>> spark-submit test.py
File "<stdin>", line 1
spark-submit test.py
 ^
SyntaxError: invalid syntax

share|improve this question

asked Mar 8 at 17:23

Spencer ChedidSpencer Chedid

42

share|improve this question

asked Mar 8 at 17:23

Spencer ChedidSpencer Chedid

42

asked Mar 8 at 17:23

Spencer ChedidSpencer Chedid

42

asked Mar 8 at 17:23

Spencer ChedidSpencer Chedid

42