pyspark-sql: print alias of an expression2019 Community Moderator ElectionHow to get name of dataframe column in pyspark?How do I add a new column to a Spark DataFrame (using PySpark)?Transforming PySpark RDD with Scaladeleting particular row in particular condition in pysparkUsing a Scala UDF in PySparkPySpark streaming: window and transformReplacing blanks with Null in PySparkTrouble porting Scala spark code to PySparkDecoding a Column and Extracting Into Several Columns using PySparkWrite PySpark Dataframe to SQL DB as batchStateful aggregation function in PySpark

Can the Witch Sight warlock invocation see through the Mirror Image spell?

If nine coins are tossed, what is the probability that the number of heads is even?

Locked Away- What am I?

"If + would" conditional in present perfect tense

The (Easy) Road to Code

Is there stress on two letters on the word стоят

Either of .... (Plural/Singular)

Do Paladin Auras of Differing Oaths Stack?

What would be the most expensive material to an intergalactic society?

Why is there an extra space when I type "ls" on the Desktop?

Having the player face themselves after the mid-game

Why aren't there more Gauls like Obelix?

Does the US political system, in principle, allow for a no-party system?

Short scifi story where reproductive organs are converted to produce "materials", pregnant protagonist is "found fit" to be a mother

Logistic regression BIC: what's the right N?

What is the "determinant" of two vectors?

Why do phishing e-mails use faked e-mail addresses instead of the real one?

Was it really inappropriate to write a pull request for the company I interviewed with?

Translation of 答えを知っている人はいませんでした

Movie: boy escapes the real world and goes to a fantasy world with big furry trolls

What is Tony Stark injecting into himself in Iron Man 3?

Why does Central Limit Theorem break down in my simulation?

How do you make a gun that shoots melee weapons and/or swords?

Do black holes violate the conservation of mass?



pyspark-sql: print alias of an expression



2019 Community Moderator ElectionHow to get name of dataframe column in pyspark?How do I add a new column to a Spark DataFrame (using PySpark)?Transforming PySpark RDD with Scaladeleting particular row in particular condition in pysparkUsing a Scala UDF in PySparkPySpark streaming: window and transformReplacing blanks with Null in PySparkTrouble porting Scala spark code to PySparkDecoding a Column and Extracting Into Several Columns using PySparkWrite PySpark Dataframe to SQL DB as batchStateful aggregation function in PySpark










0















In pyspark I have the following:



import pyspark.sql.functions as F
cc = F.lit(1).alias("A")

print(cc)
print(cc._jc.toString())


I get :



Column<b'1 AS `A`'>
1 AS `A`


Is there any way for me to just print "A" from cc ? it seems I'm unable to extract the alias easily.



Also I think that in spark-sql in scala, if I print "cc" it would just print "A" instead










share|improve this question
























  • This post will guide you in the right direction stackoverflow.com/questions/39746752/…. However,based on the docs and this answer it seems like there is no way to do this without parsing the "AS" etc.

    – Nadim Younes
    Mar 7 at 0:59
















0















In pyspark I have the following:



import pyspark.sql.functions as F
cc = F.lit(1).alias("A")

print(cc)
print(cc._jc.toString())


I get :



Column<b'1 AS `A`'>
1 AS `A`


Is there any way for me to just print "A" from cc ? it seems I'm unable to extract the alias easily.



Also I think that in spark-sql in scala, if I print "cc" it would just print "A" instead










share|improve this question
























  • This post will guide you in the right direction stackoverflow.com/questions/39746752/…. However,based on the docs and this answer it seems like there is no way to do this without parsing the "AS" etc.

    – Nadim Younes
    Mar 7 at 0:59














0












0








0








In pyspark I have the following:



import pyspark.sql.functions as F
cc = F.lit(1).alias("A")

print(cc)
print(cc._jc.toString())


I get :



Column<b'1 AS `A`'>
1 AS `A`


Is there any way for me to just print "A" from cc ? it seems I'm unable to extract the alias easily.



Also I think that in spark-sql in scala, if I print "cc" it would just print "A" instead










share|improve this question
















In pyspark I have the following:



import pyspark.sql.functions as F
cc = F.lit(1).alias("A")

print(cc)
print(cc._jc.toString())


I get :



Column<b'1 AS `A`'>
1 AS `A`


Is there any way for me to just print "A" from cc ? it seems I'm unable to extract the alias easily.



Also I think that in spark-sql in scala, if I print "cc" it would just print "A" instead







pyspark apache-spark-sql pyspark-sql






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 6 at 23:08







lezebulon

















asked Mar 6 at 22:45









lezebulonlezebulon

3,28373063




3,28373063












  • This post will guide you in the right direction stackoverflow.com/questions/39746752/…. However,based on the docs and this answer it seems like there is no way to do this without parsing the "AS" etc.

    – Nadim Younes
    Mar 7 at 0:59


















  • This post will guide you in the right direction stackoverflow.com/questions/39746752/…. However,based on the docs and this answer it seems like there is no way to do this without parsing the "AS" etc.

    – Nadim Younes
    Mar 7 at 0:59

















This post will guide you in the right direction stackoverflow.com/questions/39746752/…. However,based on the docs and this answer it seems like there is no way to do this without parsing the "AS" etc.

– Nadim Younes
Mar 7 at 0:59






This post will guide you in the right direction stackoverflow.com/questions/39746752/…. However,based on the docs and this answer it seems like there is no way to do this without parsing the "AS" etc.

– Nadim Younes
Mar 7 at 0:59













0






active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55033390%2fpyspark-sql-print-alias-of-an-expression%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55033390%2fpyspark-sql-print-alias-of-an-expression%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Can't initialize raids on a new ASUS Prime B360M-A motherboard2019 Community Moderator ElectionSimilar to RAID config yet more like mirroring solution?Can't get motherboard serial numberWhy does the BIOS entry point start with a WBINVD instruction?UEFI performance Asus Maximus V Extreme

Identity Server 4 is not redirecting to Angular app after login2019 Community Moderator ElectionIdentity Server 4 and dockerIdentityserver implicit flow unauthorized_clientIdentityServer Hybrid Flow - Access Token is null after user successful loginIdentity Server to MVC client : Page Redirect After loginLogin with Steam OpenId(oidc-client-js)Identity Server 4+.NET Core 2.0 + IdentityIdentityServer4 post-login redirect not working in Edge browserCall to IdentityServer4 generates System.NullReferenceException: Object reference not set to an instance of an objectIdentityServer4 without HTTPS not workingHow to get Authorization code from identity server without login form

2005 Ahvaz unrest Contents Background Causes Casualties Aftermath See also References Navigation menue"At Least 10 Are Killed by Bombs in Iran""Iran"Archived"Arab-Iranians in Iran to make April 15 'Day of Fury'"State of Mind, State of Order: Reactions to Ethnic Unrest in the Islamic Republic of Iran.10.1111/j.1754-9469.2008.00028.x"Iran hangs Arab separatists"Iran Overview from ArchivedConstitution of the Islamic Republic of Iran"Tehran puzzled by forged 'riots' letter""Iran and its minorities: Down in the second class""Iran: Handling Of Ahvaz Unrest Could End With Televised Confessions""Bombings Rock Iran Ahead of Election""Five die in Iran ethnic clashes""Iran: Need for restraint as anniversary of unrest in Khuzestan approaches"Archived"Iranian Sunni protesters killed in clashes with security forces"Archived