How to pass column names having spaces to sqoop --map-column-java2019 Community Moderator ElectionHow to support column names with spaces using sqoop import?Facing issue for the below command in sqoopsqoop to transfer data to HDFS from TeradataIncreasing number of mappers in sqoop command gives java heap space errorsqoop to mysql importing issuesqoop - connect to oracle and import data to HDFS in IBM BigInsightssqoop export failed after successful mapSqoop import error message ERROR tool.ImportTool: Import failed: ENOENT: No such file or directory Optionssqoop error : Direct import is not compatible with HCatalog operationssqoop incremental append last modifiedTo use this version of Sqoop, you must downgrade your metadata schema

Is a party consisting of only a bard, a cleric, and a warlock functional long-term?

If curse and magic is two sides of the same coin, why the former is forbidden?

Look at your watch and tell me what time is it. vs Look at your watch and tell me what time it is

Interplanetary conflict, some disease destroys the ability to understand or appreciate music

Charles Hockett - 'F' article?

How to simplify this time periods definition interface?

A Cautionary Suggestion

A sequence that has integer values for prime indexes only:

What is the rarity of this homebrew magic staff?

Should we release the security issues we found in our product as CVE or we can just update those on weekly release notes?

How to deal with taxi scam when on vacation?

Hacking a Safe Lock after 3 tries

Python if-else code style for reduced code for rounding floats

How can you use ICE tables to solve multiple coupled equilibria?

Recruiter wants very extensive technical details about all of my previous work

Have researchers managed to "reverse time"? If so, what does that mean for physics?

A link redirect to http instead of https: how critical is it?

Are ETF trackers fundamentally better than individual stocks?

Why do Australian milk farmers need to protest supermarkets' milk price?

Do the common programs (for example: "ls", "cat") in Linux and BSD come from the same source code?

How can I track script which gives me "command not found" right after the login?

Does Mathematica reuse previous computations?

How to change two letters closest to a string and one letter immediately after a string using notepad++

How to create the Curved texte?



How to pass column names having spaces to sqoop --map-column-java



2019 Community Moderator ElectionHow to support column names with spaces using sqoop import?Facing issue for the below command in sqoopsqoop to transfer data to HDFS from TeradataIncreasing number of mappers in sqoop command gives java heap space errorsqoop to mysql importing issuesqoop - connect to oracle and import data to HDFS in IBM BigInsightssqoop export failed after successful mapSqoop import error message ERROR tool.ImportTool: Import failed: ENOENT: No such file or directory Optionssqoop error : Direct import is not compatible with HCatalog operationssqoop incremental append last modifiedTo use this version of Sqoop, you must downgrade your metadata schema










0















I have to import data using sqoop, my source column names are having spaces in between them, so while I am adding it in --map-column-java parameter getting the error.



Sample Sqoop import:




sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java data col1=String, data col2=String, data col3=String --as-avrodatafile




Column names:



data col1,
data col2,
data col3


Error:




19/03/07 07:31:55 DEBUG sqoop.Sqoop: Malformed mapping. Column mapping should be the form key=value[,key=value]*
java.lang.IllegalArgumentException: Malformed mapping. Column mapping should be the form key=value[,key=value]*
at org.apache.sqoop.SqoopOptions.parseColumnMapping(SqoopOptions.java:1355)
at org.apache.sqoop.SqoopOptions.setMapColumnJava(SqoopOptions.java:1375)
at org.apache.sqoop.tool.BaseSqoopTool.applyCodeGenOptions(BaseSqoopTool.java:1363)
at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:1011)
at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:435)
at org.apache.sqoop.Sqoop.run(Sqoop.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Malformed mapping. Column mapping should be the form key=value[,key=value]*











share|improve this question
























  • This answer will help you. sqoop issue with space column name

    – H Roy
    Mar 7 at 14:18











  • I tried with that by passing data col getting col not found error.

    – Prabhanj
    Mar 7 at 14:34











  • Tried as below : sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "datacol1=String, datacol2=String, datacol3=String"--as-avrodatafile getting error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 7 at 14:34












  • Tried passing all columns in --map-column-java , still throwing error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 8 at 6:08











  • we have 3 columns of type 2005: Found COLNAME of type [2005, 2147483647, 0]. I see that the datatype for that COL in SQL server is nvarchar with max_length = -1 so i tried with map-col-java on those col to convert them to string. Still getting same error.

    – Prabhanj
    Mar 8 at 10:43















0















I have to import data using sqoop, my source column names are having spaces in between them, so while I am adding it in --map-column-java parameter getting the error.



Sample Sqoop import:




sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java data col1=String, data col2=String, data col3=String --as-avrodatafile




Column names:



data col1,
data col2,
data col3


Error:




19/03/07 07:31:55 DEBUG sqoop.Sqoop: Malformed mapping. Column mapping should be the form key=value[,key=value]*
java.lang.IllegalArgumentException: Malformed mapping. Column mapping should be the form key=value[,key=value]*
at org.apache.sqoop.SqoopOptions.parseColumnMapping(SqoopOptions.java:1355)
at org.apache.sqoop.SqoopOptions.setMapColumnJava(SqoopOptions.java:1375)
at org.apache.sqoop.tool.BaseSqoopTool.applyCodeGenOptions(BaseSqoopTool.java:1363)
at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:1011)
at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:435)
at org.apache.sqoop.Sqoop.run(Sqoop.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Malformed mapping. Column mapping should be the form key=value[,key=value]*











share|improve this question
























  • This answer will help you. sqoop issue with space column name

    – H Roy
    Mar 7 at 14:18











  • I tried with that by passing data col getting col not found error.

    – Prabhanj
    Mar 7 at 14:34











  • Tried as below : sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "datacol1=String, datacol2=String, datacol3=String"--as-avrodatafile getting error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 7 at 14:34












  • Tried passing all columns in --map-column-java , still throwing error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 8 at 6:08











  • we have 3 columns of type 2005: Found COLNAME of type [2005, 2147483647, 0]. I see that the datatype for that COL in SQL server is nvarchar with max_length = -1 so i tried with map-col-java on those col to convert them to string. Still getting same error.

    – Prabhanj
    Mar 8 at 10:43













0












0








0








I have to import data using sqoop, my source column names are having spaces in between them, so while I am adding it in --map-column-java parameter getting the error.



Sample Sqoop import:




sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java data col1=String, data col2=String, data col3=String --as-avrodatafile




Column names:



data col1,
data col2,
data col3


Error:




19/03/07 07:31:55 DEBUG sqoop.Sqoop: Malformed mapping. Column mapping should be the form key=value[,key=value]*
java.lang.IllegalArgumentException: Malformed mapping. Column mapping should be the form key=value[,key=value]*
at org.apache.sqoop.SqoopOptions.parseColumnMapping(SqoopOptions.java:1355)
at org.apache.sqoop.SqoopOptions.setMapColumnJava(SqoopOptions.java:1375)
at org.apache.sqoop.tool.BaseSqoopTool.applyCodeGenOptions(BaseSqoopTool.java:1363)
at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:1011)
at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:435)
at org.apache.sqoop.Sqoop.run(Sqoop.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Malformed mapping. Column mapping should be the form key=value[,key=value]*











share|improve this question
















I have to import data using sqoop, my source column names are having spaces in between them, so while I am adding it in --map-column-java parameter getting the error.



Sample Sqoop import:




sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java data col1=String, data col2=String, data col3=String --as-avrodatafile




Column names:



data col1,
data col2,
data col3


Error:




19/03/07 07:31:55 DEBUG sqoop.Sqoop: Malformed mapping. Column mapping should be the form key=value[,key=value]*
java.lang.IllegalArgumentException: Malformed mapping. Column mapping should be the form key=value[,key=value]*
at org.apache.sqoop.SqoopOptions.parseColumnMapping(SqoopOptions.java:1355)
at org.apache.sqoop.SqoopOptions.setMapColumnJava(SqoopOptions.java:1375)
at org.apache.sqoop.tool.BaseSqoopTool.applyCodeGenOptions(BaseSqoopTool.java:1363)
at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:1011)
at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:435)
at org.apache.sqoop.Sqoop.run(Sqoop.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Malformed mapping. Column mapping should be the form key=value[,key=value]*








hadoop sqoop avro






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 7 at 14:04









emix

6,76553150




6,76553150










asked Mar 7 at 13:47









PrabhanjPrabhanj

2818




2818












  • This answer will help you. sqoop issue with space column name

    – H Roy
    Mar 7 at 14:18











  • I tried with that by passing data col getting col not found error.

    – Prabhanj
    Mar 7 at 14:34











  • Tried as below : sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "datacol1=String, datacol2=String, datacol3=String"--as-avrodatafile getting error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 7 at 14:34












  • Tried passing all columns in --map-column-java , still throwing error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 8 at 6:08











  • we have 3 columns of type 2005: Found COLNAME of type [2005, 2147483647, 0]. I see that the datatype for that COL in SQL server is nvarchar with max_length = -1 so i tried with map-col-java on those col to convert them to string. Still getting same error.

    – Prabhanj
    Mar 8 at 10:43

















  • This answer will help you. sqoop issue with space column name

    – H Roy
    Mar 7 at 14:18











  • I tried with that by passing data col getting col not found error.

    – Prabhanj
    Mar 7 at 14:34











  • Tried as below : sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "datacol1=String, datacol2=String, datacol3=String"--as-avrodatafile getting error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 7 at 14:34












  • Tried passing all columns in --map-column-java , still throwing error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

    – Prabhanj
    Mar 8 at 6:08











  • we have 3 columns of type 2005: Found COLNAME of type [2005, 2147483647, 0]. I see that the datatype for that COL in SQL server is nvarchar with max_length = -1 so i tried with map-col-java on those col to convert them to string. Still getting same error.

    – Prabhanj
    Mar 8 at 10:43
















This answer will help you. sqoop issue with space column name

– H Roy
Mar 7 at 14:18





This answer will help you. sqoop issue with space column name

– H Roy
Mar 7 at 14:18













I tried with that by passing data col getting col not found error.

– Prabhanj
Mar 7 at 14:34





I tried with that by passing data col getting col not found error.

– Prabhanj
Mar 7 at 14:34













Tried as below : sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "datacol1=String, datacol2=String, datacol3=String"--as-avrodatafile getting error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

– Prabhanj
Mar 7 at 14:34






Tried as below : sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "datacol1=String, datacol2=String, datacol3=String"--as-avrodatafile getting error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

– Prabhanj
Mar 7 at 14:34














Tried passing all columns in --map-column-java , still throwing error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

– Prabhanj
Mar 8 at 6:08





Tried passing all columns in --map-column-java , still throwing error: ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005

– Prabhanj
Mar 8 at 6:08













we have 3 columns of type 2005: Found COLNAME of type [2005, 2147483647, 0]. I see that the datatype for that COL in SQL server is nvarchar with max_length = -1 so i tried with map-col-java on those col to convert them to string. Still getting same error.

– Prabhanj
Mar 8 at 10:43





we have 3 columns of type 2005: Found COLNAME of type [2005, 2147483647, 0]. I see that the datatype for that COL in SQL server is nvarchar with max_length = -1 so i tried with map-col-java on those col to convert them to string. Still getting same error.

– Prabhanj
Mar 8 at 10:43












1 Answer
1






active

oldest

votes


















0














Able to resolve this issue:



1. Spaces issue:
sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile



2. ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005:
3 columns in source are having 2005 and nvarchar added them in --map-column-java resolved this issue



3. org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 1****



This is causing due to using * in select query, so modified sqoop query as:



sqoop import --connect jdbc-con --username "user1" --query "select [col1,data col2,data col3] from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile






share|improve this answer






















    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55045356%2fhow-to-pass-column-names-having-spaces-to-sqoop-map-column-java%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    Able to resolve this issue:



    1. Spaces issue:
    sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile



    2. ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005:
    3 columns in source are having 2005 and nvarchar added them in --map-column-java resolved this issue



    3. org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 1****



    This is causing due to using * in select query, so modified sqoop query as:



    sqoop import --connect jdbc-con --username "user1" --query "select [col1,data col2,data col3] from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile






    share|improve this answer



























      0














      Able to resolve this issue:



      1. Spaces issue:
      sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile



      2. ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005:
      3 columns in source are having 2005 and nvarchar added them in --map-column-java resolved this issue



      3. org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 1****



      This is causing due to using * in select query, so modified sqoop query as:



      sqoop import --connect jdbc-con --username "user1" --query "select [col1,data col2,data col3] from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile






      share|improve this answer

























        0












        0








        0







        Able to resolve this issue:



        1. Spaces issue:
        sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile



        2. ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005:
        3 columns in source are having 2005 and nvarchar added them in --map-column-java resolved this issue



        3. org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 1****



        This is causing due to using * in select query, so modified sqoop query as:



        sqoop import --connect jdbc-con --username "user1" --query "select [col1,data col2,data col3] from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile






        share|improve this answer













        Able to resolve this issue:



        1. Spaces issue:
        sqoop import --connect jdbc-con --username "user1" --query "select * from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile



        2. ERROR tool.ImportTool: Import failed: Cannot convert SQL type 2005:
        3 columns in source are having 2005 and nvarchar added them in --map-column-java resolved this issue



        3. org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 1****



        This is causing due to using * in select query, so modified sqoop query as:



        sqoop import --connect jdbc-con --username "user1" --query "select [col1,data col2,data col3] from table where $CONDITIONS" --target-dir /target/path/ -m 1 --map-column-java "data col1=String, data col2=String, data col3=String" --as-avrodatafile







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Mar 8 at 11:44









        PrabhanjPrabhanj

        2818




        2818





























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55045356%2fhow-to-pass-column-names-having-spaces-to-sqoop-map-column-java%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            How to get text form Clipboard with JavaScript in Firefox 56?How to validate an email address in JavaScript?How do JavaScript closures work?How do I remove a property from a JavaScript object?How do you get a timestamp in JavaScript?How do I copy to the clipboard in JavaScript?How do I include a JavaScript file in another JavaScript file?Get the current URL with JavaScript?How to replace all occurrences of a string in JavaScriptHow to check whether a string contains a substring in JavaScript?How do I remove a particular element from an array in JavaScript?

            Can't initialize raids on a new ASUS Prime B360M-A motherboard2019 Community Moderator ElectionSimilar to RAID config yet more like mirroring solution?Can't get motherboard serial numberWhy does the BIOS entry point start with a WBINVD instruction?UEFI performance Asus Maximus V Extreme

            List of MPs elected to the English parliament in 1640 (April) Contents List of constituencies and members See also Notes References Navigation menueNational Archives – The Glynde Place ArchivesCobbett's Parliamentary history of England, from the Norman Conquest in 1066 to the year 1803'Aldermen in Parliament', The Aldermen of the City of London: Temp. Henry III – 1912onepage&q&f&#61, false 229