Spark-shell Git Bash











up vote
1
down vote

favorite
1












I cannot get the spark-shell command to work on Git bash. I am fairly sure my environment is set up correctly as if I run spark-shell in any directory in command prompt, it works just fine for me.



However, when I run spark-shell in bash, it outputs this value instead of running the actual shell:



"C:Program FilesJavajdk1.8.0_191binjava" -cp "C:sparkspark-2.4.0-bin- 

hadoop2.7/conf;C:sparkspark-2.4.0-bin-hadoop2.7jars*" "- 

Dscala.usejavacp=true"-Xmx1g org.apache.spark.deploy.SparkSubmit --class 

org.apache.spark.repl.Main --name "Spark shell" spark-shell





java bash scala apache-spark git-bash







share|improve this question






















  • Do you mean you are running spark from within a shell script file? If so, can you show us that script? If not, how are you running it?
    – dave
    2 days ago










  • No, I'm just calling spark-shell without a script. I just call spark-shell in any directory and it runs in command prompt, however, for git bash, it displays the above output.
    – Foxlooo
    yesterday















up vote
1
down vote

favorite
1












I cannot get the spark-shell command to work on Git bash. I am fairly sure my environment is set up correctly as if I run spark-shell in any directory in command prompt, it works just fine for me.



However, when I run spark-shell in bash, it outputs this value instead of running the actual shell:



"C:Program FilesJavajdk1.8.0_191binjava" -cp "C:sparkspark-2.4.0-bin- 

hadoop2.7/conf;C:sparkspark-2.4.0-bin-hadoop2.7jars*" "- 

Dscala.usejavacp=true"-Xmx1g org.apache.spark.deploy.SparkSubmit --class 

org.apache.spark.repl.Main --name "Spark shell" spark-shell





java bash scala apache-spark git-bash







share|improve this question






















  • Do you mean you are running spark from within a shell script file? If so, can you show us that script? If not, how are you running it?
    – dave
    2 days ago










  • No, I'm just calling spark-shell without a script. I just call spark-shell in any directory and it runs in command prompt, however, for git bash, it displays the above output.
    – Foxlooo
    yesterday













up vote
1
down vote

favorite
1









up vote
1
down vote

favorite
1






1





I cannot get the spark-shell command to work on Git bash. I am fairly sure my environment is set up correctly as if I run spark-shell in any directory in command prompt, it works just fine for me.



However, when I run spark-shell in bash, it outputs this value instead of running the actual shell:



"C:Program FilesJavajdk1.8.0_191binjava" -cp "C:sparkspark-2.4.0-bin- 

hadoop2.7/conf;C:sparkspark-2.4.0-bin-hadoop2.7jars*" "- 

Dscala.usejavacp=true"-Xmx1g org.apache.spark.deploy.SparkSubmit --class 

org.apache.spark.repl.Main --name "Spark shell" spark-shell





java bash scala apache-spark git-bash







share|improve this question













I cannot get the spark-shell command to work on Git bash. I am fairly sure my environment is set up correctly as if I run spark-shell in any directory in command prompt, it works just fine for me.



However, when I run spark-shell in bash, it outputs this value instead of running the actual shell:



"C:Program FilesJavajdk1.8.0_191binjava" -cp "C:sparkspark-2.4.0-bin- 

hadoop2.7/conf;C:sparkspark-2.4.0-bin-hadoop2.7jars*" "- 

Dscala.usejavacp=true"-Xmx1g org.apache.spark.deploy.SparkSubmit --class 

org.apache.spark.repl.Main --name "Spark shell" spark-shell





java bash scala apache-spark git-bash




java bash scala apache-spark git-bash






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 2 days ago









Foxlooo

145




145












  • Do you mean you are running spark from within a shell script file? If so, can you show us that script? If not, how are you running it?
    – dave
    2 days ago










  • No, I'm just calling spark-shell without a script. I just call spark-shell in any directory and it runs in command prompt, however, for git bash, it displays the above output.
    – Foxlooo
    yesterday


















  • Do you mean you are running spark from within a shell script file? If so, can you show us that script? If not, how are you running it?
    – dave
    2 days ago










  • No, I'm just calling spark-shell without a script. I just call spark-shell in any directory and it runs in command prompt, however, for git bash, it displays the above output.
    – Foxlooo
    yesterday
















Do you mean you are running spark from within a shell script file? If so, can you show us that script? If not, how are you running it?
– dave
2 days ago




Do you mean you are running spark from within a shell script file? If so, can you show us that script? If not, how are you running it?
– dave
2 days ago












No, I'm just calling spark-shell without a script. I just call spark-shell in any directory and it runs in command prompt, however, for git bash, it displays the above output.
– Foxlooo
yesterday




No, I'm just calling spark-shell without a script. I just call spark-shell in any directory and it runs in command prompt, however, for git bash, it displays the above output.
– Foxlooo
yesterday












1 Answer
1






active

oldest

votes

















up vote
0
down vote













In a git bash, try and use:




  • simple quotes ' (instead of double-quotes ")


  • / instead of : C:/.../... or /c/.../...


That would be more compatible with the shell used by Git bash.






share|improve this answer





















  • To clarify, the above is the output that I am returned, I am just calling spark-shell
    – Foxlooo
    7 hours ago












  • @Foxlooo Any chance to modify the spark-shell script in order to test the syntax I propose?
    – VonC
    7 hours ago










  • I'm not sure what script it's running, I'm running it from the default built-in command from the installation of spark and scala. Also, keep in mind it works in command prompt
    – Foxlooo
    6 hours ago










  • @Foxlooo OK. I am not surprised it does work in a CMD command prompt, and not in a bash (where quotes are interpreted differently)
    – VonC
    6 hours ago











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53346696%2fspark-shell-git-bash%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
0
down vote













In a git bash, try and use:




  • simple quotes ' (instead of double-quotes ")


  • / instead of : C:/.../... or /c/.../...


That would be more compatible with the shell used by Git bash.






share|improve this answer





















  • To clarify, the above is the output that I am returned, I am just calling spark-shell
    – Foxlooo
    7 hours ago












  • @Foxlooo Any chance to modify the spark-shell script in order to test the syntax I propose?
    – VonC
    7 hours ago










  • I'm not sure what script it's running, I'm running it from the default built-in command from the installation of spark and scala. Also, keep in mind it works in command prompt
    – Foxlooo
    6 hours ago










  • @Foxlooo OK. I am not surprised it does work in a CMD command prompt, and not in a bash (where quotes are interpreted differently)
    – VonC
    6 hours ago















up vote
0
down vote













In a git bash, try and use:




  • simple quotes ' (instead of double-quotes ")


  • / instead of : C:/.../... or /c/.../...


That would be more compatible with the shell used by Git bash.






share|improve this answer





















  • To clarify, the above is the output that I am returned, I am just calling spark-shell
    – Foxlooo
    7 hours ago












  • @Foxlooo Any chance to modify the spark-shell script in order to test the syntax I propose?
    – VonC
    7 hours ago










  • I'm not sure what script it's running, I'm running it from the default built-in command from the installation of spark and scala. Also, keep in mind it works in command prompt
    – Foxlooo
    6 hours ago










  • @Foxlooo OK. I am not surprised it does work in a CMD command prompt, and not in a bash (where quotes are interpreted differently)
    – VonC
    6 hours ago













up vote
0
down vote










up vote
0
down vote









In a git bash, try and use:




  • simple quotes ' (instead of double-quotes ")


  • / instead of : C:/.../... or /c/.../...


That would be more compatible with the shell used by Git bash.






share|improve this answer












In a git bash, try and use:




  • simple quotes ' (instead of double-quotes ")


  • / instead of : C:/.../... or /c/.../...


That would be more compatible with the shell used by Git bash.







share|improve this answer












share|improve this answer



share|improve this answer










answered yesterday









VonC

820k28325763090




820k28325763090












  • To clarify, the above is the output that I am returned, I am just calling spark-shell
    – Foxlooo
    7 hours ago












  • @Foxlooo Any chance to modify the spark-shell script in order to test the syntax I propose?
    – VonC
    7 hours ago










  • I'm not sure what script it's running, I'm running it from the default built-in command from the installation of spark and scala. Also, keep in mind it works in command prompt
    – Foxlooo
    6 hours ago










  • @Foxlooo OK. I am not surprised it does work in a CMD command prompt, and not in a bash (where quotes are interpreted differently)
    – VonC
    6 hours ago


















  • To clarify, the above is the output that I am returned, I am just calling spark-shell
    – Foxlooo
    7 hours ago












  • @Foxlooo Any chance to modify the spark-shell script in order to test the syntax I propose?
    – VonC
    7 hours ago










  • I'm not sure what script it's running, I'm running it from the default built-in command from the installation of spark and scala. Also, keep in mind it works in command prompt
    – Foxlooo
    6 hours ago










  • @Foxlooo OK. I am not surprised it does work in a CMD command prompt, and not in a bash (where quotes are interpreted differently)
    – VonC
    6 hours ago
















To clarify, the above is the output that I am returned, I am just calling spark-shell
– Foxlooo
7 hours ago






To clarify, the above is the output that I am returned, I am just calling spark-shell
– Foxlooo
7 hours ago














@Foxlooo Any chance to modify the spark-shell script in order to test the syntax I propose?
– VonC
7 hours ago




@Foxlooo Any chance to modify the spark-shell script in order to test the syntax I propose?
– VonC
7 hours ago












I'm not sure what script it's running, I'm running it from the default built-in command from the installation of spark and scala. Also, keep in mind it works in command prompt
– Foxlooo
6 hours ago




I'm not sure what script it's running, I'm running it from the default built-in command from the installation of spark and scala. Also, keep in mind it works in command prompt
– Foxlooo
6 hours ago












@Foxlooo OK. I am not surprised it does work in a CMD command prompt, and not in a bash (where quotes are interpreted differently)
– VonC
6 hours ago




@Foxlooo OK. I am not surprised it does work in a CMD command prompt, and not in a bash (where quotes are interpreted differently)
– VonC
6 hours ago


















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53346696%2fspark-shell-git-bash%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

Create new schema in PostgreSQL using DBeaver

Image
0 At the beginning of my current job I had limited access to DB but recently my role is changed to superuser, admin, and owner. My colleague who was the previous DB admin is able to create a schema or change the permissions as there is a permission tab in the properties and he can change permissions by clicking easily. Would you please let me know how I can create schema and change permissions as well? Why his dbeaver looks different from me? He can easily right click on the schema folder on top of all schemas on the left side column and select create new schema while I do not have that, like the following picture I found on the internet. postgresql database-schema dbeaver share | improve this question
Read more

Deepest pit of an array with Javascript: test on Codility

Image
0 $begingroup$ This is the question from a Codility test, as a task in an interview: DeepestPit - problem description A non-empty zero-indexed array A consisting of N integers is given. A pit in this array is any triplet of integers (P, Q, R) such that: ·         0 ≤ P < Q < R < N; ·         sequence [A[P], A[P+1], ..., A[Q]] is strictly decreasing, i.e. A[P] > A[P+1] > ... > A[Q]; ·         sequence A[Q], A[Q+1], ..., A[R] is strictly increasing, i.e. A[Q] < A[Q+1] < ... < A[R]. The depth of a pit (P, Q, R) is the number min {A[P] − A[Q], A[R] − A[Q]}. For example, consider array A consisting of 10 elements such that: A[0] = 0 A[1] = 1 A[2] = 3 A[3] = -2 A[4] = 0 A[5] = 1 A[6] = 0 A[7] = -3 A[8] = 2 A[9] = 3 Triplet (2, 3, 4) is
Read more

Costa Masnaga

Image
.mw-parser-output .avviso .mbox-text-div>div,.mw-parser-output .avviso .mbox-text-full-div>div{font-size:90%}.mw-parser-output .avviso .mbox-image div{width:52px}.mw-parser-output .avviso .mbox-text-full-div .hide-when-compact{display:block} Questa voce o sezione sull'argomento Lombardia non cita le fonti necessarie o quelle presenti sono insufficienti . Puoi migliorare questa voce aggiungendo citazioni da fonti attendibili secondo le linee guida sull'uso delle fonti. Segui i suggerimenti del progetto di riferimento. Costa Masnaga comune Localizzazione Stato   Italia Regione Lombardia Provincia Lecco Amministrazione Sindaco Sabina Panzeri (Al centro il cittadino) dall'08/06/2009 Territorio Coordinate 45°46′N 9°17′E  /  45.766667°N 9.283333°E 45.766667; 9.283333  ( Costa Masnaga ) Coordinate: 45°46′N 9°17′E  /  45.766667°N 9.283333°E 45.766667; 9.283333  ( Costa Masnaga ) Altitudine 318 m s.l.m
Read more