Reflection error while creating spark session through Livy
I am creating a spark session through LivyClient.
The code for creating the session is -
var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}
While creating a session, I am getting the error -
Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here
I am using the following versions -
- Apache Livy - 0.5.0-incubating
- Apache Spark - 2.3.0
- Scala - 2.11 (2.11.8)
I have checked the scala-reflect version in all the three and they are same.
I am unable to understand what is causing the below error
scala apache-spark livy
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
add a comment |
I am creating a spark session through LivyClient.
The code for creating the session is -
var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}
While creating a session, I am getting the error -
Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here
I am using the following versions -
- Apache Livy - 0.5.0-incubating
- Apache Spark - 2.3.0
- Scala - 2.11 (2.11.8)
I have checked the scala-reflect version in all the three and they are same.
I am unable to understand what is causing the below error
scala apache-spark livy
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.
– user6910411
Nov 22 '18 at 15:01
you should pass Livy Job toclient.submitmethod
– morsik
Nov 22 '18 at 20:28
add a comment |
I am creating a spark session through LivyClient.
The code for creating the session is -
var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}
While creating a session, I am getting the error -
Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here
I am using the following versions -
- Apache Livy - 0.5.0-incubating
- Apache Spark - 2.3.0
- Scala - 2.11 (2.11.8)
I have checked the scala-reflect version in all the three and they are same.
I am unable to understand what is causing the below error
scala apache-spark livy
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
I am creating a spark session through LivyClient.
The code for creating the session is -
var client: LivyClient = null
try {
client = new LivyClientBuilder()
.setURI(new URI("http://127.0.0.1:4141"))
.build();
val pi = client.submit(new FileStream()).get
} finally {
client.stop(true)
}
While creating a session, I am getting the error -
Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
enter image description here
I am using the following versions -
- Apache Livy - 0.5.0-incubating
- Apache Spark - 2.3.0
- Scala - 2.11 (2.11.8)
I have checked the scala-reflect version in all the three and they are same.
I am unable to understand what is causing the below error
scala apache-spark livy
scala apache-spark livy
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
asked Nov 22 '18 at 14:49
Naina JainNaina Jain
1
1
Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.
– user6910411
Nov 22 '18 at 15:01
you should pass Livy Job toclient.submitmethod
– morsik
Nov 22 '18 at 20:28
add a comment |
Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.
– user6910411
Nov 22 '18 at 15:01
you should pass Livy Job toclient.submitmethod
– morsik
Nov 22 '18 at 20:28
Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.
– user6910411
Nov 22 '18 at 15:01
Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.
– user6910411
Nov 22 '18 at 15:01
you should pass Livy Job to
client.submit method– morsik
Nov 22 '18 at 20:28
you should pass Livy Job to
client.submit method– morsik
Nov 22 '18 at 20:28
add a comment |
1 Answer
1
active
oldest
votes
What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433459%2freflection-error-while-creating-spark-session-through-livy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
add a comment |
What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
add a comment |
What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
What version of Java are you using? It appears that changes in Java 9 regarding dropping of deprecated features stops Spark with errors referring to reflection. The home page says that the Java must be at least version 8.
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
answered Dec 11 '18 at 15:48
Bradley RossBradley Ross
43028
43028
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433459%2freflection-error-while-creating-spark-session-through-livy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Welcome to Stack Overflow. Images of errors are highly discouraged.Please edit your question and include the error as a text.
– user6910411
Nov 22 '18 at 15:01
you should pass Livy Job to
client.submitmethod– morsik
Nov 22 '18 at 20:28