Questions regarding the submission of jar files #7337
Unanswered
shashihoushengqia
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I found that every time I submitted a Spark on K8s task through Kyuubi, a file named "kyuubi-spark-sql-engine_2.12-1.10.3.jar" would be uploaded to S3/HDFS. This would occupy a large amount of space on S3/HDFS, and uploading it every time would also slow down the startup speed of Spark. So I placed this jar file in /opt/spark/jars/ in all the Spark containers, and also in /opt/kyuubi/jars in the Kyuubi container. However, Kyuubi still uploads this jar file to S3/HDFS. So, experts, how should we solve this problem?
Beta Was this translation helpful? Give feedback.
All reactions