消息关闭
    暂无新消息!
请问大家有没有在cloudera hue界面创建oozie workflow跑pyspark程序?
我模仿官网例子试
workflow详细如下:
<workflow-app name="spark-python" xmlns="uri:oozie:workflow:0.5">
    <start to="spark-3806"/>
    <kill name="Kill">
        <message>操作失败,错误消息[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <action name="spark-3806">
        <spark xmlns="uri:oozie:spark-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <master>local[*]</master>
            <mode>yarn-client</mode>
            <name>MySpark</name>
            <jar>/user/hue/oozie/workspaces/workflows/spark-python/lib/DataTest.py</jar>
              <spark-opts>--conf spark.yarn.historyServer.address=http://clouderamanager/:18088 --conf spark.eventLog.dir=user/spark/applicationHistory --conf spark.eventLog.enabled=true </spark-opts>
        </spark>
        <ok to="End"/>
        <error to="Kill"/>
    </action>
    <end name="End"/>
</workflow-app>

总是报错:Main class [org.apache.oozie.action.hadoop.SparkMain], exit code [1]

官网只看到了调度.jar包的例子https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html#Spark_on_YARN

求指导,谢谢!!

3个回答