消息关闭
    暂无新消息!

问题来源是这样,我使用sqoop向Hive中导入mysql的表

sqoop import --connect jdbc:mysql://127.0.0.1:3306/employees_db --table titles --username root -P --hive-import -- --default-character-set=utf-8

然后发生了报错:

16/08/10 22:08:36 ERROR tool.ImportTool: Encountered IOException running import job: java.io.FileNotFoundException: File does not exist: hdfs://192.168.1.128:9000/usr/local/sqoop/lib/mysql-connector-java-5.1.39-bin.jar

于是我Google了一下,发现清一色的回答均为在hdfs中创建一个/usr/local/sqoop,然后将本地的sqoopput上去,即

[root@Master local]# hadoop dfs -put /usr/local/sqoop /usr/local
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

16/08/10 22:23:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@Master local]# hadoop dfs -ls /usr/local
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

16/08/10 22:25:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 1 items
drwxr-xr-x   - root supergroup          0 2016-08-10 22:25 /usr/local/sqoop

没错,这样是成功解决了问题
但是为什么会把sqoop存到hdfs中?我看到了有些人并不需要把sqoop移到hdfs中也能成功运行,是我哪里的配置出错还是只能这么做?hdfs是分布式储存应该是存数据文件,怎么会把一个程序存进去呢?这里令我很不解,希望能得到解答。


1个回答

︿ 2

关于这个问题,貌似找到了一个解释,这里报错是缺少hdfs://192.168.1.128:9000/usr/local/sqoop/lib/mysql-connector-java-5.1.39-bin.jar,所以我的理解就是并不是sqoop在HDFS上运行,而是sqoop的jar要放在HDFS上,所以我们只用把jar上传到HDFS即可