java第一次调用 Hadoop Java API

环境：Hadoop1.2.1

例：直接使用FileSystem以标准输出格式显示Hadoop文件系统中的文件

编写JAVA文件

import org.Apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import java.io.InputStream;
import java.net.URI;
public class FileSystemCat {
 public static void main(String[] args) throws Exception {
 String uri = args[0];
 Configuration conf = new Configuration();
 FileSystem fs = FileSystem.get(URI.create(uri), conf);
 InputStream in = null;
 try {
 in = fs.open(new Path(uri));
 IOUtils.copyBytes(in, System.out, 4096, false);
 } finally {
 IOUtils.closeStream(in);
 }
 }
}

设置JAVA_HOME环境变量，将$JAVA_HOME/bin添加到PATH环境变量

设置Hadoop的类目录( HADOOP_CLASSPATH)

[grid@hadoop1 hadoop-1.2.1]# vi conf/hadoop-env.sh
# The java implementation to use. Required.
export JAVA_HOME=/usr/java/jdk1.7.0_72
# Extra Java CLASSPATH elements. Optional.
export HADOOP_CLASSPATH=/home/grid/hadoop-1.2.1/myclasses

上传编辑好的JAVA文件

[grid@hadoop1 hadoop-1.2.1]$ mkdir myclasses
[grid@hadoop1 hadoop-1.2.1]$ cd myclasses
[grid@hadoop1 myclasses]$ rz

编译java文件

[grid@hadoop1 myclasses]$ javac -classpath ../hadoop-core-1.2.1.jar FileSystemCat.java

执行：

[grid@hadoop1 hadoop-1.2.1]$ ./bin/hadoop FileSystemCat hdfs://hadoop1:9000/user/grid/in/test3.txt
Hello Hadoop Java API