30

修改配置文件,使用block类型的压缩,压缩方式有几种可以选择,这里用的是gzip压缩。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<property>
  <name>mapred.output.compress</name>
  <value>true</value>
  <description>Should the job outputs be compressed?
  </description>
</property>
 
<property>
  <name>mapred.output.compression.type</name>
  <value>BLOCK</value>
  <description>If the job outputs are to compressed as SequenceFiles, how should
               they be compressed? Should be one of NONE, RECORD or BLOCK.
  </description>
</property>
 
<property>
  <name>mapred.output.compression.codec</name>
  <value>org.apache.hadoop.io.compress.GzipCodec</value>
  <description>If the job outputs are compressed, how should they be compressed?
  </description>
</property>

BZip2Codec,DefaultCodec

作者:Jock

30

执行hadoop balancer出现空指针异常

配置文件fs.default.name
加上端口号,问题解决。。。

作者:Jock

Switch to our mobile site