六 30
Hadoop对输入输出进行压缩
Hadoop 492 views, No Comments
修改配置文件,使用block类型的压缩,压缩方式有几种可以选择,这里用的是gzip压缩。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | <property> <name>mapred.output.compress</name> <value>true</value> <description>Should the job outputs be compressed? </description> </property> <property> <name>mapred.output.compression.type</name> <value>BLOCK</value> <description>If the job outputs are to compressed as SequenceFiles, how should they be compressed? Should be one of NONE, RECORD or BLOCK. </description> </property> <property> <name>mapred.output.compression.codec</name> <value>org.apache.hadoop.io.compress.GzipCodec</value> <description>If the job outputs are compressed, how should they be compressed? </description> </property> |
BZip2Codec,DefaultCodec
近期评论