|
五 05
|
IMG_3744照片 182Views 1 Comment »
|
|
四 28
|
假设有Service方法类似UserService
1 2 3 4 5 6 7 | package com.hjide.aop; public interface UserService { public User getUser(int id) throws Exception; public void save(User u) throws Exception; public int delete(User u) throws Exception; } |
实现类似如下
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | package com.hjide.aop; public class UserServiceImpl implements UserService { public int delete(User u) throws Exception{ System.out.println("delete"); return 0; } public User getUser(int id) throws Exception { User u = new User(); u.setId(1); u.setName("new user"); u.setType(2); System.out.println("getUser"); return u; } public void save(User u) throws Exception { System.out.println("save"); } } |
|
四 22
|
1、下載kevinweil-hadoop-lzo
2、rpm -ivh liblzo2_2-2.03-6.el4.x86_64.rpm
3、rpm -ivh libminilzo2-2.03-6.el4.x86_64.rpm
4、rpm -ivh lzo-2.03-6.el4.x86_64.rpm
5、rpm -ivh lzo-devel-2.03-6.el4.i386.rpm
6、cd kevinweil-hadoop-lzo
ant compile-native tar
7、進入build目錄下,把hadoop-lzo-0.4.0.jar拷貝到hadoop的lib目錄,native目錄也拷貝到hadoop的lib目錄下
編輯core-stie.xml文件,增加
<property>
<name>io.compression.codecs</name>
<value>com.hadoop.compression.lzo.LzopCodec</value>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
使map的中間結果也是用lzo壓縮,編輯mapred-site.xml文件,增加
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
</property>
<property>
<name>mapred.map.output.compression.codec</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
7、安裝lzop
rpm -ivh –force lzo-1.08-4.2.el4.rf.x86_64.rpm
rpm -ivh lzop-1.01-2.el4.rf.x86_64.rpm
8、Indexing LZO Files
使用lzop壓縮日誌文件后傳到hdfs上
index it in-process via:
hadoop jar /path/to/your/hadoop-lzo.jar com.hadoop.compression.lzo.LzoIndexer big_file.lzo
index it in a map-reduce job via:
hadoop jar /path/to/your/hadoop-lzo.jar com.hadoop.compression.lzo.DistributedLzoIndexer big_file.lzo
將TextInputFormat 修改為 LzoTextInputFormat
|
四 22
|
1、下載安裝fuse。Linux kernel > 2.6.9
downloaad fuse-2.8.3.tar.gz
解壓configure & make & make install
2、modprobe fuse
3、export JAVA_HOME=
4、export HADOOP_HOME=
5、vi $HADOOP_HOME/src/contrib/fuse-dfs/src/fuse_dfs_wrapper.sh
修改HADOOP_HOME, JAVA_HOME
6、cd $HADOOP_HOME
7、ant compile -Dcompile.c++=true -Dlibhdfs=true
確保你的autoconf版本>2.61,ant 版本>1.7.1,給$HADOOP_HOME/src/contrib/fuse-dfs/configure,$HADOOP_HOME/src/c++/pipes/configure,$HADOOP_HOMEsrc/c++/utils/configure增加執行權限 (chmod +x FILE)
8、ln -s $HADOOP_HOME/build/c++/Linux-amd64-64/lib $HADOOP_HOME/build/libhdfs
9、export LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:$HADOOP_HOME/build/libhdfs:$JAVA_HOME/jre/lib/amd64/server
10、ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1
如果以上都執行正常那就可以開始試著mount了
11、mkdir /export/hdfs (or wherever you want to mount it)
12、$HADOOP_HOME/build/contrib/fuse-dfs/fuse_dfs_wrapper.sh dfs://hadoop_server1.foo.com:9000 /export/hdfs
如果有錯誤看看這個目錄下這2個文件有沒有執行權fuse_dfs fuse_dfs_wrapper.sh
|
四 04
|










近期评论