四季如風

原创内容转载请注明来源 http://www.hjide.com/article/479.htm

从网上找的源代码，发现有很多问题。

首先是每次查找都是通过RandomAccessFile的文件指针的移动来定位，修改为增加一个MappedByteBuffer的内存映射文件试图提高查找效率，

接着发现在多线程环境下会出现问题，发现用来记录指针的变量是类的成员对象，多线程不能共用一个实例，增加一个同步锁就有违修改的初衷。

于是经过大量修改后有了现在这个版本，源代码请移步https://github.com/jockhuang/iplocation 下载。

作者：Jock

2个文件需要修改

第一个文件：/root/nginx-1.0.11/src/core/ngx_times.c

1、（计算字符串的长度？）

static u_char cached_http_log_time[NGX_TIME_SLOTS]
[sizeof("28/Sep/1970:12:00:00 +0600")];

修改为

static u_char cached_http_log_time[NGX_TIME_SLOTS]
[sizeof("1970-09-28 12:00:00")];

2、（计算字符串的长度？）

ngx_cached_http_log_time.len = sizeof(“28/Sep/1970:12:00:00 +0600“) – 1;

修改为

ngx_cached_http_log_time.len = sizeof(“1970-09-28 12:00:00“) – 1;

3、（关键的地方，修改格式）

p2 = &cached_http_log_time[slot][0];

(void) ngx_sprintf(p2, “%02d/%s/%d:%02d:%02d:%02d %c%02d%02d”,
tm.ngx_tm_mday, months[tm.ngx_tm_mon - 1],
tm.ngx_tm_year, tm.ngx_tm_hour,
tm.ngx_tm_min, tm.ngx_tm_sec,

修改为

p2 = &cached_http_log_time[slot][0];

(void) ngx_sprintf(p2, “%4d-%02d-%02d %02d:%02d:%02d”,
tm.ngx_tm_year, tm.ngx_tm_mon,
tm.ngx_tm_mday, tm.ngx_tm_hour,
tm.ngx_tm_min, tm.ngx_tm_sec,

第二个文件：/root/nginx-1.0.11/src/http/modules/ngx_http_log_module.c

{ ngx_string(“time_local”), sizeof(“28/Sep/1970:12:00:00 +0600“) – 1,
修改为
{ ngx_string(“time_local”), sizeof(“1970-09-28 12:00:00“) – 1,

作者：Jock

Lucene是可以做到的，利用lucene的Filter，具体可以查看lucene的api中的org.apache.lucene.search.CachingWrapperFilter，它可以缓存上次的搜索结果，从而实现在结果中的搜索。

测试实例：
package com.wsjava;
import java.io.IOException;
import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.queryParser.ParseException;
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.search.CachingWrapperFilter;
import org.apache.lucene.search.Filter;
import org.apache.lucene.search.Hits;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.QueryFilter;

public class IndexTest {

        /**
         * @param args
         * @throws ParseException
         * @throws IOException
         */
        public static void main(String[] args) throws IOException, ParseException {
                index();
                search(“day”); //简单搜索
                searchInResult(“day”, “you”); //在结果集中搜索
        }
        
        public static void index() throws IOException {
                IndexWriter writer = new IndexWriter(“d:/tesindex”,new SimpleAnalyzer(), true);
                writer.setMaxMergeDocs(1000);
                writer.setMergeFactor(100);
                for (int i = 0; i < 10; i++) {
                        Document doc = new Document();
                        String content = “How do you do?”;
                        if (i >= 5) {
                                content = “What's a good day. “;
                        }
                        if (i >= 7) {
                                content = “Nice day. Thanks you!”;
                        }
                        doc.add(new Field(“content”, content, Field.Store.YES,Field.Index.TOKENIZED));
                        writer.addDocument(doc);
                }

        }
        
        //简单实现对qw的搜索.
        public static void search(String qw) throws IOException, ParseException {
                QueryParser queryParser = new QueryParser(“content”,new SimpleAnalyzer());
                Query query = queryParser.parse(qw.trim());
                QueryFilter filter = new QueryFilter(query);
                
                search(query, filter);
        }
        
        //在搜索oldqw的结果集中搜索qw.
        public static void searchInResult(String qw, String oldqw) throws ParseException, IOException {                
                QueryParser queryParser = new QueryParser(“content”,new SimpleAnalyzer());
                Query query = queryParser.parse(qw.trim());
                Query oldQuery = queryParser.parse(oldqw.trim());
                QueryFilter oldFilter = new QueryFilter(oldQuery);
                CachingWrapperFilter filter = new CachingWrapperFilter(oldFilter);
                
                search(query, filter);
        }
        
        private static void search(Query query, Filter filter) throws IOException, ParseException {
                IndexSearcher ins = new IndexSearcher(“d:/tesindex”);
                Hits hits = ins.search(query, filter);
                for (int i = 0; i < hits.length(); i++) {
                        Document doc = hits.doc(i);
                        System.out.println(doc.get(“content”));
                }
                System.out.println();
        }
}

上面是简单的测试程序。当然在实际应用中可以做得比较复杂。

作者：Jock

200W以下的数据量全部加载到内存最简单的方式是修改Lucene（1.9版本）源码 org.apache.lucene.index.IndexReader文件的第127行 将
return open(FSDirectory.getDirectory(path, false), true);
修改为
return open(new RAMDirectory(FSDirectory.getDirectory(path, false)), true);

将133行的
return open(FSDirectory.getDirectory(path, false), true);
修改为
return open(new RAMDirectory( FSDirectory.getDirectory(path, false)), true);
这样就可以在查询的时候将所有索引一次加载到内存，查询响应时间即使是第一次查询，也会小于0.1秒，但仅适用于索引文档数量不超过200W个，并且平均的页面大小不超过10K，文献检索系统 ，这种方式尤其适合。
创建索引的时候也能够使用 RAMDirectory ， 比FSDirectory 要快很多 。

令一种方法：

FSDirectory directory = FSDirectory.getDirectory(args[0], false);   1

RAMDirectory directory = new RAMDirectory(args[0]);                 2

用2代替1即可

作者：Jock

PriorityQueue  实现一优先队列框架，实例非常简单，只需实现lessThan(Object a, Object b)方法即可，通过该方法可以控制大优先或小优先。
更多详细内容 »

作者：Jock

HitCollector 的作用很简单，通过collect（）方法控制检索返回的结果，下面是lucene自带的一个例子—-使用一个优先队
列，返回指定数目的Top n Doc。
更多详细内容 »

作者：Jock

首先当然是引入webworkxxx.jar,xwork.jar,common logging….

web.xml加入如下配置


webwork

com.opensymphony.webwork.dispatcher.FilterDispatcher

webwork
/*

更多详细内容 »

作者：Jock

首先来2个最简单的bean

package example;
 
/**
* @author jockhuang
*
*/
public class TestA {
 
public void sayHello(){
    System.out.println("TestA");
}
 
}

package example;
 
public class TestB {
    public void sayHi(){
        System.out.println("TestB");
    }
}

2个很简单的bean，applicationContext.xml中的配置如下：

< ?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:aop="http://www.springframework.org/schema/aop"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
<bean id="TestA" class="example.TestA"></bean>
<bean id="TestB" class="example.TestB"></bean>
</beans>

来个测试运行这2个bean

package example;
 
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
 
public final class TestAOP {
 
  public static void main(String[] args) {
    ApplicationContext ctx = new ClassPathXmlApplicationContext("applicationContext.xml");
    TestA a = (TestA) ctx.getBean("TestA");
    a.sayHello();
    TestB b = (TestB) ctx.getBean("TestB");
    b.sayHi();
  }
 
}

更多详细内容 »

作者：Jock

[codes=java]
/**
*
*/
package cn.org.jock.filter;

import javax.servlet.Filter;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.FilterChain;
import java.io.IOException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpSession;
import javax.servlet.http.HttpServletResponse;

/**
* @author jock
*
*/
public class AuthorizationFilter implements Filter {

 /*
  * (non-Javadoc)
  *
  * @see javax.servlet.Filter#destroy()
  */
 public void destroy() {
   // TODO Auto-generated method stub

 }

 /*
  * (non-Javadoc)
  *
  * @see javax.servlet.Filter#doFilter(javax.servlet.ServletRequest,
  *      javax.servlet.ServletResponse, javax.servlet.FilterChain)
  */
 public void doFilter(ServletRequest request, ServletResponse response,
     FilterChain chain) throws IOException, ServletException {
   HttpServletRequest req = (HttpServletRequest) request;
   HttpServletResponse res = (HttpServletResponse) response;

   HttpSession session = req.getSession(true);

   String username = (String) session.getAttribute(“username”);

   if (username == null || “”.equals(username)) {
     // 跳转到登陆页面
     res.sendRedirect(“/login.jsp”);
   } else {
     chain.doFilter(request, response);
   }

 }

 /*
  * (non-Javadoc)
  *
  * @see javax.servlet.Filter#init(javax.servlet.FilterConfig)
  */
 public void init(FilterConfig arg0) throws ServletException {
   // TODO Auto-generated method stub

 }

}

[/codes]
更多详细内容 »

作者：堕落天使

[codes=java]
package cn.org.jock.filter;

import java.io.IOException;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

public final class LogFilter implements Filter {
 private FilterConfig filterConfig = null;

 private static final Log log = LogFactory.getLog(LogFilter.class);

 public void doFilter(ServletRequest request, ServletResponse response,
     FilterChain chain) throws IOException, ServletException {

   final long start = System.currentTimeMillis();
   final String address = request.getServerName();
   
   final String file = ((HttpServletRequest) request).getRequestURI();
   chain.doFilter(request, response);

   log.info(“["+address + "_" + file + "]||[time="
       + (System.currentTimeMillis() - start)+"]“);
 }

 public void destroy() {
 }

 public void init(FilterConfig filterConfig) {
   this.filterConfig = filterConfig;
 }
}

[/codes]

web.xml的配置 更多详细内容 »

作者：堕落天使