最新国产好看的视频,伊人天堂AV在线,国产Aaaaaa视频,蜜臀视频在线观看一区,人妻av色图,密臀久久久精品影片,青青视频免费观看毛片,久草在线观看视,国产三级精品色情在线

java多線程處理執(zhí)行solr創(chuàng)建索引示例

 更新時(shí)間:2014年02月26日 10:29:57   作者:  
這篇文章主要介紹了java多線程處理執(zhí)行solr創(chuàng)建索引示例,需要的朋友可以參考下

復(fù)制代碼 代碼如下:

public class SolrIndexer implements Indexer, Searcher, DisposableBean {
 //~ Static fields/initializers =============================================

 static final Logger logger = LoggerFactory.getLogger(SolrIndexer.class);

 private static final long SHUTDOWN_TIMEOUT    = 5 * 60 * 1000L; // long enough

 private static final int  INPUT_QUEUE_LENGTH  = 16384;

 //~ Instance fields ========================================================

 private CommonsHttpSolrServer server;

 private BlockingQueue<Operation> inputQueue;

 private Thread updateThread;
 volatile boolean running = true;
 volatile boolean shuttingDown = false;

 //~ Constructors ===========================================================

 public SolrIndexer(String url) throws MalformedURLException {
  server = new CommonsHttpSolrServer(url);

  inputQueue = new ArrayBlockingQueue<Operation>(INPUT_QUEUE_LENGTH);

  updateThread = new Thread(new UpdateTask());
  updateThread.setName("SolrIndexer");
  updateThread.start();
 }

 //~ Methods ================================================================

 public void setSoTimeout(int timeout) {
  server.setSoTimeout(timeout);
 }

 public void setConnectionTimeout(int timeout) {
  server.setConnectionTimeout(timeout);
 }

 public void setAllowCompression(boolean allowCompression) {
  server.setAllowCompression(allowCompression);
 }


 public void addIndex(Indexable indexable) throws IndexingException {
  if (shuttingDown) {
   throw new IllegalStateException("SolrIndexer is shutting down");
  }
  inputQueue.offer(new Operation(indexable, OperationType.UPDATE));
 }
 

 public void delIndex(Indexable indexable) throws IndexingException {
  if (shuttingDown) {
   throw new IllegalStateException("SolrIndexer is shutting down");
  }
  inputQueue.offer(new Operation(indexable, OperationType.DELETE));
 }

 
 private void updateIndices(String type, List<Indexable> indices) throws IndexingException {
  if (indices == null || indices.size() == 0) {
   return;
  }

  logger.debug("Updating {} indices", indices.size());

  UpdateRequest req = new UpdateRequest("/" + type + "/update");
  req.setAction(UpdateRequest.ACTION.COMMIT, false, false);

  for (Indexable idx : indices) {
   Doc doc = idx.getDoc();

   SolrInputDocument solrDoc = new SolrInputDocument();
   solrDoc.setDocumentBoost(doc.getDocumentBoost());
   for (Iterator<Field> i = doc.iterator(); i.hasNext();) {
    Field field = i.next();
    solrDoc.addField(field.getName(), field.getValue(), field.getBoost());
   }

   req.add(solrDoc);   
  }

  try {
   req.process(server);   
  } catch (SolrServerException e) {
   logger.error("SolrServerException occurred", e);
   throw new IndexingException(e);
  } catch (IOException e) {
   logger.error("IOException occurred", e);
   throw new IndexingException(e);
  }
 }

 
 private void delIndices(String type, List<Indexable> indices) throws IndexingException {
  if (indices == null || indices.size() == 0) {
   return;
  }

  logger.debug("Deleting {} indices", indices.size());

  UpdateRequest req = new UpdateRequest("/" + type + "/update");
  req.setAction(UpdateRequest.ACTION.COMMIT, false, false);
  for (Indexable indexable : indices) {   
   req.deleteById(indexable.getDocId());
  }

  try {
   req.process(server);
  } catch (SolrServerException e) {
   logger.error("SolrServerException occurred", e);
   throw new IndexingException(e);
  } catch (IOException e) {
   logger.error("IOException occurred", e);
   throw new IndexingException(e);
  }
 }

 
 public QueryResult search(Query query) throws IndexingException {
  SolrQuery sq = new SolrQuery();
  sq.setQuery(query.getQuery());
  if (query.getFilter() != null) {
   sq.addFilterQuery(query.getFilter());
  }
  if (query.getOrderField() != null) {
   sq.addSortField(query.getOrderField(), query.getOrder() == Query.Order.DESC ? SolrQuery.ORDER.desc : SolrQuery.ORDER.asc);
  }
  sq.setStart(query.getOffset());
  sq.setRows(query.getLimit());

  QueryRequest req = new QueryRequest(sq);
  req.setPath("/" + query.getType() + "/select");

  try {
   QueryResponse rsp = req.process(server);
   SolrDocumentList docs = rsp.getResults();

   QueryResult result = new QueryResult();
   result.setOffset(docs.getStart());
   result.setTotal(docs.getNumFound());
   result.setSize(sq.getRows());

   List<Doc> resultDocs = new ArrayList<Doc>(result.getSize());
   for (Iterator<SolrDocument> i = docs.iterator(); i.hasNext();) {
    SolrDocument solrDocument = i.next();

    Doc doc = new Doc();
    for (Iterator<Map.Entry<String, Object>> iter = solrDocument.iterator(); iter.hasNext();) {
     Map.Entry<String, Object> field = iter.next();
     doc.addField(field.getKey(), field.getValue());
    }

    resultDocs.add(doc);
   }

   result.setDocs(resultDocs);
   return result;

  } catch (SolrServerException e) {
   logger.error("SolrServerException occurred", e);
   throw new IndexingException(e);
  }
 }
 

 public void destroy() throws Exception {
  shutdown(SHUTDOWN_TIMEOUT, TimeUnit.MILLISECONDS);  
 }

 public boolean shutdown(long timeout, TimeUnit unit) {
  if (shuttingDown) {
   logger.info("Suppressing duplicate attempt to shut down");
   return false;
  }
  shuttingDown = true;
  String baseName = updateThread.getName();
  updateThread.setName(baseName + " - SHUTTING DOWN");
  boolean rv = false;
  try {
   // Conditionally wait
   if (timeout > 0) {
    updateThread.setName(baseName + " - SHUTTING DOWN (waiting)");
    rv = waitForQueue(timeout, unit);
   }
  } finally {
   // But always begin the shutdown sequence
   running = false;
   updateThread.setName(baseName + " - SHUTTING DOWN (informed client)");
  }
  return rv;
 }

 /**
  * @param timeout
  * @param unit
  * @return
  */
 private boolean waitForQueue(long timeout, TimeUnit unit) {
  CountDownLatch latch = new CountDownLatch(1);
  inputQueue.add(new StopOperation(latch));
  try {
   return latch.await(timeout, unit);
  } catch (InterruptedException e) {
   throw new RuntimeException("Interrupted waiting for queues", e);
  }
 }

 

 class UpdateTask implements Runnable {
  public void run() {
   while (running) {
    try {
     syncIndices();
    } catch (Throwable e) {
     if (shuttingDown) {
      logger.warn("Exception occurred during shutdown", e);
     } else {
      logger.error("Problem handling solr indexing updating", e);
     }
    }
   }
   logger.info("Shut down SolrIndexer");
  }
 }

 void syncIndices() throws InterruptedException {
  Operation op = inputQueue.poll(1000L, TimeUnit.MILLISECONDS);

  if (op == null) {
   return;
  }

  if (op instanceof StopOperation) {
   ((StopOperation) op).stop();
   return;
  }

  // wait 1 second
  try {
   Thread.sleep(1000);
  } catch (InterruptedException e) {

  }

  List<Operation> ops = new ArrayList<Operation>(inputQueue.size() + 1);
  ops.add(op);
  inputQueue.drainTo(ops);

  Map<String, List<Indexable>> deleteMap = new HashMap<String, List<Indexable>>(4);
  Map<String, List<Indexable>> updateMap = new HashMap<String, List<Indexable>>(4);

  for (Operation o : ops) {
   if (o instanceof StopOperation) {
    ((StopOperation) o).stop();
   } else {
    Indexable indexable = o.indexable;
    if (o.type == OperationType.DELETE) {
     List<Indexable> docs = deleteMap.get(indexable.getType());
     if (docs == null) {
      docs = new LinkedList<Indexable>();
      deleteMap.put(indexable.getType(), docs);
     }
     docs.add(indexable);
    } else {
     List<Indexable> docs = updateMap.get(indexable.getType());
     if (docs == null) {
      docs = new LinkedList<Indexable>();
      updateMap.put(indexable.getType(), docs);
     }
     docs.add(indexable);
    }
   }
  }

  for (Iterator<Map.Entry<String, List<Indexable>>> i = deleteMap.entrySet().iterator(); i.hasNext();) {
   Map.Entry<String, List<Indexable>> entry = i.next();
   delIndices(entry.getKey(), entry.getValue());
  }

  for (Iterator<Map.Entry<String, List<Indexable>>> i = updateMap.entrySet().iterator(); i.hasNext();) {
   Map.Entry<String, List<Indexable>> entry = i.next();
   updateIndices(entry.getKey(), entry.getValue());
  }
 }

 enum OperationType { DELETE, UPDATE, SHUTDOWN }

 static class Operation {
  OperationType type;
  Indexable indexable;

  Operation() {}

  Operation(Indexable indexable, OperationType type) {
   this.indexable = indexable;
   this.type = type;
  }
 }

 static class StopOperation extends Operation {
  CountDownLatch latch;

  StopOperation(CountDownLatch latch) {
   this.latch = latch;
   this.type = OperationType.SHUTDOWN;
  }

  public void stop() {
   latch.countDown();
  }
 }

 //~ Accessors ===============

}

相關(guān)文章

  • Spring Boot日志技術(shù)logback原理及配置解析

    Spring Boot日志技術(shù)logback原理及配置解析

    這篇文章主要介紹了Spring Boot日志技術(shù)logback原理及用法解析,文中通過(guò)示例代碼介紹的非常詳細(xì),對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友可以參考下
    2020-07-07
  • spring 聲明式事務(wù)實(shí)現(xiàn)過(guò)程解析

    spring 聲明式事務(wù)實(shí)現(xiàn)過(guò)程解析

    這篇文章主要介紹了spring 聲明式事務(wù)實(shí)現(xiàn)過(guò)程解析,文中通過(guò)示例代碼介紹的非常詳細(xì),對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友可以參考下
    2019-10-10
  • JAVA中的基本數(shù)據(jù)類型

    JAVA中的基本數(shù)據(jù)類型

    本文主要介紹了JAVA中的基本數(shù)據(jù)類型。具有很好的參考價(jià)值,下面跟著小編一起來(lái)看下吧
    2017-02-02
  • SpringBoot中使用MongoDB的連接池配置

    SpringBoot中使用MongoDB的連接池配置

    由于MongoDB的客戶端本身就是一個(gè)連接池,因此,我們只需要配置客戶端即可,文中通過(guò)示例代碼介紹的非常詳細(xì),具有一定的參考價(jià)值,感興趣的小伙伴們可以參考一下
    2021-10-10
  • SpringBoot 請(qǐng)求參數(shù)忽略大小寫的實(shí)例

    SpringBoot 請(qǐng)求參數(shù)忽略大小寫的實(shí)例

    這篇文章主要介紹了SpringBoot 請(qǐng)求參數(shù)忽略大小寫的實(shí)例,具有很好的參考價(jià)值,希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧
    2021-01-01
  • SpringEvents與異步事件驅(qū)動(dòng)案例詳解

    SpringEvents與異步事件驅(qū)動(dòng)案例詳解

    本文深入探討了SpringBoot中的事件驅(qū)動(dòng)架構(gòu),特別是通過(guò)Spring事件機(jī)制實(shí)現(xiàn)組件解耦和系統(tǒng)擴(kuò)展性增強(qiáng),介紹了事件的發(fā)布者、事件本身、事件監(jiān)聽器和事件處理器的概念,感興趣的朋友跟隨小編一起看看吧
    2024-09-09
  • Java使用新浪微博API開發(fā)微博應(yīng)用的基本方法

    Java使用新浪微博API開發(fā)微博應(yīng)用的基本方法

    這篇文章主要介紹了Java使用新浪微博API開發(fā)微博應(yīng)用的基本方法,文中還給出了一個(gè)不使用任何SDK實(shí)現(xiàn)Oauth授權(quán)并實(shí)現(xiàn)簡(jiǎn)單的發(fā)布微博功能的實(shí)現(xiàn)方法,需要的朋友可以參考下
    2015-11-11
  • SpringBoot數(shù)據(jù)層測(cè)試事務(wù)回滾的實(shí)現(xiàn)流程

    SpringBoot數(shù)據(jù)層測(cè)試事務(wù)回滾的實(shí)現(xiàn)流程

    這篇文章主要介紹了SpringBoot數(shù)據(jù)層測(cè)試事務(wù)回滾的實(shí)現(xiàn),文中通過(guò)示例代碼介紹的非常詳細(xì),對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友們下面隨著小編來(lái)一起學(xué)習(xí)吧
    2022-10-10
  • IDEA整合SSM框架實(shí)現(xiàn)網(wǎng)頁(yè)上顯示數(shù)據(jù)

    IDEA整合SSM框架實(shí)現(xiàn)網(wǎng)頁(yè)上顯示數(shù)據(jù)

    最近做了個(gè)小項(xiàng)目,該項(xiàng)目包在intellij idea中實(shí)現(xiàn)了ssm框架的整合以及實(shí)現(xiàn)訪問(wèn),具有一定的參考價(jià)值,感興趣的小伙伴們可以參考一下
    2021-05-05
  • SpringBoot基于Redis實(shí)現(xiàn)生成全局唯一ID的方法

    SpringBoot基于Redis實(shí)現(xiàn)生成全局唯一ID的方法

    在項(xiàng)目中生成全局唯一ID有很多好處,生成全局唯一ID有助于提高系統(tǒng)的可用性、數(shù)據(jù)的完整性和安全性,同時(shí)也方便數(shù)據(jù)的管理和分析,所以本文給大家介紹了SpringBoot基于Redis實(shí)現(xiàn)生成全局唯一ID的方法,文中有詳細(xì)的代碼講解,需要的朋友可以參考下
    2023-12-12

最新評(píng)論

临澧县| 石家庄市| 扶余县| 皋兰县| 佳木斯市| 阳信县| 蕲春县| 襄城县| 新安县| 遂宁市| 广宁县| 南昌市| 抚远县| 正定县| 青海省| 唐山市| 富平县| 于田县| 眉山市| 隆回县| 奉化市| 长白| 泸州市| 大渡口区| 天峨县| 德州市| 商都县| 马山县| 梅河口市| 湾仔区| 沐川县| 开化县| 迭部县| 长垣县| 黄陵县| 南平市| 中阳县| 湘潭县| 民勤县| 晋江市| 贺兰县|