對(duì)python 操作solr索引數(shù)據(jù)的實(shí)例詳解
更新時(shí)間：2018年12月07日 11:11:23 作者：shaomine
今天小編就為大家分享一篇對(duì)python 操作solr索引數(shù)據(jù)的實(shí)例詳解，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧
測(cè)試代碼1：
def test(self):
data = {"add": {"doc": {"id": "100001", "*字段名*": u"我是一個(gè)大好人"}}}
params = {"boost": 1.0, "overwrite": "true", "commitWithin": 1000}
url = 'http://127.0.0.1:8983/solr/mycore/update?wt=json'
headers = {"Content-Type": "application/json"}
r = requests.post(url, json=data, params=params, headers=headers)
print r.text
def Index_data(self):
solr = pysolr.Solr('http://127.0.0.1:8983/solr/mycore/', timeout=10)
# How you'd index data.
result = solr.add([
{
"id": "doc_1",
"title": "A test document",
},
{
"id": "doc_2",
"title": "The Banana: Tasty or Dangerous?",
},
])
print result
測(cè)試代碼2：
實(shí)際數(shù)據(jù)：
def Index_Data_FromCSV(self, csvfile):
'''
從CSV文件中讀取數(shù)據(jù)，并索引到solr中
:param csvfile: csv文件，包括完整路徑
:return:
'''
list = CSVOP.ReadCSV(csvfile)
index = 0
doc = {}
params = {"boost": 1.0, "overwrite": "true", "commitWithin": 1000}
url = 'http://127.0.0.1:8983/solr/mycore/update?wt=json'
headers = {"Content-Type": "application/json"}
for item in list:
if index > 0: # 第一行是標(biāo)題
try:
doc['title'] = item[0].decode('GB2312')
doc['link'] = item[1]
# doc['date'] = item[2]
doc['source'] = item[3].decode('GB2312')
doc['keyword'] = item[4].decode('GB2312')
data = {"add": {"doc": doc}}
r = requests.post(url, json=data, params=params, headers=headers)
print r.text
except Exception,e:
print e.message
print index
index += 1
#pysolr客戶(hù)端代碼
def pysolr_Index_Data_FromCSV(self, csvfile,url='http://127.0.0.1:8983/solr/mycore/'):
'''
從CSV文件中讀取數(shù)據(jù)，并索引到solr中
:param csvfile: csv文件，包括完整路徑
:return:
'''
list = CSVOP.ReadCSV(csvfile)
index = 0
listdocs = []
for item in list:
if index > 0: # 第一行是標(biāo)題
doc = {}
try:
doc['title'] = item[0].decode('GB2312')
doc['link'] = item[1]
# doc['date'] = item[2]
doc['source'] = item[3].decode('GB2312')
doc['keyword'] = item[4].decode('GB2312')
listdocs.append(doc)
except Exception,e:
print e.message
index += 1
solr = pysolr.Solr(url, timeout=10)
result = solr.add(listdocs)
print result
查詢(xún)代碼：
def search_data(self,message='視頻'):
url = 'http://127.0.0.1:8983/solr/mycore/select?q=title:"\%s"&wt=json&indent=true' % message
r = requests.get(url, verify=False)
print r.text
r = r.json()['response']['numFound']
print message + ":" + str(r)
#pysolr客戶(hù)端
def search_data(self,where='視頻',url='http://127.0.0.1:8983/solr/mycore/'):
solr = pysolr.Solr(url, timeout=10)
dict = {'start':10,'rows': 30,'fl':'title,keyword,source,link'}
result = solr.search('title:視頻',**dict)
# result = solr.search('title:視頻')
# print result.raw_response['response']['numFound']
for item in result:
print 'keyword: %s'% item['keyword']
print 'title: %s'% item['title']
print 'source: %s'% item['source']
print 'link: %s'% item['link']
print '
'
輸出結(jié)果：
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"title:\"\\視頻\"",
"indent":"true",
"wt":"json"}},
"response":{"numFound":123,"start":0,"docs":[
{
"source":"中彩網(wǎng)",
"link":"http://www.zhcw.com/video/kaijiangshipin-3D/11981126.shtml",
"keyword":"視頻",
"title":"福彩3D開(kāi)獎(jiǎng) 視頻 -中彩 視頻",
"id":"2f0a9d21-3771-4efa-a0cc-e0484cc97993",
"_version_":1584214368617234432},
{
"source":"新浪視頻",
"link":"http://video.sina.com.cn/news/spj/topvideoes20170707/?opsubject_id=top1",
"keyword":"視頻",
"title":"今日熱門(mén) 視頻 匯總20170707",
"id":"c8aae0af-01e9-491f-b999-24b97004a4ba",
"_version_":1584214367507841024},
{
"source":"網(wǎng)易新聞",
"link":"http://news.163.com/17/0707/13/COOCNUIE00018AOR.html",
"keyword":"視頻",
"title":"網(wǎng)傳＂蘭桂坊附近不雅 視頻 ＂ 警方：傳播 視頻 將追責(zé)",
"id":"353de48d-ede7-481b-89d3-bc20ab4b3884",
"_version_":1584214367821365248},
{
"source":"鳳凰視頻",
"link":"http://v.ifeng.com/video_7480871.shtml",
"keyword":"視頻",
"title":"創(chuàng)想動(dòng)畫(huà)片：花粉過(guò)敏癥的痛誰(shuí)懂-鳳凰 視頻 -最具媒體品質(zhì)的綜合 視頻 ...",
"id":"dc5f19c4-180f-4004-a0db-4499d875a60f",
"_version_":1584214366819975168},
{
"source":"鳳凰視頻",
"link":"http://v.ifeng.com/video_7805858.shtml",
"keyword":"視頻",
"title":"節(jié)氣說(shuō)：小暑時(shí)節(jié)就該這樣養(yǎng)生-鳳凰 視頻 -最具媒體品質(zhì)的綜合 視頻 門(mén)...",
"id":"5e9eb7a7-48b8-4e41-9514-7712ae619d9a",
"_version_":1584214367516229632},
{
"source":"鳳凰視頻",
"link":"http://v.ifeng.com/video_7483506.shtml",
"keyword":"視頻",
"title":"聽(tīng)導(dǎo)演講《神奇女俠》的故事 -鳳凰 視頻 -最具媒體品質(zhì)的綜合 視頻 門(mén)戶(hù)-...",
"id":"6b1482f1-c0c9-479f-bef7-7de324fb9372",
"_version_":1584214367647301632},
{
"source":"汽車(chē)雜志",
"link":"http://www.jiemian.com/article/1445267.html",
"keyword":"視頻",
"title":"【視頻】歐寶最近找了一堆穿睡衣的辣媽拍了一段超牛的視頻",
"id":"1d327555-a6f3-4513-9a21-43d59418ab82",
"_version_":1584214368157958144},
{
"source":"味覺(jué)大師",
"link":"http://www.jiemian.com/article/1453545.html",
"keyword":"視頻",
"title":"【視頻】大董沒(méi)有肉的肉味燒茄子",
"id":"7d777870-93cb-4c18-a32b-734af8f133f1",
"_version_":1584213891451191296},
{
"source":"新浪汽車(chē)",
"link":"http://auto.sina.com.cn/video/zz/2017-07-07/detail-ifyhwehx5311889.shtml",
"keyword":"視頻",
"title":"視頻 ：兩大神車(chē)pk！高爾夫思域怎么選？",
"id":"3a50b303-6b54-4da3-aee1-a61c678c752d",
"_version_":1584213892090822656},
{
"source":"味覺(jué)大師",
"link":"http://www.jiemian.com/article/1453545.html",
"keyword":"視頻",
"title":"【視頻】大董沒(méi)有肉的肉味燒茄子",
"id":"01da8e11-77bc-4c31-ba3a-ba668e846d9d",
"_version_":1584214366191878144}]
}}
完整代碼：
#-*- coding: UTF-8 -*-
import csv
import os
import codecs
def ReadCSV(filename):
if os.path.exists(filename):
with open(filename, 'r') as f:
reader = csv.reader(f)
list = []
for item in reader:
list.append(item)
return list
#################################################
#coding=utf-8
import json
import requests
import os
import time
from os import walk
import CSVOP
from datetime import datetime
import pysolr
import math
class SolrClientObj:
def test(self):
data = {"add": {"doc": {"id": "100001", "*字段名*": u"我是一個(gè)大好人"}}}
params = {"boost": 1.0, "overwrite": "true", "commitWithin": 1000}
url = 'http://127.0.0.1:8983/solr/mycore/update?wt=json'
headers = {"Content-Type": "application/json"}
r = requests.post(url, json=data, params=params, headers=headers)
print r.text
def pysolr_Index_Data_FromCSV(self, csvfile,url='http://127.0.0.1:8983/solr/mycore/'):
'''
從CSV文件中讀取數(shù)據(jù)，并索引到solr中
:param csvfile: csv文件，包括完整路徑
:return:
'''
list = CSVOP.ReadCSV(csvfile)
index = 0
listdocs = []
for item in list:
if index > 0: # 第一行是標(biāo)題
doc = {}
try:
doc['title'] = item[0].decode('GB2312')
doc['link'] = item[1]
# doc['date'] = item[2]
doc['source'] = item[3].decode('GB2312')
doc['keyword'] = item[4].decode('GB2312')
listdocs.append(doc)
except Exception,e:
print e.message
index += 1
solr = pysolr.Solr(url, timeout=10)
result = solr.add(listdocs)
print result
def Index_Data_FromCSV(self, csvfile):
'''
從CSV文件中讀取數(shù)據(jù)，并索引到solr中
:param csvfile: csv文件，包括完整路徑
:return:
'''
list = CSVOP.ReadCSV(csvfile)
index = 0
doc = {}
params = {"boost": 1.0, "overwrite": "true", "commitWithin": 1000}
url = 'http://127.0.0.1:8983/solr/mycore/update?wt=json'
headers = {"Content-Type": "application/json"}
for item in list:
if index > 0: # 第一行是標(biāo)題
try:
doc['title'] = item[0].decode('GB2312')
doc['link'] = item[1]
# doc['date'] = item[2]
doc['source'] = item[3].decode('GB2312')
doc['keyword'] = item[4].decode('GB2312')
data = {"add": {"doc": doc}}
r = requests.post(url, json=data, params=params, headers=headers)
print r.text
except Exception,e:
print e.message
print index
index += 1
def Index_data(self):
solr = pysolr.Solr('http://127.0.0.1:8983/solr/mycore/', timeout=10)
# How you'd index data.
result = solr.add([
{
"id": "doc_1",
"title": "A test document",
},
{
"id": "doc_2",
"title": "The Banana: Tasty or Dangerous?",
},
])
print result
def search_data(self,where='視頻',url='http://127.0.0.1:8983/solr/mycore/'):
solr = pysolr.Solr(url, timeout=10)
dict = {'start':10,'rows': 30,'fl':'title,keyword,source,link'}
result = solr.search('title:視頻',**dict)
# result = solr.search('title:視頻')
# print result.raw_response['response']['numFound']
for item in result:
print 'keyword: %s'% item['keyword']
print 'title: %s'% item['title']
print 'source: %s'% item['source']
print 'link: %s'% item['link']
print ' '
def delete_index_data(self,where,url='http://127.0.0.1:8983/solr/mycore/'):
'''
刪除索引
:param where: 刪除的條件
:param url: url
:return:
'''
solr = pysolr.Solr(url, timeout=10)
# solr.delete(id=where) #id='id1':刪除id為“id1”的索引
result = solr.delete(q=where) #q='*:*'刪除所有索引
print result
obj = SolrClientObj()
# obj.delete_index_data('*:*') #刪除所有索引
# obj.Index_data()
# obj.search_data()
# obj.delete_index_data('doc_1')
obj.search_data('視頻')
# csvfile = 'D:/work/Solr/other/exportExcels/2017-07-07_info.csv'
# obj.pysolr_Index_Data_FromCSV(csvfile)
以上這篇對(duì)python 操作solr索引數(shù)據(jù)的實(shí)例詳解就是小編分享給大家的全部?jī)?nèi)容了，希望能給大家一個(gè)參考，也希望大家多多支持腳本之家。
您可能感興趣的文章:
- 在Python的gevent框架下執(zhí)行異步的Solr查詢(xún)的教程
- Python 讀取串口數(shù)據(jù),動(dòng)態(tài)繪圖的示例
- python調(diào)用動(dòng)態(tài)鏈接庫(kù)的基本過(guò)程詳解
- python 實(shí)現(xiàn)在tkinter中動(dòng)態(tài)顯示label圖片的方法
- Python使用MyQR制作專(zhuān)屬動(dòng)態(tài)彩色二維碼功能
- 【python】matplotlib動(dòng)態(tài)顯示詳解
- python 動(dòng)態(tài)遷移solr數(shù)據(jù)過(guò)程解析
相關(guān)文章
這篇文章主要介紹了模型訓(xùn)練時(shí)GPU利用率太低的原因及解決方案，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。如有錯(cuò)誤或未考慮完全的地方，望不吝賜教2023-02-02
基于telepath庫(kù)實(shí)現(xiàn)Python和JavaScript之間交換數(shù)據(jù)
telepath是一個(gè)Django庫(kù)，用于在Python和JavaScript之間交換數(shù)據(jù)，使您可以構(gòu)建具有豐富客戶(hù)端接口的應(yīng)用程序，同時(shí)將業(yè)務(wù)邏輯保留在服務(wù)器端代碼中。2021-05-05
Python?PaddleGAN實(shí)現(xiàn)調(diào)整照片人物年齡
這篇文章主要介紹了通過(guò)PaddleGAN實(shí)現(xiàn)照片人物的老年化和年輕化處理，文中的示例代碼講解有效，對(duì)我們學(xué)習(xí)或工作有一定的幫助，感興趣的可以學(xué)習(xí)一下2021-12-12
Python中UserWarning：The NumPy module was
在 Python 項(xiàng)目中,我們經(jīng)常需要導(dǎo)入許多庫(kù)來(lái)完成各種任務(wù),NumPy 作為一個(gè)核心的科學(xué)計(jì)算庫(kù),被廣泛應(yīng)用于數(shù)據(jù)處理和分析,然而,有時(shí)我們會(huì)遇到 NumPy 重載的警告,本文將詳細(xì)講解這一警告的原因,并提供解決方案,需要的朋友可以參考下2024-07-07
PyTorch線性回歸和邏輯回歸實(shí)戰(zhàn)示例
這篇文章主要介紹了PyTorch線性回歸和邏輯回歸實(shí)戰(zhàn)示例,小編覺(jué)得挺不錯(cuò)的，現(xiàn)在分享給大家，也給大家做個(gè)參考。一起跟隨小編過(guò)來(lái)看看吧2018-05-05
中文分詞是中文文本處理的一個(gè)基礎(chǔ)性工作，然而長(zhǎng)久以來(lái)，在Python編程領(lǐng)域，一直缺少高準(zhǔn)確率、高效率的分詞組件。下面這篇文章主要給大家介紹了關(guān)于python中文分詞教程之前向最大正向匹配算法的相關(guān)資料，需要的朋友可以參考下。2017-11-11
在本篇文章里小編給大家分享的是關(guān)于python編寫(xiě)win程序的實(shí)例內(nèi)容，有需要的朋友們可以學(xué)習(xí)下。2020-06-06
python tkinter之 復(fù)選、文本、下拉的實(shí)現(xiàn)
這篇文章主要介紹了python tkinter之 復(fù)選、文本、下拉的實(shí)現(xiàn)，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧2020-03-03
本文詳細(xì)介紹了Pycharm中多種提高編程效率的快捷鍵操作,包括代碼格式化、代碼合并、修正代碼警告等,適合Python開(kāi)發(fā)者使用,感興趣的可以了解一下2024-11-11
Python內(nèi)置函數(shù)all()的實(shí)現(xiàn)
Python內(nèi)置函數(shù)?all()?用于判斷可迭代對(duì)象中的所有元素是否都為真值（Truthy）,是邏輯判斷的重要工具,下面就來(lái)具體介紹如何使用,感興趣的可以了解一下2025-04-04