最新国产好看的视频,伊人天堂AV在线,国产Aaaaaa视频,蜜臀视频在线观看一区,人妻av色图,密臀久久久精品影片,青青视频免费观看毛片,久草在线观看视,国产三级精品色情在线

Python lxml模塊的基本使用方法分析

 更新時(shí)間：2019年12月21日 11:21:51   作者：Dylan HU  
這篇文章主要介紹了Python lxml模塊的基本使用方法,結(jié)合實(shí)例形式分析了Python安裝與使用lxml模塊常見操作技巧與相關(guān)注意事項(xiàng),需要的朋友可以參考下

本文實(shí)例講述了Python lxml模塊的基本使用方法。分享給大家供大家參考，具體如下：

1 lxml的安裝

安裝方式：pip install lxml

2 lxml的使用

2.1 lxml模塊的入門使用

導(dǎo)入lxml 的 etree 庫 (導(dǎo)入沒有提示不代表不能用)

from lxml import etree

利用etree.HTML，將字符串轉(zhuǎn)化為Element對象,Element對象具有xpath的方法,返回結(jié)果的列表，能夠接受bytes類型的數(shù)據(jù)和str類型的數(shù)據(jù)

html = etree.HTML(text) 
ret_list = html.xpath("xpath字符串")

把轉(zhuǎn)化后的element對象轉(zhuǎn)化為字符串，返回bytes類型結(jié)果 etree.tostring(element)

假設(shè)我們現(xiàn)有如下的html字符換，嘗試對他進(jìn)行操作

<div> <ul> 
<li class="item-1"><a href="link1.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >first item</a></li> 
<li class="item-1"><a href="link2.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >second item</a></li> 
<li class="item-inactive"><a href="link3.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >third item</a></li> 
<li class="item-1"><a href="link4.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fourth item</a></li> 
<li class="item-0"><a href="link5.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fifth item</a> # 注意，此處缺少一個(gè) </li> 閉合標(biāo)簽 
</ul> </div>


from lxml import etree
text = ''' <div> <ul> 
    <li class="item-1"><a href="link1.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >first item</a></li> 
    <li class="item-1"><a href="link2.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >second item</a></li> 
    <li class="item-inactive"><a href="link3.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >third item</a></li> 
    <li class="item-1"><a href="link4.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fourth item</a></li> 
    <li class="item-0"><a href="link5.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fifth item</a> 
    </ul> </div> '''
html = etree.HTML(text)
print(type(html)) 
handeled_html_str = etree.tostring(html).decode()
print(handeled_html_str)

輸出為

<class 'lxml.etree._Element'>
<html><body><div> <ul>
        <li class="item-1"><a href="link1.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >first item</a></li>
        <li class="item-1"><a href="link2.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >second item</a></li>
        <li class="item-inactive"><a href="link3.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >third item</a></li>
        <li class="item-1"><a href="link4.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fourth item</a></li>
        <li class="item-0"><a href="link5.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fifth item</a>
        </li></ul> </div> </body></html>

可以發(fā)現(xiàn)，lxml確實(shí)能夠把確實(shí)的標(biāo)簽補(bǔ)充完成，但是請注意lxml是人寫的，很多時(shí)候由于網(wǎng)頁不夠規(guī)范，或者是lxml的bug，即使參考url地址對應(yīng)的響應(yīng)去提取數(shù)據(jù)，任然獲取不到，這個(gè)時(shí)候我們需要使用etree.tostring的方法，觀察etree到底把html轉(zhuǎn)化成了什么樣子，即根據(jù)轉(zhuǎn)化后的html字符串去進(jìn)行數(shù)據(jù)的提取。

2.2 lxml的深入練習(xí)

接下來我們繼續(xù)操作，假設(shè)每個(gè)class為item-1的li標(biāo)簽是1條新聞數(shù)據(jù)，如何把這條新聞數(shù)據(jù)組成一個(gè)字典

from lxml import etree
text = ''' <div> <ul> 
    <li class="item-1"><a href="link1.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >first item</a></li> 
    <li class="item-1"><a href="link2.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >second item</a></li> 
    <li class="item-inactive"><a href="link3.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >third item</a></li> 
    <li class="item-1"><a href="link4.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fourth item</a></li> 
    <li class="item-0"><a href="link5.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fifth item</a> 
    </ul> </div> '''
html = etree.HTML(text)
#獲取href的列表和title的列表
href_list = html.xpath("http://li[@class='item-1']/a/@href")
title_list = html.xpath("http://li[@class='item-1']/a/text()")
#組裝成字典
for href in href_list:
  item = {}
  item["href"] = href
  item["title"] = title_list[href_list.index(href)]
  print(item)

輸出為

{'href': 'link1.html', 'title': 'first item'}
{'href': 'link2.html', 'title': 'second item'}
{'href': 'link4.html', 'title': 'fourth item'}

假設(shè)在某種情況下，某個(gè)新聞的href沒有，那么會怎樣呢？

from lxml import etree
text = ''' <div> <ul> 
    <li class="item-1"><a>first item</a></li> 
    <li class="item-1"><a href="link2.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >second item</a></li> 
    <li class="item-inactive"><a href="link3.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >third item</a></li> 
    <li class="item-1"><a href="link4.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fourth item</a></li> 
    <li class="item-0"><a href="link5.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fifth item</a> 
    </ul> </div> '''

結(jié)果是

{'href': 'link2.html', 'title': 'first item'}
{'href': 'link4.html', 'title': 'second item'}

數(shù)據(jù)的對應(yīng)全部錯(cuò)了，這不是我們想要的，接下來通過2.3小節(jié)的學(xué)習(xí)來解決這個(gè)問題

2.3 lxml模塊的進(jìn)階使用

前面我們?nèi)〉綄傩?，或者是文本的時(shí)候，返回字符串 但是如果我們?nèi)〉降氖且粋€(gè)節(jié)點(diǎn)，返回什么呢?

返回的是element對象，可以繼續(xù)使用xpath方法，對此我們可以在后面的數(shù)據(jù)提取過程中：先根據(jù)某個(gè)標(biāo)簽進(jìn)行分組，分組之后再進(jìn)行數(shù)據(jù)的提取

示例如下：

from lxml import etree
text = ''' <div> <ul> 
    <li class="item-1"><a>first item</a></li> 
    <li class="item-1"><a href="link2.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >second item</a></li> 
    <li class="item-inactive"><a href="link3.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >third item</a></li> 
    <li class="item-1"><a href="link4.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fourth item</a></li> 
    <li class="item-0"><a href="link5.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fifth item</a> 
    </ul> </div> '''
html = etree.HTML(text)
li_list = html.xpath("http://li[@class='item-1']")
print(li_list)

結(jié)果為：

[<Element li at 0x11106cb48>, <Element li at 0x11106cb88>, <Element li at 0x11106cbc8>]

可以發(fā)現(xiàn)結(jié)果是一個(gè)element對象，這個(gè)對象能夠繼續(xù)使用xpath方法

先根據(jù)li標(biāo)簽進(jìn)行分組，之后再進(jìn)行數(shù)據(jù)的提取

from lxml import etree
text = ''' <div> <ul> 
    <li class="item-1"><a>first item</a></li> 
    <li class="item-1"><a href="link2.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >second item</a></li> 
    <li class="item-inactive"><a href="link3.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >third item</a></li> 
    <li class="item-1"><a href="link4.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fourth item</a></li> 
    <li class="item-0"><a href="link5.html" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >fifth item</a> 
    </ul> </div> '''
#根據(jù)li標(biāo)簽進(jìn)行分組
html = etree.HTML(text)
li_list = html.xpath("http://li[@class='item-1']")
#在每一組中繼續(xù)進(jìn)行數(shù)據(jù)的提取
for li in li_list:
  item = {}
  item["href"] = li.xpath("./a/@href")[0] if len(li.xpath("./a/@href"))>0 else None
  item["title"] = li.xpath("./a/text()")[0] if len(li.xpath("./a/text()"))>0 else None
  print(item)

結(jié)果是：

{'href': None, 'title': 'first item'}
{'href': 'link2.html', 'title': 'second item'}
{'href': 'link4.html', 'title': 'fourth item'}

前面的代碼中，進(jìn)行數(shù)據(jù)提取需要判斷，可能某些一面不存在數(shù)據(jù)的情況，對應(yīng)的可以使用三元運(yùn)算符來解決

PS：這里再為大家提供幾款關(guān)于xml操作的在線工具供大家參考使用：

在線XML/JSON互相轉(zhuǎn)換工具：
http://tools.jb51.net/code/xmljson

在線格式化XML/在線壓縮XML：
http://tools.jb51.net/code/xmlformat

XML在線壓縮/格式化工具：
http://tools.jb51.net/code/xml_format_compress

XML代碼在線格式化美化工具：
http://tools.jb51.net/code/xmlcodeformat

更多關(guān)于Python相關(guān)內(nèi)容感興趣的讀者可查看本站專題：《Python操作xml數(shù)據(jù)技巧總結(jié)》、《Python數(shù)據(jù)結(jié)構(gòu)與算法教程》、《Python Socket編程技巧總結(jié)》、《Python函數(shù)使用技巧總結(jié)》、《Python字符串操作技巧匯總》、《Python入門與進(jìn)階經(jīng)典教程》及《Python文件與目錄操作技巧匯總》

希望本文所述對大家Python程序設(shè)計(jì)有所幫助。

相關(guān)文章

最新評論

中牟县| 时尚| 察哈| 苍山县| 遵化市| 嘉义市| 宝鸡市| 化德县| 屯留县| 沽源县| 渭南市| 板桥市| 长葛市| 和龙市| 新昌县| 黑山县| 城固县| 建宁县| 北票市| 姚安县| 禹州市| 互助| 台前县| 买车| 镇巴县| 全南县| 萨迦县| 浏阳市| 天峻县| 姜堰市| 高雄市| 夏津县| 和田市| 北票市| 罗山县| 丽江市| 外汇| 平湖市| 武宣县| 锦州市| 遂昌县|