快捷導(dǎo)航

python 如何獲取頁(yè)面所有a標(biāo)簽下href的值

更新時(shí)間：2021年05月06日 11:47:02 作者：不愿透露姓名的菜鳥(niǎo)

這篇文章主要介紹了python 獲取頁(yè)面所有a標(biāo)簽下href的值操作，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧

看代碼吧~

# -*- coding:utf-8 -*-
#python 2.7
#http://tieba.baidu.com/p/2460150866
#標(biāo)簽操作 
 
from bs4 import BeautifulSoup
import urllib.request
import re 
 
#如果是網(wǎng)址，可以用這個(gè)辦法來(lái)讀取網(wǎng)頁(yè)
#html_doc = "http://tieba.baidu.com/p/2460150866"
#req = urllib.request.Request(html_doc)  
#webpage = urllib.request.urlopen(req)  
#html = webpage.read() 
 
html="""
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a  rel="external nofollow"  rel="external nofollow"  class="sister" id="xiaodeng"><!-- Elsie --></a>,
<a  rel="external nofollow"  rel="external nofollow"  class="sister" id="link2">Lacie</a> and
<a  rel="external nofollow"  class="sister" id="link3">Tillie</a>;
<a  rel="external nofollow"  rel="external nofollow"  class="sister" id="xiaodeng">Lacie</a>
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
soup = BeautifulSoup(html, 'html.parser')   #文檔對(duì)象 
 
#查找a標(biāo)簽,只會(huì)查找出一個(gè)a標(biāo)簽
#print(soup.a)#<a class="sister"  rel="external nofollow"  rel="external nofollow"  id="xiaodeng"><!-- Elsie --></a>
 
for k in soup.find_all('a'):
    print(k)
    print(k['class'])#查a標(biāo)簽的class屬性
    print(k['id'])#查a標(biāo)簽的id值
    print(k['href'])#查a標(biāo)簽的href值
    print(k.string)#查a標(biāo)簽的string

如果，標(biāo)簽<a>中含有其他標(biāo)簽，比如<em>..</em>，此時(shí)要提取<a>中的數(shù)據(jù)，需要用k.get_text()

soup = BeautifulSoup(html, 'html.parser')   #文檔對(duì)象
#查找a標(biāo)簽,只會(huì)查找出一個(gè)a標(biāo)簽
for k in soup.find_all('a'):
    print(k)
    print(k['class'])#查a標(biāo)簽的class屬性
    print(k['id'])#查a標(biāo)簽的id值
    print(k['href'])#查a標(biāo)簽的href值
    print(k.string)#查a標(biāo)簽的string

如果，標(biāo)簽<a>中含有其他標(biāo)簽，比如<em>..</em>，此時(shí)要提取<a>中的數(shù)據(jù)，需要用k.get_text()

通常我們使用下面這種模式也是能夠處理的，下面的方法使用了get()。

 html = urlopen(url)
 soup = BeautifulSoup(html, 'html.parser')
 t1 = soup.find_all('a')
 print t1
 href_list = []
 for t2 in t1:
    t3 = t2.get('href')
    href_list.append(t3)

補(bǔ)充：python爬蟲(chóng)獲取任意頁(yè)面的標(biāo)簽和屬性（包括獲取a標(biāo)簽的href屬性）

看代碼吧~

# coding=utf-8 
from bs4 import BeautifulSoup 
import requests 
# 定義一個(gè)獲取url頁(yè)面下label標(biāo)簽的attr屬性的函數(shù) 
def getHtml(url, label, attr): 
    response = requests.get(url) 
    response.encoding = 'utf-8' 
    html = response.text 
    soup = BeautifulSoup(html, 'html.parser'); 
    for target in soup.find_all(label):
 
        try: 
            value = target.get(attr)
 
        except: 
            value = ''
 
        if value: 
            print(value)
 
url = 'https://baidu.com/' 
label = 'a' 
attr = 'href' 
getHtml(url, label, attr)