python實現(xiàn)數(shù)據(jù)庫跨服務(wù)器遷移
基于Python2.7的版本環(huán)境,Python實現(xiàn)的數(shù)據(jù)庫跨服務(wù)器(跨庫)遷移, 每以5000條一查詢一提交,代碼中可以自行更改每次查詢提交數(shù)目.
# -*- coding: utf-8 -*-
import MySQLdb
import time
import warnings
warnings.filterwarnings("ignore")
class ConnectMysql(object):
def __init__(self):
# 這里設(shè)置分頁查詢, 每頁查詢多少數(shù)據(jù)
self.page_size = 5000
def getTable(self):
conn = MySQLdb.connect(
host="***.***.**.**",
user="****",
passwd="*************",
db='****',
charset='utf8'
)
conn_local = MySQLdb.connect(
host="********************************",
user="**********",
passwd="********",
db='*******',
charset='utf8'
)
cur = conn.cursor()
cur_local = conn_local.cursor()
cur.execute('show tables')
tables = cur.fetchall()
for table in tables:
print str(table[0]).lower()
# 需要遷移的數(shù)據(jù)庫查詢表的列數(shù)
cur.execute("SELECT COUNT(*) FROM information_schema.COLUMNS WHERE table_schema='china' AND table_name='" + table[0] + "'")
table_col_count = cur.fetchone()
# print table_col_count[0]
# 需要遷移的數(shù)據(jù)庫查詢表的結(jié)構(gòu)
cur.execute('show create table ' + table[0])
result = cur.fetchall()
create_sql = result[0][1]
# 查詢需要遷移的數(shù)據(jù)庫表的數(shù)據(jù)條數(shù)
cur.execute('select count(*) from ' + table[0])
total = cur.fetchone()
page = total[0] / self.page_size
page1 = total[0] % self.page_size
if page1 != 0:
page = page + 1
# 阿里云數(shù)據(jù)庫創(chuàng)建表
cur_local.execute("SELECT table_name FROM information_schema.`TABLES` WHERE table_schema='user' AND table_name='" + str(table[0]).lower() + "'")
table_name = cur_local.fetchone()
if table_name is None:
cur_local.execute(create_sql)
for p in range(0, page):
while True:
try:
print '開始', table[0], '的第', p + 1, '頁查詢'
if p == 0:
limit_param = ' limit ' + str(p * self.page_size) + ',' + str(self.page_size)
else:
limit_param = ' limit ' + str(p * self.page_size + 1) + ',' + str(self.page_size)
cur.execute('select * from ' + table[0] + limit_param)
inserts = cur.fetchall()
print '查詢成功'
param = ''
for i in range(0, table_col_count[0]):
param = param + '%s,'
print '開始插入'
cur_local.executemany('replace into ' + table[0] + ' values (' + param[0:-1] + ')', inserts)
print table[0], '的第', p + 1, '頁, 插入完成, 還有', page - p - 1, '頁, 任重而道遠(yuǎn)'
conn_local.commit()
break
except Exception as e:
print e
time.sleep(60)
cur = conn.cursor()
cur_local = conn_local.cursor()
print table[0], ' 插入完成'
print '\n \n ======================================================================== \n\n'
cur_local.close()
conn_local.close()
cur.close()
conn.close()
if __name__ == '__main__':
conn_mysql = ConnectMysql()
conn_mysql.getTable()
以上就是本文的全部內(nèi)容,希望對大家的學(xué)習(xí)有所幫助,也希望大家多多支持腳本之家。
相關(guān)文章
pandas 如何保存數(shù)據(jù)到excel,csv
這篇文章主要介紹了pandas 如何保存數(shù)據(jù)到excel,csv的實現(xiàn)方式,具有很好的參考價值,希望對大家有所幫助。如有錯誤或未考慮完全的地方,望不吝賜教2022-07-07
詳解向scrapy中的spider傳遞參數(shù)的幾種方法(2種)
這篇文章主要介紹了詳解向scrapy中的spider傳遞參數(shù)的幾種方法,文中通過示例代碼介紹的非常詳細(xì),對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值,需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2020-09-09
使用Python進(jìn)行Excel文件xls/xlsx/xsv格式互相轉(zhuǎn)換
在MAC上搭建python數(shù)據(jù)分析開發(fā)環(huán)境
Tensorflow 訓(xùn)練自己的數(shù)據(jù)集將數(shù)據(jù)直接導(dǎo)入到內(nèi)存
對python3中的RE(正則表達(dá)式)-詳細(xì)總結(jié)

