wangyu8460958's recent timeline updates
wangyu8460958

wangyu8460958

V2EX member #227278, joined on 2017-04-22 10:01:56 +08:00
wangyu8460958's recent replies
Apr 27, 2017
Replied to a topic by wangyu8460958 Python Python 如何分批次导入 100W 条 mysql 数据
下面是我自己写的脚本,下面的脚本还能够优化吗?

#coding=utf-8

import MySQLdb

import numpy as np

conn = MySQLdb.connect(host='localhost',port = 3306,user='root',passwd='123456',db='google')
cur = conn.cursor()
conn.autocommit(1)

query_sql = "select id from google.Video where created_at < date_sub(now(), interval 1 year);"
insert_sql = "insert ignore into tmp.Video_2017bak (select * from google.Video where id = %s);"
delete_sql = "delete from google.Video where id = %s;"
cur.execute(query_sql)
dataList = cur.fetchall()
aaa = np.array(dataList)
ids = []

for i in range(len(aaa)):
ids.append(aaa[i])
if (i+1)%100==0 :
cur.executemany(insert_sql,ids)
ids = []
cur.executemany(insert_sql,ids)
ids = []

for i in range(len(aaa)):
ids.append(aaa[i])
if (i+1)%100==0 :
cur.executemany(delete_sql,ids)
ids = []
cur.executemany(delete_sql,ids)
ids = []

cur.close()
conn.close()
Apr 22, 2017
Replied to a topic by wangyu8460958 Python Python 如何分批次导入 100W 条 mysql 数据
@skydiver 我想使用脚本让它每周执行一次,所以想使用 python 脚本来做。
About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   3566 Online   Highest 6679   ·     Select Language
创意工作者们的社区
World is powered by solitude
VERSION: 3.9.8.5 · 19ms · UTC 04:55 · PVG 12:55 · LAX 21:55 · JFK 00:55
♥ Do have faith in what you're doing.