最新国产好看的视频,伊人天堂AV在线,国产Aaaaaa视频,蜜臀视频在线观看一区,人妻av色图,密臀久久久精品影片,青青视频免费观看毛片,久草在线观看视,国产三级精品色情在线

Python實現(xiàn)批量將CSV按行或按列拆分

 更新時間:2026年03月04日 08:27:20   作者:小莊-Python辦公  
這篇文章主要介紹了一個基于Python標(biāo)準(zhǔn)庫csv和PyQt5的高性能CSV拆分工具,文中的示例代碼講解詳細(xì),感興趣的小伙伴可以跟隨小編一起學(xué)習(xí)一下

本文完整拆解一個可用于生產(chǎn)的 CSV 拆分工具,從架構(gòu)、核心算法、GUI 與多線程到異常與性能邊界,全部基于 Python 標(biāo)準(zhǔn)庫 csv 與 PyQt5 實現(xiàn)。

效果圖

需求與目標(biāo)

核心目標(biāo)是實現(xiàn)一個桌面 GUI:

  • 多文件選擇與拖拽列表
  • 按行拆分與按列拆分兩種模式,參數(shù)聯(lián)動
  • 可配置輸出目錄、編碼、分隔符、命名規(guī)則與覆蓋策略
  • 支持 1GB / 500 萬行規(guī)模,內(nèi)存占用低,GUI 不阻塞
  • 實時進(jìn)度、日志與異常提示,支持取消與清理

總體設(shè)計

項目結(jié)構(gòu)極簡,僅一個入口腳本:

  • GUI 與業(yè)務(wù)邏輯全部在 [main.py]
  • 核心拆分函數(shù) split_csv_file 完成流式拆分與異常捕獲
  • Worker 在 QThread 中執(zhí)行,保證 GUI 響應(yīng)
  • 日志、進(jìn)度通過 Qt 信號回傳 UI

設(shè)計重點是把“耗時 IO 與 CPU”放在后臺線程,把“交互與展示”留給主線程。

CSV 拆分核心邏輯

核心函數(shù)是 [split_csv_file],僅使用 csv.reader/csv.writer,滿足低內(nèi)存與高兼容性。

1. 統(tǒng)一設(shè)置與結(jié)果模型

通過 SplitSettingsFileResult 描述輸入配置與輸出狀態(tài):

  • 選擇拆分模式、編碼、分隔符
  • 命名規(guī)則(前綴/后綴/序號格式)
  • 是否保留表頭、覆蓋策略
  • 統(tǒng)一錯誤信息與 traceback

2. 按行拆分

按行拆分包含兩種子模式:

  • 按固定行數(shù)切分
  • 按唯一列值分組輸出

按固定行數(shù)

關(guān)鍵點:

  • 每達(dá)到 rows_per_file 就關(guān)閉當(dāng)前輸出文件,啟動下一個
  • 表頭按需寫入
  • 通過 row_index 控制進(jìn)度刷新,避免頻繁更新 UI

示例片段(僅為理解):

if count == 0:
    seq_text = format_sequence(seq, settings.seq_format)
    filename = f"{settings.prefix}{seq_text}{settings.suffix}.csv"
    out_file = os.path.join(settings.output_dir, filename)

按唯一列值

核心策略是使用 WriterCache 復(fù)用輸出文件句柄,避免每行開關(guān)文件:

  • WriterCache 維護(hù)有限數(shù)量打開文件
  • 超出上限后按 LRU 關(guān)閉句柄
  • 已創(chuàng)建文件用 append 追加,避免重復(fù)寫表頭

3. 按列拆分與二次按行

按列拆分本質(zhì)上也是“按唯一值分組輸出”,并可選二次按行拆分:

  • 先按列值生成文件名
  • 若開啟二次拆分,按行數(shù)再切分出 value_序號.csv

4. 文件名安全與序號格式

為防止非法字符,統(tǒng)一使用 sanitize_filename 處理列值:

  • 替換系統(tǒng)禁止字符
  • 空值統(tǒng)一成 EMPTY
  • 過長截斷

序號格式支持兩種:

  • 純數(shù)字模板,如 0001
  • format 模式,如 {0:04d}

GUI 設(shè)計與交互流

GUI 核心在拆分成多個可復(fù)用面板。

1. 文件選擇與拖拽

FileListWidget 繼承 QListWidget,實現(xiàn)拖拽導(dǎo)入:

  • 過濾 csv 后綴
  • 自動去重

2. 模式切換

按行/按列通過 QStackedWidget 互斥顯示:

  • self.mode_rowself.mode_col 綁定
  • _switch_mode 切換索引

3. 參數(shù)聯(lián)動與列名讀取

當(dāng)文件列表變化或編碼/分隔符變化時,自動刷新列名:

  • 讀取首行作為表頭
  • 更新“按行唯一列”和“按列拆分列”下拉框

4. 輸出配置與命名規(guī)則

統(tǒng)一面板提供:

  • 輸出目錄選擇
  • 分隔符(逗號/分號/制表符/豎線)
  • 編碼(UTF-8/UTF-8-sig/GBK)
  • 覆蓋或跳過策略
  • 命名規(guī)則(前綴/后綴/序號格式)

多線程與任務(wù)生命周期

耗時任務(wù)在 QThread 中執(zhí)行:

  • run() 循環(huán)處理文件
  • 向 UI 發(fā)射總體進(jìn)度與單文件進(jìn)度
  • 每個文件獨立成功/失敗計數(shù),互不阻塞

調(diào)用流程:

  • _start() 創(chuàng)建線程與 Worker
  • 綁定信號:log、progress、finished
  • worker_thread.start() 開始后臺執(zhí)行

進(jìn)度與日志設(shè)計

進(jìn)度條

進(jìn)度分兩層:

  • 當(dāng)前文件進(jìn)度:讀取 file.tell() 與文件大小比例
  • 總體進(jìn)度:按文件大小加權(quán)

日志

日志輸出包含時間戳與異常堆棧:

  • timestamp() 統(tǒng)一格式
  • 失敗時記錄 traceback
  • 支持導(dǎo)出為文本文件

異常處理與容錯策略

異常處理貫穿拆分流程:

  • 編碼錯誤 UnicodeDecodeError
  • 列名缺失 ValueError
  • 磁盤空間不足 OSError + errno.ENOSPC

設(shè)計要點:

  • 單文件失敗不影響其他文件
  • 失敗信息寫入日志并彈窗提示
  • 失敗數(shù)在任務(wù)結(jié)束匯總

性能與內(nèi)存控制

針對 1GB 大文件設(shè)計要點:

  • csv.reader 流式讀取,行級處理
  • 輸出文件使用 WriterCache 限制句柄數(shù)量
  • 不緩存整文件或整列內(nèi)容

內(nèi)存占用主要來自:

  • 現(xiàn)有行字符串對象
  • 已打開的輸出句柄
  • GUI 層的日志與列表

對于“按唯一列值”的模式,如果唯一值極多:

  • 可適當(dāng)降低 max_open_files
  • 采用“跳過已存在文件”減少 IO

源代碼

'''
作者:小莊-Python辦公
公眾號:小莊-Python學(xué)習(xí)
'''
import sys
import os
import csv
import traceback
import errno
from dataclasses import dataclass, field
from datetime import datetime
from typing import Callable, Dict, List, Optional, Tuple

from PyQt5.QtCore import Qt, QObject, pyqtSignal, QThread
from PyQt5.QtGui import QFont
from PyQt5.QtWidgets import (
    QApplication,
    QMainWindow,
    QWidget,
    QFileDialog,
    QListWidget,
    QListWidgetItem,
    QVBoxLayout,
    QHBoxLayout,
    QLabel,
    QPushButton,
    QRadioButton,
    QButtonGroup,
    QStackedWidget,
    QGroupBox,
    QLineEdit,
    QSpinBox,
    QComboBox,
    QCheckBox,
    QTextEdit,
    QMessageBox,
    QProgressBar,
)


def timestamp() -> str:
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")


def sanitize_filename(value: str) -> str:
    if value is None:
        value = ""
    value = str(value).strip()
    if not value:
        value = "EMPTY"
    for ch in ['\\', '/', ':', '*', '?', '"', '<', '>', '|']:
        value = value.replace(ch, "_")
    value = value.replace("\n", "_").replace("\r", "_").replace("\t", "_")
    value = value.strip(" .")
    if not value:
        value = "EMPTY"
    if len(value) > 120:
        value = value[:120]
    return value


def format_sequence(seq: int, pattern: str) -> str:
    if "{" in pattern and "}" in pattern:
        return pattern.format(seq)
    width = max(1, len(pattern))
    return str(seq).zfill(width)


def ensure_dir(path: str) -> None:
    if not os.path.isdir(path):
        os.makedirs(path, exist_ok=True)


@dataclass
class SplitSettings:
    mode: str
    row_mode: str
    rows_per_file: int
    unique_column: str
    col_column: str
    col_secondary_split: bool
    col_secondary_rows: int
    prefix: str
    suffix: str
    seq_format: str
    keep_header: bool
    delimiter: str
    encoding: str
    output_dir: str
    on_exists: str
    max_open_files: int = 64


@dataclass
class FileResult:
    file_path: str
    success: bool
    created_files: List[str] = field(default_factory=list)
    skipped_files: List[str] = field(default_factory=list)
    error_message: str = ""
    exception: Optional[Exception] = None
    traceback_text: str = ""


def read_header(file_path: str, encoding: str, delimiter: str) -> List[str]:
    with open(file_path, "r", newline="", encoding=encoding) as f:
        reader = csv.reader(f, delimiter=delimiter)
        return next(reader, [])


class WriterCache:
    def __init__(self, max_open: int, created_set: set):
        self.max_open = max_open
        self._order: List[str] = []
        self._writers: Dict[str, Tuple[object, csv.writer]] = {}
        self._created_set = created_set

    def get_writer(self, key: str, file_path: str, encoding: str, delimiter: str, header: Optional[List[str]], on_exists: str, created_files: List[str], skipped_files: List[str]) -> Optional[csv.writer]:
        if key in self._writers:
            if key in self._order:
                self._order.remove(key)
            self._order.append(key)
            return self._writers[key][1]

        file_exists = os.path.exists(file_path)
        if file_exists and file_path not in self._created_set and on_exists == "skip":
            skipped_files.append(file_path)
            return None

        ensure_dir(os.path.dirname(file_path))
        if file_exists and file_path in self._created_set:
            mode = "a"
            header_to_write = None
        else:
            mode = "w"
            header_to_write = header
            self._created_set.add(file_path)
            created_files.append(file_path)
        f = open(file_path, mode, newline="", encoding=encoding)
        writer = csv.writer(f, delimiter=delimiter)
        if header_to_write:
            writer.writerow(header_to_write)
        self._writers[key] = (f, writer)
        self._order.append(key)
        while len(self._order) > self.max_open:
            evict_key = self._order.pop(0)
            handle, _ = self._writers.pop(evict_key)
            try:
                handle.close()
            except Exception:
                pass
        return writer

    def close_all(self) -> None:
        for handle, _ in self._writers.values():
            try:
                handle.close()
            except Exception:
                pass
        self._writers.clear()
        self._order.clear()


def split_csv_file(
    file_path: str,
    settings: SplitSettings,
    log_cb: Callable[[str], None],
    progress_cb: Callable[[float], None],
    cancel_cb: Callable[[], bool],
) -> FileResult:
    result = FileResult(file_path=file_path, success=True)
    file_size = os.path.getsize(file_path) if os.path.exists(file_path) else 0
    file_size = max(1, file_size)

    def update_progress(file_obj) -> None:
        try:
            ratio = min(1.0, file_obj.tell() / file_size)
            progress_cb(ratio)
        except Exception:
            pass

    try:
        with open(file_path, "r", newline="", encoding=settings.encoding) as f:
            reader = csv.reader(f, delimiter=settings.delimiter)
            header = next(reader, None)
            if header is None:
                return result
            header_row = header if settings.keep_header else None

            created_set: set = set()
            if settings.mode == "row":
                if settings.row_mode == "fixed":
                    seq = 1
                    count = 0
                    row_index = 0
                    writer = None
                    handle = None
                    for row in reader:
                        if cancel_cb():
                            result.success = False
                            result.error_message = "用戶取消"
                            break
                        if count == 0:
                            seq_text = format_sequence(seq, settings.seq_format)
                            filename = f"{settings.prefix}{seq_text}{settings.suffix}.csv"
                            out_file = os.path.join(settings.output_dir, filename)
                            if os.path.exists(out_file) and settings.on_exists == "skip":
                                result.skipped_files.append(out_file)
                                writer = None
                                handle = None
                            else:
                                ensure_dir(settings.output_dir)
                                handle = open(out_file, "w", newline="", encoding=settings.encoding)
                                writer = csv.writer(handle, delimiter=settings.delimiter)
                                if header_row:
                                    writer.writerow(header_row)
                                result.created_files.append(out_file)
                            seq += 1
                        if writer:
                            writer.writerow(row)
                        count += 1
                        row_index += 1
                        if count >= settings.rows_per_file:
                            count = 0
                            if handle:
                                try:
                                    handle.close()
                                except Exception:
                                    pass
                            writer = None
                            handle = None
                        if row_index % 2000 == 0:
                            update_progress(f)
                    if handle:
                        try:
                            handle.close()
                        except Exception:
                            pass
                    update_progress(f)
                else:
                    if settings.unique_column not in header:
                        raise ValueError(f"列名不存在: {settings.unique_column}")
                    col_index = header.index(settings.unique_column)
                    cache = WriterCache(settings.max_open_files, created_set)
                    skipped_values: set = set()
                    for row in reader:
                        if cancel_cb():
                            result.success = False
                            result.error_message = "用戶取消"
                            break
                        key = sanitize_filename(row[col_index] if col_index < len(row) else "")
                        if key in skipped_values:
                            continue
                        filename = f"{settings.prefix}{key}{settings.suffix}.csv"
                        out_file = os.path.join(settings.output_dir, filename)
                        writer = cache.get_writer(key, out_file, settings.encoding, settings.delimiter, header_row, settings.on_exists, result.created_files, result.skipped_files)
                        if writer is None:
                            skipped_values.add(key)
                            continue
                        writer.writerow(row)
                        if len(result.created_files) % 100 == 0:
                            update_progress(f)
                    cache.close_all()
                    update_progress(f)
            else:
                if settings.col_column not in header:
                    raise ValueError(f"列名不存在: {settings.col_column}")
                col_index = header.index(settings.col_column)
                cache = WriterCache(settings.max_open_files, created_set)
                value_state: Dict[str, Dict[str, int]] = {}
                skipped_values: set = set()
                for row in reader:
                    if cancel_cb():
                        result.success = False
                        result.error_message = "用戶取消"
                        break
                    value = sanitize_filename(row[col_index] if col_index < len(row) else "")
                    if value in skipped_values:
                        continue
                    if settings.col_secondary_split:
                        state = value_state.setdefault(value, {"seq": 1, "count": 0})
                        if state["count"] == 0:
                            seq_text = format_sequence(state["seq"], settings.seq_format)
                            filename = f"{settings.prefix}{value}_{seq_text}{settings.suffix}.csv"
                            out_file = os.path.join(settings.output_dir, filename)
                            writer = cache.get_writer(f"{value}:{state['seq']}", out_file, settings.encoding, settings.delimiter, header_row, settings.on_exists, result.created_files, result.skipped_files)
                            if writer is None:
                                skipped_values.add(value)
                                continue
                            state["seq"] += 1
                        else:
                            writer = cache.get_writer(f"{value}:{state['seq']-1}", os.path.join(settings.output_dir, f"{settings.prefix}{value}_{format_sequence(state['seq']-1, settings.seq_format)}{settings.suffix}.csv"), settings.encoding, settings.delimiter, None, settings.on_exists, result.created_files, result.skipped_files)
                        if writer:
                            writer.writerow(row)
                        state["count"] += 1
                        if state["count"] >= settings.col_secondary_rows:
                            state["count"] = 0
                    else:
                        filename = f"{settings.prefix}{value}{settings.suffix}.csv"
                        out_file = os.path.join(settings.output_dir, filename)
                        writer = cache.get_writer(value, out_file, settings.encoding, settings.delimiter, header_row, settings.on_exists, result.created_files, result.skipped_files)
                        if writer is None:
                            skipped_values.add(value)
                            continue
                        writer.writerow(row)
                    if len(result.created_files) % 100 == 0:
                        update_progress(f)
                cache.close_all()
                update_progress(f)
    except UnicodeDecodeError as e:
        result.success = False
        result.error_message = f"編碼錯誤: {e}"
        result.exception = e
        result.traceback_text = traceback.format_exc()
    except Exception as e:
        result.success = False
        result.error_message = str(e)
        result.exception = e
        result.traceback_text = traceback.format_exc()

    if result.exception and isinstance(result.exception, OSError) and getattr(result.exception, "errno", None) == errno.ENOSPC:
        log_cb(f"{timestamp()} 磁盤空間不足: {result.exception}")
    return result


class Worker(QObject):
    overall_progress = pyqtSignal(int)
    file_progress = pyqtSignal(int)
    log = pyqtSignal(str)
    finished = pyqtSignal(dict)
    show_error = pyqtSignal(str, str)

    def __init__(self, files: List[str], settings: SplitSettings):
        super().__init__()
        self.files = files
        self.settings = settings
        self._cancel = False
        self._created_files: List[str] = []

    def cancel(self) -> None:
        self._cancel = True

    def _is_cancelled(self) -> bool:
        return self._cancel

    def run(self) -> None:
        total_size = sum(max(1, os.path.getsize(p)) for p in self.files)
        processed_size = 0
        success_count = 0
        fail_count = 0

        for file_path in self.files:
            if self._cancel:
                break
            file_size = max(1, os.path.getsize(file_path))
            self.log.emit(f"{timestamp()} 開始處理: {file_path}")

            def log_cb(msg: str) -> None:
                self.log.emit(msg)

            def progress_cb(ratio: float) -> None:
                self.file_progress.emit(int(ratio * 100))
                overall_ratio = min(1.0, (processed_size + ratio * file_size) / total_size)
                self.overall_progress.emit(int(overall_ratio * 100))

            result = split_csv_file(file_path, self.settings, log_cb, progress_cb, self._is_cancelled)
            self._created_files.extend(result.created_files)
            if result.success:
                success_count += 1
                self.log.emit(f"{timestamp()} 完成: {file_path} 輸出 {len(result.created_files)} 個文件")
            else:
                fail_count += 1
                detail = result.error_message
                if result.traceback_text:
                    detail = f"{result.error_message}\n{result.traceback_text}"
                self.log.emit(f"{timestamp()} 失敗: {file_path}\n{detail}")
                if isinstance(result.exception, OSError) and getattr(result.exception, "errno", None) == errno.ENOSPC:
                    self.show_error.emit("磁盤空間不足", detail)
            processed_size += file_size
            self.file_progress.emit(100)
            self.overall_progress.emit(int(min(1.0, processed_size / total_size) * 100))

        summary = {
            "success": success_count,
            "failed": fail_count,
            "cancelled": self._cancel,
        }
        if self._cancel:
            for path in self._created_files:
                try:
                    if os.path.exists(path):
                        os.remove(path)
                except Exception:
                    pass
        self.finished.emit(summary)


class FileListWidget(QListWidget):
    def __init__(self):
        super().__init__()
        self.setSelectionMode(QListWidget.ExtendedSelection)
        self.setAcceptDrops(True)

    def dragEnterEvent(self, event):
        if event.mimeData().hasUrls():
            event.acceptProposedAction()
        else:
            super().dragEnterEvent(event)

    def dropEvent(self, event):
        if event.mimeData().hasUrls():
            for url in event.mimeData().urls():
                path = url.toLocalFile()
                if path and path.lower().endswith(".csv"):
                    self.add_file(path)
            event.acceptProposedAction()
        else:
            super().dropEvent(event)

    def add_file(self, path: str) -> None:
        for i in range(self.count()):
            if self.item(i).text() == path:
                return
        self.addItem(QListWidgetItem(path))

    def files(self) -> List[str]:
        return [self.item(i).text() for i in range(self.count())]


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("CSV 拆分工具 作者:小莊-Python辦公")
        self.resize(1000, 720)

        self.file_list = FileListWidget()
        self.add_btn = QPushButton("添加文件")
        self.remove_btn = QPushButton("移除選中")
        self.clear_btn = QPushButton("清空列表")

        self.mode_row = QRadioButton("按行拆分")
        self.mode_col = QRadioButton("按列拆分")
        self.mode_row.setChecked(True)
        self.mode_group = QButtonGroup()
        self.mode_group.addButton(self.mode_row)
        self.mode_group.addButton(self.mode_col)

        self.stack = QStackedWidget()
        self.row_panel = self._build_row_panel()
        self.col_panel = self._build_col_panel()
        self.stack.addWidget(self.row_panel)
        self.stack.addWidget(self.col_panel)

        self.prefix_input = QLineEdit("part_")
        self.suffix_input = QLineEdit("")
        self.seq_input = QLineEdit("0001")
        self.keep_header = QCheckBox("保留表頭")
        self.keep_header.setChecked(True)

        self.output_dir_input = QLineEdit("")
        self.output_browse_btn = QPushButton("選擇輸出目錄")
        self.delimiter_combo = QComboBox()
        self.delimiter_combo.addItems([",", ";", "\\t", "|"])
        self.encoding_combo = QComboBox()
        self.encoding_combo.addItems(["UTF-8", "UTF-8-sig", "GBK"])
        self.exists_overwrite = QRadioButton("覆蓋")
        self.exists_skip = QRadioButton("跳過")
        self.exists_overwrite.setChecked(True)

        self.start_btn = QPushButton("開始")
        self.cancel_btn = QPushButton("取消")
        self.cancel_btn.setEnabled(False)
        self.export_log_btn = QPushButton("導(dǎo)出日志")

        self.total_progress = QProgressBar()
        self.file_progress = QProgressBar()
        self.log_text = QTextEdit()
        self.log_text.setReadOnly(True)
        self.log_text.setFont(QFont("Consolas", 10))

        layout = QVBoxLayout()
        layout.addWidget(self._build_files_group())
        layout.addWidget(self._build_mode_group())
        layout.addWidget(self._build_naming_group())
        layout.addWidget(self._build_output_group())
        layout.addWidget(self._build_progress_group())
        layout.addWidget(self._build_log_group())

        container = QWidget()
        container.setLayout(layout)
        self.setCentralWidget(container)

        self.add_btn.clicked.connect(self._add_files)
        self.remove_btn.clicked.connect(self._remove_selected)
        self.clear_btn.clicked.connect(self.file_list.clear)
        self.mode_row.toggled.connect(self._switch_mode)
        self.mode_col.toggled.connect(self._switch_mode)
        self.output_browse_btn.clicked.connect(self._choose_output_dir)
        self.start_btn.clicked.connect(self._start)
        self.cancel_btn.clicked.connect(self._cancel)
        self.export_log_btn.clicked.connect(self._export_log)
        self.delimiter_combo.currentIndexChanged.connect(self._refresh_headers)
        self.encoding_combo.currentIndexChanged.connect(self._refresh_headers)
        self.file_list.itemSelectionChanged.connect(self._refresh_headers)

        self.worker_thread: Optional[QThread] = None
        self.worker: Optional[Worker] = None

    def _build_files_group(self) -> QGroupBox:
        group = QGroupBox("文件選擇")
        layout = QVBoxLayout()
        btn_layout = QHBoxLayout()
        btn_layout.addWidget(self.add_btn)
        btn_layout.addWidget(self.remove_btn)
        btn_layout.addWidget(self.clear_btn)
        layout.addLayout(btn_layout)
        layout.addWidget(self.file_list)
        group.setLayout(layout)
        return group

    def _build_mode_group(self) -> QGroupBox:
        group = QGroupBox("拆分模式")
        layout = QVBoxLayout()
        mode_layout = QHBoxLayout()
        mode_layout.addWidget(self.mode_row)
        mode_layout.addWidget(self.mode_col)
        layout.addLayout(mode_layout)
        layout.addWidget(self.stack)
        group.setLayout(layout)
        return group

    def _build_row_panel(self) -> QWidget:
        panel = QWidget()
        layout = QVBoxLayout()
        self.row_fixed = QRadioButton("按固定行數(shù)")
        self.row_unique = QRadioButton("按唯一列值")
        self.row_fixed.setChecked(True)
        self.row_group = QButtonGroup()
        self.row_group.addButton(self.row_fixed)
        self.row_group.addButton(self.row_unique)
        row_mode_layout = QHBoxLayout()
        row_mode_layout.addWidget(self.row_fixed)
        row_mode_layout.addWidget(self.row_unique)
        layout.addLayout(row_mode_layout)

        fixed_layout = QHBoxLayout()
        fixed_layout.addWidget(QLabel("每文件行數(shù)"))
        self.rows_per_file = QSpinBox()
        self.rows_per_file.setRange(1, 10_000_000)
        self.rows_per_file.setValue(100000)
        fixed_layout.addWidget(self.rows_per_file)
        layout.addLayout(fixed_layout)

        unique_layout = QHBoxLayout()
        unique_layout.addWidget(QLabel("列名"))
        self.row_unique_column = QComboBox()
        unique_layout.addWidget(self.row_unique_column)
        layout.addLayout(unique_layout)

        panel.setLayout(layout)
        return panel

    def _build_col_panel(self) -> QWidget:
        panel = QWidget()
        layout = QVBoxLayout()
        col_layout = QHBoxLayout()
        col_layout.addWidget(QLabel("列名"))
        self.col_column = QComboBox()
        col_layout.addWidget(self.col_column)
        layout.addLayout(col_layout)

        split_layout = QHBoxLayout()
        self.col_secondary = QCheckBox("每子文件再按行數(shù)拆分")
        split_layout.addWidget(self.col_secondary)
        split_layout.addWidget(QLabel("行數(shù)"))
        self.col_secondary_rows = QSpinBox()
        self.col_secondary_rows.setRange(1, 10_000_000)
        self.col_secondary_rows.setValue(100000)
        split_layout.addWidget(self.col_secondary_rows)
        layout.addLayout(split_layout)
        panel.setLayout(layout)
        return panel

    def _build_naming_group(self) -> QGroupBox:
        group = QGroupBox("命名規(guī)則")
        layout = QVBoxLayout()
        row1 = QHBoxLayout()
        row1.addWidget(QLabel("前綴"))
        row1.addWidget(self.prefix_input)
        row1.addWidget(QLabel("后綴"))
        row1.addWidget(self.suffix_input)
        layout.addLayout(row1)
        row2 = QHBoxLayout()
        row2.addWidget(QLabel("序號格式"))
        row2.addWidget(self.seq_input)
        row2.addWidget(self.keep_header)
        layout.addLayout(row2)
        group.setLayout(layout)
        return group

    def _build_output_group(self) -> QGroupBox:
        group = QGroupBox("輸出配置")
        layout = QVBoxLayout()
        row1 = QHBoxLayout()
        row1.addWidget(QLabel("輸出目錄"))
        row1.addWidget(self.output_dir_input)
        row1.addWidget(self.output_browse_btn)
        layout.addLayout(row1)

        row2 = QHBoxLayout()
        row2.addWidget(QLabel("分隔符"))
        row2.addWidget(self.delimiter_combo)
        row2.addWidget(QLabel("編碼"))
        row2.addWidget(self.encoding_combo)
        layout.addLayout(row2)

        row3 = QHBoxLayout()
        row3.addWidget(QLabel("已存在文件"))
        row3.addWidget(self.exists_overwrite)
        row3.addWidget(self.exists_skip)
        layout.addLayout(row3)
        group.setLayout(layout)
        return group

    def _build_progress_group(self) -> QGroupBox:
        group = QGroupBox("進(jìn)度")
        layout = QVBoxLayout()
        layout.addWidget(QLabel("總體進(jìn)度"))
        layout.addWidget(self.total_progress)
        layout.addWidget(QLabel("當(dāng)前文件進(jìn)度"))
        layout.addWidget(self.file_progress)
        row = QHBoxLayout()
        row.addWidget(self.start_btn)
        row.addWidget(self.cancel_btn)
        row.addWidget(self.export_log_btn)
        layout.addLayout(row)
        group.setLayout(layout)
        return group

    def _build_log_group(self) -> QGroupBox:
        group = QGroupBox("日志")
        layout = QVBoxLayout()
        layout.addWidget(self.log_text)
        group.setLayout(layout)
        return group

    def _switch_mode(self) -> None:
        self.stack.setCurrentIndex(0 if self.mode_row.isChecked() else 1)

    def _add_files(self) -> None:
        files, _ = QFileDialog.getOpenFileNames(self, "選擇 CSV 文件", "", "CSV Files (*.csv)")
        for path in files:
            self.file_list.add_file(path)
        self._refresh_headers()

    def _remove_selected(self) -> None:
        for item in self.file_list.selectedItems():
            row = self.file_list.row(item)
            self.file_list.takeItem(row)
        self._refresh_headers()

    def _choose_output_dir(self) -> None:
        path = QFileDialog.getExistingDirectory(self, "選擇輸出目錄")
        if path:
            self.output_dir_input.setText(path)

    def _delimiter_value(self) -> str:
        text = self.delimiter_combo.currentText()
        return "\t" if text == "\\t" else text

    def _refresh_headers(self) -> None:
        files = self.file_list.files()
        if not files:
            self.row_unique_column.clear()
            self.col_column.clear()
            return
        path = files[0]
        try:
            header = read_header(path, self.encoding_combo.currentText(), self._delimiter_value())
            self.row_unique_column.clear()
            self.col_column.clear()
            self.row_unique_column.addItems(header)
            self.col_column.addItems(header)
        except Exception as e:
            self.log_text.append(f"{timestamp()} 讀取表頭失敗: {e}")
            self.row_unique_column.clear()
            self.col_column.clear()

    def _build_settings(self) -> Optional[SplitSettings]:
        files = self.file_list.files()
        if not files:
            QMessageBox.warning(self, "提示", "請先選擇 CSV 文件")
            return None
        output_dir = self.output_dir_input.text().strip()
        if not output_dir:
            QMessageBox.warning(self, "提示", "請指定輸出目錄")
            return None
        delimiter = self._delimiter_value()
        encoding = self.encoding_combo.currentText()
        on_exists = "overwrite" if self.exists_overwrite.isChecked() else "skip"
        mode = "row" if self.mode_row.isChecked() else "col"
        row_mode = "fixed" if self.row_fixed.isChecked() else "unique"
        settings = SplitSettings(
            mode=mode,
            row_mode=row_mode,
            rows_per_file=self.rows_per_file.value(),
            unique_column=self.row_unique_column.currentText(),
            col_column=self.col_column.currentText(),
            col_secondary_split=self.col_secondary.isChecked(),
            col_secondary_rows=self.col_secondary_rows.value(),
            prefix=self.prefix_input.text(),
            suffix=self.suffix_input.text(),
            seq_format=self.seq_input.text() or "0001",
            keep_header=self.keep_header.isChecked(),
            delimiter=delimiter,
            encoding=encoding,
            output_dir=output_dir,
            on_exists=on_exists,
        )
        return settings

    def _start(self) -> None:
        settings = self._build_settings()
        if not settings:
            return
        files = self.file_list.files()
        ensure_dir(settings.output_dir)
        self.total_progress.setValue(0)
        self.file_progress.setValue(0)
        self.start_btn.setEnabled(False)
        self.cancel_btn.setEnabled(True)
        self.log_text.append(f"{timestamp()} 任務(wù)開始,文件數(shù): {len(files)}")

        self.worker_thread = QThread()
        self.worker = Worker(files, settings)
        self.worker.moveToThread(self.worker_thread)
        self.worker_thread.started.connect(self.worker.run)
        self.worker.log.connect(self.log_text.append)
        self.worker.file_progress.connect(self.file_progress.setValue)
        self.worker.overall_progress.connect(self.total_progress.setValue)
        self.worker.finished.connect(self._finish)
        self.worker.show_error.connect(self._show_error)
        self.worker_thread.start()

    def _finish(self, summary: dict) -> None:
        if self.worker_thread:
            self.worker_thread.quit()
            self.worker_thread.wait()
        self.start_btn.setEnabled(True)
        self.cancel_btn.setEnabled(False)
        if summary.get("cancelled"):
            QMessageBox.information(self, "已取消", "任務(wù)已取消,臨時文件已清理")
        else:
            QMessageBox.information(
                self,
                "完成",
                f"成功: {summary.get('success', 0)},失敗: {summary.get('failed', 0)}",
            )

    def _cancel(self) -> None:
        if self.worker:
            self.worker.cancel()
            self.log_text.append(f"{timestamp()} 用戶取消任務(wù)")
            self.cancel_btn.setEnabled(False)

    def _show_error(self, title: str, detail: str) -> None:
        QMessageBox.critical(self, title, detail)

    def _export_log(self) -> None:
        path, _ = QFileDialog.getSaveFileName(self, "導(dǎo)出日志", "log.txt", "Text Files (*.txt)")
        if not path:
            return
        try:
            with open(path, "w", encoding="utf-8") as f:
                f.write(self.log_text.toPlainText())
            QMessageBox.information(self, "成功", "日志已導(dǎo)出")
        except Exception as e:
            QMessageBox.critical(self, "失敗", str(e))


def main() -> None:
    app = QApplication(sys.argv)
    window = MainWindow()
    window.show()
    sys.exit(app.exec_())


if __name__ == "__main__":
    main()

可擴展方向

如果繼續(xù)增強,可考慮:

  • 增加“預(yù)覽前 N 行”功能
  • 支持保存與加載配置
  • 增加“輸出文件壓縮”為 zip
  • 提供“拆分后統(tǒng)計報表”

結(jié)語

這個工具的核心理念是“穩(wěn)定、低內(nèi)存、高可用”。通過標(biāo)準(zhǔn)庫 csv 與 PyQt5 的組合,達(dá)成了性能與交互的平衡,并保持項目結(jié)構(gòu)簡單、易維護(hù)。對于日常辦公與大規(guī)模數(shù)據(jù)拆分,已具備可直接使用的工程級質(zhì)量。

以上就是Python實現(xiàn)批量將CSV按行或按列拆分的詳細(xì)內(nèi)容,更多關(guān)于Python拆分CSV的資料請關(guān)注腳本之家其它相關(guān)文章!

相關(guān)文章

  • Pytorch+PyG實現(xiàn)EdgeCNN過程示例詳解

    Pytorch+PyG實現(xiàn)EdgeCNN過程示例詳解

    這篇文章主要為大家介紹了Pytorch+PyG實現(xiàn)EdgeCNN過程示例詳解,有需要的朋友可以借鑒參考下,希望能夠有所幫助,祝大家多多進(jìn)步,早日升職加薪
    2023-04-04
  • 一文學(xué)會如何將Python打包后的exe還原成.py

    一文學(xué)會如何將Python打包后的exe還原成.py

    反編譯的第一步就是要將exe文件轉(zhuǎn)換成py文件,下面這篇文章主要給大家介紹了如何通過一文學(xué)會將Python打包后的exe還原成.py的相關(guān)資料,文中通過實例代碼介紹的非常詳細(xì),需要的朋友可以參考下
    2022-11-11
  • Python中Dict兩種實現(xiàn)的原理詳解

    Python中Dict兩種實現(xiàn)的原理詳解

    在Python中,?Dict是一系列由鍵和值配對組成的元素的集合,?它是一個可變?nèi)萜髂P?,可以存儲任意類型對象。本文主要介紹了Dict兩種實現(xiàn)的原理,感興趣的可以了解一下
    2023-03-03
  • python列表刪除元素的三種實現(xiàn)方法

    python列表刪除元素的三種實現(xiàn)方法

    本文主要介紹了python列表刪除元素的三種實現(xiàn)方法,主要包括pop方法,remove方法,del方法這三種,具有一定的參考價值,感興趣的可以了解一下
    2024-01-01
  • Python自動化辦公之Word文檔的創(chuàng)建與生成

    Python自動化辦公之Word文檔的創(chuàng)建與生成

    這篇文章主要為大家詳細(xì)介紹了如何通過python腳本來自動生成一個?word文檔,文中的示例代碼講解詳細(xì),感興趣的小伙伴可以了解一下
    2022-05-05
  • python實現(xiàn)上傳下載文件功能

    python實現(xiàn)上傳下載文件功能

    這篇文章主要為大家詳細(xì)介紹了python實現(xiàn)上傳下載文件功能,文中示例代碼介紹的非常詳細(xì),具有一定的參考價值,感興趣的小伙伴們可以參考一下
    2017-07-07
  • Python實現(xiàn)在Word中創(chuàng)建,讀取和刪除列表詳解

    Python實現(xiàn)在Word中創(chuàng)建,讀取和刪除列表詳解

    在Word中,列表是一種用于組織和呈現(xiàn)信息的有效工具,這篇文章將探討一下如何使用Python在Word文檔中創(chuàng)建,讀取和刪除列表,需要的可以了解下
    2025-03-03
  • Python基礎(chǔ)知識點 初識Python.md

    Python基礎(chǔ)知識點 初識Python.md

    在本篇文章中我們給大家總結(jié)了關(guān)于Python基礎(chǔ)知識點,通過初識Python.md的相關(guān)內(nèi)容分享給Python初學(xué)者,一起來看下吧。
    2019-05-05
  • Python文件的寫入操作write與writelines方法詳解

    Python文件的寫入操作write與writelines方法詳解

    在Python編程中,文件操作是一個非常重要的基礎(chǔ)知識,無論是處理數(shù)據(jù)、保存配置還是記錄日志,我們都需要頻繁地進(jìn)行文件讀寫操作,今天我們就來深入探討Python中的文件寫入操作,特別是write()和writelines()這兩個核心方法,需要的朋友可以參考下
    2026-05-05
  • python裝飾器深入學(xué)習(xí)

    python裝飾器深入學(xué)習(xí)

    這篇文章主要深入學(xué)習(xí)了python裝飾器的相關(guān)資料,什么是裝飾器?裝飾器遵循的原則等,具有一定的參考價值,感興趣的小伙伴們可以參考一下
    2018-04-04

最新評論

韩城市| 陈巴尔虎旗| 铁岭县| 崇左市| 漠河县| 中阳县| 文化| 贵溪市| 江永县| 黄陵县| 肇源县| 永顺县| 南阳市| 铜鼓县| 淄博市| 道孚县| 渭南市| 平顶山市| 永嘉县| 斗六市| 边坝县| 衡东县| 昌平区| 景洪市| 漳浦县| 青河县| 佛冈县| 乌兰浩特市| 彰化市| 邯郸市| 济阳县| 台安县| 儋州市| 甘肃省| 惠东县| 黔西| 舒兰市| 日照市| 炎陵县| 象州县| 甘孜县|