Apache ShardingSphere簡介及使用詳解
一 什么是 ShardingSphere?
1.1 背景:為什么需要分庫分表?
隨著系統(tǒng)數(shù)據(jù)量和用戶量不斷增長,傳統(tǒng)單體數(shù)據(jù)庫容易遇到瓶頸:
寫入/查詢壓力大:單表千萬級(jí)數(shù)據(jù),索引效率下降。
存儲(chǔ)超限:單機(jī)磁盤空間和 IOPS 不堪重負(fù)。
水平擴(kuò)展困難:業(yè)務(wù)難以橫向拓展。
為了解決這些問題,開發(fā)者往往采用 分庫分表 技術(shù),將一張大表分成多個(gè)小表,分布在不同數(shù)據(jù)庫中,實(shí)現(xiàn)水平擴(kuò)展和負(fù)載均衡。
1.2 什么是 Apache ShardingSphere?
Apache ShardingSphere 是一款開源的 分布式數(shù)據(jù)庫中間件框架,可以幫助開發(fā)者輕松實(shí)現(xiàn):
- 分庫分表
- 讀寫分離
- 分布式事務(wù)
- 數(shù)據(jù)脫敏
- 可觀測(cè)性 & 擴(kuò)展治理能力
ShardingSphere 核心組件包括:
| 組件 | 說明 |
|---|---|
| ShardingSphere-JDBC | JDBC 封裝層,嵌入應(yīng)用中,適用于微服務(wù)架構(gòu) |
| ShardingSphere-Proxy | 獨(dú)立部署的數(shù)據(jù)庫代理,支持多語言訪問 |
| ShardingSphere-Sidecar(規(guī)劃中) | 面向云原生場(chǎng)景,Service Mesh 結(jié)構(gòu) |
1.3 ShardingSphere-JDBC 的核心功能
| 功能 | 簡要說明 |
|---|---|
| 分庫分表 | 按字段配置路由策略,自動(dòng)將 SQL 分發(fā)到對(duì)應(yīng)的數(shù)據(jù)庫/表 |
| 讀寫分離 | 自動(dòng)區(qū)分讀寫請(qǐng)求,路由到主/從數(shù)據(jù)庫 |
| 靈活分片策略 | 支持取模、范圍、Hint、自定義算法等多種策略 |
| 事務(wù)支持 | 支持本地事務(wù)和分布式事務(wù)(Seata/XA) |
| 透明化使用 | 對(duì)開發(fā)者來說使用方式與普通 JDBC 幾乎一致,只需配置邏輯表名 |
二、實(shí)踐部分:Java + H2 實(shí)現(xiàn)分庫分表
示例:使用 Java 原生程序 + ShardingSphere-JDBC + H2 內(nèi)存數(shù)據(jù)庫,模擬一個(gè)電商訂單系統(tǒng)按 user_id 分庫,order_id 分表。
2.1 項(xiàng)目結(jié)構(gòu)
- 使用 H2 數(shù)據(jù)庫模擬兩個(gè)庫
ds0,ds1 - 每個(gè)庫中創(chuàng)建兩個(gè)分表
t_order_0,t_order_1 - 邏輯表名:
t_order
2.2 Maven 依賴配置
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc-core</artifactId>
<version>5.4.1</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.2.220</version>
</dependency>
<dependency>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
<version>5.0.1</version>
</dependency>
</dependencies>2.3 核心代碼結(jié)構(gòu)
1. 創(chuàng)建真實(shí)表(物理分表)
public static void createTables() throws SQLException {
for (int i = 0; i < 2; i++) {
String db = "ds" + i;
try (Connection conn = DriverManager.getConnection("jdbc:h2:mem:" + db + ";DB_CLOSE_DELAY=-1;MODE=MySQL", "sa", "")) {
Statement stmt = conn.createStatement();
for (int j = 0; j < 2; j++) {
stmt.execute(String.format("""
CREATE TABLE IF NOT EXISTS t_order_%d (
order_id BIGINT PRIMARY KEY,
user_id INT,
status VARCHAR(50)
)
""", j));
}
}
}
}2. 配置分庫分表數(shù)據(jù)源
public static DataSource createDataSource() throws SQLException {
Map<String, DataSource> dataSourceMap = new HashMap<>();
for (int i = 0; i < 2; i++) {
HikariDataSource ds = new HikariDataSource();
ds.setJdbcUrl("jdbc:h2:mem:ds" + i + ";DB_CLOSE_DELAY=-1;MODE=MySQL");
ds.setUsername("sa");
ds.setPassword("");
dataSourceMap.put("ds" + i, ds);
}
ShardingTableRuleConfiguration orderTableRule = new ShardingTableRuleConfiguration(
"t_order", "ds${0..1}.t_order_${0..1}");
orderTableRule.setDatabaseShardingStrategy(
new StandardShardingStrategyConfiguration("user_id", "dbSharding"));
orderTableRule.setTableShardingStrategy(
new StandardShardingStrategyConfiguration("order_id", "tableSharding"));
ShardingRuleConfiguration config = new ShardingRuleConfiguration();
config.getTables().add(orderTableRule);
config.getShardingAlgorithms().put("dbSharding",
new AlgorithmConfiguration("INLINE", props("algorithm-expression", "ds${user_id % 2}")));
config.getShardingAlgorithms().put("tableSharding",
new AlgorithmConfiguration("INLINE", props("algorithm-expression", "t_order_${order_id % 2}")));
return ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, List.of(config), new Properties());
}3. 插入與查詢數(shù)據(jù)
public static void insertOrder(DataSource ds, long orderId, int userId, String status) throws SQLException {
try (Connection conn = ds.getConnection()) {
PreparedStatement ps = conn.prepareStatement("INSERT INTO t_order (order_id, user_id, status) VALUES (?, ?, ?)");
ps.setLong(1, orderId);
ps.setInt(2, userId);
ps.setString(3, status);
ps.executeUpdate();
}
}
public static void queryOrders(DataSource ds) throws SQLException {
try (Connection conn = ds.getConnection()) {
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM t_order");
while (rs.next()) {
System.out.printf("order_id=%d, user_id=%d, status=%s%n",
rs.getLong("order_id"),
rs.getInt("user_id"),
rs.getString("status"));
}
}
}2.4 程序入口
public static void main(String[] args) throws Exception {
createTables(); // 創(chuàng)建實(shí)際分表
DataSource shardingDataSource = createDataSource();
insertOrder(shardingDataSource, 1001, 1, "INIT");
insertOrder(shardingDataSource, 1002, 2, "PAID");
insertOrder(shardingDataSource, 1003, 3, "SHIPPED");
queryOrders(shardingDataSource);
}輸出示例
order_id=1001, user_id=1, status=INIT
order_id=1002, user_id=2, status=PAID
order_id=1003, user_id=3, status=SHIPPED
ShardingSphere 已自動(dòng)根據(jù)你配置的策略將數(shù)據(jù)路由到對(duì)應(yīng)的庫和表!
總結(jié)
Apache ShardingSphere 提供了強(qiáng)大、靈活的分庫分表能力,通過合理配置可以極大提升系統(tǒng)的性能與擴(kuò)展性:
- 分片策略靈活,支持多種規(guī)則或自定義算法
- 配置簡單,無需改動(dòng)業(yè)務(wù) SQL
- 兼容性強(qiáng),支持 JDBC、Spring Boot、YAML 等多種使用方式
無論你是中小項(xiàng)目的快速原型,還是大規(guī)模高并發(fā)系統(tǒng),ShardingSphere 都是一個(gè)值得一試的解決方案。
完整代碼
package org.example;
import com.zaxxer.hikari.HikariDataSource;
import org.apache.shardingsphere.driver.api.ShardingSphereDataSourceFactory;
import org.apache.shardingsphere.infra.config.algorithm.AlgorithmConfiguration;
import org.apache.shardingsphere.sharding.api.config.ShardingRuleConfiguration;
import org.apache.shardingsphere.sharding.api.config.rule.ShardingTableRuleConfiguration;
import org.apache.shardingsphere.sharding.api.config.strategy.sharding.StandardShardingStrategyConfiguration;
import javax.sql.DataSource;
import java.sql.*;
import java.util.*;
public class ShardingH2Example {
public static void main(String[] args) throws Exception {
createTables(); // 注意:用 DriverManager 直連底層 db 建表
DataSource shardingDataSource = createDataSource();
// 插入測(cè)試數(shù)據(jù)
insertOrder(shardingDataSource, 1001, 1, "INIT");
insertOrder(shardingDataSource, 1002, 2, "PAID");
insertOrder(shardingDataSource, 1003, 3, "SHIPPED");
// 查詢測(cè)試數(shù)據(jù)
queryOrders(shardingDataSource);
}
public static DataSource createDataSource() throws Exception {
Map<String, DataSource> dataSourceMap = new HashMap<>();
// 模擬兩個(gè)庫:ds0 和 ds1(分別用兩個(gè)內(nèi)存 H2 實(shí)例模擬)
for (int i = 0; i < 2; i++) {
HikariDataSource ds = new HikariDataSource();
ds.setJdbcUrl("jdbc:h2:mem:ds" + i + ";DB_CLOSE_DELAY=-1;MODE=MySQL");
ds.setUsername("sa");
ds.setPassword("");
dataSourceMap.put("ds" + i, ds);
}
// 分表規(guī)則配置
ShardingTableRuleConfiguration orderTableRule = new ShardingTableRuleConfiguration(
"t_order", "ds${0..1}.t_order_${0..1}");
// 分庫策略(user_id)
orderTableRule.setDatabaseShardingStrategy(new StandardShardingStrategyConfiguration(
"user_id", "dbSharding"));
// 分表策略(order_id)
orderTableRule.setTableShardingStrategy(new StandardShardingStrategyConfiguration(
"order_id", "tableSharding"));
ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
shardingRuleConfig.getTables().add(orderTableRule);
Properties dbProps = new Properties();
dbProps.setProperty("algorithm-expression", "ds${user_id % 2}");
shardingRuleConfig.getShardingAlgorithms().put("dbSharding",
new AlgorithmConfiguration("INLINE", dbProps));
Properties tableProps = new Properties();
tableProps.setProperty("algorithm-expression", "t_order_${order_id % 2}");
shardingRuleConfig.getShardingAlgorithms().put("tableSharding",
new AlgorithmConfiguration("INLINE", tableProps));
return ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, List.of(shardingRuleConfig), new Properties());
}
public static void createTables() throws SQLException {
for (int i = 0; i < 2; i++) {
String dbName = "ds" + i;
try (Connection conn = DriverManager.getConnection("jdbc:h2:mem:" + dbName + ";DB_CLOSE_DELAY=-1;MODE=MySQL", "sa", "")) {
Statement stmt = conn.createStatement();
for (int j = 0; j < 2; j++) {
String sql = String.format("""
CREATE TABLE IF NOT EXISTS t_order_%d (
order_id BIGINT PRIMARY KEY,
user_id INT,
status VARCHAR(50)
)
""", j);
stmt.execute(sql);
}
}
}
}
// 輔助方法用于手動(dòng)連接底層 H2 數(shù)據(jù)源
private static HikariDataSource getH2DataSource(String name) {
HikariDataSource ds = new HikariDataSource();
ds.setJdbcUrl("jdbc:h2:mem:" + name + ";DB_CLOSE_DELAY=-1;MODE=MySQL");
ds.setUsername("sa");
ds.setPassword("");
return ds;
}
public static void insertOrder(DataSource ds, long orderId, int userId, String status) throws SQLException {
try (Connection conn = ds.getConnection()) {
PreparedStatement ps = conn.prepareStatement("INSERT INTO t_order (order_id, user_id, status) VALUES (?, ?, ?)");
ps.setLong(1, orderId);
ps.setInt(2, userId);
ps.setString(3, status);
ps.executeUpdate();
}
}
public static void queryOrders(DataSource ds) throws SQLException {
try (Connection conn = ds.getConnection()) {
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM t_order");
System.out.println("Query Results:");
while (rs.next()) {
System.out.printf("order_id: %d, user_id: %d, status: %s%n",
rs.getLong("order_id"),
rs.getInt("user_id"),
rs.getString("status"));
}
}
}
}到此這篇關(guān)于Apache ShardingSphere 初識(shí)使用的文章就介紹到這了,更多相關(guān)Apache ShardingSphere使用內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!
- SpringBoot?整合?ShardingSphere4.1.1實(shí)現(xiàn)分庫分表功能
- springboot如何使用yml文件方式配置shardingsphere
- ShardingSphere如何進(jìn)行sql重寫示例詳解
- SpringBoot3和ShardingSphere5框架實(shí)現(xiàn)數(shù)據(jù)分庫分表
- springboot整合shardingsphere和seata實(shí)現(xiàn)分布式事務(wù)的實(shí)踐
- Java基于ShardingSphere實(shí)現(xiàn)分庫分表的實(shí)例詳解
- ShardingSphere jdbc集成多數(shù)據(jù)源的實(shí)現(xiàn)步驟
- SpringBoot整合ShardingSphere的示例代碼
- Java中ShardingSphere分庫分表實(shí)戰(zhàn)
相關(guān)文章
Apache2.4.x版wampserver本地php服務(wù)器如何讓外網(wǎng)訪問及啟用.htaccess
這篇文章主要介紹了Apache2.4.x版wampserver本地php服務(wù)器如何讓外網(wǎng)訪問及啟用.htaccess,需要的朋友可以參考下2015-02-02
#實(shí)踐筆記#Ubuntu配置Apache+PHP+MySQL
在我早先的一篇文章《#實(shí)踐筆記#本地配置PHPnow與WordPress》,已經(jīng)討論過PHPnow的本地安裝教程。但是PHPnow套件是Windows平臺(tái)下的軟件,不適用我現(xiàn)在使用的平臺(tái)2012-03-03
Linux環(huán)境(CentOS6.7 64位)下安裝subversion1.9.5的方法
這篇文章主要介紹了Linux環(huán)境(CentOS6.7 64位)下安裝subversion1.9.5的方法,結(jié)合實(shí)例形式分析了CentOS下安裝subversion1.9.5的相關(guān)步驟、命令及操作注意事項(xiàng),需要的朋友可以參考下2018-04-04
Ubuntu徹底卸載MySQL、Apache2和Php的方法教程
這篇文章主要給大家介紹了關(guān)于在Ubuntu系統(tǒng)下徹底卸載MySQL、Apache2和Php的方法教程,文中通過示例代碼介紹的非常詳細(xì),對(duì)大家具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友們下面跟著小編來一起看看吧。2017-08-08
如何關(guān)閉http Methods中的Trace 提高安全意識(shí)
在配置文件http.conf 添加 TraceEnable off 即可關(guān)閉2013-02-02
linux環(huán)境配置nginx導(dǎo)致頁面不刷新的解決方法
這篇文章介紹了linux環(huán)境配置nginx導(dǎo)致頁面不刷新的解決方法,有需要的朋友可以參考一下2013-09-09
Linux中/var/spool/postfix/maildrop占空間很大的原因解析
這篇文章主要給大家介紹了關(guān)于Linux中/var/spool/postfix/maildrop占空間很大的原因解析,文中介紹的非常詳細(xì),需要的朋友可以參考借鑒,下面來一起看看吧。2017-04-04

