ShardingSphere:Sharding-Proxy实现分库分表

发布于 2022-12-04  1327 次阅读


前置工作:

ShardingSphere-Proxy就是代理,类似网关,对外暴露一个入口,请求过来时,将请求转发到配置的真实的mysql实例。
转发之前做了一些处理工作,如果请求(也就是sql语句)带了分片键,则根据分片算法计算出真实的mysql实例以及表名,查询效率高。
如果sql是范围查询,则会进行全表扫描,也就是广播,查询完之后进行合并。

下载:

sharding-proxy(5.x版):

https://apache.website-solution.net/shardingsphere/5.0.0-alpha/apache-shardingsphere-5.0.0-alpha-shardingsphere-proxy-bin.tar.gz​apache.website-solution.net/shardingsphere/5.0.0-alpha/apache-shardingsphere-5.0.0-alpha-shardingsphere-proxy-bin.tar.gz

proxy解压后如下:

我这里是mysql,下载mysql驱动,复制到lib目录下

编辑 conf下文件,config-sharding.yaml

vim config-sharding.yaml

释放 If you want to connect to MySQL, you should manually copy MySQL driver to lib directory. 下注释

# 对外数据库名称
schemaName: sharding_db
# 数据库链接公共参数
dataSourceCommon:
 username: root
 password: 123456
 connectionTimeoutMilliseconds: 30000
 idleTimeoutMilliseconds: 60000
 maxLifetimeMilliseconds: 1800000
 maxPoolSize: 50
 minPoolSize: 1
 maintenanceIntervalMilliseconds: 30000
# 数据库地址,这里配置了两个库
dataSources:
 ds_0:
   url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
 ds_1:
   url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
# 分片规则
rules:
- !SHARDING
 tables:
   # 虚拟表名称,最后登陆 proxy之后,sharding_db库下只有一张表 test(我这里只分了一个表,用于测试)
   test:
     # ds_${0..1} 分库规则,库索引从0到1,一共两个,前缀为:ds_, test,分成3张表,索引从0到2,前缀为:test_
     actualDataNodes: ds_${0..1}.test_${0..2}
     tableStrategy:
       standard:
         # 分片键
         shardingColumn: id
         shardingAlgorithmName: test_inline
     keyGenerateStrategy:
       column: id
       keyGeneratorName: snowflake  #主键生成策略 -- 雪花算法
# 默认数据库分片规则
 defaultDatabaseStrategy:
   standard:
     # 依据 id 进行分片
     shardingColumn: id
     shardingAlgorithmName: database_inline
 defaultTableStrategy:
   none:
 
 shardingAlgorithms:
   database_inline:
     type: INLINE
     props:
       # 数据库分片规则, id取模2,结果有 0 和 1,路由到 0 和 1这两个数据库
       algorithm-expression: ds_${id % 2}
   test_inline:
     type: INLINE
     props:
       # 数据库表分表规则:id 模 3,结果有:0、1、2, 得到 test_0、test_1、test_2这三张表
       algorithm-expression: test_${id % 3}
 keyGenerators:
   snowflake:
     type: SNOWFLAKE
     props:
       worker-id: 123

配置规则已经记录在注释中

接下来启动 sharding-proxy :

cd apache-shardingsphere-5.0.0-alpha-shardingsphere-proxy-bin/bin

./start.sh

查看log验证是否启动成功

cd ..
cd logs
tail -f stdout.log
如果出现 o.a.s.p.frontend.ShardingSphereProxy - ShardingSphere-Proxy start success., 则启动成功

连接mysql

mysql -uroot -proot -P3307

show databases;
mysql> show databases;
+-------------+
| Database    |
+-------------+
| sharding_db |
+-------------+
1 row in set (0.02 sec)
这就是对外暴露的库
mysql> use sharding_db;
mysql> show tables;
+---------------------+
| Tables_in_demo_ds_0 |
+---------------------+
| test                |
+---------------------+
1 row in set (0.01 sec)
这就是我们配置的虚拟表

到这里已经我们的代理层已经搭建成功了,如果要集成项目,把代理层的jdbc地址当常规的mysql地址写在yml中即可。

测试数据

新建表:

 CREATE TABLE `test` (
  `id` bigint(20) NOT NULL COMMENT '主键id',
  `name` varchar(64) NOT NULL DEFAULT '' COMMENT '名称',
  `age` int(11) NOT NULL DEFAULT '0' COMMENT '年龄',
  `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

验证插入数据是否自动按照分片规则存储数据:

mysql> insert into test(name, age, create_time) values('张三', 20, now()), ('王五', 30, now());
Query OK, 2 rows affected (0.02 sec)

mysql> select * from test;
+--------------------+--------+-----+---------------------+
| id                 | name   | age | create_time         |
+--------------------+--------+-----+---------------------+
| 561687339941212162 | 王五   |  30 | 2021-01-28 23:04:57 |
| 561687339941212161 | 张三   |  20 | 2021-01-28 23:04:57 |
+--------------------+--------+-----+---------------------+
2 rows in set (0.07 sec)

id为561687339941212162的,按照分片算法:561687339941212162 % 2 = 0,561687339941212162 % 3 = 1,所以在demo_ds_0号数据库,表数据在test_1表

mysql> show tables;
+---------------------+
| Tables_in_demo_ds_0 |
+---------------------+
| test_0              |
| test_1              |
+---------------------+
2 rows in set (0.00 sec)

mysql> select * from test_1;
+--------------------+--------+-----+---------------------+
| id                 | name   | age | create_time         |
+--------------------+--------+-----+---------------------+
| 561687339941212162 | 王五   |  30 | 2021-01-28 23:04:57 |
+--------------------+--------+-----+---------------------+
1 row in set (0.00 sec)

id 为 561687339941212161 的数据,561687339941212161 % 2 = 1, 561687339941212161 % 3 = 0,所以在 demo_ds_1库,表数据在:test_0表:

mysql> show tables;
+---------------------+
| Tables_in_demo_ds_1 |
+---------------------+
| test_0              |
| test_1              |
+---------------------+
2 rows in set (0.00 sec)

mysql> select * from test_0;
+--------------------+--------+-----+---------------------+
| id                 | name   | age | create_time         |
+--------------------+--------+-----+---------------------+
| 561687339941212161 | 张三   |  20 | 2021-01-28 23:04:57 |
+--------------------+--------+-----+---------------------+
1 row in set (0.00 sec)

欢迎欢迎~热烈欢迎~