MySQL Server层的 max_connections 和引擎层的 innodb_thread_concurrency

2023-09-16 18:52:00

MySQL 数据库服务端是由 Server 层 和 引擎层组成

  • Server 层包括连接器、查询缓存、分析器、优化器、执行器等,涵盖 MySQL
    的大多数核心服务功能,以及所有的内置函数(如日期、时间、数学和加密函数等),所有跨存储引擎的功能都在这一层实现,比如存储过程、触发器、视图等。
  • 存储引擎层负责数据的存储和提取。其架构模式是插件式的,支持 InnoDB、MyISAM、Memory 等多个存储引擎。从 MySQL 5.5.5 版本开始InnoDB成为了默认存储引擎。

max_connections: Server层的最大连接并发数

通过 mysql client 或者 驱动程序连接 MySQL时,每创建一个连接,MySQL 的 Server层 就会创建一个线程来处理该连接(实际情况更可能是 MySQL server 是用 thread pool 线程池来处理连接请求的:当客户端1的连接断开时,对应的线程进入线程池而不是销毁;客户端2马上建立连接就会发现客户端1用过的THREAD_OS_ID在处理客户端2的连接;站在操作系统角度看对于建立和断开链接不会存在线程的创建和销毁,当然前提是线程池里有空闲线程);此处的线程就是通过 show processlist 查看到的列表,你可能会有疑问 show processlist 的 id 和操作系统 ps 查看到的进程或线程id 有什么关系? 其实是一一对应的,比如 show processlist 的 id 844227 就是 performance_schema.threads 表的列 PROCESSLIST_ID 从而可以得知对应的 THREAD_OS_ID 为6789,通过 ps -eLf 就可以看到该值。(具体可以并发处理多少连接是由环境变量 max_connections 决定的,这是 MySQL Server层的行为还未进入引擎层,也就是说和环境变量 innodb_thread_concurrency 还没有关系呢) 具体如下:

	mysql> show processlist;
	+--------+------+-----------+------+---------+------+----------+------------------+
	| Id     | User | Host      | db   | Command | Time | State    | Info             |
	+--------+------+-----------+------+---------+------+----------+------------------+
	| 844227 | root | localhost | NULL | Sleep   |  530 |          | NULL             |
	| 844228 | root | localhost | NULL | Query   |    0 | starting | show processlist |
	| 844230 | root | localhost | NULL | Sleep   |  505 |          | NULL             |
	| 844231 | root | localhost | NULL | Sleep   |  959 |          | NULL             |
	| 844232 | root | localhost | NULL | Sleep   |  930 |          | NULL             |
	| 844233 | root | localhost | NULL | Sleep   |  845 |          | NULL             |
	| 844234 | root | localhost | NULL | Sleep   |  834 |          | NULL             |
	| 844236 | root | localhost | NULL | Sleep   |  836 |          | NULL             |
	| 844237 | root | localhost | NULL | Sleep   |  583 |          | NULL             |
	+--------+------+-----------+------+---------+------+----------+------------------+
	9 rows in set (0.00 sec)
	mysql> 
	mysql> //show processlist 为9个连接,对应的Threads_connected 也为9,
	mysql> //同时 Threads_cached 为0代表线程池无空闲线程;第10个连接进来时会创建新的线程
	mysql> SHOW STATUS LIKE 'Threads%'; 
	+-------------------+-------+
	| Variable_name     | Value |
	+-------------------+-------+
	| Threads_cached    | 0     |
	| Threads_connected | 9     |
	| Threads_created   | 2477  |
	| Threads_running   | 1     |
	+-------------------+-------+
	4 rows in set (0.01 sec)
	mysql>
	mysql> select *  from performance_schema.threads  where type = 'FOREGROUND' and name like '%conn%' ;
	+-----------+---------------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+-------------------+---------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+
	| THREAD_ID | NAME                      | TYPE       | PROCESSLIST_ID | PROCESSLIST_USER | PROCESSLIST_HOST | PROCESSLIST_DB | PROCESSLIST_COMMAND | PROCESSLIST_TIME | PROCESSLIST_STATE | PROCESSLIST_INFO                                                                            | PARENT_THREAD_ID | ROLE | INSTRUMENTED | HISTORY | CONNECTION_TYPE | THREAD_OS_ID |
	+-----------+---------------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+-------------------+---------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+
	|    844252 | thread/sql/one_connection | FOREGROUND |         844227 | root             | localhost        | NULL           | Sleep               |              581 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |         6789 |
	|    844253 | thread/sql/one_connection | FOREGROUND |         844228 | root             | localhost        | NULL           | Query               |                0 | Sending data      | select *  from performance_schema.threads  where type = 'FOREGROUND' and name like '%conn%' |             NULL | NULL | YES          | YES     | Socket          |        28020 |
	|    844255 | thread/sql/one_connection | FOREGROUND |         844230 | root             | localhost        | NULL           | Sleep               |              556 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28276 |
	|    844256 | thread/sql/one_connection | FOREGROUND |         844231 | root             | localhost        | NULL           | Sleep               |             1010 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28036 |
	|    844257 | thread/sql/one_connection | FOREGROUND |         844232 | root             | localhost        | NULL           | Sleep               |              981 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28356 |
	|    844258 | thread/sql/one_connection | FOREGROUND |         844233 | root             | localhost        | NULL           | Sleep               |              896 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |         6785 |
	|    844259 | thread/sql/one_connection | FOREGROUND |         844234 | root             | localhost        | NULL           | Sleep               |              885 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28277 |
	|    844261 | thread/sql/one_connection | FOREGROUND |         844236 | root             | localhost        | NULL           | Sleep               |              887 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28222 |
	|    844262 | thread/sql/one_connection | FOREGROUND |         844237 | root             | localhost        | NULL           | Sleep               |              634 | NULL              | NULL                                                                                        |             NULL | NULL | YES          | YES     | Socket          |        28360 |
	+-----------+---------------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+-------------------+---------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+
	9 rows in set (0.00 sec)
	mysql>
	[root@dbs-test-rob ~]# //看最后一行的线程的启动时间 16:43 可知,这就是在线程池中无空闲线程时为接下来的新建连接(第10个连接)而创建的线程
	[root@dbs-test-rob ~]# ps -eLf | grep mysqld 
	mysql     4105     1 28020  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1 28036  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1 28222  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1 28276  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1 28277  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1 28356  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1 28360  0   37 9月06 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1  6785  0   37 9月08 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1  6789  0   37 9月08 ?       00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	mysql     4105     1 23536  0   37 16:43 ?        00:00:00 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
	[root@dbs-test-rob ~]#

innodb_thread_concurrency: InnoDB 引擎层的最大并发执行的线程数

innodb_thread_concurrency:InnoDB tries to keep the number of operating system threads concurrently inside InnoDB less than or equal to the limit given by this variable (InnoDB uses operating system threads to process user transactions). Once the number of threads reaches this limit, additional threads are placed into a wait state within a “First In, First Out” (FIFO) queue for execution(一旦数量达到了限制,额外的线程被放到了 FIFO队列里). Threads waiting for locks are not counted in the number of concurrently executing threads.

innodb_thread_sleep_delay:Defines how long InnoDB threads sleep before joining the InnoDB queue, in microseconds. The default value is 10000. A value of 0 disables sleep. You can set innodb_adaptive_max_sleep_delay to the highest value you would allow for innodb_thread_sleep_delay, and InnoDB automatically adjusts innodb_thread_sleep_delay up or down depending on current thread-scheduling activity. This dynamic adjustment helps the thread scheduling mechanism to work smoothly during times when the system is lightly loaded or when it is operating near full capacity.

innodb_concurrency_tickets:Determines the number of threads that can enter InnoDB concurrently. A thread is placed in a queue when it tries to enter InnoDB if the number of threads has already reached the concurrency limit. When a thread is permitted to enter InnoDB, it is given a number of “ tickets” equal to the value of innodb_concurrency_tickets, and the thread can enter and leave InnoDB freely until it has used up its tickets. After that point, the thread again becomes subject to the concurrency check (and possible queuing) the next time it tries to enter InnoDB. The default value is 5000.

With a small innodb_concurrency_tickets value, small transactions that only need to process a few rows compete fairly with larger transactions that process many rows. The disadvantage of a small innodb_concurrency_tickets value is that large transactions must loop through the queue many times before they can complete, which extends the amount of time required to complete their task.(大事务在完成之前必须多次通过队列,这增大了他们完成事务的总时间)

With a large innodb_concurrency_tickets value, large transactions spend less time waiting for a position at the end of the queue (controlled by innodb_thread_concurrency) and more time retrieving rows. Large transactions also require fewer trips through the queue to complete their task. The disadvantage of a large innodb_concurrency_tickets value is that too many large transactions running at the same time can starve smaller transactions by making them wait a longer time before executing.

With a nonzero innodb_thread_concurrency value, you may need to adjust the innodb_concurrency_tickets value up or down to find the optimal balance between larger and smaller transactions. The SHOW ENGINE INNODB STATUS report shows the number of tickets remaining(剩下的tickets) for an executing transaction in its current pass through the queue. This data may also be obtained from the TRX_CONCURRENCY_TICKETS column of the Information Schema INNODB_TRX table.

进一步的思考:
如果 MySQL 系统中,突然慢查询陡增或者普通的一个查询也需要耗时很久,有可能是并发执行的线程达到了 innodb_thread_concurrency :因为并发执行的线程达到了 innodb_thread_concurrency 后,再有新连接过来执行一个哪怕最简单的按ID查询,该查询也会需要很长时间才能返回;因为只能等到“正在执行的线程”把自己的 tickets(innodb_concurrency_tickets)消耗完了,才会换出;进而从FIFO中取出一个待执行的 thread 比如按照ID的查询

mysql> show engine innodb status \G
*************************** 1. row ***************************
  Type: InnoDB
  Name:
Status:
=====================================
2023-09-18 10:51:17 0x7f1c3171f700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 19 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 494 srv_active, 0 srv_shutdown, 49571163 srv_idle
srv_master_thread log flush and writes: 49571657
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 152
OS WAIT ARRAY INFO: signal count 129
RW-shared spins 0, rounds 194, OS waits 84
RW-excl spins 0, rounds 142, OS waits 0
RW-sx spins 15, rounds 193, OS waits 1
Spin rounds per wait: 194.00 RW-shared, 142.00 RW-excl, 12.87 RW-sx
------------
TRANSACTIONS
------------
Trx id counter 21627
Purge done for trx's n:o < 21625 undo n:o < 0 state: running but idle
History list length 0
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421234659759952, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421234659760864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
 ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
515 OS file reads, 2656 OS file writes, 587 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 0, seg size 2, 0 merges
merged operations:
 insert 0, delete mark 0, delete 0
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
Hash table size 34673, node heap has 1 buffer(s)
Hash table size 34673, node heap has 1 buffer(s)
Hash table size 34673, node heap has 0 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number 13325540
Log flushed up to   13325540
Pages flushed up to 13325540
Last checkpoint at  13325531
0 pending log flushes, 0 pending chkp writes
399 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 137428992
Dictionary memory allocated 239644
Buffer pool size   8191
Free buffers       7657
Database pages     532
Old database pages 211
Modified db pages  0
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 472, created 60, written 2191
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 532, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=4105, Main thread ID=139759309551360, state: sleeping
Number of rows inserted 20397, updated 78, deleted 3, read 24652
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================

1 row in set (0.00 sec)
mysql>
mysql>
mysql> select trx_id,trx_state,trx_query,trx_operation_state,trx_concurrency_tickets from information_schema.innodb_trx \G
*************************** 1. row ***************************
                 trx_id: 162612
              trx_state: RUNNING
              trx_query: insert into testti3 select * from testti3
    trx_operation_state: NULL
trx_concurrency_tickets: 10
*************************** 2. row ***************************
                 trx_id: 422212176322720
              trx_state: RUNNING
              trx_query: insert into testui select * from testui
    trx_operation_state: sleeping before entering InnoDB
trx_concurrency_tickets: 0
2 rows in set (0.32 sec)

从trx_operation_state中可以看到他们不断的在进行轮换的进入的innodb层次,同时我们还能看到
活跃事物trx_concurrency_tickets这个tickets不断的减少,而处于sleeping before entering InnoDB
的事物其trx_concurrency_tickets为0。
更多推荐

【Python】pyecharts 模块 ① ( ECharts 简介 | pyecharts 简介 | pyecharts 中文网站 | pyecharts 画廊网站 | pyecharts 画 )

文章目录一、pyecharts模块1、ECharts简介2、pyecharts简介3、pyecharts中文网站4、pyecharts画廊网站5、pyecharts画廊用法pyecharts画廊网站:https://gallery.pyecharts.org/#/一、pyecharts模块1、ECharts简介ECha

u盘内容防止复制(U盘内数据防拷贝的方法)

随着科技的发展,U盘已经成为我们日常生活和工作中不可或缺的一部分。然而,U盘的普及也带来了一些问题,如数据泄露、病毒传播等。因此,保护U盘中的数据安全变得尤为重要。方法一:设置文件权限打开U盘,找到需要保护的文件或文件夹。右键点击文件或文件夹,选择“属性”。在弹出的属性窗口中,切换到“安全”选项卡。点击“编辑”按钮,打

解决vue项目导出当前页Table为Excel

解决vue项目中导出当前页表格为Excel表格的方案用到的技术:Vue2Element-uifile-saverxlsx1、创建vue项目,安装element-ui2、创建一个组件,组件内放入表格,和导出按钮<template><div><!--导出的按钮--><el-buttonsize="small"type="p

SpringSecurity

SpringSecurity从入门到精通参考代码0.简介​SpringSecurity是Spring家族中的一个安全管理框架。相比与另外一个安全框架Shiro,它提供了更丰富的功能,社区资源也比Shiro丰富。​一般来说中大型的项目都是使用SpringSecurity来做安全框架。小项目有Shiro的比较多,因为相比与

深入JavaScript的运行原理

一、深入V8引擎原理1.JavaScript代码的执行JavaScript代码下载好之后,是如何一步步被执行的呢?我们知道,浏览器内核是由两部分组成的,以webkit为例:WebCore:负责HTML解析、布局、渲染等等相关的工作;JavaScriptCore:解析、执行JavaScript代码;另外一个强大的Java

邮件营销中为什么要细分联系人?

在电子商务行业,邮件营销成为了各大企业吸引客户、推广产品的主要方式之一。然而,有效进行邮件营销需要一套完善的联系人管理系统。本文将从以下五点详细探讨邮件营销联系人管理有必要吗?一、精确定位目标用户邮件营销联系人管理是通过收集、分析和管理用户信息的过程。通过建立一个详尽的联系人数据库,企业可以对客户进行细致的分类和分组。

计算机网络第五节 网络层

一,网络引入的目的1.网络层以下层次解决的问题,未解决的问题从7层结构上看,网络层下是数据链路层从4层结构上看,网络层下面是网络接口层至少我们看到的网络层下面是以太网以太网解决了什么问题?答:以太网解决了具体网络上主机间数据传输的问题;主机之间可以以物理地址,以广播的传输方式进行数据的交换传输没有解决人心不足答的问题:

Springboot定时任务 Spring task

文章目录SpringTask简单操作SpringBoot注解开始1.fixDelay2.fixedRate单线程3.fixedRate多线程4.initialDelay5.cron(推荐)6.任务调度配置SpringTask简单操作SpringBoot注解开始@EnableScheduling@SpringBootAp

【JavaEE】多线程(三)

多线程(三)续上文,多线程(二),我们已经讲了创建线程Thread的一些重要的属性和方法那么接下来,我们继续来体会了解多线程吧~文章目录多线程(三)线程启动startstart与run的区别中断线程interrupt方法一方法二线程等待join线程状态线程安全线程安全问题的原因synchronized线程启动start

Scrum敏捷开发企业培训大纲介绍-企业内训

课程简介Scrum是目前运用最为广泛的敏捷开发方法,是一个轻量级的项目管理和产品研发管理框架。这是一个两天的实训课程,面向研发管理者、项目经理、产品经理、研发团队等,旨在帮助学员全面系统地学习Scrum和敏捷开发,帮助企业快速启动敏捷实施。课程采用案例讲解+沙盘演练的方式授课,通过两天的强化训练学员将学会基于Scrum

Java笔记:JVM优化分析

1.我们为什么要对jvm做优化?在本地开发环境中我们很少会遇到需要对jvm进行优化的需求,但是到了生产环境,我们可能将有下面的需求:运行的应用“卡住了”,日志不输出,程序没有反应服务器的CPU负载突然升高在多线程应用下,如何分配线程的数量?……说明:使用的jdk版本为1.8。2.jvm的运行参数在jvm中有很多的参数可

热文推荐