Tag Archives: tuning

继续说Apache的优化

经过了前面一段时间的探索,我们对ApacheMySql的性能调优有了个初步的认识,我本来以为问题到这里应该就解决了,但是运行了十几天后,还是出现了负载达到50+的情况。于是,网站又挂了。

分析access.log与之前并无不同,被盗链的请求还是很多,不过这些都被302重定向了啊,死活想不到办法,一度怀疑是不是因为302或者404的错误日志过多的引起的IO问题导致服务器资源紧张。

除此之外,还是有个现象引起了注意,每次挂掉的不是Apache,而是数据库。为什么盗链会引起数据库问题,这显然无法解释。

不绕圈子了,最后发现的问题,就是为了防盗链而特意写的Rewrite。原本的rewrite是这样的,对于盗链的资源,重定向到首页。

这看起来没有问题,可是瞬间请求数百次到首页,然后运行脚本,执行数据库查询……

还有一点,返回的是302,这就意味着搜索引擎和下载工具会认为资源是存在的,不会停止响应。

所以,最终的解决方案。重定向所有盗链的资源到一个静态LOGO,同时,设定返回Code为403.

整个世界清净了。

性能调优:MySql Server at Linux

我们继续性能调优系列。

从过前两天的努力,再加上Cache和其他优化,系统的负载由原来的动不动10+下降到了1左右。这应当是巨大的变化,但是一段时间后发现,系统还是有不稳定的情况,CPU有时会莫名的高涨,虽然之前对Apache和PHP的限制使得系统有一点的稳定性,在大多数情况下可以在一段时间后恢复。不过凡事都应有个结局,对吧,找出问题是怪黍蜀的天性……

轮到了MySql, 现代系统,数据库基本上是应用的核心。系统后期再去调优基本上已经是木已成舟司马当作活马医,不过验尸为以后积攒经验还是必要的。再次多余一句,请尽可能优雅的设计数据库

设计到达一定阶段,就是应用了。这个时候可以进行SQL的Debug, 善用执行计划,调整索引和存储策略,参看Wiki 查询计划

好了,如果这些都没问题了,或者你没办法做这些事情了,就剩下数据库系统的调优了。Database Tuning 这是个多少有点复杂的主题,不过今天介绍的东西只要理解相关概念就可以了。

用于MySql调优的诊断脚本

MySqlTuner 是一个Perl脚本,简单好用。

下载脚本

wget http://mysqltuner.com/releases/mysqltuner-1.0.0.pl

修改脚本,增加执行权限

chmod +x mysqltuner-1.0.0.pl

运行

./mysqltuner-1.0.0.pl

程序输出

>>  MySQLTuner 1.0.0 – Major Hayden <major@mhtx.net>
>>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
>>  Run with ‘–help’ for additional options and output filtering
Please enter your MySQL administrative login: root
Please enter your MySQL administrative password:

——– General Statistics ————————————————–
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.0.45-log
[OK] Operating on 32-bit architecture with less than 2GB RAM

——– Storage Engine Statistics ——————————————-
[--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 89M (Tables: 147)
[--] Data in InnoDB tables: 27M (Tables: 157)
[--] Data in MEMORY tables: 126K (Tables: 2)
[!!] Total fragmented tables: 12

——– Performance Metrics ————————————————-
[--] Up for: 16h 33m 50s (480K q [8.050 qps], 28K conn, TX: 94M, RX: 95M)
[--] Reads / Writes: 75% / 25%
[--] Total buffers: 88.0M global + 2.7M per thread (60 max threads)
[OK] Maximum possible memory usage: 249.2M (49% of installed RAM)
[OK] Slow queries: 0% (18/480K)
[OK] Highest usage of available connections: 33% (20/60)
[OK] Key buffer size / total MyISAM indexes: 32.0M/17.2M
[OK] Key buffer hit rate: 99.6% (3M cached / 15K reads)
[OK] Query cache efficiency: 72.3% (248K cached / 344K selects)
[!!] Query cache prunes per day: 36459
[OK] Sorts requiring temporary tables: 0% (3 temp sorts / 26K sorts)
[OK] Temporary tables created on disk: 21% (3K on disk / 14K total)
[OK] Thread cache hit rate: 97% (765 created / 28K connections)
[OK] Table cache hit rate: 28% (351 open / 1K opened)
[OK] Open file limit used: 31% (343/1K)
[OK] Table locks acquired immediately: 99% (193K immediate / 193K locks)
[OK] InnoDB data size / buffer pool: 27.1M/30.0M

——– Recommendations —————————————————–
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours – recommendations may be inaccurate
Variables to adjust:
    query_cache_size (> 8M)

 

注意标注!!!的提示,同时脚本给出了若干优化意见。根据经验分析和修改就可以了。

顺带定时检查和优化表的cron

mysqlcheck -Aao -auto-repair -u root –password=*password* >/dev/null

同时给出  /etc/my.cnf参考配置

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
#skip-locking
skip-bdb
# MySQL 4.x has query caching available.
# Enable it for vast improvement and it may be all you need to tweak.
query_cache_type=1
query_cache_limit=1M
query_cache_size=8M
# max_connections=500
# Reduced to 200 as memory will not be enough for 500 connections.
# memory=key_buffer+(sort_buffer_size+read_buffer_size)*max_connections
# which is now: 64 + (1 + 1) * 200 = 464 MB
# max_connections = approx. MaxClients setting in httpd.conf file
# Default set to 100.
max_connections=60
#interactive_timeout=180
interactive_timeout=100
#wait_timeout=180
#wait_timeout=100
# Reduced wait_timeout to prevent idle clients holding connections.
#wait_timeout=30
wait_timeout=15
connect_timeout=10
# max_connect_errors is set to 10 by default
#max_connect_errors=10
#table_cache=512
# Checked opened tables and adjusted accordingly after running for a while.
table_cache=512
#tmp_table_size=32M
#max_heap_table_size=32M
#thread_cache=128
# Reduced it to 32 to prevent memory hogging. Also, see notes below.
thread_cache=4
thread_cache_size=4
# key_buffer=258M
# Reduced it by checking current size of *.MYI files, see notes below.
key_buffer=32M
# Commented out the buffer sizes and keeping the default.
# sort_buffer_size=2M by default.
#sort_buffer_size=1M
# read_buffer_size=128K by default.
#read_buffer_size=1M
# 1Mb of read_rnd_buffer_size for 1GB RAM -- see notes below.
# read_rnd_buffer_size=256K by default.
#read_rnd_buffer_size=1M
# myisam_sort_buffer_size used for ALTER, OPTIMIZE, REPAIR TABLE commands.
# myisam_sort_buffer_size=8M by default.
#myisam_sort_buffer_size=64M
# thread_concurrency = 2 * (no. of CPU)
thread_concurrency=2
innodb_buffer_pool_size=30M
# log slow queries is a must. Many queries that take more than 2 seconds.
# If so, then your tables need enhancement.
log_slow_queries=/var/log/mysqld.slow.log
long_query_time=2

Apache性能调优

说来不好意思,服务器的性能一直都是个大问题,访问量不大,却经常死机。

调优了很多次,也没什么效果。

晚上加晚班,现在无事,又搜索了一番,修改了下,这下貌似有效果了,分享如下。

问题描述

RHL Linux 5.0 + Apache Httpd 2.2 + Mysql5.0 + PHP 5.0 环境

问题描述:服务器运行若干时间后,会无法响应请求,分析日志文件和进程,有如下情况

  1. 大量的404错误:盗链和后续处理有关
  2. 一些比较奇怪的Apache程序异常
    [Sat Apr 25 02:06:40 2009] [notice] child pid 5534 exit signal Floating point exception (8)
    [Sat Apr 25 02:06:41 2009] [notice] child pid 5499 exit signal Floating point exception (8)
    [Sat Apr 25 02:06:41 2009] [notice] child pid 5527 exit signal Floating point exception (8)
    ………
    强制重启进程时还有*** glibc detected *** /usr/sbin/httpd: free(): invalid pointer: 0×00359170 ***
    *** glibc detected *** /usr/sbin/httpd: free(): invalid pointer: 0×00359170 ***
  3. 系统负载平衡很高,甚至达到60
  4. CPU使用很高,物理内存使用达到90%,交换内存也将近满了
  5. 有大量的Apache Httpd进程,且多为Sleeping状态

原因分析

常见的引起服务器不稳定的原因:

  1. 程序问题引起的内存泄漏
  2. 不恰当的配置导致资源死锁

关于Apache Httpd的调优,有很多说法,不过个人推荐两篇文章,应该是通俗易懂的,呃,E文

Open Source: Configuring Apache – Don’t Succumb To The “Slashdot Effect” http://blogcritics.org/archives/2006/01/27/175740.php

Tuning Apache, part 1 http://virtualthreads.blogspot.com/2006_01_01_archive.html

还有有中文的

Apache Prefork和Worker模式性能比较

http://www.5dlinux.com/article/9/2009/linux_29722.html

官方文档的中文翻译 性能方面的提示

http://lamp.linux.gov.cn/Apache/ApacheMenu/misc/perf-tuning.html

说说对上面两个问题的方案。

内存泄漏。程序哪里错了,不好说,得跟踪定位,每个系统都有内存跟踪的工具,除此之外还要分析日志,可以通过Apache的日志配置来跟踪脚本运行的时间,找出可能的元凶。

除此之外,还有个很土的办法:定时重启。实在没招了可以参考。

服务器的配置么,那更有说法了。Apache httpd默认是Prefork模式(参考上面),这里就是论事的介绍几个参数:事实上我只是简单摘译一下上面的某篇文章,后面的设置是作者的推荐值,中文里面是我的推荐

  • MaxKeepAliveRequests 0 The KeepAlive directive in httpd.conf allows persistent connections to the web server, so that new connection does not have to be initiated for each request. Setting the MaxKeepAliveRequests directive to 0 enables unlimited number of requests per connection, which makes sense if you think about it. Why allow persistent connections but then terminate them after a short period of time?
  • 最大保持连接请求数:0 – 无限制。保持连接意味着不必为同一客户端的多次请求重新释放和分配资源,也意味着这些资源不会马上释放。这在现在这个Ajax漫天飞,外加无数图片和JS的时代很重要,但是如果控制不好资源的保持与释放,就意味着系统资源的耗尽。不过这里只是能复用的请求数,无限就好了。
  • KeepAliveTimeout 15
    Because persistent connections are allowed, it is important that they are not kept open indefinitely. This directive will close the connection after 15 seconds of inactivity.
  • 保持连接超时 3 秒
    如果连接请求无活动,就有必要关掉它,默认15秒太长了点,3秒钟就够了。
  • MinSpareServers 15
    This is the minimum number of spare servers you want running at any given time. This way, if multiple simultaneous requests are received there will already be child processes running to handle them. Setting this number too high is a waste of system resources and setting it too low will cause the system to slow down.
  • MaxSpareServers 65
    Same as above, but the maximum child processes running at any given time.
  • StartServers 15
    This is the number of servers Apache will start initially. As more servers handle requests a minimum of 15 spare servers will run up to the maximum of 64.
  • 上面这三个参数放在一起说:其实就是Apache用于响应用户的子进程,一般一个子进程消耗20M左右内存,上面三个数字可以控制子进程的数量。于是根据可用的系统资源,就可以计算出个合理的取值了。
  • MaxClients 500
    This is the maximum number of simultaneous clients that can connect to the server at any given time. Setting this number too low will result in users being locked out of the server under normal traffic situations and setting it too high will result in your server being so overloaded that all the requests timeout anyway. I think 500 is about right for most people’s needs.
  • 最大客户端,其实就是并发处理量,这也要根据情况来配置,太大会导致服务器崩溃,太小会导致请求排队,响应缓慢。
  • MaxRequestsPerChild 100000
    Sets the maximum number of requests each child process will handle. This is mostly to prevent memory leaks and other mishaps but is important nonetheless. Setting this too low will cause a large portion of child processes to end for no real reason, thus slowing down the site. This could be set to 0 (unlimited) but that would negate any protection from valid issues like memory leaks.
  • 每个子进程可以处理的最多请求。做限制的好处就是,如果某个程序内存泄漏,那么一段时间后就可以把它释放掉。限制小会导致频繁创建进程,拖慢系统。
  • HostnameLookups off
    This prevents DNS lookups of all the visitors to the site, I am pretty sure it’s off by default. If it’s on in your httpd.conf I would recommend turning it off.这没啥好说的

默认的配置

<IfModule prefork.c>

ServerLimit 10000
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 10000
MaxRequestsPerChild 0
</IfModule>

这个默认配置的缺点就是不能防止内存泄漏,结合KeepAlive的设置,占用过多资源。

KeepAlive on
KeepAliveTimeout 2
MaxKeepAliveRequests 0
MaxClients 150
MinSpareServers 5
MaxSpareServers 12
StartServers 5
MaxRequestsPerChild 100000

较短的保持连接释放,有利于应对无效的盗链请求。

客户端限制用于保护服务器正常运行。

实验继续进行中,本文将不定期更新。