idea's blog

做个笔记.

# nginx.conf

server {
    listen 8080 ;
    server_name localhost;

    location / {
        root           /home/work/htdocs;
        expires 1d;
        autoindex on;
        index  index.php index.html;
    }
    location ~* \.php$ {
        root           /home/work/htdocs;
        fastcgi_pass   127.0.0.1:30000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  /home/work/htdocs/$fastcgi_script_name;
        client_max_body_size       100m;
        include        fastcgi_params;
        fastcgi_connect_timeout 1000s;
        fastcgi_send_timeout 1000s;
        fastcgi_read_timeout 1000s;
    }
}

# nginx.sh ngix 启动脚本

#!/bin/sh
case "$1" in
    'start')
        sudo /usr/local/nginx/sbin/nginx -s start
        ;;
    'stop')
        sudo /usr/local/nginx/sbin/nginx -s stop
        ;;
    'restart')
        sudo /usr/local/nginx/sbin/nginx -s reload
        ;;
    *)
        echo "Usage: $0 {start|stop}"
        exit 1
    ;;
esac

# spaw-php.sh php-cgi 启动脚本
#!/bin/sh
dir=`dirname $0`
PID_FILE=/home/work/htdocs/spaw-php.pid
PHP_FCGI="/home/work/php/bin/php-cgi -f /home/work/htdocs/php.ini"

case "$1" in
    'start')
        spawn-fcgi -C 3 -p 30000 -f "$PHP_FCGI" -P $PID_FILE
        ;;
    'stop')
        kill `cat $PID_FILE`
        ;;
    'restart')
        kill `cat $PID_FILE`
        spawn-fcgi -C 3 -p 30000 -f "$PHP_FCGI" -P $PID_FILE
        ;;
    *)
        echo "Usage: $0 {start|stop}"
        exit 1
    ;;
esac

Written by ideawu at 2012-04-12 15:52:17

我收集到一些文章和视频, 可以带你窥探 Facebook 的架构. Facebook 承载了几十亿的用户, 它的架构(包括思想和实现)是非常值得参考的. 当然, 你要小心不要照搬 Facebook 的每一字一句, 因为任何思想和实现都是有自己的应用场景的.

• 文章: BigPipe: Pipelining web pages for high performance
• 文章: Needle in a haystack: efficient storage of billions of photos
• 文章: http://www.facebook.com/note.php?note_id=10150148835363920
• 文章: The Underlying Technology of Messages
• 文章: Facebook Chat
• 文章: Building Efficient Data Centers with the Open Compute Project
• 文章: Scaling Facebook to 500 Million Users and Beyond
• 文章: Making Facebook 2x Faster
• 文章: Hacking Conventional Computing Infrastructure
• 视频: The Underlying Technology of Messages Tech Talk (12/7/10)
• 视频: Typeahead Search Tech Talk (6/15/2010)

Written by ideawu at 2012-04-06 13:17:22

Redis 是一个性能非常高效的内存 Key-Value 存储服务, 同时它还具有两个非常重要的特性: 1. 持久化; 2. Value 数据结构. 这两个特性让它在不少场景轻松击败了 Memcached 和 Casandra 等.

Redis 的持久化在两种方式: Snapshotting(快照) 和 Append-only file(aof). 在一个采用了 aof 模式的 Redis 服务器上, 当执行 bgrewriteaof 对 aof 进行归并优化时, 出现了 Redis 被阻塞的问题, 此时, Redis 无法提供任何读取和写入操作.

按字面理解, bgrewriteaof 是在后台进行操作, 不应该影响 Redis 的正常服务. 原理也确实是这样的, Redis 首先 fork 一个子进程, 并在该子进程里进行归并和写持久化存储设备(如硬盘)的. 按照正常逻辑, 在一台多核的机器上, 即使子进程占满 CPU 和硬盘, 也不应该导致 Redis 服务阻塞啊!

其实, 问题就出在硬盘上.

Continue reading »

Written by ideawu at 2012-03-23 13:50:34 | tags: aof, appendfsync, bgrewriteaof, bgsave, no-appendfsync-on-rewrite, Redis, Redis性能优化, Redis阻塞

以前偶尔被人问到, 为什么你(和大部分互联网公司)做Web开发要选择PHP, PHP有什么好处. 简单的回答便是”PHP简单,开发快速”.

但是, ASP.NET不简单吗? ASP.NET上手不快吗? Python呢?

有些人用各种PHP框架能快速搭建简单博客的例子来说明PHP的优势, 但这全是狗屎! 实际的业务不可能是博客那么简单.

我认为, PHP的最大优势便是他的数据结构和内置函数, 具体地说便是字符串和数组, 以及字符串和数组的函数.

PHP的字符串既能表示一般文本, 也能表示任意二进制数据, 也就是说, PHP的字符串就是一段内存. PHP的的字符串操作函数囊括了大部分常见和不常见的文本操作: 截取, 查找, 正则, 字符集编码转换…每一个都是一把利器.

PHP的数组是整合了列表和哈希表的数据结构. 由于”树”是最能描述现实世界的数据结构, 而PHP的数组可以(轻松地)表示任意树, 所以, PHP的数组也最能描述现实世界(建模).

如果要用一句话来描述PHP的优势, 我会用这一句:”PHP语言的数据结构和内置函数, 可以几乎直接地描述和处理实际业务. PHP是计算机与现实业务的最直接胶合剂.”

Written by ideawu at 2012-03-08 10:43:18

query_cache_size, tmp_table_size 这两个选项一定要设置!

# The following options will be passed to all MySQL clients
[client]
#password	= your_password
port		= 3306
socket		= /home/work/mysql/tmp/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
port		= 3306
bind-address = 127.0.0.1
socket		= /home/work/mysql/tmp/mysql.sock
#skip-locking
max_allowed_packet = 1M
table_open_cache = 16
sort_buffer_size = 8M
read_buffer_size = 512K
read_rnd_buffer_size = 256K
net_buffer_length = 8K
thread_stack = 128K

# Size of the Key Buffer, used to cache index blocks for MyISAM tables.
# Do not set it larger than 30% of your available memory, as some memory
# is also required by the OS to cache rows. Even if you're not using
# MyISAM tables, you should still set it to 8-64M as it will also be
# used for internal temporary disk tables.
key_buffer_size = 8M

# If the temporary file used for fast index creation would be bigger
# than using the key cache by the amount specified here, then prefer the
# key cache method.  This is mainly used to force long character keys in
# large tables to use the slower key cache method to create the index.
myisam_sort_buffer_size = 8M

# Query cache is used to cache SELECT results and later return them
# without actual executing the same query once again. Having the query
# cache enabled may result in significant speed improvements, if your
# have a lot of identical queries and rarely changing tables. See the
# "Qcache_lowmem_prunes" status variable to check if the current value
# is high enough for your load.
# Note: In case your tables change very often or if your queries are
# textually different every time, the query cache may result in a
# slowdown instead of a performance improvement.
query_cache_size=16M

# Maximum size for internal (in-memory) temporary tables. If a table
# grows larger than this value, it is automatically converted to disk
# based table This limitation is for a single table. There can be many
# of them.
tmp_table_size=16M

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (using the "enable-named-pipe" option) will render mysqld useless!
#
skip-networking
server-id	= 1

# Uncomment the following if you want to log updates
#log-bin=mysql-bin

# binary logging format - mixed recommended
#binlog_format=mixed

# Uncomment the following if you are using InnoDB tables
#default_storage_engine=InnoDB

innodb_data_home_dir = /home/work/mysql/var/
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /home/work/mysql/var/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 32M
innodb_additional_mem_pool_size = 4M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 10M
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[myisamchk]
key_buffer_size = 8M
sort_buffer_size = 8M

[mysqlhotcopy]
interactive-timeout

Written by ideawu at 2012-03-01 10:41:23 | tags: MySQL

网站如果禁止非浏览器访问, 一般主要还是防止自动 Spam, 主要是:

• 防止网站被机器人恶意抓取
• 防止网站被机器人恶意提交

判断恶意请求(程序和软件直接发送 HTTP 请求)的方法有很多, 比如同 IP 请求频率是. 但一旦判定某个请求为异常行为之后, 并不适合直接关闭该连接, 或者返回出错页. 因为绝大多数的判定方法都无法做到很高的正确率, 其主要原来就是缺少”交互性”. “交互性”是判断人和机器的主要因素.

这里判断一种利用交互性来判别是否浏览器正常访问的方法:

1. 当初步判定请求为异常时, 给该请求返回一个动态令牌, 要求对方在下次重新带着该令牌来访问原先请求的资源;
2. 如何告知用户进行这个行为, 根据交互性的强弱依次为:
   1. 设置 Cookie
   2. 通过 JavaScript 脚本的执行(设置 Cookie, 生成带令牌的链接并访问)
   3. 显示网页表单, 要求用户进行点击和输入等操作(单纯的无输入表单, 图片验证码, 声音验证码)
3. 第一种设置 Cookie 太过于简单, 因为一般的机器人都能执行该行为. 第二种 JavaScript 脚本的方式, 其实已经能阻挡大部分机器人. 第三种是非常好的识别那漏掉的一小撮高级机器人, 但同时也可能影响正常用户体验.

Written by ideawu at 2012-02-18 11:50:43