Perbandingan CMS

Tadi siang, temen saya, rizqinova putra, pemilik dari kemudian.com. Dia mengetes kehebatan dari beberapa cms yang dia pakai. CMS-CMS yang dites kali ini adalah, Drupal (www.kemudian.com), wordpress (www.inkuiri.com), dan vbulletin (www.kemudian.com). ia mengetes pada server dengan kemampuan :
Memory : 8GB
Processor : Q9300 Intel
Apache : Prefork mode
PHP : 5.2.4, FastCGI, APC
OS : Ubuntu 8.04.2

dengan Benchmark:
10.000 request with 5 concurrent users

Berikut ini adalah hasil pengetesan yang beliau lakukan :

Drupal kemudian.com :
Dengan memcache :

Concurrency Level: 5
Time taken for tests: 50.333074 seconds
Complete requests: 10000
Failed requests: 19
(Connect: 0, Length: 19, Exceptions: 0)
Write errors: 0
Non-2xx responses: 19
Total transferred: 397318877 bytes
HTML transferred: 392194833 bytes
Requests per second: 198.68 [#/sec] (mean)
Time per request: 25.167 [ms] (mean)
Time per request: 5.033 [ms] (mean, across all concurrent requests)
Transfer rate: 7708.77 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 1
Processing: 5 24 368.6 9 19018
Waiting: 5 24 368.6 9 19018
Total: 5 24 368.6 9 19018

Percentage of the requests served within a certain time (ms)
50% 9
66% 13
75% 15
80% 17
90% 24
95% 31
98% 42
99% 53
100% 19018 (longest request)

Tanpa Memcache :

test 1
root@node1:/home/kemudian/public_html/pro/sites/kemudian.com# ab -n 10000 -c 5 http://kemudian.com/index.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking kemudian.com (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests


Server Software: Apache
Server Hostname: kemudian.com
Server Port: 80

Document Path: /index.php
Document Length: 39091 bytes

Concurrency Level: 5
Time taken for tests: 56.115547 seconds
Complete requests: 10000
Failed requests: 22
(Connect: 0, Length: 22, Exceptions: 0)
Write errors: 0
Non-2xx responses: 22
Total transferred: 395186298 bytes
HTML transferred: 390063198 bytes
Requests per second: 178.20 [#/sec] (mean)
Time per request: 28.058 [ms] (mean)
Time per request: 5.612 [ms] (mean, across all concurrent requests)
Transfer rate: 6877.31 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 4 27 481.1 6 20091
Waiting: 4 27 481.1 6 20091
Total: 4 27 481.1 6 20091

Percentage of the requests served within a certain time (ms)
50% 6
66% 7
75% 7
80% 9
90% 14
95% 19
98% 26
99% 34
100% 20091 (longest request)



Test 2
root@node1:/home/kemudian/public_html/pro/sites/kemudian.com# ab -n 10000 -c 5 http://kemudian.com/index.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking kemudian.com (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests


Server Software: Apache
Server Hostname: kemudian.com
Server Port: 80

Document Path: /index.php
Document Length: 39091 bytes

Concurrency Level: 5
Time taken for tests: 41.62125 seconds
Complete requests: 10000
Failed requests: 18
(Connect: 0, Length: 18, Exceptions: 0)
Write errors: 0
Non-2xx responses: 18
Total transferred: 395341510 bytes
HTML transferred: 390217162 bytes
Requests per second: 243.53 [#/sec] (mean)
Time per request: 20.531 [ms] (mean)
Time per request: 4.106 [ms] (mean, across all concurrent requests)
Transfer rate: 9402.22 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 4 19 360.1 6 19008
Waiting: 4 19 360.1 6 19008
Total: 4 19 360.1 6 19008

Percentage of the requests served within a certain time (ms)
50% 6
66% 8
75% 10
80% 11
90% 16
95% 22
98% 29
99% 39
100% 19008 (longest request)


Inkuiri (wordpress)
root@node1:~# ab -n 10000 -c 5 http://www.inkuiri.com/index.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking www.inkuiri.com (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests


Server Software: Apache
Server Hostname: www.inkuiri.com
Server Port: 80

Document Path: /index.php
Document Length: 0 bytes

Concurrency Level: 5
Time taken for tests: 53.45823 seconds
Complete requests: 10000
Failed requests: 19
(Connect: 0, Length: 19, Exceptions: 0)
Write errors: 0
Non-2xx responses: 10000
Total transferred: 3638341 bytes
HTML transferred: 11457 bytes
Requests per second: 188.52 [#/sec] (mean)
Time per request: 26.523 [ms] (mean)
Time per request: 5.305 [ms] (mean, across all concurrent requests)
Transfer rate: 66.98 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 9 25 68.3 21 3126
Waiting: 9 25 68.3 21 3126
Total: 9 25 68.3 21 3126

Percentage of the requests served within a certain time (ms)
50% 21
66% 26
75% 28
80% 30
90% 36
95% 41
98% 49
99% 57
100% 3126 (longest request)

Vbulletin

root@node1:~# ab -n 10000 -c 5 http://www.kners.com/index.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking www.kners.com (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests


Server Software: Apache
Server Hostname: www.kners.com
Server Port: 80

Document Path: /index.php
Document Length: 40537 bytes

Concurrency Level: 5
Time taken for tests: 80.772915 seconds
Complete requests: 10000
Failed requests: 2446
(Connect: 0, Length: 2446, Exceptions: 0)
Write errors: 0
Non-2xx responses: 21
Total transferred: 408209408 bytes
HTML transferred: 403534669 bytes
Requests per second: 123.80 [#/sec] (mean)
Time per request: 40.386 [ms] (mean)
Time per request: 8.077 [ms] (mean, across all concurrent requests)
Transfer rate: 4935.34 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 5 39 69.4 33 1867
Waiting: 5 39 69.4 32 1867
Total: 5 39 69.4 33 1867

Percentage of the requests served within a certain time (ms)
50% 33
66% 40
75% 45
80% 47
90% 54
95% 62
98% 77
99% 97
100% 1867 (longest request)

dari beberapa percobaan diatas, jelas terlihat bahwa Drupal lebih baik daripada tiga pesaingnya. ini terjadi karena arsitektur drupal yang mirip dengan OOP (hal-hal semacam inheritance, polimorphysm ditangani dengan baik oleh drupal) sehingga dia bisa menerapkan strategi cache yang lebih aggressif (dengan pendekatan polimorphysm hal ini dapat diperlakukan dengan baik) baik terhadap fungsi-fungsi dalam modul, maupun terhadap query database dibanding dengan dua pesaingnya.

sebagai contoh dalam wordpress dan vbulletin, setiap fungsi harus di-include kedalam mesin yang utama, sehingga menimbulkan keterlambatan yang lebih lama (wp supercache belum dienable, kemungkinan hal ini yang menimbulkan drawback pada wordpress).

yap, sekian laporan pengujian kali ini, semoga dapat membantu anda untuk menentukan CMS yang terbaik diantara mereka bertiga.

NB : katanya codeigniter ngadain sayembara yang bisa bikin framework yang lebih cepet dari dia dikasih hadiah, coba ah :D