Release 0.5.3

fix sync response 204
add links in History.md
2014-06-13 18:08:12 +08:00 · 2014-06-13 18:05:09 +08:00 · 2014-06-08 00:59:54 +08:00 · 2014-06-08 00:59:22 +08:00 · 2014-06-04 17:37:19 +08:00 · 2014-06-04 17:29:52 +08:00
51 changed files with 1639 additions and 360 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -16,11 +16,14 @@ results
 node_modules
 npm-debug.log
 public/dist/
+.dist
 config/config.js
 backup/*.json
 backup/*.gz
 docs/web/history.md
+docs/web/_readme.md
 view/web/_layout.html
 bin/mysql.js
 bin/test.sql
 coverage/
+config/web_readme.md
--- a/.jshintrc
+++ b/.jshintrc
@@ -58,7 +58,7 @@
    "shadow"        : true,      // true: Allows re-define variables later in code e.g. `var x=1; x=2;`
    "sub"           : false,     // true: Tolerate using `[]` notation when it can still be expressed in dot notation
    "supernew"      : false,     // true: Tolerate `new function () { ... };` and `new Object;`
-    "validthis"     : false,     // true: Tolerate using this in a non-constructor function
+    "validthis"     : true,      // true: Tolerate using this in a non-constructor function

    // Environments
    "browser"       : true,     // Web Browser (window, document, etc)
--- a/.npmignore
+++ b/.npmignore
@@ -8,6 +8,8 @@ logo.png
 public/dist/
 backup/*.json
 backup/*.gz
+docs/web/history.md
+docs/web/_readme.md
 view/web/_layout.html
 bin/mysql.js
 bin/test.sql
@@ -15,3 +17,5 @@ coverage/
 .jshintrc
 .jshintignore
 .DS_Store
+config/web_readme.md
+.dist/
--- a/History.md
+++ b/History.md
@@ -1,21 +1,91 @@

+0.5.3 / 2014-06-13
+==================
+
+  * fix sync response 204
+  * add links in History.md
+  * bump koa
+  * fix test-cov
+  * bump koa and should
+
+0.5.2 / 2014-06-04
+==================
+
+ * sync hotfix
+ * sync phantomjs downloads pkg. close [#348](https://github.com/cnpm/cnpmjs.org/issues/348)
+ * add restart, fixed [#346](https://github.com/cnpm/cnpmjs.org/issues/346)
+
+0.5.1 / 2014-05-28
+==================
+
+ * fix attack on /-/all/since?stale=update_after&startkey=2 close [#336](https://github.com/cnpm/cnpmjs.org/issues/336)
+ * bump thunkify-wrap
+ * bump koa-middlewares
+ * remove outputError
+ * bump dependencies
+ * use svg badge
+ * add package/notfound page
+ * add dist mirror link to home page
+ * fix sync listdiff and add more test cases
+
+0.5.0 / 2014-05-13
+==================
+
+ * filter /nightlies/*
+ * use koa setter instead of set()
+ * add more info on error email
+ * add sync dist to sync/index.js
+ * show dist page
+ * sync dist file and save it to database
+ * disable gzip before [#335](https://github.com/cnpm/cnpmjs.org/issues/335) has fix
+
+0.4.3 / 2014-04-18
+==================
+
+  * Merge pull request [#334](https://github.com/cnpm/cnpmjs.org/issues/334) from cnpm/fix-permission
+  * add permission check to /:name/:tag
+  * Merge pull request [#333](https://github.com/cnpm/cnpmjs.org/issues/333) from cnpm/issue332-tag
+  * fix space
+  * add put /:name/:tag, close [#332](https://github.com/cnpm/cnpmjs.org/issues/332)
+
+0.4.2 / 2014-04-17
+==================
+
+ * sync interval config
+ * fix fav ico and show pkg size on pkg info page. fix [#318](https://github.com/cnpm/cnpmjs.org/issues/318)
+ * sync work sync one done must wait for a defer.setImmediate. fix [#328](https://github.com/cnpm/cnpmjs.org/issues/328)
+ * bump dep versions
+ * if download tarball 404, throw err better than ignore it. fixed [#325](https://github.com/cnpm/cnpmjs.org/issues/325)
+ * refator sync
+ * hotfix, close [#321](https://github.com/cnpm/cnpmjs.org/issues/321)
+ * hotfix, close [#319](https://github.com/cnpm/cnpmjs.org/issues/319)
+ * support custom web home page
+ * npm get short only can read from cnpm now
+ * if using reverted proxy like nginx, only binding on local host
+ * fix redis detect logic
+
+0.4.1 / 2014-04-10
+==================
+
+ * fix sync status code error
+
 0.4.0 / 2014-04-09
 ==================

 * fix test cases to run on local machine
 * add contribute guidelines
- * use local mysql for dev env. fix #308
+ * use local mysql for dev env. fix [#308](https://github.com/cnpm/cnpmjs.org/issues/308)
 * use copy to
 * use koa-compress and koa-conditional-get
- * maintainers is string, fix #301
+ * maintainers is string, fix [#301](https://github.com/cnpm/cnpmjs.org/issues/301)

 0.3.13 / 2014-03-27
 ==================

 * fix npm adduser update 409 bug
 * fix multiline coverage
- * show package engines. fixed #280
- * dont sync local package field. fix #295
+ * show package engines. fixed [#280](https://github.com/cnpm/cnpmjs.org/issues/280)
+ * dont sync local package field. fix [#295](https://github.com/cnpm/cnpmjs.org/issues/295)

 0.3.12 / 2014-03-26
 ==================
@@ -23,53 +93,53 @@
 * fix result.successes not exist error
 * fix search list
 * add simple request for listall
- * only return package name in /-/all and /-/all/since, fixed #291
+ * only return package name in /-/all and /-/all/since, fixed [#291](https://github.com/cnpm/cnpmjs.org/issues/291)
 * refine docs foloder
- * use module gmt_modified as etag. fix #288
+ * use module gmt_modified as etag. fix [#288](https://github.com/cnpm/cnpmjs.org/issues/288)
 * fix typo, remove unused config in package.json
 * web page only list cnpm registry related info
 * use generator in qnfs

-0.3.11 / 2014-03-20 
+0.3.11 / 2014-03-20
 ==================

-  * use common.isMaintainer, fixed #283
+  * use common.isMaintainer, fixed [#283](https://github.com/cnpm/cnpmjs.org/issues/283)
  * update dependencies
-  * use co-mocha for test, fixed #279
+  * use co-mocha for test, fixed [#279](https://github.com/cnpm/cnpmjs.org/issues/279)
  * update thunkify-wrap, breaking change in thunkify-wrap
  * refactor SQLs by using multiline
  * use multiline to refactor sqls
  * ignore contributors

-0.3.10 / 2014-03-16 
+0.3.10 / 2014-03-16
 ==================

-  * Only /_session request send the authSession. fixed #223
-  * sync npm user info when maintainers and contributors not exists. fixed #82
+  * Only /_session request send the authSession. fixed [#223](https://github.com/cnpm/cnpmjs.org/issues/223)
+  * sync npm user info when maintainers and contributors not exists. fixed [#82](https://github.com/cnpm/cnpmjs.org/issues/82)
  * save npm user to mysql
  * password salt always be randoms
-  * remove session access in /name and /name/version, fixed #274
+  * remove session access in /name and /name/version, fixed [#274](https://github.com/cnpm/cnpmjs.org/issues/274)
  * fix update maintainer session error
  * update koa-middlewares
  * fix test, fix sync_by_install
  * use defer session
-  * Support npm owner|author add [name] [pkg]. fixed #271
+  * Support npm owner|author add [name] [pkg]. fixed [#271](https://github.com/cnpm/cnpmjs.org/issues/271)

-0.3.9 / 2014-03-14 
+0.3.9 / 2014-03-14
 ==================

  * custom user-agent
  * use co-urllib instead of thunkify urllib; fix mock http.request test cases
  * request limit custom message
  * add config.redis check
-  * add koa-limit, fixed #267
+  * add koa-limit, fixed [#267](https://github.com/cnpm/cnpmjs.org/issues/267)

-0.3.8 / 2014-03-11 
+0.3.8 / 2014-03-11
 ==================

-  * update middlewares, fixed missing charset bug #264
+  * update middlewares, fixed missing charset bug [#264](https://github.com/cnpm/cnpmjs.org/issues/264)

-0.3.7 / 2014-03-11 
+0.3.7 / 2014-03-11
 ==================

  * show worker die date time
@@ -79,63 +149,63 @@
  * fix return versions
  * fix makefile, remove eventproxy
  * refactor sync_module_worker
-  * add make test-dev, fixed #259
+  * add make test-dev, fixed [#259](https://github.com/cnpm/cnpmjs.org/issues/259)
  * change npm.js to generator
  * update urllib, proxy/npm.js use generator
  * sync_all and sync_exist to generator
  * change function to generator
  * need node >= v0.11.9

-0.3.6 / 2014-03-06 
+0.3.6 / 2014-03-06
 ==================

-  * install missing package should sync it from source npm. fixed #252
+  * install missing package should sync it from source npm. fixed [#252](https://github.com/cnpm/cnpmjs.org/issues/252)
  * npm publish dont contains .jshint*
  * npm test run jshint
  * Add jshint check: $ make jshint
  * use `yield* next` instead of `yield next`
  * replace dist.u.qiniudn.com with cnpmjs.org/dist

-0.3.5 / 2014-03-05 
+0.3.5 / 2014-03-05
 ==================

-  * redirect /dist/xxx.tgz => http://dist.u.qiniudn.com/xxx.tgz fixed #249
-  * redirect /name to /package/name when /name is 404. fixed #245
-  * Add missing properies and sync missing star users. fixed #235
+  * redirect /dist/xxx.tgz => http://dist.u.qiniudn.com/xxx.tgz fixed [#249](https://github.com/cnpm/cnpmjs.org/issues/249)
+  * redirect /name to /package/name when /name is 404. fixed [#245](https://github.com/cnpm/cnpmjs.org/issues/245)
+  * Add missing properies and sync missing star users. fixed [#235](https://github.com/cnpm/cnpmjs.org/issues/235)

-0.3.4 / 2014-03-04 
+0.3.4 / 2014-03-04
 ==================

  * add cov
  * use istanbul run test coverage
-  * gzip support. fix #241
+  * gzip support. fix [#241](https://github.com/cnpm/cnpmjs.org/issues/241)
  * readme spelling patch (@stanzheng)
-  * default readme to null, fixed #233
+  * default readme to null, fixed [#233](https://github.com/cnpm/cnpmjs.org/issues/233)
  * remove readme in versions

-0.3.3 / 2014-02-28 
+0.3.3 / 2014-02-28
 ==================

-  * Merge pull request #232 from cnpm/host-hotfix
+  * Merge pull request [#232](https://github.com/cnpm/cnpmjs.org/issues/232) from cnpm/host-hotfix
  * get request host from request.headers
-  * Merge pull request #231 from cnpm/bug-fix
-  * fix deps display bug#230 and nsf.url TypeError#229
+  * Merge pull request [#231](https://github.com/cnpm/cnpmjs.org/issues/231) from cnpm/bug-fix
+  * fix deps display bug[#230](https://github.com/cnpm/cnpmjs.org/issues/230) and nsf.url TypeError[#229](https://github.com/cnpm/cnpmjs.org/issues/229)

-0.3.2 / 2014-02-28 
+0.3.2 / 2014-02-28
 ==================

  * update koa-sess and koa-redis
  * fix sync all test
  * remove nfs.downloadStream first, fix tmppath error
-  * fix fengmk2/giturl#1 bug
+  * fix fengmk2/giturl[#1](https://github.com/cnpm/cnpmjs.org/issues/1) bug

-0.3.1 / 2014-02-27 
+0.3.1 / 2014-02-27
 ==================

-  * add etag fixed #224
+  * add etag fixed [#224](https://github.com/cnpm/cnpmjs.org/issues/224)
  * travis ci install on source npm

-0.3.0 / 2014-02-27 
+0.3.0 / 2014-02-27
 ==================

  * fix typo and dont sync not exists pkgs
@@ -156,162 +226,162 @@
  * fix all the test of registry module.test.js
  * convert registry/module.js to koa type
  * fix auth middleware
-  * finish registry user controller koa and update mm to support thunkify. fixed #196
+  * finish registry user controller koa and update mm to support thunkify. fixed [#196](https://github.com/cnpm/cnpmjs.org/issues/196)
  * change controllers/user.js to koa
  * thunkify all proxy
  * convert all middlewares to koa type
  * change regsitry sync to koa
  * addd koa-jsonp, koa-bodyparser, fix / controller
  * first koa run registry home page /
-  * Merge pull request #212 from cnpm/fix-sync-404
+  * Merge pull request [#212](https://github.com/cnpm/cnpmjs.org/issues/212) from cnpm/fix-sync-404
  * return friendly 404 reason
-  * Merge pull request #211 from cnpm/bug-fix
-  * override json limit to default 10mb. fixed #209
-  * fix #210 addPackageAndDist package version detect bug
+  * Merge pull request [#211](https://github.com/cnpm/cnpmjs.org/issues/211) from cnpm/bug-fix
+  * override json limit to default 10mb. fixed [#209](https://github.com/cnpm/cnpmjs.org/issues/209)
+  * fix [#210](https://github.com/cnpm/cnpmjs.org/issues/210) addPackageAndDist package version detect bug

-0.2.27 / 2014-02-19 
+0.2.27 / 2014-02-19
 ==================

-  * support json result in search, fixed #189
+  * support json result in search, fixed [#189](https://github.com/cnpm/cnpmjs.org/issues/189)

-0.2.26 / 2014-02-19 
+0.2.26 / 2014-02-19
 ==================

  * npm publish also need to add deps

-0.2.25 / 2014-02-19 
+0.2.25 / 2014-02-19
 ==================

  * max handle number of package.json `dependencies` property
-  * Dependents support. fixed #190
+  * Dependents support. fixed [#190](https://github.com/cnpm/cnpmjs.org/issues/190)

-0.2.24 / 2014-02-13 
+0.2.24 / 2014-02-13
 ==================

  * fix if delete all the versions
-  * refactor remove module, fixed #186
+  * refactor remove module, fixed [#186](https://github.com/cnpm/cnpmjs.org/issues/186)

-0.2.23 / 2014-01-26 
+0.2.23 / 2014-01-26
 ==================

-  * system admin can add, publish, remove the packages. fixed #176
+  * system admin can add, publish, remove the packages. fixed [#176](https://github.com/cnpm/cnpmjs.org/issues/176)

-0.2.22 / 2014-01-26 
+0.2.22 / 2014-01-26
 ==================

-  * add keyword and search support keyword. #181
+  * add keyword and search support keyword. [#181](https://github.com/cnpm/cnpmjs.org/issues/181)

-0.2.21 / 2014-01-24 
+0.2.21 / 2014-01-24
 ==================

  * refactor code styles on package.html
  * nav-tabs e.preventDefault
-  * Show registry server error response. fixed #178
+  * Show registry server error response. fixed [#178](https://github.com/cnpm/cnpmjs.org/issues/178)
  * nav-tabs for package.html (@4simple)

-0.2.20 / 2014-01-23 
+0.2.20 / 2014-01-23
 ==================

  * hotfix sync missing dependencies and readmes
-  * fix sync readme error, fixed #174
+  * fix sync readme error, fixed [#174](https://github.com/cnpm/cnpmjs.org/issues/174)
  * add updateReadme in module

-0.2.19 / 2014-01-22 
+0.2.19 / 2014-01-22
 ==================

-  * npm install no need to check authorization header. fixed #171
+  * npm install no need to check authorization header. fixed [#171](https://github.com/cnpm/cnpmjs.org/issues/171)

-0.2.18 / 2014-01-20 
+0.2.18 / 2014-01-20
 ==================

-  * Support gitlab git url to display and click. fixed #160
+  * Support gitlab git url to display and click. fixed [#160](https://github.com/cnpm/cnpmjs.org/issues/160)
  * fix redis crash

-0.2.17 / 2014-01-17 
+0.2.17 / 2014-01-17
 ==================

  * custom logo url
  * hotfix layout bug

-0.2.16 / 2014-01-16 
+0.2.16 / 2014-01-16
 ==================

  * fix publish-time bug

-0.2.15 / 2014-01-16 
+0.2.15 / 2014-01-16
 ==================

  * add publish_time to debug

-0.2.14 / 2014-01-16 
+0.2.14 / 2014-01-16
 ==================

  * add make autod
-  * update publish_time, fixed #163
+  * update publish_time, fixed [#163](https://github.com/cnpm/cnpmjs.org/issues/163)

-0.2.13 / 2014-01-15 
+0.2.13 / 2014-01-15
 ==================

  * markdown tmpl not support footer, need to wrap on app start

-0.2.12 / 2014-01-15 
+0.2.12 / 2014-01-15
 ==================

  * add footer and npm client name customable

-0.2.11 / 2014-01-15 
+0.2.11 / 2014-01-15
 ==================

  * package page contributor link to search, default is true

-0.2.10 / 2014-01-14 
+0.2.10 / 2014-01-14
 ==================

-  * fix #155 Content-Disposition wrong.
+  * fix [#155](https://github.com/cnpm/cnpmjs.org/issues/155) Content-Disposition wrong.

-0.2.9 / 2014-01-14 
+0.2.9 / 2014-01-14
 ==================

  * support startkey=c and startkey="c"
-  * support couch db search api. fixed #153
+  * support couch db search api. fixed [#153](https://github.com/cnpm/cnpmjs.org/issues/153)
  * fix fork me image link
  * support sync by query.name

-0.2.8 / 2014-01-14 
+0.2.8 / 2014-01-14
 ==================

  * dont show err stack on test env
  * add download link for package page

-0.2.7 / 2014-01-13 
+0.2.7 / 2014-01-13
 ==================

-  * add shasum when nfs.upload and hfs.uploadBuffer, fixed #148
+  * add shasum when nfs.upload and hfs.uploadBuffer, fixed [#148](https://github.com/cnpm/cnpmjs.org/issues/148)

-0.2.6 / 2014-01-13 
+0.2.6 / 2014-01-13
 ==================

-  * support custom session store, fixed #146
+  * support custom session store, fixed [#146](https://github.com/cnpm/cnpmjs.org/issues/146)

-0.2.5 / 2014-01-13 
+0.2.5 / 2014-01-13
 ==================

  * add download timeout and unit test
  * use downloadStream() first
  * nfs download to a writeable stream.

-0.2.4 / 2014-01-10 
+0.2.4 / 2014-01-10
 ==================

-  * set main script to  index.js, fixed #142
+  * set main script to  index.js, fixed [#142](https://github.com/cnpm/cnpmjs.org/issues/142)

-0.2.3 / 2014-01-10 
+0.2.3 / 2014-01-10
 ==================

  * Dont show sync button on private package
-  * Sync package as publish with no deps. fixed #138
+  * Sync package as publish with no deps. fixed [#138](https://github.com/cnpm/cnpmjs.org/issues/138)

-0.2.2 / 2014-01-10 
+0.2.2 / 2014-01-10
 ==================

  * keep compatibility
@@ -321,23 +391,23 @@
  * new npm publish in one request, add _publish_in_cnpm
  * support unsure name ufs
  * contributors maybe a object
-  * Object #<Object> has no method 'forEach' fixed #134
-  * support custom config as a module, fixed issue #132
-  * support npm new publish flow. fixed #129
+  * Object #<Object> has no method 'forEach' fixed [#134](https://github.com/cnpm/cnpmjs.org/issues/134)
+  * support custom config as a module, fixed issue [#132](https://github.com/cnpm/cnpmjs.org/issues/132)
+  * support npm new publish flow. fixed [#129](https://github.com/cnpm/cnpmjs.org/issues/129)
  * add toString and constructor to test admin
-  * fix #119 hasOwnProperty check admin bug.
+  * fix [#119](https://github.com/cnpm/cnpmjs.org/issues/119) hasOwnProperty check admin bug.

-0.2.0 / 2013-12-27 
+0.2.0 / 2013-12-27
 ==================

  * remove to lower case
-  * fix #127 execSync and execsync.
+  * fix [#127](https://github.com/cnpm/cnpmjs.org/issues/127) execSync and execsync.
  * add contributors list on package page
  * mv blanket to config
  * sync typeerror fix #statusCode
  * add disturl
-  * fix #122 admin security bug
-  * fixed #121, let pkg 404 as success
+  * fix [#122](https://github.com/cnpm/cnpmjs.org/issues/122) admin security bug
+  * fixed [#121](https://github.com/cnpm/cnpmjs.org/issues/121), let pkg 404 as success
  * fix sql insert error
  * fix typos

@@ -349,78 +419,78 @@
  * make sure all packages name are lower case
  * select ids from tag
  * fix nodejsctl
-  * fix #112 missing versions and time no sync
+  * fix [#112](https://github.com/cnpm/cnpmjs.org/issues/112) missing versions and time no sync
  * remove restart command
  * fix sync missing packages error
  * fix web/readme.md, add install
-  * fix #109 pkg no times and no versions bug.
+  * fix [#109](https://github.com/cnpm/cnpmjs.org/issues/109) pkg no times and no versions bug.

 0.1.2 / 2013-12-19
 ==================

-  * fix times not exists canot sync bug. fixed #101
+  * fix times not exists canot sync bug. fixed [#101](https://github.com/cnpm/cnpmjs.org/issues/101)
  * support npm run command
-  * remove before_install and install in travis, fixed #102
-  * split all sub queries, fixed #104
-  * fix doc, fixed #103
+  * remove before_install and install in travis, fixed [#102](https://github.com/cnpm/cnpmjs.org/issues/102)
+  * split all sub queries, fixed [#104](https://github.com/cnpm/cnpmjs.org/issues/104)
+  * fix doc, fixed [#103](https://github.com/cnpm/cnpmjs.org/issues/103)
  * fix search too slow.
  * dont email sync log level info
  * only sync missing packages at first time
  * update dependencies
-  * sync all will sync all the missing packages, fixed #97
+  * sync all will sync all the missing packages, fixed [#97](https://github.com/cnpm/cnpmjs.org/issues/97)

 0.1.0 / 2013-12-12
 ==================

  * add sync title
-  * add favicon. fixed #69
-  * refine sync page, fiexd #70
+  * add favicon. fixed [#69](https://github.com/cnpm/cnpmjs.org/issues/69)
+  * refine sync page, fiexd [#70](https://github.com/cnpm/cnpmjs.org/issues/70)
  * add app version
  * add test for sync
  * refine sync page
  * registry and web all use controllers/sync.js
-  * sync from web, fixed #58
+  * sync from web, fixed [#58](https://github.com/cnpm/cnpmjs.org/issues/58)
  * saving missing descriptions
-  * add package download info. fixed #63
+  * add package download info. fixed [#63](https://github.com/cnpm/cnpmjs.org/issues/63)
  * add avatar
  * use dependecies, fixed #issue62
-  * support open search, fixed #60
-  * make sure publish_time and author is same to source npm registry. fixed #56
+  * support open search, fixed [#60](https://github.com/cnpm/cnpmjs.org/issues/60)
+  * make sure publish_time and author is same to source npm registry. fixed [#56](https://github.com/cnpm/cnpmjs.org/issues/56)
  * add test for search
  * add a simple search by mysql like
-  * fix This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery. fixed #54
+  * fix This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery. fixed [#54](https://github.com/cnpm/cnpmjs.org/issues/54)
  * update install doc, use nodejsctl to start
  * must add limit on list by author sql
-  * fix sql, change test to fit my local database, fixed #46
+  * fix sql, change test to fit my local database, fixed [#46](https://github.com/cnpm/cnpmjs.org/issues/46)
  * use registry.cnpmjs.org
-  * add install document and total package info on home page. fix #42
-  * add module_id to tag table. #46
-  * skip error version. fixed #43
+  * add install document and total package info on home page. fix [#42](https://github.com/cnpm/cnpmjs.org/issues/42)
+  * add module_id to tag table. [#46](https://github.com/cnpm/cnpmjs.org/issues/46)
+  * skip error version. fixed [#43](https://github.com/cnpm/cnpmjs.org/issues/43)
  * sync may make a user do not exist in database, but have modules in registry
  * add user page
  * fix set license
-  * ignore 404 on sync. fixed #39
+  * ignore 404 on sync. fixed [#39](https://github.com/cnpm/cnpmjs.org/issues/39)
  * fix module page, add test
  * update urllib to 0.5.5
  * version and tag
  * add module page
  * fix download url
  * first get tag, then try version
-  * support sync triggle by install, finish #31
+  * support sync triggle by install, finish [#31](https://github.com/cnpm/cnpmjs.org/issues/31)
  * addTag error return 500
  * just one download field
  * add download total info on home page
  * add download count
  * versions empty and also check missing tags
  * remove tags on unpublish
-  * add module tag. fix #6
+  * add module tag. fix [#6](https://github.com/cnpm/cnpmjs.org/issues/6)
  * add [done] flag to check sync done on client
-  * get sync log #29
+  * get sync log [#29](https://github.com/cnpm/cnpmjs.org/issues/29)
  * fix test in module
  * rm tmp file on down request error
  * add time for debug str
  * fix pkg not exists null bug
-  * use sync module woker to handle sync process. fixed #19
+  * use sync module woker to handle sync process. fixed [#19](https://github.com/cnpm/cnpmjs.org/issues/19)
  * if private mode enable, only admin can publish module
  * add alias in readme
  * fix sql, add sort by name
@@ -429,15 +499,15 @@
  * add npm and cnpm image
  * add registry total info on home page
  * fix mods bug in module.removeAll, change module.update => module.removeWithVersions
-  * add test, fix bug. fixed #18
+  * add test, fix bug. fixed [#18](https://github.com/cnpm/cnpmjs.org/issues/18)
  * spoort unpublish
  * add web page index readme
-  * switchable nfs #21
+  * switchable nfs [#21](https://github.com/cnpm/cnpmjs.org/issues/21)
  * change file path to match npm file path
-  * use qn cdn to store tarball file fixed #16
-  * add GET /:name/:version, fixed #3
+  * use qn cdn to store tarball file fixed [#16](https://github.com/cnpm/cnpmjs.org/issues/16)
+  * add GET /:name/:version, fixed [#3](https://github.com/cnpm/cnpmjs.org/issues/3)
  * add module controller test cases; fix next module not exists logic bug.
-  * publish module flow finish #11
+  * publish module flow finish [#11](https://github.com/cnpm/cnpmjs.org/issues/11)
  * add test for controllers/registry/user.js
  * add test for middleware/auth
  * add test for proxy/user
@@ -448,9 +518,9 @@
  * add start time
  * add home page
  * remove session controller
-  * adduser() finish fixed #5
+  * adduser() finish fixed [#5](https://github.com/cnpm/cnpmjs.org/issues/5)
  * rm app.js and routes.js
-  * Mock npm adduser server response, fixing #5
+  * Mock npm adduser server response, fixing [#5](https://github.com/cnpm/cnpmjs.org/issues/5)
  * adjust project dir, separate registry and web server
  * Init rest frame for cnpmjs.org
  * init
--- a/6
+++ b/6
@@ -2,7 +2,7 @@ TESTS = $(shell ls -S `find test -type f -name "*.test.js" -print`)
 REPORTER = tap
 TIMEOUT = 30000
 MOCHA_OPTS =
-REGISTRY = --registry=http://r.cnpmjs.org
+REGISTRY = --registry=http://registry.npm.taobao.org

 install:
 	@npm install $(REGISTRY) \
@@ -23,12 +23,13 @@ test: install pretest
 		--reporter $(REPORTER) \
 		--timeout $(TIMEOUT) \
 		--require should \
+		--require should-http \
 		--require co-mocha \
 		--require ./test/init.js \
 		$(MOCHA_OPTS) \
 		$(TESTS)

-test-cov cov: install
+test-cov cov: install pretest
 	@NODE_ENV=test node --harmony \
 		node_modules/.bin/istanbul cover --preserve-comments \
 		./node_modules/.bin/_mocha \
@@ -36,6 +37,7 @@ test-cov cov: install
 		--reporter $(REPORTER) \
 		--timeout $(TIMEOUT) \
 		--require should \
+		--require should-http \
 		--require co-mocha \
 		--require ./test/init.js \
 		$(MOCHA_OPTS) \
--- a/README.md
+++ b/README.md
@@ -1,9 +1,9 @@
 cnpmjs.org
 =======

-[![Build Status](https://secure.travis-ci.org/cnpm/cnpmjs.org.png)](http://travis-ci.org/cnpm/cnpmjs.org) [![Dependency Status](https://gemnasium.com/cnpm/cnpmjs.org.png)](https://gemnasium.com/cnpm/cnpmjs.org)
+[![Build Status](https://secure.travis-ci.org/cnpm/cnpmjs.org.svg)](http://travis-ci.org/cnpm/cnpmjs.org) [![Dependency Status](https://gemnasium.com/cnpm/cnpmjs.org.svg)](https://gemnasium.com/cnpm/cnpmjs.org)

-[![NPM](https://nodei.co/npm/cnpmjs.org.png?downloads=true&stars=true)](https://nodei.co/npm/cnpmjs.org/)
+[![NPM](https://nodei.co/npm/cnpmjs.org.svg?downloads=true&stars=true)](https://nodei.co/npm/cnpmjs.org/)

 ![logo](https://raw.github.com/cnpm/cnpmjs.org/master/logo.png)

@@ -78,7 +78,7 @@ $ make test-cov
 $ make autod

 # start server
-$ node --harmony dispatch.js
+$ node --harmony_generators dispatch.js
 ```

 ## How to contribute
--- a/common/qnfs.js
+++ b/common/qnfs.js
@@ -16,10 +16,11 @@

 var thunkify = require('thunkify-wrap');
 var qn = require('qn');
+var fs = require('fs');
 var config = require('../config');
 var client = qn.create(config.qn);

-thunkify(client, ['delete', 'uploadFile', 'upload']);
+thunkify(client, ['delete', 'uploadFile', 'upload', 'download']);

 exports._client = client;

@@ -62,6 +63,14 @@ exports.url = function (key) {
  return client.resourceURL(key);
 };

+exports.download = function* (key, filepath, options) {
+  var writeStream = fs.createWriteStream(filepath);
+  yield client.download(key, {
+    timeout: options.timeout,
+    writeStream: writeStream
+  });
+};
+
 exports.remove = function *(key) {
  try {
    return yield client.delete(key);
--- a/common/redis.js
+++ b/common/redis.js
@@ -17,7 +17,7 @@
 var config = require('../config');

 // close redis by set config.redis to `null` or `{}`
-if (!config.redis || !config.redis.host || !config.redis.port) {
+if (config.redis && config.redis.host && config.redis.port) {

  var redis = require('redis');
  var wrapper = require('co-redis');
--- a/config/index.js
+++ b/config/index.js
@@ -30,6 +30,7 @@ var config = {
  version: version,
  registryPort: 7001,
  webPort: 7002,
+  bindingHost: '127.0.0.1', // only binding on 127.0.0.1 for local access
  enableCluster: false,
  numCPUs: os.cpus().length,
  debug: true, // if debug
@@ -50,18 +51,22 @@ var config = {

  sessionSecret: 'cnpmjs.org test session secret',
  redis: {
-    host: 'pub-redis-19533.us-east-1-4.3.ec2.garantiadata.com',
-    port: 19533,
-    pass: 'cnpmjs_dev'
+    // host: 'pub-redis-19533.us-east-1-4.3.ec2.garantiadata.com',
+    // port: 19533,
+    // pass: 'cnpmjs_dev'
  },
  jsonLimit: '10mb', // max request json body size
-  uploadDir: path.join(root, 'public', 'dist'),
+  uploadDir: path.join(root, '.dist'),
  // qiniu cdn: http://www.qiniu.com/, it free for dev.
  qn: {
-    accessKey: "iN7NgwM31j4-BZacMjPrOQBs34UG1maYCAQmhdCV",
-    secretKey: "6QTOr2Jg1gcZEWDQXKOGZh5PziC2MCV5KsntT70j",
-    bucket: "qtestbucket",
-    domain: "http://qtestbucket.qiniudn.com"
+    // accessKey: "iN7NgwM31j4-BZacMjPrOQBs34UG1maYCAQmhdCV",
+    // secretKey: "6QTOr2Jg1gcZEWDQXKOGZh5PziC2MCV5KsntT70j",
+    // bucket: "qtestbucket",
+    // domain: "http://qtestbucket.qiniudn.com",
+    accessKey: "5UyUq-l6jsWqZMU6tuQ85Msehrs3Dr58G-mCZ9rE",
+    secretKey: "YaRsPKiYm4nGUt8mdz2QxeV5Q_yaUzVxagRuWTfM",
+    bucket: "qiniu-sdk-test",
+    domain: "http://qiniu-sdk-test.qiniudn.com",
  },

  mail: {
@@ -75,9 +80,12 @@ var config = {
    debug: false
  },

-  disturl: 'http://dist.u.qiniudn.com',
+  disturl: 'http://nodejs.org/dist',
+  syncDist: false,
  logoURL: 'http://ww4.sinaimg.cn/large/69c1d4acgw1ebfly5kjlij208202oglr.jpg',
  registryHost: 'r.cnpmjs.org',
+  // customReadmeFile: __dirname + '/web_readme.md',
+  customReadmeFile: '', // you can use your custom readme file instead the cnpm one
  customFooter: '', // you can add copyright and site total script html here
  npmClientName: 'cnpm', // use `${name} install package`
  packagePageContributorSearch: true, // package page contributor link to search, default is true
@@ -93,6 +101,7 @@ var config = {
  backupFilePrefix: '/cnpm/backup/', // backup filepath prefix
  syncModel: 'none', // 'none', 'all', 'exist'
  syncConcurrency: 1,
+  syncInterval: '10m', // sync interval, default is 10 minutes
  maxDependencies: 200, // max handle number of package.json `dependencies` property

  limit: {
@@ -103,13 +112,14 @@ var config = {
    whiteList: [],
    blackList: [],
    message: 'request frequency limited, any question, please contact fengmk2@gmail.com',
-  }
+  },
+  enableCompress: false, // enable gzip response or not
 };

 // load config/config.js, everything in config.js will cover the same key in index.js
 var customConfig = path.join(root, 'config/config.js');
 if (fs.existsSync(customConfig)) {
-  copy(require(customConfig)).toCover(config);
+  copy(require(customConfig)).override(config);
 }

 mkdirp.sync(config.logdir);
@@ -121,5 +131,5 @@ config.loadConfig = function (customConfig) {
  if (!customConfig) {
    return;
  }
-  copy(customConfig).toCover(config);
+  copy(customConfig).override(config);
 };
--- a/controllers/registry/module.js
+++ b/controllers/registry/module.js
@@ -18,6 +18,7 @@
 var debug = require('debug')('cnpmjs.org:controllers:registry:module');
 var path = require('path');
 var fs = require('fs');
+var util = require('util');
 var crypto = require('crypto');
 var utility = require('utility');
 var coRead = require('co-read');
@@ -36,6 +37,7 @@ var SyncModuleWorker = require('../../proxy/sync_module_worker');
 var logger = require('../../common/logger');
 var ModuleDeps = require('../../proxy/module_deps');
 var ModuleStar = require('../../proxy/module_star');
+var downloadAsReadStream = require('../utils').downloadAsReadStream;

 /**
 * show all version of a module
@@ -82,8 +84,8 @@ exports.show = function *(next) {
      return;
    }
    var result = yield SyncModuleWorker.sync(name, 'sync-by-install');
-    this.status = result.ok ? 200 : result.statusCode;
    this.body = result.pkg;
+    this.status = result.ok ? 200 : (result.statusCode || 500);
    return;
  }

@@ -217,8 +219,6 @@ exports.get = function *(next) {

 var _downloads = {};

-var DOWNLOAD_TIMEOUT = ms('10m');
-
 exports.download = function *(next) {
  var name = this.params.name;
  var filename = this.params.filename;
@@ -258,28 +258,13 @@ exports.download = function *(next) {
  _downloads[name] = (_downloads[name] || 0) + 1;

  if (typeof dist.size === 'number') {
-    this.set('Content-Length', dist.size);
+    this.length = dist.size;
  }
-  this.set('Content-Type', mime.lookup(dist.key));
-  this.set('Content-Disposition', 'attachment; filename="' + filename + '"');
-  this.set('ETag', dist.shasum);
+  this.type = mime.lookup(dist.key);
+  this.attachment = filename;
+  this.etag = dist.shasum;

-  // use download file api
-  var tmpPath = path.join(config.uploadDir,
-    utility.randomString() + dist.key.replace(/\//g, '-'));
-  function cleanup() {
-    fs.unlink(tmpPath, utility.noop);
-  }
-  try {
-    yield nfs.download(dist.key, tmpPath, {timeout: DOWNLOAD_TIMEOUT});
-  } catch (err) {
-    cleanup();
-    this.throw(err);
-  }
-  var tarball = fs.createReadStream(tmpPath);
-  tarball.once('error', cleanup);
-  tarball.once('end', cleanup);
-  this.body = tarball;
+  this.body = yield* downloadAsReadStream(dist.key);
 };

 setInterval(function () {
@@ -938,3 +923,57 @@ exports.listAllModuleNames = function *() {
    return m.name;
  });
 };
+
+exports.updateTag = function *() {
+  var version = this.request.body;
+  var tag = this.params.tag;
+  var name = this.params.name;
+
+  if (!version) {
+    this.status = 400;
+    this.body = {
+      error: 'version_missed',
+      reason: 'version not found'
+    };
+    return;
+  }
+
+  if (!semver.valid(version)) {
+    this.status = 403;
+    var reason = util.format('setting tag %s to invalid version: %s: %s/%s',
+      tag, version, name, tag);
+    this.body = {
+      error: 'forbidden',
+      reason: reason
+    };
+    return;
+  }
+
+  var mod = yield Module.get(name, version);
+  if (!mod) {
+    this.status = 403;
+    var reason = util.format('setting tag %s to unknown version: %s: %s/%s',
+      tag, version, name, tag);
+    this.body = {
+      error: 'forbidden',
+      reason: reason
+    };
+    return;
+  }
+
+  // check permission
+  if (!common.isMaintainer(this.user, mod.package.maintainers)) {
+    this.status = 403;
+    this.body = {
+      error: 'forbidden',
+      reason: 'no permission to modify ' + name
+    };
+    return;
+  }
+
+  yield Module.addTag(name, tag, version);
+  this.status = 201;
+  this.body = {
+    ok: true
+  };
+};
--- a/controllers/sync.js
+++ b/controllers/sync.js
@@ -39,7 +39,7 @@ exports.sync = function *() {
  var result = yield SyncModuleWorker.sync(name, username, options);

  // friendly 404 reason info
-  if (result.staticCache === 404) {
+  if (result.statusCode === 404) {
    this.status = 404;
    this.body = {
      ok: false,
@@ -48,7 +48,7 @@ exports.sync = function *() {
    return;
  }
  if (!result.ok) {
-    this.status = result.statusCode;
+    this.status = result.statusCode || 500;
    this.body = result.pkg;
    return;
  }
--- a/controllers/total.js
+++ b/controllers/total.js
@@ -15,13 +15,12 @@
 * Module dependencies.
 */

-var microtime = require('microtime');
 var Total = require('../proxy/total');
 var Download = require('./download');
 var version = require('../package.json').version;
 var config = require('../config');

-var startTime = '' + microtime.now();
+var startTime = '' + Date.now();

 exports.show = function *() {
  var r = yield [Total.get(), Download.total()];
--- a/controllers/utils.js
+++ b/controllers/utils.js
@@ -0,0 +1,46 @@
+/**!
+ * cnpmjs.org - controllers/utils.js
+ *
+ * Copyright(c) fengmk2 and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *   fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var debug = require('debug')('cnpmjs.org:controllers:utils');
+var path = require('path');
+var fs = require('fs');
+var utility = require('utility');
+var ms = require('ms');
+var nfs = require('../common/nfs');
+var config = require('../config');
+
+var DOWNLOAD_TIMEOUT = ms('10m');
+
+exports.downloadAsReadStream = function* (key) {
+  var tmpPath = path.join(config.uploadDir,
+    utility.randomString() + key.replace(/\//g, '-'));
+  function cleanup() {
+    debug('cleanup %s', tmpPath);
+    fs.unlink(tmpPath, utility.noop);
+  }
+  debug('downloadAsReadStream() %s to %s', key, tmpPath);
+  try {
+    yield nfs.download(key, tmpPath, {timeout: DOWNLOAD_TIMEOUT});
+  } catch (err) {
+    debug('downloadAsReadStream() %s to %s error: %s', key, tmpPath, err.stack);
+    cleanup();
+    throw err;
+  }
+  var tarball = fs.createReadStream(tmpPath);
+  tarball.once('error', cleanup);
+  tarball.once('end', cleanup);
+  return tarball;
+};
--- a/controllers/web/dist.js
+++ b/controllers/web/dist.js
@@ -14,10 +14,72 @@
 * Module dependencies.
 */

+var debug = require('debug')('cnpmjs.org:controllers:web:dist');
+var mime = require('mime');
+var Dist = require('../../proxy/dist');
 var config = require('../../config');
+var downloadAsReadStream = require('../utils').downloadAsReadStream;

-exports.redirect = function *(next) {
+function padding(max, current, pad) {
+  pad = pad || ' ';
+  var left = max - current;
+  var str = '';
+  for (var i = 0; i < left; i++) {
+    str += pad;
+  }
+  return str;
+}
+
+exports.list = function* (next) {
  var params = this.params;
-  var url = config.disturl + (params[0] || '/');
-  this.redirect(url);
+  var url = params[0];
+  if (!url) {
+    // GET /dist => /dist/
+    return this.redirect('/dist/');
+  }
+
+  var isDir = url[url.length - 1] === '/';
+  if (!isDir) {
+    return yield* download.call(this, next);
+  }
+
+  var items = yield* Dist.listdir(url);
+  if (url === '/') {
+    // phantomjs/
+    items.push({
+      name: 'phantomjs/',
+      date: '',
+    });
+  }
+
+  yield this.render('dist', {
+    title: 'Mirror index of ' + config.disturl + url,
+    disturl: config.disturl,
+    dirname: url,
+    items: items,
+    padding: padding
+  });
 };
+
+ function* download(next) {
+  var fullname = this.params[0];
+  var info = yield* Dist.getfile(fullname);
+  debug('download %s got %j', fullname, info);
+  if (!info || !info.url) {
+    return yield* next;
+  }
+
+  if (info.url.indexOf('http') === 0) {
+    return this.redirect(info.url);
+  }
+
+  // download it from nfs
+  if (typeof info.size === 'number') {
+    this.length = info.size;
+  }
+  this.type = mime.lookup(info.url);
+  this.attachment = info.name;
+  this.etag = info.sha1;
+
+  this.body = yield* downloadAsReadStream(info.url);
+}
--- a/controllers/web/package.js
+++ b/controllers/web/package.js
@@ -14,6 +14,7 @@
 * Module dependencies.
 */

+var bytes = require('bytes');
 var giturl = require('giturl');
 var moment = require('moment');
 var eventproxy = require('eventproxy');
@@ -108,6 +109,10 @@ exports.display = function *(next) {

  pkg.dependents = dependents;

+  if (pkg.dist) {
+    pkg.dist.size = bytes(pkg.dist.size || 0);
+  }
+
  yield this.render('package', {
    title: 'Package - ' + pkg.name,
    package: pkg,
--- a/dispatch.js
+++ b/dispatch.js
@@ -24,6 +24,18 @@ var childProcess = require('child_process');
 var syncPath = path.join(__dirname, 'sync');

 if (config.enableCluster) {
+  forkWorker();
+  if (config.syncModel !== 'none') {
+    forkSyncer();
+  }
+} else {
+  require(workerPath);
+  if (config.syncModel !== 'none') {
+    require(syncPath);
+  }
+}
+
+function forkWorker() {
  cluster.setupMaster({
    exec: workerPath
  });
@@ -50,9 +62,16 @@ if (config.enableCluster) {
  for (var i = 0; i < config.numCPUs; i++) {
    cluster.fork();
  }
-
-  childProcess.fork(syncPath);
-} else {
-  require(workerPath);
-  require(syncPath);
+}
+
+function forkSyncer() {
+  var syncer = childProcess.fork(syncPath);
+  syncer.on('exit', function (code, signal) {
+    var err = new Error(util.format('syncer %s died (code: %s, signal: %s, stdout: %s, stderr: %s)',
+      syncer.pid, code, signal, syncer.stdout, syncer.stderr));
+    err.name = 'SyncerWorkerDiedError';
+    console.error('[%s] [master:%s] syncer exit: %s: %s',
+      Date(), process.pid, err.name, err.message);
+    setTimeout(forkSyncer, 1000);
+  });
 }
--- a/docs/db.sql
+++ b/docs/db.sql
@@ -87,10 +87,12 @@ CREATE TABLE `tag` (
 `version` varchar(30) NOT NULL COMMENT 'module version',
 `module_id` bigint(20) unsigned NOT NULL COMMENT 'module id',
 PRIMARY KEY (`id`),
- UNIQUE KEY `name` (`name`, `tag`)
+ UNIQUE KEY `name` (`name`, `tag`),
+ KEY `gmt_modified` (`gmt_modified`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='module tag';
 -- ALTER TABLE  `tag` ADD  `module_id` BIGINT( 20 ) UNSIGNED NOT NULL;
 -- ALTER TABLE  `tag` CHANGE  `name`  `name` VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT  'module name';
+-- ALTER TABLE `tag` ADD KEY `gmt_modified` (`gmt_modified`);

 CREATE TABLE `total` (
 `name` varchar(100) NOT NULL COMMENT 'total name',
@@ -137,3 +139,30 @@ CREATE TABLE `module_deps` (
 UNIQUE KEY `name_deps` (`name`,`deps`),
 KEY `name` (`name`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='module deps';
+
+CREATE TABLE `dist_dir` (
+ `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
+ `gmt_create` datetime NOT NULL COMMENT 'create time',
+ `gmt_modified` datetime NOT NULL COMMENT 'modified time',
+ `name` varchar(200) NOT NULL COMMENT 'user name',
+ `parent` varchar(200) NOT NULL COMMENT 'parent dir' DEFAULT '/',
+ `date` varchar(20) COMMENT '02-May-2014 01:06',
+ PRIMARY KEY (`id`),
+ UNIQUE KEY `name` (`parent`, `name`),
+ KEY `gmt_modified` (`gmt_modified`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='dist dir info';
+
+CREATE TABLE `dist_file` (
+ `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
+ `gmt_create` datetime NOT NULL COMMENT 'create time',
+ `gmt_modified` datetime NOT NULL COMMENT 'modified time',
+ `name` varchar(100) NOT NULL COMMENT 'user name',
+ `parent` varchar(200) NOT NULL COMMENT 'parent dir' DEFAULT '/',
+ `date` varchar(20) COMMENT '02-May-2014 01:06',
+ `size` int(10) unsigned NOT NULL COMMENT 'file size' DEFAULT '0',
+ `sha1` varchar(40) COMMENT 'sha1 hex value',
+ `url` varchar(2048),
+ PRIMARY KEY (`id`),
+ UNIQUE KEY `fullname` (`parent`, `name`),
+ KEY `gmt_modified` (`gmt_modified`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='dist file info';
--- a/docs/new.sql
+++ b/docs/new.sql
@@ -0,0 +1,27 @@
+-- http://nodejs.org/dist/ mirror
+CREATE TABLE `dist_dir` (
+ `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
+ `gmt_create` datetime NOT NULL COMMENT 'create time',
+ `gmt_modified` datetime NOT NULL COMMENT 'modified time',
+ `name` varchar(200) NOT NULL COMMENT 'user name',
+ `parent` varchar(200) NOT NULL COMMENT 'parent dir' DEFAULT '/',
+ `date` varchar(20) COMMENT '02-May-2014 01:06',
+ PRIMARY KEY (`id`),
+ UNIQUE KEY `name` (`parent`, `name`),
+ KEY `gmt_modified` (`gmt_modified`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='dist dir info';
+
+CREATE TABLE `dist_file` (
+ `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
+ `gmt_create` datetime NOT NULL COMMENT 'create time',
+ `gmt_modified` datetime NOT NULL COMMENT 'modified time',
+ `name` varchar(100) NOT NULL COMMENT 'user name',
+ `parent` varchar(200) NOT NULL COMMENT 'parent dir' DEFAULT '/',
+ `date` varchar(20) COMMENT '02-May-2014 01:06',
+ `size` int(10) unsigned NOT NULL COMMENT 'file size' DEFAULT '0',
+ `sha1` varchar(40) COMMENT 'sha1 hex value',
+ `url` varchar(2048),
+ PRIMARY KEY (`id`),
+ UNIQUE KEY `fullname` (`parent`, `name`),
+ KEY `gmt_modified` (`gmt_modified`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='dist file info';
--- a/docs/web/readme.md
+++ b/docs/web/readme.md
@@ -1,9 +1,13 @@
-# cnpmjs.org: Private npm registry and web for Enterprise
+# cnpmjs.org: Private npm registry and web for Company
+
+So `cnpm` is meaning: **Company npm**.

 ## Registry

 * Our public registry: [r.cnpmjs.org](http://r.cnpmjs.org), syncing from [registry.npmjs.org](http://registry.npmjs.org)
 * Current [cnpmjs.org](/) version: <span id="app-version"></span>
+* Mirror of [nodejs.org/dist](http://nodejs.org/dist): [/dist mirror](/dist)
+* Mirror of [phantomjs downloads](https://bitbucket.org/ariya/phantomjs/downloads): [phantomjs mirror](/dist/phantomjs/)

 <table class="downloads">
  <tbody>
@@ -166,34 +170,15 @@ $ cnpm info cnpm

@see Github [Issues](https://github.com/cnpm/cnpmjs.org/issues)

-## Authors
+## Histories

 Release [History](/history).

-```bash
-$ git summary
-
- project  : cnpmjs.org
- repo age : 4 months ago
- commits  : 472
- active   : 167 days
- files    : 104
- authors  :
-   272  fengmk2                 57.6%
-   195  dead_horse              41.3%
-     2  4simple                 0.4%
-     2  Stanley Zheng           0.4%
-     1  Alsotang                0.2%
-```
-
 ## npm and cnpm relation

 ![npm&cnpm](https://docs.google.com/drawings/d/12QeQfGalqjsB77mRnf5Iq5oSXHCIUTvZTwECMonqCmw/pub?w=383&h=284)

 ## 捐赠 Donate
-如果您觉得 [cnpmjs.org] 对您有帮助，欢迎请作者一杯咖啡.
+如果您觉得 [cnpmjs.org](/) 对您有帮助，欢迎请作者一杯咖啡.

 [![Donate](https://img.alipay.com/sys/personalprod/style/mc/btn-index.png)](https://me.alipay.com/imk2)
-
- [cnpmjs.org]: http://cnpmjs.org/
- [registry.cnpmjs.org]: http://registry.cnpmjs.org/
--- a/lib/common.js
+++ b/lib/common.js
@@ -21,6 +21,7 @@ var util = require('util');

 exports.getTarballFilepath = function (filename) {
  // ensure download file path unique
+  // TODO: not only .tgz, and also other extname
  var name = filename.replace(/\.tgz$/, '.' + crypto.randomBytes(16).toString('hex') + '.tgz');
  return path.join(config.uploadDir, name);
 };
--- a/middleware/registry_not_found.js
+++ b/middleware/registry_not_found.js
@@ -17,7 +17,7 @@
 module.exports = function *notFound(next) {
  yield *next;

-  if (this.status) {
+  if (this.status && this.status !== 404) {
    return;
  }
  this.status = 404;
--- a/middleware/static.js
+++ b/middleware/static.js
@@ -0,0 +1,29 @@
+/**!
+ * cnpmjs.org - middleware/static.js
+ *
+ * Copyright(c) fengmk2 and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *   fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var path = require('path');
+var middlewares = require('koa-middlewares');
+var config = require('../config');
+
+var staticDir = path.join(path.dirname(__dirname), 'public');
+
+module.exports = middlewares.staticCache(staticDir, {
+  buffer: config.debug ? false : true,
+  maxAge: config.debug ? 0 : 60 * 60 * 24 * 7,
+  alas: {
+    '/favicon.ico': '/favicon.png'
+  }
+});
--- a/middleware/web_not_found.js
+++ b/middleware/web_not_found.js
@@ -14,20 +14,33 @@
 * Module dependencies.
 */

+var debug = require('debug')('cnpmjs.org:middleware:web_not_found');
+
 module.exports = function *notFound(next) {
  yield *next;

-  if (this.status) {
+  if (this.status && this.status !== 404) {
    return;
  }

  var m = /^\/([\w\-\_\.]+)\/?$/.exec(this.url);
+  debug('%s match %j', this.url, m);
  if (m) {
    return this.redirect('/package/' + m[1]);
  }

+  // package not found
+  m = /\/package\/([\w\-\_\.]+)\/?$/.exec(this.url);
+  if (m) {
+    var name = m[1];
+    this.status = 404;
+    yield* this.render('404', {
+      title: 'Package - ' + name,
+      name: name
+    });
+    return;
+  }
+
  this.status = 404;
-  this.type = 'text/html';
-  this.charset = 'utf-8';
  this.body = 'Cannot GET ' + this.path;
 };
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
 {
  "name": "cnpmjs.org",
-  "version": "0.4.0",
+  "version": "0.5.3",
  "description": "Private npm registry and web for Enterprise, base on MySQL and Simple Store Service",
  "main": "index.js",
  "scripts": {
@@ -10,42 +10,44 @@
    "stop": "./bin/nodejsctl stop"
  },
  "dependencies": {
-    "co": "3.0.5",
+    "bytes": "1.0.0",
+    "cheerio": "0.17.0",
+    "co": "3.0.6",
+    "co-defer": "0.1.0",
    "co-gather": "0.0.1",
    "co-read": "0.0.2",
    "co-redis": "1.1.0",
-    "co-urllib": "0.2.0",
+    "co-urllib": "0.2.3",
    "co-write": "0.3.0",
-    "copy-to": "0.0.3",
-    "debug": "0.8.0",
+    "copy-to": "1.0.1",
+    "debug": "1.0.2",
    "eventproxy": "0.3.1",
-    "giturl": "0.0.2",
-    "graceful": "0.0.6",
+    "giturl": "0.0.3",
+    "graceful": "0.1.0",
    "gravatar": "1.0.6",
    "humanize-number": "0.0.2",
-    "koa": "0.5.2",
-    "koa-limit": "1.0.0",
+    "koa": "0.8.0",
+    "koa-limit": "1.0.2",
    "koa-markdown": "0.0.3",
-    "koa-middlewares": "0.1.3",
+    "koa-middlewares": "0.2.1",
    "logfilestream": "0.1.0",
    "marked": "0.3.2",
-    "microtime": "0.5.1",
    "mime": "1.2.11",
-    "mkdirp": "0.3.5",
-    "moment": "2.5.1",
+    "mkdirp": "0.5.0",
+    "moment": "2.7.0",
    "ms": "0.6.2",
-    "multiline": "0.3.2",
-    "mysql": "2.1.1",
-    "nodemailer": "0.6.1",
-    "qn": "0.2.1",
+    "multiline": "0.3.4",
+    "mysql": "2.3.2",
+    "nodemailer": "0.6.5",
+    "qn": "0.2.2",
    "ready": "0.1.1",
-    "redis": "0.10.1",
-    "semver": "2.2.1",
-    "thunkify-wrap": "0.1.1",
-    "utility": "0.1.12"
+    "redis": "0.10.3",
+    "semver": "2.3.0",
+    "thunkify-wrap": "0.1.2",
+    "utility": "0.1.13"
  },
  "devDependencies": {
-    "autod": ">=0.0.13",
+    "autod": "~0.2.0",
    "chunkstream": "0.0.1",
    "co-mocha": "0.0.2",
    "contributors": "*",
@@ -55,8 +57,9 @@
    "mm": "0.2.1",
    "mocha": "*",
    "pedding": "0.0.3",
-    "should": "3.3.0",
-    "supertest": "0.10.0"
+    "should": "4.0.4",
+    "should-http": "0.0.1",
+    "supertest": "0.13.0"
  },
  "homepage": "https://github.com/cnpm/cnpmjs.org",
  "repository": {
--- a/proxy/dist.js
+++ b/proxy/dist.js
@@ -0,0 +1,85 @@
+/**!
+ * cnpmjs.org - proxy/dist.js
+ *
+ * Copyright(c) fengmk2 and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *   fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var path = require('path');
+var multiline = require('multiline');
+var mysql = require('../common/mysql');
+
+var SAVE_FILE_SQL = multiline(function () {;/*
+  INSERT INTO
+    dist_file(gmt_create, gmt_modified, name, parent, date, size, url, sha1)
+  VALUES
+    (now(), now(), ?, ?, ?, ?, ?, ?)
+  ON DUPLICATE KEY UPDATE
+    name=VALUES(name),
+    parent=VALUES(parent),
+    date=VALUES(date),
+    size=VALUES(size),
+    url=VALUES(url),
+    sha1=VALUES(sha1);
+*/});
+
+exports.savefile = function* (info) {
+  return yield mysql.query(SAVE_FILE_SQL, [
+    info.name, info.parent, info.date, info.size, info.url, info.sha1
+  ]);
+};
+
+var SAVE_DIR_SQL = multiline(function () {;/*
+  INSERT INTO
+    dist_dir(gmt_create, gmt_modified, name, parent, date)
+  VALUES
+    (now(), now(), ?, ?, ?)
+  ON DUPLICATE KEY UPDATE
+    name=VALUES(name),
+    parent=VALUES(parent),
+    date=VALUES(date);
+*/});
+
+exports.savedir = function* (info) {
+  return yield mysql.query(SAVE_DIR_SQL, [
+    info.name, info.parent, info.date
+  ]);
+};
+
+var LIST_DIRS_SQL = multiline(function () {;/*
+  SELECT name, parent, date FROM dist_dir WHERE parent = ?;
+*/});
+var LIST_FILES_SQL = multiline(function () {;/*
+  SELECT name, parent, date, size, url, sha1
+  FROM dist_file WHERE parent = ?;
+*/});
+
+exports.listdir = function* (name) {
+  var rs = yield [
+    mysql.query(LIST_DIRS_SQL, [name]),
+    mysql.query(LIST_FILES_SQL, [name]),
+  ];
+  return rs[0].concat(rs[1]);
+};
+
+var GET_FILE_SQL = multiline(function () {;/*
+  SELECT name, parent, date, url, size, sha1 FROM dist_file WHERE parent = ? AND name = ?;
+*/});
+
+exports.getfile = function* (fullname) {
+  var name = path.basename(fullname);
+  var parent = path.dirname(fullname);
+  if (parent !== '/') {
+    parent += '/';
+  }
+  return yield mysql.queryOne(GET_FILE_SQL, [parent, name]);
+};
--- a/proxy/module.js
+++ b/proxy/module.js
@@ -433,38 +433,16 @@ exports.listByName = function (name, callback) {
  });
 };

-var LIST_SINCE_SQLS = [];
-LIST_SINCE_SQLS.push(multiline(function () {;/*
-  SELECT
-    module_id
-  FROM
-    tag
-  WHERE
-    tag="latest" AND gmt_modified>?;
-*/}));
-LIST_SINCE_SQLS.push(multiline(function () {;/*
+var LIST_SINCE_SQL = multiline(function () {;/*
  SELECT
    distinct(name)
  FROM
-    module
+    tag
  WHERE
-    id IN (?);
-*/}));
+    gmt_modified > ?;
+*/});
 exports.listSince = function (start, callback) {
-  var ep = eventproxy.create();
-  ep.fail(callback);
-  mysql.query(LIST_SINCE_SQLS[0], [new Date(start)], ep.done(function (rows) {
-    if (!rows || rows.length === 0) {
-      return callback(null, []);
-    }
-    ep.emit('ids', rows.map(function (r) {
-      return r.module_id;
-    }));
-  }));
-
-  ep.once('ids', function (ids) {
-    mysql.query(LIST_SINCE_SQLS[1], [ids], callback);
-  });
+  mysql.query(LIST_SINCE_SQL, [new Date(start)], callback);
 };

 var LIST_ALL_NAME_SQL = multiline(function () {;/*
--- a/proxy/npm.js
+++ b/proxy/npm.js
@@ -26,7 +26,8 @@ function *request(url, options) {
  options.headers = {
    'user-agent': USER_AGENT
  };
-  url = config.sourceNpmRegistry + url;
+  var registry = options.registry || config.sourceNpmRegistry;
+  url = registry + url;
  var r;
  try {
    r = yield *urllib.request(url, options);
@@ -70,7 +71,8 @@ exports.getAllSince = function *(startkey) {

 exports.getShort = function *() {
  var r = yield *request('/-/short', {
-    timeout: 300000
+    timeout: 300000,
+    registry: 'http://r.cnpmjs.org', // registry.npmjs.org/-/short is 404 now.
  });
  return r.data;
 };
--- a/proxy/sync_module_worker.js
+++ b/proxy/sync_module_worker.js
@@ -17,6 +17,7 @@

 var co = require('co');
 var gather = require('co-gather');
+var defer = require('co-defer');
 var thunkify = require('thunkify-wrap');
 var debug = require('debug')('cnpmjs.org:proxy:sync_module_worker');
 var EventEmitter = require('events').EventEmitter;
@@ -42,16 +43,12 @@ var USER_AGENT = 'sync.cnpmjs.org/' + config.version + ' ' + urllib.USER_AGENT;
 function SyncModuleWorker(options) {
  EventEmitter.call(this);
  this._logId = options.logId;
-  this.startName = options.name;
  if (!Array.isArray(options.name)) {
    options.name = [options.name];
  }

-  this.names = options.name;
-  // for (var i = 0; i < this.names.length; i++) {
-  //   // ensure package name is lower case
-  //   this.names[i] = this.names[i].toLowerCase();
-  // }
+  this.names = options.name || [];
+  this.startName = this.names[0];

  this.username = options.username;
  this.concurrency = options.concurrency || 1;
@@ -124,10 +121,25 @@ SyncModuleWorker.prototype.add = function (name) {
  this.log('    add dependencies: %s', name);
 };

+SyncModuleWorker.prototype._doneOne = function *(concurrencyId, name, success) {
+  if (success) {
+    this.pushSuccess(name);
+  } else {
+    this.pushFail(name);
+  }
+  delete this.syncingNames[name];
+  var that = this;
+  // relase the stack: https://github.com/cnpm/cnpmjs.org/issues/328
+  defer.setImmediate(function *() {
+    that.log('[c#%s] setImmediate after, %s done, start next...', concurrencyId, name);
+    yield *that.next(concurrencyId);
+  });
+};
+
 SyncModuleWorker.prototype.next = function *(concurrencyId) {
  var name = this.names.shift();
  if (!name) {
-    return process.nextTick(this.finish.bind(this));
+    return setImmediate(this.finish.bind(this));
  }

  var that = this;
@@ -138,20 +150,17 @@ SyncModuleWorker.prototype.next = function *(concurrencyId) {
    pkg = yield npm.get(name);
  } catch (err) {
    // if 404
-    if (err.res && err.res.statusCode === 404) {
-      that.pushSuccess(name);
-    } else {
-      that.pushFail(name);
+    if (!err.res || err.res.statusCode !== 404) {
+      var errMessage = err.name + ': ' + err.message;
+      that.log('[c#%s] [error] [%s] get package error: %s', concurrencyId, name, errMessage);
+      yield *that._doneOne(concurrencyId, name, false);
+      return;
    }
-    that.log('[error] [%s] get package error: %s', name, err.stack);
-    delete that.syncingNames[name];
-    yield *that.next(concurrencyId);
-    return;
  }
+
  if (!pkg) {
-    that.log('[error] [%s] get package error: package not exist', name);
-    delete that.syncingNames[name];
-    yield that.next(concurrencyId);
+    that.log('[c#%s] [error] [%s] get package error: package not exists', concurrencyId, name);
+    yield *that._doneOne(concurrencyId, name, true);
    return;
  }

@@ -160,17 +169,14 @@ SyncModuleWorker.prototype.next = function *(concurrencyId) {
  try {
    versions = yield that._sync(name, pkg);
  } catch (err) {
-    that.pushFail(name);
-    that.log('[error] [%s] sync error: %s', name, err.stack);
-    delete that.syncingNames[name];
-    yield *that.next(concurrencyId);
+    that.log('[c#%s] [error] [%s] sync error: %s', concurrencyId, name, err.stack);
+    yield *that._doneOne(concurrencyId, name, false);
    return;
  }
-  that.log('[%s] synced success, %d versions: %s',
-    name, versions.length, versions.join(', '));
-  that.pushSuccess(name);
-  delete that.syncingNames[name];
-  yield that.next(concurrencyId);
+
+  that.log('[c#%d] [%s] synced success, %d versions: %s',
+    concurrencyId, name, versions.length, versions.join(', '));
+  yield *that._doneOne(concurrencyId, name, true);
 };

 function *_listStarUsers(modName) {
@@ -256,7 +262,7 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
    var maintainers = p.maintainers || [];
    if (maintainers && !Array.isArray(maintainers)) {
      // http://r.cnpmjs.org/jasmine-node
-      // TODO: "maintainers": "Martin HĂ¤ger <martin.haeger@gmail.com>",
+      // TODO: "maintainers": "Martin H膫陇ger <martin.haeger@gmail.com>",
      maintainers = [maintainers];
    }

@@ -594,12 +600,14 @@ SyncModuleWorker.prototype._syncOneVersion = function *(versionIndex, sourcePack
    // get tarball
    var r = yield *urllib.request(downurl, options);
    var statusCode = r.status || -1;
-    if (statusCode === 404) {
-      shasum = sourcePackage.dist.shasum;
-      return yield afterUpload({
-        url: downurl
-      });
-    }
+    // https://github.com/cnpm/cnpmjs.org/issues/325
+    // if (statusCode === 404) {
+    //   shasum = sourcePackage.dist.shasum;
+    //   return yield *afterUpload({
+    //     url: downurl
+    //   });
+    // }
+
    if (statusCode !== 200) {
      var err = new Error('Download ' + downurl + ' fail, status: ' + statusCode);
      err.name = 'DownloadTarballError';
@@ -616,6 +624,13 @@ SyncModuleWorker.prototype._syncOneVersion = function *(versionIndex, sourcePack
    var end = thunkify.event(rs);
    yield end(); // after end event emit

+    if (dataSize === 0) {
+      var err = new Error('Download ' + downurl + ' file size is zero');
+      err.name = 'DownloadTarballSizeZeroError';
+      err.data = sourcePackage;
+      throw err;
+    }
+
    // check shasum
    shasum = shasum.digest('hex');
    if (shasum !== sourcePackage.dist.shasum) {
@@ -633,7 +648,7 @@ SyncModuleWorker.prototype._syncOneVersion = function *(versionIndex, sourcePack
    };
    // upload to NFS
    var result = yield nfs.upload(filepath, options);
-    return yield afterUpload(result);
+    return yield *afterUpload(result);
  } finally {
    // remove tmp file whatever
    fs.unlink(filepath, utility.noop);
@@ -678,13 +693,13 @@ SyncModuleWorker.prototype._syncOneVersion = function *(versionIndex, sourcePack
    mod.package.dist = dist;
    var r = yield Module.add(mod);

-    that.log('    [%s:%s] done, insertId: %s, author: %s, version: %s, ' +
-    'size: %d, publish_time: %j, publish on cnpm: %s',
-    sourcePackage.name, versionIndex,
-    r.id,
-    author, mod.version, dataSize,
-    new Date(mod.publish_time),
-    that._publish);
+    that.log('    [%s:%s] done, insertId: %s, author: %s, version: %s, '
+      + 'size: %d, publish_time: %j, publish on cnpm: %s',
+      sourcePackage.name, versionIndex,
+      r.id,
+      author, mod.version, dataSize,
+      new Date(mod.publish_time),
+      that._publish);

    return r;
  }
@@ -696,7 +711,8 @@ SyncModuleWorker.sync = function *(name, username, options) {
  if (!pkg || !pkg._rev) {
    return {
      ok: false,
-      pkg: pkg
+      pkg: pkg,
+      statusCode: 404
    };
  }
  var result = yield Log.create({name: name, username: username});
--- a/routes/registry.js
+++ b/routes/registry.js
@@ -45,6 +45,8 @@ function routes(app) {
  app.put('/:name/sync', sync.sync);
  app.get('/:name/sync/log/:id', sync.getSyncLog);

+  app.put('/:name/:tag', login, mod.updateTag);
+
  // need limit by ip
  app.get('/:name/download/:filename', limit, mod.download);

--- a/routes/web.js
+++ b/routes/web.js
@@ -36,7 +36,7 @@ function routes(app) {

  app.get('/_list/search/search', pkg.rangeSearch);

-  app.get(/^\/dist(\/.+)?/, dist.redirect);
+  app.get(/^\/dist(\/.*)?/, dist.list);
 }

 module.exports = routes;
--- a/servers/registry.js
+++ b/servers/registry.js
@@ -18,28 +18,29 @@
 var koa = require('koa');
 var app = module.exports = koa();
 var http = require('http');
-var microtime = require('microtime');
 var middlewares = require('koa-middlewares');
 var routes = require('../routes/registry');
 var logger = require('../common/logger');
 var config = require('../config');
 var session = require('../common/session');
 var auth = require('../middleware/auth');
+var staticCache = require('../middleware/static');
 var notFound = require('../middleware/registry_not_found');

-app.use(middlewares.rt({headerName: 'X-ReadTime', timer: microtime}));
-
-app.use(middlewares.rewrite('/favicon.ico', '/public/favicon.ico'));
+app.use(middlewares.rt({headerName: 'X-ReadTime'}));
+app.use(middlewares.rewrite('/favicon.ico', '/favicon.png'));
+app.use(staticCache);

 app.keys = ['todokey', config.sessionSecret];
-app.outputErrors = true;
 app.proxy = true;
 app.use(session);
 app.use(middlewares.bodyParser({jsonLimit: config.jsonLimit}));
 app.use(auth());
 app.use(notFound);

-app.use(middlewares.compress({threshold: 150}));
+if (config.enableCompress) {
+  app.use(middlewares.compress({threshold: 150}));
+}
 app.use(middlewares.conditional());
 app.use(middlewares.etag());

--- a/servers/web.js
+++ b/servers/web.js
@@ -1,4 +1,4 @@
-/*!
+/**!
 * cnpmjs.org - servers/web.js
 *
 * Copyright(c) cnpmjs.org and other contributors.
@@ -18,13 +18,13 @@
 var path = require('path');
 var http = require('http');
 var fs = require('fs');
-var microtime = require('microtime');
 var koa = require('koa');
 var middlewares = require('koa-middlewares');
 var markdown = require('koa-markdown');
 var session = require('../common/session');
 var opensearch = require('../middleware/opensearch');
 var notFound = require('../middleware/web_not_found');
+var staticCache = require('../middleware/static');
 var auth = require('../middleware/auth');
 var routes = require('../routes/web');
 var logger = require('../common/logger');
@@ -34,22 +34,22 @@ var app = koa();

 var rootdir = path.dirname(__dirname);

-app.use(middlewares.rt({headerName: 'X-ReadTime', timer: microtime}));
-app.use(middlewares.staticCache(path.join(__dirname, '..', 'public'), {
-  buffer: !config.debug,
-  maxAge: config.debug ? 0 : 60 * 60 * 24 * 7,
-  dir: path.join(rootdir, 'public')
-}));
+app.use(middlewares.rt({headerName: 'X-ReadTime'}));
+app.use(middlewares.rewrite('/favicon.ico', '/favicon.png'));
+app.use(staticCache);
+
 app.use(opensearch);
 app.keys = ['todokey', config.sessionSecret];
-app.outputErrors = true;
 app.proxy = true;
 app.use(session);
 app.use(middlewares.bodyParser());
 app.use(auth());
 app.use(notFound);

-app.use(middlewares.compress({threshold: 150}));
+if (config.enableCompress) {
+  app.use(middlewares.compress({threshold: 150}));
+}
+
 app.use(middlewares.conditional());
 app.use(middlewares.etag());

@@ -63,20 +63,30 @@ var layout = fs.readFileSync(path.join(viewDir, 'layout.html'), 'utf8')
  .replace('{{logoURL}}', config.logoURL);
 fs.writeFileSync(layoutFile, layout);

+// custom web readme home page support
+var readmeFile = path.join(docDir, '_readme.md');
+var readmeContent;
+if (config.customReadmeFile) {
+  readmeContent = fs.readFileSync(config.customReadmeFile, 'utf8');
+} else {
+  readmeContent = fs.readFileSync(path.join(docDir, 'readme.md'), 'utf8');
+}
+fs.writeFileSync(readmeFile, readmeContent);
+
 app.use(markdown({
  baseUrl: '/',
  root: docDir,
  layout: layoutFile,
  titleHolder: '<%- locals.title %>',
  bodyHolder: '<%- locals.body %>',
-  indexName: 'readme'
+  indexName: '_readme'
 }));

 var locals = {
  config: config
 };

-middlewares.render(app, {
+middlewares.ejs(app, {
  root: viewDir,
  viewExt: 'html',
  layout: '_layout',
--- a/sync/index.js
+++ b/sync/index.js
@@ -14,15 +14,16 @@
 * Module dependencies.
 */

-var config = require('../config');
+var debug = require('debug')('cnpmjs.org:sync:index');
+var co = require('co');
 var ms = require('ms');
-var mail = require('../common/mail');
 var util = require('util');
 var utility = require('utility');
-var debug = require('debug')('cnpmjs.org:sync:index');
+var config = require('../config');
+var mail = require('../common/mail');
 var Total = require('../proxy/total');
 var logger = require('../common/logger');
-var co = require('co');
+var syncDistWorker = require('./sync_dist');

 var sync = null;

@@ -73,7 +74,32 @@ var handleSync = co(function *() {

 if (sync) {
  handleSync();
-  setInterval(handleSync, ms('30m'));
+  setInterval(handleSync, ms(config.syncInterval));
+}
+
+var syncingDist = false;
+var syncDist = co(function* syncDist() {
+  if (syncingDist) {
+    return;
+  }
+  syncingDist = true;
+  logger.info('Start syncing dist...');
+  try {
+    yield* syncDistWorker();
+    yield* syncDistWorker.syncPhantomjsDir();
+  } catch (err) {
+    err.message += ' (sync dist error)';
+    logger.warn('Sync dist error: %s: %s\n%s', err.name, err.message, err.stack);
+    sendMailToAdmin(err, null, new Date());
+  }
+  syncingDist = false;
+});
+
+if (config.syncDist) {
+  syncDist();
+  setInterval(syncDist, ms(config.syncInterval));
+} else {
+  logger.info('sync dist disable');
 }

 function sendMailToAdmin(err, result, syncTime) {
@@ -91,7 +117,7 @@ function sendMailToAdmin(err, result, syncTime) {
    subject = 'Sync Error';
    type = 'error';
    html = util.format('Sync packages from official registry failed.\n' +
-      'Start sync time is %s.\nError message is %s.', syncTime, err.stack);
+      'Start sync time is %s.\nError message is %s: %s\n%s.', syncTime, err.name, err.message, err.stack);
  } else if (result.fails && result.fails.length) {
    subject = 'Sync Finished But Some Packages Failed';
    type = 'warn';
--- a/sync/status.js
+++ b/sync/status.js
@@ -47,6 +47,13 @@ Status.prototype.start = function () {
  this.timer = setInterval(this.log.bind(this), 30000);
 };

+Status.prototype.stop = function () {
+  this.log(true);
+  clearInterval(this.timer);
+  this.timer = null;
+  this.started = false;
+};
+
 Status.init = function (options, worker) {
  var status = new Status(options);
  status.start();
@@ -65,9 +72,7 @@ Status.init = function (options, worker) {
  });

  worker.on('end', function () {
-    status.started = false;
-    status.log(true);
-    clearInterval(status.timer);
+    status.stop();
  });
 };

--- a/sync/sync_all.js
+++ b/sync/sync_all.js
@@ -24,7 +24,6 @@ var Npm = require('../proxy/npm');
 var Total = require('../proxy/total');
 var SyncModuleWorker = require('../proxy/sync_module_worker');
 var Module = require('../proxy/module');
-var co = require('co');
 var thunkify = require('thunkify-wrap');

 function subtract(subtracter, minuend) {
--- a/sync/sync_dist.js
+++ b/sync/sync_dist.js
@@ -0,0 +1,340 @@
+/**!
+ * cnpmjs.org - sync/sync_dist.js
+ *
+ * Copyright(c) fengmk2 and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *   fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var debug = require('debug')('cnpmjs.org:sync:sync_dist');
+var fs = require('fs');
+var urllib = require('co-urllib');
+var co = require('co');
+var bytes = require('bytes');
+var crypto = require('crypto');
+var utility = require('utility');
+var thunkify = require('thunkify-wrap');
+var cheerio = require('cheerio');
+var urlResolve = require('url').resolve;
+var common = require('../lib/common');
+var Dist = require('../proxy/dist');
+var config = require('../config');
+var nfs = require('../common/nfs');
+var logger = require('../common/logger');
+
+var disturl = config.disturl;
+var USER_AGENT = 'distsync.cnpmjs.org/' + config.version + ' ' + urllib.USER_AGENT;
+
+module.exports = sync;
+
+function* sync(name) {
+  name = name || '/';
+  yield* syncDir(name);
+}
+
+function* syncDir(fullname, info) {
+  var news = yield* sync.listdiff(fullname);
+  var files = [];
+  var dirs = [];
+
+  for (var i = 0; i < news.length; i++) {
+    var item = news[i];
+    if (item.type === 'dir') {
+      dirs.push(item);
+    } else {
+      files.push(item);
+    }
+  }
+
+  logger.info('sync remote:%s got %d new items, %d dirs, %d files to sync',
+    fullname, news.length, dirs.length, files.length);
+
+  for (var i = 0; i < files.length; i++) {
+    yield* syncFile(files[i]);
+  }
+
+  for (var i = 0; i < dirs.length; i++) {
+    var dir = dirs[i];
+    yield* syncDir(dir.parent + dir.name, dir);
+  }
+
+  if (info) {
+    logger.info('Save dir:%s %j to database', fullname, info);
+    yield* Dist.savedir(info);
+  }
+
+  logger.info('Sync %s finished, %d dirs, %d files',
+    fullname, dirs.length, files.length);
+}
+
+function* syncFile(info) {
+  var name = info.parent + info.name;
+  name = process.pid + name.replace(/\//g, '_'); // make sure no parent dir
+  var isPhantomjsURL = false;
+  var downurl = disturl + info.parent + info.name;
+  if (info.downloadURL) {
+    downurl = info.downloadURL;
+    isPhantomjsURL = true;
+  }
+  var filepath = common.getTarballFilepath(name);
+  var ws = fs.createWriteStream(filepath);
+
+  var options = {
+    writeStream: ws,
+    followRedirect: true,
+    timeout: 6000000, // 100 minutes download
+    headers: {
+      'user-agent': USER_AGENT
+    }
+  };
+
+  try {
+    logger.info('downloading %s %s to %s, isPhantomjsURL: %s',
+      bytes(info.size), downurl, filepath, isPhantomjsURL);
+    // get tarball
+    var r = yield *urllib.request(downurl, options);
+    var statusCode = r.status || -1;
+    logger.info('download %s got status %s, headers: %j',
+      downurl, statusCode, r.headers);
+    if (statusCode !== 200) {
+      var err = new Error('Download ' + downurl + ' fail, status: ' + statusCode);
+      err.name = 'DownloadDistFileError';
+      throw err;
+    }
+
+    var shasum = crypto.createHash('sha1');
+    var dataSize = 0;
+    var rs = fs.createReadStream(filepath);
+    rs.on('data', function (data) {
+      shasum.update(data);
+      dataSize += data.length;
+    });
+    var end = thunkify.event(rs);
+    yield end(); // after end event emit
+
+    if (dataSize === 0) {
+      var err = new Error('Download ' + downurl + ' file size is zero');
+      err.name = 'DownloadDistFileZeroSizeError';
+      throw err;
+    }
+
+    if (isPhantomjsURL) {
+      debug('real size: %s, expect size: %s', dataSize, info.size);
+      if (dataSize < info.size) {
+        // phantomjs download page only show `6.7 MB`
+        var err = new Error('Download ' + downurl + ' file size is '
+          + dataSize + ' not match ' + info.size);
+        err.name = 'DownloadDistFileSizeError';
+        throw err;
+      }
+      info.size = dataSize;
+    } else if (dataSize !== info.size) {
+      var err = new Error('Download ' + downurl + ' file size is '
+        + dataSize + ' not match ' + info.size);
+      err.name = 'DownloadDistFileSizeError';
+      throw err;
+    }
+
+    shasum = shasum.digest('hex');
+    var args = {
+      key: '/dist' + info.parent + info.name,
+      size: info.size,
+      shasum: shasum,
+    };
+
+    // upload to NFS
+    logger.info('uploading %s to nfs:%s', filepath, args.key);
+    var result = yield nfs.upload(filepath, args);
+    info.url = result.url || result.key;
+    info.sha1 = shasum;
+
+    logger.info('upload %s to nfs:%s with size:%d, sha1:%s',
+      args.key, info.url, info.size, info.sha1);
+  } finally {
+    // remove tmp file whatever
+    fs.unlink(filepath, utility.noop);
+  }
+
+  logger.info('Sync dist file: %j done', info);
+  yield* Dist.savefile(info);
+}
+
+// <a href="latest/">latest/</a>                                            02-May-2014 14:45                   -
+// <a href="node-v0.4.10.tar.gz">node-v0.4.10.tar.gz</a>                                26-Aug-2011 16:22            12410018
+var FILE_RE = /^<a[^>]+>([^<]+)<\/a>\s+(\d+\-\w+\-\d+ \d+\:\d+)\s+([\-\d]+)/;
+
+function* listdir(fullname) {
+  var url = disturl + fullname;
+  var result = yield* urllib.request(url, {
+    timeout: 60000,
+  });
+  debug('listdir %s got %s, %j', url, result.status, result.headers);
+  var html = result.data && result.data.toString() || '';
+  var lines = html.split('\n');
+  var items = [];
+  for (var i = 0; i < lines.length; i++) {
+    var m = FILE_RE.exec(lines[i].trim());
+    if (!m) {
+      continue;
+    }
+    var itemName = m[1].replace(/^\/+/, '');
+    if (!itemName) {
+      continue;
+    }
+
+    // filter /nightlies/*
+    if (itemName.indexOf('nightlies/') === 0) {
+      continue;
+    }
+
+    items.push({
+      name: itemName, // 'SHASUMS.txt', 'x64/'
+      date: m[2],
+      size: m[3] === '-' ? '-' : parseInt(m[3]),
+      type: m[3] === '-' ? 'dir' : 'file',
+      parent: fullname, // '/', '/v0.10.28/'
+    });
+  }
+  return items;
+}
+
+sync.listdiff = function* (fullname) {
+  var items = yield* listdir(fullname);
+  if (items.length === 0) {
+    return items;
+  }
+  var exists = yield* Dist.listdir(fullname);
+  debug('listdiff %s got %s exists items', fullname, exists.length);
+  var map = {};
+  for (var i = 0; i < exists.length; i++) {
+    var item = exists[i];
+    map[item.name] = item;
+  }
+  var news = [];
+  for (var i = 0; i < items.length; i++) {
+    var item = items[i];
+    var exist = map[item.name];
+    if (!exist || exist.date !== item.date) {
+      news.push(item);
+      continue;
+    }
+
+    if (item.size !== '-' && item.size !== exist.size) {
+      news.push(item);
+      continue;
+    }
+
+    debug('skip %s', item.name);
+  }
+  return news;
+};
+
+function* syncPhantomjsDir() {
+  var fullname = '/phantomjs/';
+  var files = yield* sync.listPhantomjsDiff(fullname);
+
+  logger.info('sync remote:%s got %d files to sync',
+    fullname, files.length);
+
+  for (var i = 0; i < files.length; i++) {
+    yield* syncFile(files[i]);
+  }
+
+  logger.info('SyncPhantomjsDir %s finished, %d files',
+    fullname, files.length);
+}
+sync.syncPhantomjsDir = syncPhantomjsDir;
+
+// <tr class="iterable-item" id="download-301626">
+//   <td class="name"><a class="execute" href="/ariya/phantomjs/downloads/phantomjs-1.9.7-windows.zip">phantomjs-1.9.7-windows.zip</a></td>
+//   <td class="size">6.7 MB</td>
+//   <td class="uploaded-by"><a href="/Vitallium">Vitallium</a></td>
+//   <td class="count">122956</td>
+//   <td class="date">
+//     <div>
+//       <time datetime="2014-01-27T18:29:53.706942" data-title="true">2014-01-27</time>
+//     </div>
+//   </td>
+//   <td class="delete">
+//
+//   </td>
+// </tr>
+
+function* listPhantomjsDir(fullname) {
+  var url = 'https://bitbucket.org/ariya/phantomjs/downloads';
+  var result = yield* urllib.request(url, {
+    timeout: 60000,
+  });
+  debug('listPhantomjsDir %s got %s, %j', url, result.status, result.headers);
+  var html = result.data && result.data.toString() || '';
+  var $ = cheerio.load(html);
+  var items = [];
+  $('tr.iterable-item').each(function (i, el) {
+    var $el = $(this);
+    var $link = $el.find('.name a');
+    var name = $link.text();
+    var downloadURL = $link.attr('href');
+    if (!name || !downloadURL || !/\.(zip|bz2|gz)$/.test(downloadURL)) {
+      return;
+    }
+    downloadURL = urlResolve(url, downloadURL);
+    var size = parseInt(bytes($el.find('.size').text().toLowerCase().replace(/\s/g, '')));
+    if (size > 1024 * 1024) {
+      size -= 1024 * 1024;
+    } else if (size > 1024) {
+      size -= 1024;
+    } else {
+      size -= 10;
+    }
+    var date = $el.find('.date time').text();
+    items.push({
+      name: name, // 'SHASUMS.txt', 'x64/'
+      date: date,
+      size: size,
+      type: 'file',
+      parent: fullname,
+      downloadURL: downloadURL,
+    });
+  });
+  return items;
+}
+sync.listPhantomjsDir = listPhantomjsDir;
+
+sync.listPhantomjsDiff = function* (fullname) {
+  var items = yield* listPhantomjsDir(fullname);
+  if (items.length === 0) {
+    return items;
+  }
+  var exists = yield* Dist.listdir(fullname);
+  debug('listdiff %s got %s exists items', fullname, exists.length);
+  var map = {};
+  for (var i = 0; i < exists.length; i++) {
+    var item = exists[i];
+    map[item.name] = item;
+  }
+  var news = [];
+  for (var i = 0; i < items.length; i++) {
+    var item = items[i];
+    var exist = map[item.name];
+    if (!exist || exist.date !== item.date) {
+      news.push(item);
+      continue;
+    }
+
+    // if (item.size !== exist.size) {
+    //   news.push(item);
+    //   continue;
+    // }
+
+    debug('skip %s', item.name);
+  }
+  return news;
+};
--- a/test/controllers/registry/module.test.js
+++ b/test/controllers/registry/module.test.js
@@ -79,7 +79,7 @@ describe('controllers/registry/module.test.js', function () {
    it('should return module info and etag', function (done) {
      request(app)
      .get('/mk2testmodule')
-      .expect('content-type', 'application/json')
+      .expect('content-type', 'application/json; charset=utf-8')
      .expect(200, function (err, res) {
        should.not.exist(err);
        // should have etag
@@ -98,7 +98,7 @@ describe('controllers/registry/module.test.js', function () {
          'license');
        res.body.name.should.equal('mk2testmodule');
        res.body.versions[Object.keys(res.body.versions)[0]]
-          .dist.tarball.should.include('/mk2testmodule/download');
+          .dist.tarball.should.containEql('/mk2testmodule/download');
        res.body.time.should.have.property('modified');
        res.body.time.modified.should.be.a.String;
        res.body.time.should.have.property('created');
@@ -113,7 +113,7 @@ describe('controllers/registry/module.test.js', function () {
      request(app)
      .get('/mk2testmodule')
      .set('accept-encoding', 'gzip')
-      .expect('content-encoding', 'gzip')
+      // .expect('content-encoding', 'gzip')
      .expect(200, function (err, res) {
        // console.log(res.headers)
        should.not.exist(err);
@@ -131,7 +131,7 @@ describe('controllers/registry/module.test.js', function () {
          'license');
        res.body.name.should.equal('mk2testmodule');
        res.body.versions[Object.keys(res.body.versions)[0]]
-          .dist.tarball.should.include('/mk2testmodule/download');
+          .dist.tarball.should.containEql('/mk2testmodule/download');
        should.not.exist(res.headers['set-cookie']);
        done();
      });
@@ -172,7 +172,7 @@ describe('controllers/registry/module.test.js', function () {
        body.name.should.equal('mk2testmodule');
        body.version.should.equal('0.0.1');
        body._id.should.equal('mk2testmodule@0.0.1');
-        body.dist.tarball.should.include('/mk2testmodule/download/mk2testmodule-0.0.1.tgz');
+        body.dist.tarball.should.containEql('/mk2testmodule/download/mk2testmodule-0.0.1.tgz');
        done();
      });
    });
@@ -189,7 +189,7 @@ describe('controllers/registry/module.test.js', function () {
        }]
      })
      .set('authorization', baseauth)
-      .expect('content-type', 'application/json', done);
+      .expect('content-type', 'application/json; charset=utf-8', done);
    });

    it('should add new maintainers', function (done) {
@@ -206,7 +206,7 @@ describe('controllers/registry/module.test.js', function () {
      })
      .set('authorization', baseauth)
      .expect(201)
-      .expect('content-type', 'application/json', done);
+      .expect('content-type', 'application/json; charset=utf-8', done);
    });

    it('should add again new maintainers', function (done) {
@@ -223,7 +223,7 @@ describe('controllers/registry/module.test.js', function () {
      })
      .set('authorization', baseauth)
      .expect(201)
-      .expect('content-type', 'application/json', done);
+      .expect('content-type', 'application/json; charset=utf-8', done);
    });

    it('should rm maintainers', function (done) {
@@ -237,7 +237,7 @@ describe('controllers/registry/module.test.js', function () {
      })
      .set('authorization', baseauth)
      .expect(201)
-      .expect('content-type', 'application/json', done);
+      .expect('content-type', 'application/json; charset=utf-8', done);
    });

    it('should rm again maintainers', function (done) {
@@ -251,7 +251,7 @@ describe('controllers/registry/module.test.js', function () {
      })
      .set('authorization', baseauth)
      .expect(201)
-      .expect('content-type', 'application/json', done);
+      .expect('content-type', 'application/json; charset=utf-8', done);
    });
  });

@@ -399,7 +399,7 @@ describe('controllers/registry/module.test.js', function () {
    it('should upload tarball fail 404 when rev wrong', function (done) {
      var body = fs.readFileSync(path.join(fixtures, 'testputmodule-0.1.9.tgz'));
      request(app)
-      .put('/' + pkg.name + '/-/' + pkg.name + '-0.1.9.tgz/-rev/' + lastRev + '1')
+      .put('/' + pkg.name + '/-/' + pkg.name + '-0.1.9.tgz/-rev/' + '1231231')
      .set('authorization', baseauth)
      .set('content-type', 'application/octet-stream')
      .set('content-length', '' + body.length)
@@ -479,7 +479,7 @@ describe('controllers/registry/module.test.js', function () {
        should.not.exist(err);
        res.body.name.should.equal('testputmodule');
        res.body.version.should.equal('0.1.9');
-        res.body.dist.tarball.should.include('/testputmodule/download/testputmodule-0.1.9.tgz');
+        res.body.dist.tarball.should.containEql('/testputmodule/download/testputmodule-0.1.9.tgz');
        done();
      });
    });
@@ -557,6 +557,8 @@ describe('controllers/registry/module.test.js', function () {
      request(app)
      .get('/-/all/since?stale=update_after&startkey=0')
      .expect(200, function (err, res) {
+        should.not.exist(err);
+        should.exist(res.body);
        res.body.should.be.an.Object;
        res.body._updated.should.be.a.Number;
        var keys = Object.keys(res.body);
@@ -569,6 +571,8 @@ describe('controllers/registry/module.test.js', function () {
      request(app)
      .get('/-/all/since?stale=update_after&startkey=' + (Date.now() * 2))
      .expect(200, function (err, res) {
+        should.not.exist(err);
+        should.exist(res.body);
        res.body.should.be.an.Object;
        res.body._updated.should.be.a.Number;
        res.body.should.eql({
@@ -648,7 +652,7 @@ describe('controllers/registry/module.test.js', function () {
    it('should download a file with 302 redirect', function (done) {
      request(app)
      .get('/cutter/download/cutter-0.0.2.tgz')
-      .expect('Location', 'http://qtestbucket.qiniudn.com/cutter/-/cutter-0.0.2.tgz')
+      .expect('Location', config.qn.domain + '/cutter/-/cutter-0.0.2.tgz')
      .expect(302, done);
    });
  });
@@ -685,7 +689,66 @@ describe('controllers/registry/module.test.js', function () {
    });
  });

-  describe('DELETE /:name/-rev/:rev', function (done) {
+  describe('PUT /:name/:tag', function () {
+    it('should create new tag ok', function (done) {
+      request(app)
+      .put('/testputmodule/newtag')
+      .set('content-type', 'application/json')
+      .set('authorization', baseauth)
+      .send('"0.1.9"')
+      .expect(201, done);
+    });
+
+    it('should override exist tag ok', function (done) {
+      request(app)
+      .put('/testputmodule/newtag')
+      .set('content-type', 'application/json')
+      .set('authorization', baseauth)
+      .send('"0.1.9"')
+      .expect(201, done);
+    });
+
+    it('should tag invalid version 403', function (done) {
+      request(app)
+      .put('/testputmodule/newtag')
+      .set('content-type', 'application/json')
+      .set('authorization', baseauth)
+      .send('"hello"')
+      .expect(403)
+      .expect({
+        error: 'forbidden',
+        reason: 'setting tag newtag to invalid version: hello: testputmodule/newtag'
+      }, done);
+    });
+
+    it('should tag not eixst version 403', function (done) {
+      request(app)
+      .put('/testputmodule/newtag')
+      .set('content-type', 'application/json')
+      .set('authorization', baseauth)
+      .send('"5.0.0"')
+      .expect(403)
+      .expect({
+        error: 'forbidden',
+        reason: 'setting tag newtag to unknown version: 5.0.0: testputmodule/newtag'
+      }, done);
+    });
+
+    it('should tag permission 403', function (done) {
+      request(app)
+      .put('/testputmodule/newtag')
+      .set('content-type', 'application/json')
+      .set('authorization', baseauthOther)
+      .send('"0.1.9"')
+      .expect(403)
+      .expect({
+        error: 'forbidden',
+        reason: 'no permission to modify testputmodule'
+      }, done);
+    });
+  });
+
+  describe('DELETE /:name/-rev/:rev', function () {
    var baseauth = 'Basic ' + new Buffer('cnpmjstest10:cnpmjstest10').toString('base64');
    var baseauthOther = 'Basic ' + new Buffer('cnpmjstest101:cnpmjstest101').toString('base64');
    var lastRev;
--- a/test/controllers/registry/user.test.js
+++ b/test/controllers/registry/user.test.js
@@ -167,7 +167,7 @@ describe('controllers/registry/user.test.js', function () {
      }, function (err, res) {
        should.not.exist(err);
        should.exist(res.headers['set-cookie']);
-        res.headers['set-cookie'].join(';').should.include('AuthSession=');
+        res.headers['set-cookie'].join(';').should.containEql('AuthSession=');
        done();
      });
    });
--- a/test/controllers/web/dist.test.js
+++ b/test/controllers/web/dist.test.js
@@ -16,22 +16,126 @@

 var should = require('should');
 var request = require('supertest');
+var pedding = require('pedding');
+var mm = require('mm');
 var app = require('../../../servers/web');
+var Dist = require('../../../proxy/dist');

 describe('controllers/web/dist.test.js', function () {
  before(function (done) {
    app.listen(0, done);
  });
+
  after(function (done) {
    app.close(done);
  });

-  describe('GET /dist', function (done) {
-    it('should 302 to config.disturl', function (done) {
+  afterEach(mm.restore);
+
+  describe('GET /dist/*', function (done) {
+    it('should GET /dist redirect to /dist/', function (done) {
      request(app)
      .get('/dist')
-      .expect('Location', 'http://dist.u.qiniudn.com/')
-      .expect(302, done);
+      .expect(302)
+      .expect('Location', '/dist/', done);
+    });
+
+    it('should GET /dist/ show file list', function (done) {
+      request(app)
+      .get('/dist/')
+      .expect('Content-Type', 'text/html; charset=utf-8')
+      .expect(200, function (err, res) {
+        should.not.exist(err);
+        res.text.should.containEql('<title>Mirror index of http://nodejs.org/dist/</title>');
+        done();
+      });
+    });
+
+    it('should mock return files and dirs', function (done) {
+      mm(Dist, 'listdir', function* () {
+        return [
+          {name: 'ooxx/', date: '02-May-2014 00:54'},
+          {name: 'foo.txt', size: 1024, date: '02-May-2014 00:54'},
+        ];
+      });
+      request(app)
+      .get('/dist/v1.0.0/')
+      .expect('Content-Type', 'text/html; charset=utf-8')
+      .expect(200, function (err, res) {
+        should.not.exist(err);
+        res.text.should.containEql('<title>Mirror index of http://nodejs.org/dist/v1.0.0/</title>');
+        res.text.should.containEql('<h1>Mirror index of <a target="_blank" href="http://nodejs.org/dist/v1.0.0/">http://nodejs.org/dist/v1.0.0/</a></h1>');
+        res.text.should.containEql('<a href="ooxx/">ooxx/</a>                                             02-May-2014 00:54                   -\n');
+        res.text.should.containEql('<a href="foo.txt">foo.txt</a>                                           02-May-2014 00:54                1024\n');
+        done();
+      });
+    });
+
+    it('should list files and dirs', function (done) {
+      mm(Dist, 'listdir', function* () {
+        return [
+          {name: 'npm/', date: '02-May-2014 00:54'},
+          {name: 'npm-versions.txt', size: 1676, date: '02-May-2014 00:54'},
+        ];
+      });
+      request(app)
+      .get('/dist/')
+      .expect('Content-Type', 'text/html; charset=utf-8')
+      .expect(200, function (err, res) {
+        should.not.exist(err);
+        res.text.should.containEql('<title>Mirror index of http://nodejs.org/dist/</title>');
+        res.text.should.containEql('<h1>Mirror index of <a target="_blank" href="http://nodejs.org/dist/">http://nodejs.org/dist/</a></h1>');
+        res.text.should.containEql('<a href="npm/">npm/</a>                                              02-May-2014 00:54                   -\n');
+        res.text.should.containEql('<a href="npm-versions.txt">npm-versions.txt</a>                                  02-May-2014 00:54                1676\n');
+        done();
+      });
+    });
+  });
+
+  describe('GET /dist/ files', function () {
+    it('should redirect to nfs url', function (done) {
+      mm(Dist, 'getfile', function* () {
+        return {
+          name: 'foo.txt', size: 1024, date: '02-May-2014 00:54',
+          url: 'http://mock.com/dist/v0.10.28/SHASUMS.txt'
+        };
+      });
+
+      request(app)
+      .get('/dist/v0.10.28/SHASUMS.txt')
+      .expect(302)
+      .expect('Location', 'http://mock.com/dist/v0.10.28/SHASUMS.txt', done);
+    });
+
+    it('should GET /dist/npm-versions.txt redirect to nfs url', function (done) {
+      mm(Dist, 'getfile', function* (fullname) {
+        fullname.should.equal('/npm-versions.txt');
+        return {
+          name: 'npm-versions.txt', size: 1024, date: '02-May-2014 00:54',
+          url: 'http://mock.com/dist/npm-versions.txt'
+        };
+      });
+
+      request(app)
+      .get('/dist/npm-versions.txt')
+      .expect(302)
+      .expect('Location', 'http://mock.com/dist/npm-versions.txt', done);
+    });
+
+    it('should download nfs file and send it', function (done) {
+      mm(Dist, 'getfile', function* () {
+        return {
+          name: 'foo.txt',
+          size: 1264,
+          date: '02-May-2014 00:54',
+          url: '/dist/v0.10.28/SHASUMS.txt'
+        };
+      });
+
+      request(app)
+      .get('/dist/v0.10.28/SHASUMS.txt')
+      .expect(200)
+      .expect(/6eff580cc8460741155d42ef1ef537961194443f/, done);
    });
  });
 });
--- a/test/controllers/web/package.test.js
+++ b/test/controllers/web/package.test.js
@@ -51,7 +51,7 @@ describe('controllers/web/package.test.js', function () {
    it('should search with "m"', function (done) {
      request(app)
      .get('/_list/search/search?startkey="m"&limit=2')
-      .expect('content-type', 'application/json')
+      .expect('content-type', 'application/json; charset=utf-8')
      .expect(200, function (err, res) {
        should.not.exist(err);
        res.body.should.have.keys('rows');
@@ -95,14 +95,14 @@ describe('controllers/web/package.test.js', function () {
      request(app)
      .get('/package/mk2testmodule')
      .expect(200)
-      .expect('content-encoding', 'gzip')
+      // .expect('content-encoding', 'gzip')
      .expect('content-type', 'text/html; charset=utf-8')
      .expect(/<div id="package">/)
      .expect(/<th>Maintainers<\/th>/)
      .expect(/<th>Version<\/th>/, function (err, res) {
        should.not.exist(err);
        res.should.have.header('etag');
-        res.text.should.include('<meta charset="utf-8">');
+        res.text.should.containEql('<meta charset="utf-8">');
        done();
      });
    });
--- a/test/middleware/static.test.js
+++ b/test/middleware/static.test.js
@@ -0,0 +1,60 @@
+/**!
+ * cnpmjs.org - test/middleware/static.test.js
+ *
+ * Copyright(c) cnpmjs.org and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *  fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var should = require('should');
+var request = require('supertest');
+var registry = require('../../servers/registry');
+var web = require('../../servers/registry');
+
+describe('middleware/static.test.js', function () {
+  before(function (done) {
+    registry = registry.listen(0, function () {
+      web = web.listen(0, done);
+    });
+  });
+
+  describe('registry', function () {
+    it('should /favicon.ico rewrite to /favicon.png', function (done) {
+      request(registry)
+      .get('/favicon.ico')
+      // .expect('content-type', 'image/png')
+      .expect(200, done);
+    });
+
+    it('should 200 /favicon.png', function (done) {
+      request(registry)
+      .get('/favicon.png')
+      .expect('content-type', 'image/png')
+      .expect(200, done);
+    });
+  });
+
+  describe('web', function () {
+    it('should /favicon.ico rewrite to /favicon.png', function (done) {
+      request(registry)
+      .get('/favicon.ico')
+      // .expect('content-type', 'image/png')
+      .expect(200, done);
+    });
+
+    it('should 200 /favicon.png', function (done) {
+      request(registry)
+      .get('/favicon.png')
+      .expect('content-type', 'image/png')
+      .expect(200, done);
+    });
+  });
+});
--- a/test/proxy/dist.test.js
+++ b/test/proxy/dist.test.js
@@ -0,0 +1,52 @@
+/**!
+ * cnpmjs.org - test/proxy/dist.test.js
+ *
+ * Copyright(c) cnpmjs.org and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *  fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var should = require('should');
+var Dist = require('../../proxy/dist');
+
+describe('proxy/dist.test.js', function () {
+  describe('savefile() and getfile', function () {
+    it('should save and get /npm-versions.txt', function* () {
+      var info = {
+        name: 'npm-versions.txt',
+        parent: '/',
+        date: '15-Sep-2011 23:48',
+        size: 1676,
+        url: 'http://cnpmjs.org/dist/npm-versions.txt',
+        sha1: '104731881047318810473188'
+      };
+      yield* Dist.savefile(info);
+      var got = yield* Dist.getfile('/npm-versions.txt');
+      should.exist(got);
+      got.should.eql(info);
+    });
+
+    it('should save and get /v1.0.0/npm-versions.txt', function* () {
+      var info = {
+        name: 'npm-versions.txt',
+        parent: '/v1.0.0/',
+        date: '15-Sep-2011 23:48',
+        size: 1676,
+        url: 'http://cnpmjs.org/dist/v1.0.0/npm-versions.txt',
+        sha1: '104731881047318810473188'
+      };
+      yield* Dist.savefile(info);
+      var got = yield* Dist.getfile('/v1.0.0/npm-versions.txt');
+      should.exist(got);
+      got.should.eql(info);
+    });
+  });
+});
--- a/test/sync/sync_all.test.js
+++ b/test/sync/sync_all.test.js
@@ -25,13 +25,14 @@ describe('sync/sync_all.test.js', function () {
    afterEach(mm.restore);

    it('should sync first time ok', function *() {
-      mm.data(Npm, 'getShort', ['mk2testmodule']);
+      mm.data(Npm, 'getShort', ['mk2testmodule', 'mk2testmodule-not-exists']);
      mm.data(Total, 'getTotalInfo', {last_sync_time: 0});
      var data = yield sync;
-      data.successes.should.eql(['mk2testmodule']);
+      data.successes.should.eql(['mk2testmodule', 'mk2testmodule-not-exists']);
      mm.restore();
      var result = yield Total.getTotalInfo();
-      result.last_sync_module.should.equal('mk2testmodule');
+      should.exist(result);
+      result.last_sync_module.should.equal('mk2testmodule-not-exists');
    });

    it('should sync common ok', function *() {
--- a/test/sync/sync_dist.test.js
+++ b/test/sync/sync_dist.test.js
@@ -0,0 +1,129 @@
+/**!
+ * cnpmjs.org - test/sync/sync_dist.test.js
+ *
+ * Copyright(c) fengmk2 and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *   fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var should = require('should');
+var mm = require('mm');
+var urllib = require('co-urllib');
+var Dist = require('../../proxy/dist');
+var distsync = require('../../sync/sync_dist');
+
+describe('sync/sync_dist.test.js', function () {
+  afterEach(mm.restore);
+
+  describe('listPhantomjsDir()', function () {
+    it('should list all phantomjs download infos', function* () {
+      var items = yield* distsync.listPhantomjsDir('/phantomjs');
+      items.length.should.above(1);
+      items.forEach(function (item) {
+        item.should.have.keys('name', 'date', 'size', 'type', 'parent', 'downloadURL');
+      });
+    });
+  });
+
+  describe('listdiff()', function () {
+    it('should got all news', function* () {
+      mm(urllib, 'request', function* () {
+        return {
+          status: 200,
+          data: '<a href="npm/">npm/</a>   06-May-2014 01:18     -\n<a href="npm-versions.txt">npm-versions.txt</a>  27-Feb-2014 00:01         1676',
+          headers: {},
+        };
+      });
+
+      mm(Dist, 'listdir', function* () {
+        return [];
+      });
+
+      var items = yield* distsync.listdiff('/');
+      items.should.eql([
+        { name: 'npm/',
+          date: '06-May-2014 01:18',
+          size: '-',
+          type: 'dir',
+          parent: '/' },
+        { name: 'npm-versions.txt',
+          date: '27-Feb-2014 00:01',
+          size: 1676,
+          type: 'file',
+          parent: '/' }
+      ]);
+    });
+
+    it('should got empty when all exists', function* () {
+      mm(urllib, 'request', function* () {
+        return {
+          status: 200,
+          data: '<a href="npm/">npm/</a>   06-May-2014 01:18     -\n<a href="npm-versions.txt">npm-versions.txt</a>  27-Feb-2014 00:01         1676',
+          headers: {},
+        };
+      });
+
+      mm(Dist, 'listdir', function* () {
+        return [
+          {
+            name: 'npm/',
+            date: '06-May-2014 01:18',
+            parent: '/'
+          },
+          {
+            name: 'npm-versions.txt',
+            date: '27-Feb-2014 00:01',
+            size: 1676,
+            parent: '/'
+          },
+        ];
+      });
+
+      var items = yield* distsync.listdiff('/');
+      items.should.length(0);
+    });
+
+    it('should got date change dir', function* () {
+      mm(urllib, 'request', function* () {
+        return {
+          status: 200,
+          data: '<a href="npm/">npm/</a>   06-May-2014 01:18     -\n<a href="npm-versions.txt">npm-versions.txt</a>  27-Feb-2014 00:01         1676',
+          headers: {},
+        };
+      });
+
+      mm(Dist, 'listdir', function* () {
+        return [
+          {
+            name: 'npm/',
+            date: '06-May-2014 01:17',
+            parent: '/'
+          },
+          {
+            name: 'npm-versions.txt',
+            date: '27-Feb-2014 00:01',
+            size: 1676,
+            parent: '/'
+          },
+        ];
+      });
+
+      var items = yield* distsync.listdiff('/');
+      items.should.eql([
+        { name: 'npm/',
+          date: '06-May-2014 01:18',
+          size: '-',
+          type: 'dir',
+          parent: '/' }
+      ]);
+    });
+  });
+});
--- a/test/sync_dist.js
+++ b/test/sync_dist.js
@@ -0,0 +1,25 @@
+/**!
+ * cnpmjs.org - test/sync_dist.js
+ *
+ * Copyright(c) fengmk2 and other contributors.
+ * MIT Licensed
+ *
+ * Authors:
+ *   fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
+ */
+
+'use strict';
+
+/**
+ * Module dependencies.
+ */
+
+var debug = require('debug');
+debug.enable('cnpmjs.org*');
+var co = require('co');
+var syncdist = require('../sync/sync_dist');
+
+co(function* () {
+  // yield* syncdist('/v0.10.28/');
+  yield* syncdist.syncPhantomjsDir();
+})();
--- a/test/zlib_write_after_close.js
+++ b/test/zlib_write_after_close.js
@@ -0,0 +1,9 @@
+var zlib = require('zlib');
+
+var stream = zlib.createGzip();
+
+stream.write('foo');
+stream.close();
+
+// Assertion failed: (!ctx->pending_close_ && "close is pending"), function Write, file ../src/node_zlib.cc, line 150.
+stream.write('again');
--- a/view/web/404.html
+++ b/view/web/404.html
@@ -0,0 +1,5 @@
+<div class="alert alert-warning">
+  Can not found package match <%= name %>. You can
+  <a href="/sync/<%= name %>" target="_blank">SYNC</a> from official npm registry or
+  <a href="https://npmjs.org/search?q=<%= name %>" target="_blank">SEARCH</a> in official npm website.
+</div>
--- a/view/web/dist.html
+++ b/view/web/dist.html
@@ -0,0 +1,9 @@
+<h1>Mirror index of <a target="_blank" href="<%= disturl %><%= dirname %>"><%= disturl %><%= dirname %></a></h1>
+<hr>
+<pre><a href="../">../</a><% for (var i = 0; i < items.length; i++) {
+  var item = items[i];
+  var sizestr = '' + (item.size || '-');
+%>
+<a href="<%= item.name %>"><%= item.name %></a><%- padding(50, item.name.length, " ") %><%= item.date %><%- padding(20, sizestr.length, " ") %><%= sizestr %><% } %>
+</pre>
+<hr>
--- a/view/web/layout.html
+++ b/view/web/layout.html
@@ -4,7 +4,7 @@
    <meta charset="utf-8">
    <title><%- locals.title %></title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
-    <link rel="shortcut icon" href="/public/favicon.png">
+    <link rel="shortcut icon" href="/favicon.png">
    <!-- Bootstrap -->
    <link href="http://cdn.staticfile.org/twitter-bootstrap/3.0.0-rc2/css/bootstrap.min.css" rel="stylesheet" media="screen">
    <link href="http://cdn.staticfile.org/prettify/r298/prettify.min.css" rel="stylesheet" media="screen">
--- a/view/web/package.html
+++ b/view/web/package.html
@@ -252,6 +252,7 @@
            <a class="downloadlink" target="_blank" href="<%= package.dist.tarball %>">
              <%= package.dist.tarball %>
            </a>
+            (<%- package.dist.size %>)
          </td>
        </tr>
        <% } %>
--- a/worker.js
+++ b/worker.js
@@ -21,12 +21,14 @@ var config = require('./config');
 var registry = require('./servers/registry');
 var web = require('./servers/web');

-registry.listen(config.registryPort);
-web.listen(config.webPort);
+registry.listen(config.registryPort, config.bindingHost);
+web.listen(config.webPort, config.bindingHost);

-console.log('[%s] [worker:%d] Server started, registry server listen at %d, web listen at %d, cluster: %s',
+console.log('[%s] [worker:%d] Server started, registry server listen at %s:%d, web listen at %s:%d, cluster: %s',
  new Date(), process.pid,
-  config.registryPort, config.webPort, config.enableCluster);
+  config.bindingHost, config.registryPort,
+  config.bindingHost, config.webPort,
+  config.enableCluster);

 graceful({
  server: [registry, web],
Author	SHA1	Message	Date
dead_horse	f11b4e9954	Release 0.5.3	2014-06-13 18:08:12 +08:00
dead_horse	6e775eed47	fix sync response 204	2014-06-13 18:05:09 +08:00
dead_horse	ca6841c042	add links in History.md	2014-06-08 00:59:54 +08:00
dead_horse	652b900fc4	bump koa	2014-06-08 00:59:22 +08:00
dead_horse	43379cc66d	fix test-cov	2014-06-04 17:37:19 +08:00
dead_horse	8f9b3db029	bump koa and should	2014-06-04 17:29:52 +08:00
fengmk2	884b75f6a3	Release 0.5.2	2014-06-04 15:41:01 +08:00
dead_horse	a958ee40c4	Merge pull request #350 from cnpm/hotfix sync hotfix	2014-06-04 15:39:05 +08:00
fengmk2	1a01909750	sync hotfix	2014-06-04 15:35:50 +08:00
dead_horse	6901a5c994	Merge pull request #349 from cnpm/dist-sync-phantomjs sync phantomjs downloads pkg. close #348	2014-06-04 15:12:31 +08:00
fengmk2	33b13dc30e	sync phantomjs downloads pkg. close #348	2014-06-04 14:35:20 +08:00
fengmk2	73d9d2c3d0	Merge pull request #347 from cnpm/issue346-restart-syncer add restart, fixed #346	2014-06-04 12:39:34 +08:00
dead_horse	8b955adb42	add restart, fixed #346	2014-06-04 10:57:50 +08:00
fengmk2	1f219cbd1b	Release 0.5.1	2014-05-28 09:18:15 +08:00
dead_horse	17025d42b4	Merge pull request #344 from cnpm/list-since-from-tag fix attack on /-/all/since?stale=update_after&startkey=2 close #336	2014-05-28 09:09:43 +08:00
fengmk2	922f26b052	fix attack on /-/all/since?stale=update_after&startkey=2 close #336 Also add modified_time key to tag	2014-05-28 00:19:26 +08:00
dead_horse	9371685188	bump thunkify-wrap	2014-05-26 21:44:51 +08:00
dead_horse	ad0b66c7e5	bump koa-middlewares	2014-05-25 01:03:19 +08:00
dead_horse	5588880ec0	remove outputError	2014-05-25 00:44:26 +08:00
dead_horse	9ad72cbdcb	bump dependencies	2014-05-25 00:43:16 +08:00
dead_horse	4d033471fc	use svg badge	2014-05-25 00:36:58 +08:00
fengmk2	590b7f7eee	Merge pull request #340 from cnpm/404-page add package/notfound page	2014-05-19 19:26:34 +08:00
dead_horse	0ce817f9f3	add package/notfound page	2014-05-19 19:16:15 +08:00
fengmk2	59af2bfd56	add dist mirror link to home page	2014-05-13 09:37:46 +08:00
fengmk2	a83f6bb183	fix sync listdiff and add more test cases	2014-05-13 09:28:54 +08:00
fengmk2	b6f1531743	Release 0.5.0	2014-05-13 08:32:51 +08:00
dead_horse	42d0b538ca	Merge pull request #337 from cnpm/dist-sync Dist sync	2014-05-12 23:48:52 +08:00
fengmk2	a240eb9922	filter /nightlies/*	2014-05-12 17:32:06 +08:00
fengmk2	6853b73fb2	use koa setter instead of set()	2014-05-12 17:09:51 +08:00
fengmk2	a887acec05	update co-urllib	2014-05-12 10:19:19 +08:00
fengmk2	1d3f0f0ad3	add more info on error email	2014-05-12 10:03:01 +08:00
fengmk2	20e04065df	add sync dist to sync/index.js	2014-05-12 07:38:48 +08:00
fengmk2	1cee298bc3	show dist page	2014-05-11 20:56:40 +08:00
fengmk2	854bc6c9d8	sync dist file and save it to database	2014-05-10 18:59:13 +08:00
fengmk2	b70c1c421a	disable gzip before #335 has fix	2014-05-04 20:05:55 +08:00
dead_horse	b1b6172892	Release 0.4.3	2014-04-18 16:15:50 +08:00
fengmk2	e5a77a4368	Merge pull request #334 from cnpm/fix-permission add permission check to /:name/:tag	2014-04-18 14:34:27 +08:00
dead_horse	b74cccd342	add permission check to /:name/:tag	2014-04-18 14:29:42 +08:00
fengmk2	6aae538f49	Merge pull request #333 from cnpm/issue332-tag add put /:name/:tag, close #332	2014-04-18 14:12:56 +08:00
dead_horse	74c8b25374	fix space	2014-04-18 14:07:49 +08:00
dead_horse	0cec2edea6	add put /:name/:tag, close #332	2014-04-18 12:18:07 +08:00
fengmk2	b59a0a99cf	Release 0.4.2	2014-04-17 17:33:17 +08:00
dead_horse	c7e82809e3	Merge pull request #330 from cnpm/package-size-show fix fav ico and show pkg size on pkg info page. fix #318	2014-04-17 17:31:06 +08:00
fengmk2	206ee505a8	sync interval config	2014-04-17 17:28:27 +08:00
fengmk2	d39838f930	fix fav ico and show pkg size on pkg info page. fix #318	2014-04-17 14:59:38 +08:00
dead_horse	512e21aaf5	Merge pull request #329 from cnpm/fix-stack-over sync work sync one done must wait for a defer.setImmediate. fix #328	2014-04-17 14:12:22 +08:00
fengmk2	f7904ee699	sync work sync one done must wait for a defer.setImmediate. fix #328	2014-04-17 09:11:46 +08:00
dead_horse	7007be6ef4	bump dep versions	2014-04-16 15:17:14 +08:00
dead_horse	70aa3299e5	Merge pull request #326 from cnpm/noattachment if download tarball 404, throw err better than ignore it. fixed #325	2014-04-16 12:00:29 +08:00
fengmk2	a04f5d68eb	if download tarball 404, throw err better than ignore it. fixed #325	2014-04-16 11:44:28 +08:00
fengmk2	7cfe3b58ce	refator sync	2014-04-16 11:19:32 +08:00
fengmk2	afdf0bc653	Merge pull request #323 from cnpm/hot-fix-status hotfix, close #319	2014-04-16 09:39:15 +08:00
fengmk2	42055ac91e	Merge pull request #324 from cnpm/hotfix-dist hotfix, close #321	2014-04-16 09:31:48 +08:00
dead_horse	b93e0dab41	hotfix, close #321	2014-04-16 09:28:22 +08:00
dead_horse	01f2187830	hotfix, close #319	2014-04-16 09:23:59 +08:00
fengmk2	c2f49fcdd9	support custom web home page	2014-04-14 15:14:28 +08:00
dead_horse	3925ef6044	Merge pull request #316 from cnpm/first-full-sync npm get short only can read from cnpm now	2014-04-14 14:04:59 +08:00
fengmk2	85494d2ba0	npm get short only can read from cnpm now	2014-04-14 13:57:04 +08:00
dead_horse	660ca394d6	Merge pull request #315 from cnpm/local-bindding Local bindding and redis fix	2014-04-14 10:47:39 +08:00
fengmk2	ad7eeac00c	if using reverted proxy like nginx, only binding on local host	2014-04-14 10:43:49 +08:00
fengmk2	f63b72891b	fix redis detect logic	2014-04-14 10:27:46 +08:00
fengmk2	28d31b093d	Release 0.4.1	2014-04-10 14:46:01 +08:00
fengmk2	466e14e92a	Merge pull request #313 from cnpm/hotfix-sync-status fix sync status code error	2014-04-10 07:46:57 +08:00
dead_horse	3646c2ce50	fix sync status code error	2014-04-10 02:39:02 +08:00