Compare commits

...

44 Commits
0.5.3 ... 0.8.2

Author SHA1 Message Date
fengmk2
c0295ecc77 Release 0.8.2 2014-07-22 01:00:38 +08:00
fengmk2
aaf7f04d08 fix default scope detect 2014-07-22 01:00:22 +08:00
fengmk2
beceac10dc Release 0.8.1 2014-07-21 23:54:45 +08:00
dead_horse
fad58273b6 Merge pull request #377 from cnpm/default-scope
support default @org. close #376
2014-07-21 17:16:21 +08:00
fengmk2
e0361dc1f2 add more test cases 2014-07-21 16:18:28 +08:00
fengmk2
d8c337acc5 support default @org. close #376 2014-07-21 16:13:43 +08:00
dead_horse
9d7683bd61 hotfix redis init error 2014-07-21 14:14:52 +08:00
fengmk2
994b2e47aa Release 0.8.0 2014-07-21 13:58:24 +08:00
dead_horse
b14faad3fb Merge pull request #375 from cnpm/scope-package
support scoped package
2014-07-21 13:56:13 +08:00
fengmk2
9163f793a3 update readme show scoped packages 2014-07-21 13:53:33 +08:00
fengmk2
1c0d4fdcdc support "scoped" packages. close #352 2014-07-21 13:51:43 +08:00
dead_horse
20fce9f55f Merge pull request #373 from cnpm/stop-old-publish-flow
Stop old publish flow
2014-07-19 01:21:22 +08:00
fengmk2
6b73a7762f use safe jsonp 2014-07-19 01:16:30 +08:00
fengmk2
fd34e0512c Stop support old publish flow. fix #368 2014-07-19 01:11:33 +08:00
fengmk2
a84715167e Merge pull request #372 from cnpm/sql
update SQLs
2014-07-13 07:07:59 +08:00
dead_horse
db1cb0d0f8 update SQLs 2014-07-12 23:20:55 +08:00
dead_horse
dc137f0821 bump redis 2014-07-12 14:57:23 +08:00
fengmk2
76d270fc77 Merge pull request #371 from cnpm/update-logger
use sync_info and sync_error categories
2014-07-11 12:06:28 +08:00
dead_horse
b5190b56e8 use sync_info and sync_error categories 2014-07-11 10:03:41 +08:00
fengmk2
d2d238db59 add categories to loggers. fix #370 2014-07-11 09:07:48 +08:00
fengmk2
4c02d84fec fix get latest tag always not exists bug 2014-07-11 01:04:52 +08:00
dead_horse
74585f0b8a Merge pull request #369 from cnpm/publish-with-tag
support `npm publish --tag beta`. fix #366
2014-07-10 23:47:48 +08:00
fengmk2
a6ac632e73 support npm publish --tag beta. fix #366 2014-07-10 23:34:25 +08:00
fengmk2
0b59547ac1 Merge pull request #365 from cnpm/logger
use mini-logger and error-formater
2014-07-09 11:30:48 +08:00
dead_horse
b5929c94e0 use mini-logger and error-formater 2014-07-09 11:23:20 +08:00
fengmk2
4c96682603 Release 0.7.0 2014-07-07 12:11:12 +08:00
dead_horse
78af0fc15b Merge pull request #364 from cnpm/maintainer
Maintainer logic improve
2014-07-07 11:42:53 +08:00
fengmk2
ce381618a0 use module_maintainers on GET /pakcage/:name page 2014-07-07 10:25:06 +08:00
fengmk2
87486c484f use new module_maintainers on GET /:name 2014-07-07 10:23:53 +08:00
fengmk2
4608712e05 admin user should never publish to other user's packages. fix #363
but admin user need have the permission to delete other's packages.
2014-07-07 01:41:03 +08:00
fengmk2
3c6576bcab Add a new table for module-maintainers.
* fix #362 update maintianers change the lastmodified. remove version and change last modified.
* #363 change isMaintainer detect logic.
2014-07-07 00:56:44 +08:00
fengmk2
ba1986b931 gravatar use https 2014-07-02 00:09:56 +08:00
fengmk2
5efa508376 support https 2014-07-01 23:26:48 +08:00
dead_horse
38617d4572 bump dependencies 2014-06-30 17:31:04 +08:00
fengmk2
a9a04b370d Release 0.6.1 2014-06-18 12:27:47 +08:00
fengmk2
ec1d924dc1 hot fix removeTagsByNames() 2014-06-17 10:41:41 +08:00
fengmk2
234d1ec4d6 fix _rev not exists 2014-06-16 15:52:56 +08:00
dead_horse
b8f4958d8f Merge pull request #357 from cnpm/fix-sync-request
sync unpublished on GET /sync/:name
2014-06-16 15:23:22 +08:00
fengmk2
e56a21e890 sync unpublished on GET /sync/:name 2014-06-16 15:21:53 +08:00
fengmk2
e3434a58c5 Release 0.6.0 2014-06-16 15:06:17 +08:00
dead_horse
cd9d38aadb Merge pull request #356 from cnpm/unpublished
sync unpublished info. close #353
2014-06-16 13:50:05 +08:00
fengmk2
0ff22ccc60 sync unpublished info. close #353 2014-06-16 10:54:02 +08:00
dead_horse
e4cfb13383 Merge pull request #354 from cnpm/delete-not-exists-versions
Delete not exists versions on sync worker. #353
2014-06-16 09:39:58 +08:00
fengmk2
23ad3a29b4 Delete not exists versions on sync worker. #353 2014-06-15 00:57:18 +08:00
59 changed files with 2215 additions and 906 deletions

View File

@@ -1,4 +1,52 @@
0.8.2 / 2014-07-22
==================
* fix default scope detect
0.8.1 / 2014-07-21
==================
* add more test cases
* support default @org. close #376
* hotfix redis init error
0.8.0 / 2014-07-21
==================
* support "scoped" packages. close #352
* use safe jsonp
* Stop support old publish flow. fix #368
* update SQLs
* use sync_info and sync_error categories
* add categories to loggers. fix #370
* fix get latest tag always not exists bug
* support `npm publish --tag beta`. fix #366
* use mini-logger and error-formater
0.7.0 / 2014-07-07
==================
* use module_maintainers on GET /pakcage/:name page
* use new module_maintainers on GET /:name
* admin user should never publish to other user's packages. fix #363
* Add a new table for module-maintainers.
* gravatar use https
* support https
0.6.1 / 2014-06-18
==================
* hot fix removeTagsByNames()
* fix _rev not exists
* sync unpublished on GET /sync/:name
0.6.0 / 2014-06-16
==================
* sync unpublished info. close #353
* Delete not exists versions on sync worker. #353
0.5.3 / 2014-06-13
==================

View File

@@ -1,12 +1,12 @@
TESTS = $(shell ls -S `find test -type f -name "*.test.js" -print`)
REPORTER = tap
REPORTER = spec
TIMEOUT = 30000
MOCHA_OPTS =
REGISTRY = --registry=http://registry.npm.taobao.org
REGISTRY = --registry=https://registry.npm.taobao.org
install:
@npm install $(REGISTRY) \
--disturl=http://dist.cnpmjs.org
--disturl=https://npm.taobao.org/dist
jshint: install
@-./node_modules/.bin/jshint ./
@@ -19,7 +19,7 @@ pretest:
test: install pretest
@NODE_ENV=test ./node_modules/.bin/mocha \
--harmony-generators \
--harmony \
--reporter $(REPORTER) \
--timeout $(TIMEOUT) \
--require should \

View File

@@ -22,6 +22,7 @@ Our goal is to provide a low cost maintenance and easy to use solution for priva
### Features
* **Support "scoped" packages**: [npm/npm#5239](https://github.com/npm/npm/issues/5239)
* **Simple to deploy**: only need `mysql` and a [simple store system](https://github.com/cnpm/cnpmjs.org/wiki/NFS-Guide).
You can get the source code through `npm` or `git`.
* **Low cost and easy maintenance**: `package.json` info store in MySQL, tarball(tgz file) store in CDN or other store systems.
@@ -50,7 +51,7 @@ only need to change the registry in config. Even include manual synchronization
### Dependencies
* [node](http://nodejs.org) >=0.11.9
* [node](http://nodejs.org) =0.11.12
* [mysql](http://dev.mysql.com/downloads/) >= 0.5.0, include `mysqld` and `mysql cli`. I test on `mysql@5.6.16`.
### Start MySQL

View File

@@ -7,7 +7,7 @@ export NODE_ENV='production'
ulimit -c unlimited
cd `dirname $0`/..
NODEJS='node --harmony-generators'
NODEJS='node --harmony'
BASE_HOME=`pwd`
PROJECT_NAME=`basename ${BASE_HOME}`
STDOUT_LOG=`$NODEJS -e "console.log(require('path').join(require('$BASE_HOME/config').logdir, 'nodejs_stdout.log'));\

View File

@@ -15,73 +15,48 @@
* Module dependencies.
*/
var util = require('util');
var moment = require('moment');
var logstream = require('logfilestream');
var ms = require('ms');
var formater = require('error-formater');
var Logger = require('mini-logger');
var config = require('../config');
var utility = require('utility');
var mail = require('./mail');
var util = require('util');
var isTEST = process.env.NODE_ENV === 'test';
var ONE_DAY = ms('1d');
var levels = ['info', 'warn', 'error'];
var categories = ['sync_info', 'sync_error'];
levels.forEach(function (catetory) {
var options = {
logdir: config.logdir,
duration: ONE_DAY,
nameformat: '[' + catetory + '.]YYYY-MM-DD[.log]'
};
var stream = logstream(options);
function write(msg) {
var time = moment().format('YYYY-MM-DD HH:mm:ss.SSS');
var subject = null;
if (msg instanceof Error) {
subject = msg.name;
var err = {
name: msg.name,
code: msg.code,
message: msg.message,
stack: msg.stack,
host: msg.host,
url: msg.url,
data: msg.data
};
if (err.name === 'Error' && typeof err.code === 'string') {
err.name = err.code + err.name;
}
err.name += 'Exception';
if (err.host) {
err.message += ' (' + err.host + ')';
}
msg = util.format('%s nodejs.%s: %s\nURL: %s\nData: %j\n%s\n\n',
time,
err.name,
err.stack,
err.url,
err.data,
time
);
} else {
msg = time + ' ' + util.format.apply(util, arguments) + '\n';
}
if (!isTEST) {
stream.write(msg);
if (config.debug) {
var level = catetory;
console.log('[' + level + '] ' + msg);
} else {
if (catetory === 'error' && subject) {
// send error email
var to = [];
for (var name in config.admins) {
to.push(config.admins[name]);
}
mail.error(to, subject, msg);
}
}
}
}
exports[catetory] = write;
var logger = module.exports = Logger({
categories: categories,
dir: config.logdir,
duration: '1d',
format: '[{category}.]YYYY-MM-DD[.log]',
stdout: config.debug && !isTEST,
errorFormater: errorFormater
});
var to = [];
for (var name in config.admins) {
to.push(config.admins[name]);
}
function errorFormater(err) {
var msg = formater.both(err);
mail.error(to, msg.json.name, msg.text);
return msg.text;
}
logger.syncInfo = function () {
var args = [].slice.call(arguments);
if (typeof args[0] === 'string') {
args[0] = util.format('[%s][%s] ', utility.logDate(), process.pid) + args[0];
}
logger.sync_info.apply(logger, args);
};
logger.syncError =function () {
var args = [].slice.call(arguments);
if (typeof args[0] === 'string') {
args[0] = util.format('[%s][%s] ', utility.logDate(), process.pid) + args[0];
}
logger.sync_error.apply(logger, arguments);
};

View File

@@ -22,8 +22,7 @@ if (config.redis && config.redis.host && config.redis.port) {
var redis = require('redis');
var wrapper = require('co-redis');
var logger = require('./logger');
var _client = redis.createClient(config.redis);
var _client = redis.createClient(config.redis.port, config.redis.host);
_client.on('error', function (err) {
logger.error(err);

View File

@@ -80,9 +80,10 @@ var config = {
debug: false
},
noticeSyncDistError: true,
disturl: 'http://nodejs.org/dist',
syncDist: false,
logoURL: 'http://ww4.sinaimg.cn/large/69c1d4acgw1ebfly5kjlij208202oglr.jpg',
logoURL: '//ww4.sinaimg.cn/large/69c1d4acgw1ebfly5kjlij208202oglr.jpg',
registryHost: 'r.cnpmjs.org',
// customReadmeFile: __dirname + '/web_readme.md',
customReadmeFile: '', // you can use your custom readme file instead the cnpm one
@@ -92,6 +93,7 @@ var config = {
sourceNpmRegistry: 'http://registry.npmjs.org',
enablePrivate: true, // enable private mode, only admin can publish, other use just can sync package from source npm
admins: {
// name: email
fengmk2: 'fengmk2@gmail.com',
admin: 'admin@cnpmjs.org',
dead_horse: 'dead_horse@qq.com',
@@ -114,6 +116,7 @@ var config = {
message: 'request frequency limited, any question, please contact fengmk2@gmail.com',
},
enableCompress: false, // enable gzip response or not
defaultScope: '', // default scope name
};
// load config/config.js, everything in config.js will cover the same key in index.js

View File

@@ -37,16 +37,48 @@ var SyncModuleWorker = require('../../proxy/sync_module_worker');
var logger = require('../../common/logger');
var ModuleDeps = require('../../proxy/module_deps');
var ModuleStar = require('../../proxy/module_star');
var ModuleUnpublished = require('../../proxy/module_unpublished');
var packageService = require('../../services/package');
var downloadAsReadStream = require('../utils').downloadAsReadStream;
/**
* show all version of a module
* GET /:name
*/
exports.show = function *(next) {
var name = this.params.name;
var modifiedTime = yield *Module.getLastModified(name);
debug('show %s, last modified: %s', name, modifiedTime);
exports.show = function* (next) {
var orginalName = this.params.name || this.params[0];
var name = orginalName;
var rs = yield [
Module.getLastModified(name),
Module.listTags(name)
];
var modifiedTime = rs[0];
var tags = rs[1];
var adaptDefaultScope = false;
if (tags.length === 0 && config.defaultScope && name.indexOf(config.defaultScope + '/') === 0) {
// remove default scope name and retry
name = name.split('/')[1];
rs = yield [
Module.getLastModified(name),
Module.listTags(name)
];
modifiedTime = rs[0];
tags = rs[1];
adaptDefaultScope = true;
}
debug('show %s(%s), last modified: %s, tags: %j', name, orginalName, modifiedTime, tags);
if (modifiedTime) {
// find out the latest modfied time
// because update tags only modfied tag, wont change module gmt_modified
for (var i = 0; i < tags.length; i++) {
var tag = tags[i];
if (tag.gmt_modified > modifiedTime) {
modifiedTime = tag.gmt_modified;
}
}
// use modifiedTime as etag
this.set('ETag', '"' + modifiedTime.getTime() + '"');
@@ -60,22 +92,46 @@ exports.show = function *(next) {
}
var r = yield [
Module.listTags(name),
Module.listByName(name),
ModuleStar.listUsers(name)
ModuleStar.listUsers(name),
packageService.listMaintainers(name),
];
var tags = r[0];
var rows = r[1];
var users = r[2];
var rows = r[0];
var users = r[1];
var maintainers = r[2];
debug('show %s got %d rows, %d tags, %d star users, maintainers: %j',
name, rows.length, tags.length, users.length, maintainers);
var userMap = {};
for (var i = 0; i < users.length; i++) {
userMap[users[i]] = true;
}
users = userMap;
if (rows.length === 0) {
// check if unpublished
var unpublishedInfo = yield* ModuleUnpublished.get(name);
debug('show unpublished %j', unpublishedInfo);
if (unpublishedInfo) {
this.status = 404;
this.body = {
_id: orginalName,
name: orginalName,
time: {
modified: unpublishedInfo.package.time,
unpublished: unpublishedInfo.package,
},
_attachments: {}
};
return;
}
}
// if module not exist in this registry,
// sync the module backend and return package info from official registry
if (rows.length === 0) {
if (!this.allowSync) {
if (!this.allowSync || adaptDefaultScope) {
this.status = 404;
this.body = {
error: 'not_found',
@@ -120,10 +176,20 @@ exports.show = function *(next) {
readme = pkg.readme;
}
delete pkg.readme;
if (maintainers.length > 0) {
// TODO: need to use newer maintainers
pkg.maintainers = maintainers;
}
if (!createdTime || t < createdTime) {
createdTime = t;
}
if (adaptDefaultScope) {
// change to orginal name for default scope was removed above
pkg.name = orginalName;
pkg._id = orginalName + '@' + pkg.version;
}
}
if (modifiedTime && createdTime) {
@@ -152,10 +218,16 @@ exports.show = function *(next) {
var pkg = latestMod.package;
if (tags.length === 0 && pkg.version !== 'next') {
// some sync error reason, will cause tags missing
// set latest tag at least
distTags.latest = pkg.version;
}
var info = {
_id: name,
_id: orginalName,
_rev: rev,
name: name,
name: orginalName,
description: pkg.description,
"dist-tags": distTags,
maintainers: pkg.maintainers,
@@ -173,24 +245,36 @@ exports.show = function *(next) {
info.bugs = pkg.bugs;
info.license = pkg.license;
debug('show module %s: %s, latest: %s', name, rev, latestMod.version);
debug('show module %s: %s, latest: %s', orginalName, rev, latestMod.version);
this.body = info;
};
/**
* get the special version or tag of a module
*
* GET /:name/:version
* GET /:name/:tag
*/
exports.get = function *(next) {
var name = this.params.name;
var tag = this.params.version;
exports.get = function* (next) {
var name = this.params.name || this.params[0];
var tag = this.params.version || this.params[1];
var version = semver.valid(tag);
var method = version ? 'get' : 'getByTag';
var queryLabel = version ? version : tag;
debug('%s %s with %j', method, name, this.params);
var mod = yield Module[method](name, queryLabel);
var rs = yield [
Module[method](name, queryLabel),
packageService.listMaintainers(name),
];
var mod = rs[0];
if (mod) {
common.setDownloadURL(mod.package, this);
mod.package._cnpm_publish_time = mod.publish_time;
var maintainers = rs[1];
if (maintainers.length > 0) {
mod.package.maintainers = maintainers;
}
this.body = mod.package;
return;
}
@@ -311,104 +395,6 @@ setInterval(function () {
next();
}, 5000);
exports.upload = function *(next) {
var length = Number(this.get('content-length')) || 0;
if (!length || !this.is('application/octet-stream')) {
debug('request length or type error');
return yield *next;
}
var username = this.user.name;
var name = this.params.name;
var id = Number(this.params.rev);
var filename = this.params.filename;
var version = filename.substring(name.length + 1);
version = version.replace(/\.tgz$/, '');
// save version on pkg upload
debug('%s: upload %s, file size: %d', username, this.url, length);
var mod = yield Module.getById(id);
if (!mod) {
debug('can not get this module');
return yield* next;
}
if (!common.isMaintainer(this.user, mod.package.maintainers) || mod.name !== name) {
this.status = 403;
this.body = {
error: 'no_perms',
reason: 'Current user can not publish this module'
};
return;
}
if (mod.version !== 'next') {
// rev wrong
this.status = 403;
this.body = {
error: 'rev_wrong',
reason: 'rev not match next module'
};
return;
}
var filepath = common.getTarballFilepath(filename);
var ws = fs.createWriteStream(filepath);
var shasum = crypto.createHash('sha1');
var dataSize = 0;
var buf;
while (buf = yield coRead(this.req)) {
shasum.update(buf);
dataSize += buf.length;
yield coWrite(ws, buf);
}
ws.end();
if (dataSize !== length) {
this.status = 403;
this.body = {
error: 'size_wrong',
reason: 'Header size ' + length + ' not match download size ' + dataSize,
};
return;
}
shasum = shasum.digest('hex');
var options = {
key: common.getCDNKey(name, filename),
size: length,
shasum: shasum
};
var result;
try {
result = yield nfs.upload(filepath, options);
} catch (err) {
fs.unlink(filepath, utility.noop);
this.throw(err);
}
fs.unlink(filepath, utility.noop);
var dist = {
shasum: shasum,
size: length
};
// if nfs upload return a key, record it
if (result.url) {
dist.tarball = result.url;
} else if (result.key) {
dist.key = result.key;
dist.tarball = result.key;
}
mod.package.dist = dist;
mod.package.version = version;
debug('%s module: save file to %s, size: %d, sha1: %s, dist: %j, version: %s',
id, filepath, length, shasum, dist, version);
var updateResult = yield Module.update(mod);
this.status = 201;
this.body = {
ok: true,
rev: String(updateResult.id)
};
};
function _addDepsRelations(pkg) {
var dependencies = Object.keys(pkg.dependencies || {});
if (dependencies.length > config.maxDependencies) {
@@ -421,77 +407,13 @@ function _addDepsRelations(pkg) {
});
}
exports.updateLatest = function *(next) {
var username = this.user.name;
var name = this.params.name;
var version = semver.valid(this.params.version);
if (!version) {
this.status = 400;
this.body = {
error: 'Params Invalid',
reason: 'Invalid version: ' + this.params.version,
};
return;
}
var nextMod = yield Module.get(name, 'next');
if (!nextMod) {
debug('can not get nextMod');
return yield* next;
}
if (!common.isMaintainer(this.user, nextMod.package.maintainers)) {
this.status = 401;
this.body = {
error: 'noperms',
reason: 'Current user can not publish this module'
};
return;
}
// check version if not match pkg upload
if (nextMod.package.version !== version) {
this.status = 403;
this.body = {
error: 'version_wrong',
reason: 'version not match'
};
return;
}
var body = this.request.body;
nextMod.version = version;
nextMod.author = username;
body.dist = nextMod.package.dist;
body.maintainers = nextMod.package.maintainers;
if (!body.author) {
body.author = {
name: username,
};
}
body._publish_on_cnpm = true;
nextMod.package = body;
_addDepsRelations(body);
// reset publish time
nextMod.publish_time = Date.now();
debug('update %s:%s %j', nextMod.package.name, nextMod.package.version, nextMod.package.dist);
// change latest to version
try {
yield Module.update(nextMod);
} catch (err) {
debug('update nextMod %s error: %s', name, err);
return this.throw(err);
}
yield Module.addTag(name, 'latest', version);
nextMod.version = 'next';
var addResult = yield Module.add(nextMod);
this.status = 201;
this.body = {
ok: true,
rev: String(addResult.id)
};
};
// old flows:
// 1. add()
// 2. upload()
// 3. updateLatest()
//
// new flows: only one request
// PUT /:name
exports.addPackageAndDist = function *(next) {
// 'dist-tags': { latest: '0.0.2' },
// _attachments:
@@ -523,16 +445,36 @@ exports.addPackageAndDist = function *(next) {
tags.push([t, distTags[t]]);
}
debug('addPackageAndDist %s:%s, attachment size: %s', name, version, attachment.length);
if (tags.length === 0) {
this.status = 400;
this.body = {
error: 'invalid',
reason: 'dist-tags should not be empty'
};
return;
}
debug('%s addPackageAndDist %s:%s, attachment size: %s, maintainers: %j, distTags: %j',
username, name, version, attachment.length, versionPackage.maintainers, distTags);
var exists = yield Module.get(name, version);
var shasum;
if (exists) {
this.status = 409;
this.status = 403;
this.body = {
error: 'conflict',
reason: 'Document update conflict.'
error: 'forbidden',
reason: 'cannot modify pre-existing version: ' + version
};
return;
}
// check maintainers
var isMaintainer = yield* packageService.isMaintainer(name, username);
if (!isMaintainer) {
this.status = 403;
this.body = {
error: 'forbidden user',
reason: username + ' not authorized to modify ' + name
};
return;
}
@@ -545,11 +487,22 @@ exports.addPackageAndDist = function *(next) {
this.status = 403;
this.body = {
error: 'size_wrong',
reason: 'Attachment size ' + attachment.length + ' not match download size ' + tarballBuffer.length,
reason: 'Attachment size ' + attachment.length
+ ' not match download size ' + tarballBuffer.length,
};
return;
}
if (!distTags.latest) {
// need to check if latest tag exists or not
var latest = yield Module.getByTag(name, 'latest');
if (!latest) {
// auto add latest
tags.push(['latest', tags[0][1]]);
debug('auto add latest tag: %j', tags);
}
}
shasum = crypto.createHash('sha1');
shasum.update(tarballBuffer);
shasum = shasum.digest('hex');
@@ -601,141 +554,81 @@ exports.addPackageAndDist = function *(next) {
};
};
exports.add = function *(next) {
var username = this.user.name;
var name = this.params.name;
var pkg = this.request.body || {};
if (!common.isMaintainer(this.user, pkg.maintainers)) {
this.status = 403;
this.body = {
error: 'no_perms',
reason: 'Current user can not publish this module'
};
return;
}
if (pkg._attachments && Object.keys(pkg._attachments).length > 0) {
return yield exports.addPackageAndDist.call(this, next);
}
var r = yield [Module.getLatest(name), Module.get(name, 'next')];
var latestMod = r[0];
var nextMod = r[1];
if (nextMod) {
nextMod.exists = true;
} else {
nextMod = {
name: name,
version: 'next',
author: username,
package: {
name: name,
version: 'next',
description: pkg.description,
readme: pkg.readme,
maintainers: pkg.maintainers,
}
};
debug('add next module: %s', name);
var result = yield Module.add(nextMod);
nextMod.id = result.id;
}
var maintainers = latestMod && latestMod.package.maintainers.length > 0 ?
latestMod.package.maintainers : nextMod.package.maintainers;
if (!common.isMaintainer(this.user, maintainers)) {
this.status = 403;
this.body = {
error: 'no_perms',
reason: 'Current user can not publish this module'
};
return;
}
debug('add %s rev: %s, version: %s', name, nextMod.id, nextMod.version);
if (latestMod || nextMod.version !== 'next') {
this.status = 409;
this.body = {
error: 'conflict',
reason: 'Document update conflict.'
};
return;
}
this.status = 201;
this.body = {
ok: true,
id: name,
rev: String(nextMod.id),
};
};
exports.updateOrRemove = function *(next) {
debug('updateOrRemove module %s, %j', this.params.name, this.request.body);
// PUT /:name/-rev/:rev
exports.updateOrRemove = function* (next) {
debug('updateOrRemove module %s, %s, %j', this.url, this.params.name, this.request.body);
var body = this.request.body;
if (body.versions) {
yield *exports.removeWithVersions.call(this, next);
} else if (body.maintainers && body.maintainers.length > 0) {
yield *exports.updateMaintainers.call(this, next);
yield* exports.removeWithVersions.call(this, next);
} else if (body.maintainers) {
yield* exports.updateMaintainers.call(this, next);
} else {
yield *next;
yield* next;
}
};
exports.updateMaintainers = function *(next) {
exports.updateMaintainers = function* (next) {
var name = this.params.name;
var body = this.request.body;
debug('updateMaintainers module %s, %j', name, body);
var latestMod = yield Module.getLatest(name);
var isMaintainer = yield* packageService.isMaintainer(name, this.user.name);
if (!latestMod || !latestMod.package) {
return yield *next;
}
if (!common.isMaintainer(this.user, latestMod.package.maintainers)) {
if (!isMaintainer && !this.user.isAdmin) {
this.status = 403;
this.body = {
error: 'no_perms',
reason: 'Current user can not publish this module'
error: 'forbidden user',
reason: this.user.name + ' not authorized to modify ' + name
};
return;
}
var r = yield *Module.updateMaintainers(latestMod.id, body.maintainers);
var usernames = body.maintainers.map(function (user) {
return user.name;
});
if (usernames.length === 0) {
this.status = 403;
this.body = {
error: 'invalid operation',
reason: 'Can not remove all maintainers'
};
return;
}
var r = yield *packageService.updateMaintainers(name, usernames);
debug('result: %j', r);
this.status = 201;
this.body = {
ok: true,
id: name,
rev: String(latestMod.id),
rev: this.params.rev,
};
};
exports.removeWithVersions = function *(next) {
debug('removeWithVersions module %s, with info %j', this.params.name, this.request.body);
exports.removeWithVersions = function* (next) {
var username = this.user.name;
var name = this.params.name;
// left versions
var versions = this.request.body.versions || {};
debug('removeWithVersions module %s, with versions %j', name, Object.keys(versions));
// step1: list all the versions
var mods = yield Module.listByName(name);
debug('removeWithVersions module %s, left versions %j, %s mods',
name, Object.keys(versions), mods && mods.length);
if (!mods || !mods.length) {
return yield *next;
return yield* next;
}
// step2: check permission
var firstMod = mods[0];
if (!common.isMaintainer(this.user, firstMod.package.maintainers) || firstMod.name !== name) {
var isMaintainer = yield* packageService.isMaintainer(name, username);
// admin can delete the module
if (!isMaintainer && !this.user.isAdmin) {
this.status = 403;
this.body = {
error: 'no_perms',
reason: 'Current user can not update this module'
error: 'forbidden user',
reason: username + ' not authorized to modify ' + name
};
return;
}
@@ -795,27 +688,36 @@ exports.removeWithVersions = function *(next) {
} else {
debug('no tag need to be remove');
}
// step 7: update last modified, make sure etag change
yield* Module.updateLastModified(name);
this.status = 201;
this.body = { ok: true };
};
exports.removeTar = function *(next) {
exports.removeTar = function* (next) {
debug('remove tarball with filename: %s, id: %s', this.params.filename, this.params.rev);
var id = Number(this.params.rev);
var filename = this.params.filename;
var name = this.params.name;
var username = this.user.name;
var mod = yield Module.getById(id);
if (!mod) {
if (isNaN(id)) {
return yield* next;
}
if (!common.isMaintainer(this.user, mod.package.maintainers) || mod.name !== name) {
var mod = yield Module.getById(id);
if (!mod || mod.name !== name) {
return yield* next;
}
var isMaintainer = yield* packageService.isMaintainer(name, username);
if (!isMaintainer && !this.user.isAdmin) {
this.status = 403;
this.body = {
error: 'no_perms',
reason: 'Current user can not delete this tarball'
error: 'forbidden user',
reason: username + ' not authorized to modify ' + name
};
return;
}
@@ -825,10 +727,10 @@ exports.removeTar = function *(next) {
this.body = { ok: true };
};
exports.removeAll = function *(next) {
exports.removeAll = function* (next) {
debug('remove all the module with name: %s, id: %s', this.params.name, this.params.rev);
// var id = Number(this.params.rev);
var name = this.params.name;
var username = this.user.name;
var mods = yield Module.listByName(name);
debug('removeAll module %s: %d', name, mods.length);
@@ -837,11 +739,13 @@ exports.removeAll = function *(next) {
return yield* next;
}
if (!common.isMaintainer(this.user, mod.package.maintainers) || mod.name !== name) {
var isMaintainer = yield* packageService.isMaintainer(name, username);
// admin can delete the module
if (!isMaintainer && !this.user.isAdmin) {
this.status = 403;
this.body = {
error: 'no_perms',
reason: 'Current user can not delete this tarball'
error: 'forbidden user',
reason: username + ' not authorized to modify ' + name
};
return;
}
@@ -849,16 +753,24 @@ exports.removeAll = function *(next) {
yield [Module.removeByName(name), Module.removeTags(name)];
var keys = [];
for (var i = 0; i < mods.length; i++) {
var key = urlparse(mods[i].dist_tarball).path;
var row = mods[i];
var dist = row.package.dist;
var key = dist.key;
if (!key) {
key = urlparse(dist.tarball).pathname;
}
key && keys.push(key);
}
try {
yield keys.map(function (key) {
return nfs.remove(key);
});
} catch (err) {
// ignore error here
if (keys.length > 0) {
try {
yield keys.map(function (key) {
return nfs.remove(key);
});
} catch (err) {
// ignore error here
}
}
this.body = { ok: true };
};
@@ -924,10 +836,12 @@ exports.listAllModuleNames = function *() {
});
};
exports.updateTag = function *() {
// PUT /:name/:tag
exports.updateTag = function* () {
var version = this.request.body;
var tag = this.params.tag;
var name = this.params.name;
debug('updateTag: %s %s to %s', name, version, tag);
if (!version) {
this.status = 400;
@@ -962,11 +876,12 @@ exports.updateTag = function *() {
}
// check permission
if (!common.isMaintainer(this.user, mod.package.maintainers)) {
var isMaintainer = yield* packageService.isMaintainer(name, this.user.name);
if (!isMaintainer) {
this.status = 403;
this.body = {
error: 'forbidden',
reason: 'no permission to modify ' + name
error: 'forbidden user',
reason: this.user.name + ' not authorized to modify ' + name
};
return;
}

View File

@@ -1,5 +1,5 @@
/**!
* cnpmjs.org - controllers/download.js
* cnpmjs.org - controllers/sync.js
*
* Copyright(c) cnpmjs.org and other contributors.
* MIT Licensed
@@ -14,14 +14,16 @@
* Module dependencies.
*/
var debug = require('debug')('cnpmjs.org:controllers:sync');
var Log = require('../proxy/module_log');
var SyncModuleWorker = require('../proxy/sync_module_worker');
exports.sync = function *() {
exports.sync = function* () {
var username = this.user.name || 'anonymous';
var name = this.params.name;
var name = this.params.name || this.params[0];
var publish = this.query.publish === 'true';
var noDep = this.query.nodeps === 'true';
debug('sync %s with query: %j', name, this.query);
if (publish && !this.user.isAdmin) {
this.status = 403;
this.body = {
@@ -37,6 +39,7 @@ exports.sync = function *() {
};
var result = yield SyncModuleWorker.sync(name, username, options);
debug('sync %s got %j', name, result);
// friendly 404 reason info
if (result.statusCode === 404) {
@@ -59,8 +62,9 @@ exports.sync = function *() {
};
};
exports.getSyncLog = function *(next) {
var logId = this.params.id;
exports.getSyncLog = function* (next) {
// params: [$name, $id] on scope package
var logId = this.params.id || this.params[1];
var offset = Number(this.query.offset) || 0;
var row = yield Log.get(logId);
if (!row) {

View File

@@ -14,6 +14,7 @@
* Module dependencies.
*/
var debug = require('debug')('cnpmjs.org:controllers:web:package');
var bytes = require('bytes');
var giturl = require('giturl');
var moment = require('moment');
@@ -30,11 +31,16 @@ var Log = require('../../proxy/module_log');
var ModuleDeps = require('../../proxy/module_deps');
var setDownloadURL = require('../../lib/common').setDownloadURL;
var ModuleStar = require('../../proxy/module_star');
var packageService = require('../../services/package');
exports.display = function *(next) {
exports.display = function* (next) {
var params = this.params;
var name = params.name;
var tag = params.version;
// normal: {name: $name, version: $version}
// scope: [$name, $version]
var orginalName = params.name || params[0];
var name = orginalName;
var tag = params.version || params[1];
debug('display %s with %j', name, params);
var getPackageMethod;
var getPackageArgs;
@@ -46,23 +52,30 @@ exports.display = function *(next) {
getPackageMethod = 'getByTag';
getPackageArgs = [name, tag || 'latest'];
}
var r = yield [
Module[getPackageMethod].apply(Module, getPackageArgs),
down.total(name),
ModuleDeps.list(name),
ModuleStar.listUsers(name),
];
var pkg = r[0];
var download = r[1];
var dependents = (r[2] || []).map(function (item) {
return item.deps;
});
var users = r[3];
var pkg = yield Module[getPackageMethod].apply(Module, getPackageArgs);
if (!pkg && config.defaultScope && name.indexOf(config.defaultScope + '/') === 0) {
name = name.split('/')[1];
pkg = yield Module[getPackageMethod].apply(Module, [name, getPackageArgs[1]]);
}
if (!pkg || !pkg.package) {
return yield* next;
}
var r = yield [
down.total(name),
ModuleDeps.list(name),
ModuleStar.listUsers(name),
packageService.listMaintainers(name)
];
var download = r[0];
var dependents = (r[1] || []).map(function (item) {
return item.deps;
});
var users = r[2];
var maintainers = r[3];
pkg.package.fromNow = moment(pkg.publish_time).fromNow();
pkg = pkg.package;
pkg.users = users;
@@ -71,11 +84,15 @@ exports.display = function *(next) {
pkg.readme = pkg.description || '';
}
if (maintainers.length > 0) {
pkg.maintainers = maintainers;
}
if (pkg.maintainers) {
for (var i = 0; i < pkg.maintainers.length; i++) {
var maintainer = pkg.maintainers[i];
if (maintainer.email) {
maintainer.gravatar = gravatar.url(maintainer.email, {s: '50', d: 'retro'}, false);
maintainer.gravatar = gravatar.url(maintainer.email, {s: '50', d: 'retro'}, true);
}
}
}
@@ -88,7 +105,7 @@ exports.display = function *(next) {
for (var i = 0; i < pkg.contributors.length; i++) {
var contributor = pkg.contributors[i];
if (contributor.email) {
contributor.gravatar = gravatar.url(contributor.email, {s: '50', d: 'retro'}, false);
contributor.gravatar = gravatar.url(contributor.email, {s: '50', d: 'retro'}, true);
}
if (config.packagePageContributorSearch || !contributor.url) {
contributor.url = '/~' + encodeURIComponent(contributor.name);
@@ -113,6 +130,10 @@ exports.display = function *(next) {
pkg.dist.size = bytes(pkg.dist.size || 0);
}
if (pkg.name !== orginalName) {
pkg.name = orginalName;
}
yield this.render('package', {
title: 'Package - ' + pkg.name,
package: pkg,
@@ -122,7 +143,8 @@ exports.display = function *(next) {
exports.search = function *(next) {
var params = this.params;
var word = params.word;
var word = params.word || params[0];
debug('search %j', word);
var result = yield Module.search(word);
var match = null;
@@ -185,8 +207,8 @@ exports.rangeSearch = function *(next) {
};
};
exports.displaySync = function *(next) {
var name = this.params.name || this.query.name;
exports.displaySync = function* (next) {
var name = this.params.name || this.params[0] || this.query.name;
yield this.render('sync', {
name: name,
title: 'Sync - ' + name

View File

@@ -40,6 +40,16 @@ CREATE TABLE `module_star` (
KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='module star';
CREATE TABLE `module_maintainer` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime NOT NULL COMMENT 'create time',
`user` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'user name',
`name` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'module name',
PRIMARY KEY (`id`),
UNIQUE KEY `user_module_name` (`user`,`name`),
KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='module maintainers';
CREATE TABLE `module` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime NOT NULL COMMENT 'create time',
@@ -94,6 +104,17 @@ CREATE TABLE `tag` (
-- ALTER TABLE `tag` CHANGE `name` `name` VARCHAR( 100 ) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'module name';
-- ALTER TABLE `tag` ADD KEY `gmt_modified` (`gmt_modified`);
CREATE TABLE `module_unpublished` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime NOT NULL COMMENT 'create time',
`gmt_modified` datetime NOT NULL COMMENT 'modified time',
`name` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'module name',
`package` longtext CHARACTER SET utf8 COLLATE utf8_general_ci COMMENT 'base info: tags, time, maintainers, description, versions',
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`),
KEY `gmt_modified` (`gmt_modified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='module unpublished info';
CREATE TABLE `total` (
`name` varchar(100) NOT NULL COMMENT 'total name',
`gmt_modified` datetime NOT NULL COMMENT 'modified time',

View File

@@ -1,27 +1,9 @@
-- http://nodejs.org/dist/ mirror
CREATE TABLE `dist_dir` (
CREATE TABLE `module_maintainer` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime NOT NULL COMMENT 'create time',
`gmt_modified` datetime NOT NULL COMMENT 'modified time',
`name` varchar(200) NOT NULL COMMENT 'user name',
`parent` varchar(200) NOT NULL COMMENT 'parent dir' DEFAULT '/',
`date` varchar(20) COMMENT '02-May-2014 01:06',
`user` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'user name',
`name` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'module name',
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`parent`, `name`),
KEY `gmt_modified` (`gmt_modified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='dist dir info';
CREATE TABLE `dist_file` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime NOT NULL COMMENT 'create time',
`gmt_modified` datetime NOT NULL COMMENT 'modified time',
`name` varchar(100) NOT NULL COMMENT 'user name',
`parent` varchar(200) NOT NULL COMMENT 'parent dir' DEFAULT '/',
`date` varchar(20) COMMENT '02-May-2014 01:06',
`size` int(10) unsigned NOT NULL COMMENT 'file size' DEFAULT '0',
`sha1` varchar(40) COMMENT 'sha1 hex value',
`url` varchar(2048),
PRIMARY KEY (`id`),
UNIQUE KEY `fullname` (`parent`, `name`),
KEY `gmt_modified` (`gmt_modified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='dist file info';
UNIQUE KEY `user_module_name` (`user`,`name`),
KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='module maintainers';

View File

@@ -20,6 +20,10 @@ module.exports = function *notFound(next) {
if (this.status && this.status !== 404) {
return;
}
if (this.body && this.body.name) {
return;
}
this.status = 404;
this.body = {
error: 'not_found',

View File

@@ -20,21 +20,22 @@ var config = require('../config');
* this.allowSync - allow sync triggle by cnpm install
*/
module.exports = function *syncByInstall(next) {
module.exports = function* syncByInstall(next) {
if (!config.syncByInstall || !config.enablePrivate) {
// only config.enablePrivate should enable sync on install
return yield *next;
return yield* next;
}
// request not by node, consider it request from web
var ua = this.get('user-agent');
if (!ua || ua.indexOf('node') < 0) {
return yield *next;
return yield* next;
}
// if request with `/xxx?write=true`, meaning the read request using for write
if (this.query.write) {
return yield *next;
return yield* next;
}
this.allowSync = true;
yield *next;
yield* next;
};

View File

@@ -22,8 +22,15 @@ module.exports = function *notFound(next) {
if (this.status && this.status !== 404) {
return;
}
if (this.body) {
return;
}
var m = /^\/([\w\-\_\.]+)\/?$/.exec(this.url);
var m = /^\/([\w\-\.]+)\/?$/.exec(this.path);
if (!m) {
// scoped packages
m = /^\/(@[\w\-\.]+\/[\w\-\.]+)$/.exec(this.path);
}
debug('%s match %j', this.url, m);
if (m) {
return this.redirect('/package/' + m[1]);

View File

@@ -1,6 +1,6 @@
{
"name": "cnpmjs.org",
"version": "0.5.3",
"version": "0.8.2",
"description": "Private npm registry and web for Enterprise, base on MySQL and Simple Store Service",
"main": "index.js",
"scripts": {
@@ -15,36 +15,37 @@
"co": "3.0.6",
"co-defer": "0.1.0",
"co-gather": "0.0.1",
"co-read": "0.0.2",
"co-read": "0.1.0",
"co-redis": "1.1.0",
"co-urllib": "0.2.3",
"co-write": "0.3.0",
"copy-to": "1.0.1",
"debug": "1.0.2",
"debug": "1.0.4",
"error-formater": "1.0.3",
"eventproxy": "0.3.1",
"giturl": "0.0.3",
"graceful": "0.1.0",
"gravatar": "1.0.6",
"humanize-number": "0.0.2",
"koa": "0.8.0",
"koa": "0.8.1",
"koa-limit": "1.0.2",
"koa-markdown": "0.0.3",
"koa-middlewares": "0.2.1",
"logfilestream": "0.1.0",
"koa-middlewares": "1.2.0",
"marked": "0.3.2",
"mime": "1.2.11",
"mini-logger": "0.3.0",
"mkdirp": "0.5.0",
"moment": "2.7.0",
"ms": "0.6.2",
"multiline": "0.3.4",
"mysql": "2.3.2",
"nodemailer": "0.6.5",
"mysql": "2.4.1",
"nodemailer": "0.7.1",
"qn": "0.2.2",
"ready": "0.1.1",
"redis": "0.10.3",
"semver": "2.3.0",
"thunkify-wrap": "0.1.2",
"utility": "0.1.13"
"redis": "0.11.0",
"semver": "2.3.1",
"thunkify-wrap": "1.0.1",
"utility": "0.1.16"
},
"devDependencies": {
"autod": "~0.2.0",
@@ -56,7 +57,7 @@
"jshint": "*",
"mm": "0.2.1",
"mocha": "*",
"pedding": "0.0.3",
"pedding": "1.0.0",
"should": "4.0.4",
"should-http": "0.0.1",
"supertest": "0.13.0"

View File

@@ -21,9 +21,6 @@ var config = require('../config');
var mysql = require('../common/mysql');
var multiline = require('multiline');
var MODULE_COLUMNS = 'id, publish_time, gmt_create, gmt_modified, author, name, \
version, description, package, dist_tarball, dist_shasum, dist_size';
var INSERT_MODULE_SQL = multiline(function () {;/*
INSERT INTO
module(gmt_create, gmt_modified, publish_time, author, name, version,
@@ -184,19 +181,7 @@ exports.updateReadme = function (id, readme, callback) {
});
};
var UPDATE_DIST_SQL = multiline(function () {;/*
UPDATE
module
SET
publish_time=?,
version=?,
package=?,
dist_tarball=?,
dist_shasum=?,
dist_size=?
WHERE
id=?;
*/});
var UPDATE_DIST_SQL = 'UPDATE module SET ? WHERE id=?';
exports.update = function (mod, callback) {
var pkg;
try {
@@ -205,8 +190,18 @@ exports.update = function (mod, callback) {
return callback(e);
}
var dist = mod.package.dist;
var arg = {
publish_time: mod.publish_time,
version: mod.version,
package: pkg,
dist_tarball: dist.tarball,
dist_shasum: dist.shasum,
dist_size: dist.size
};
mysql.query(UPDATE_DIST_SQL,
[mod.publish_time, mod.version, pkg, dist.tarball, dist.shasum, dist.size, mod.id],
[arg, mod.id],
function (err, result) {
if (err) {
return callback(err);
@@ -350,7 +345,7 @@ var DELETE_TAGS_BY_IDS_SQL = multiline(function () {;/*
DELETE FROM
tag
WHERE
id in (?);
id IN (?);
*/});
exports.removeTagsByIds = function (ids, callback) {
mysql.query(DELETE_TAGS_BY_IDS_SQL, [ids], callback);
@@ -653,13 +648,6 @@ exports.search = function (word, options, callback) {
thunkify(exports);
exports.updateMaintainers = function *(id, maintainers) {
var mod = yield exports.getById(id);
mod.package.maintainers = maintainers;
var pkg = stringifyPackage(mod.package);
return yield mysql.query(UPDATE_PACKAGE_SQL, [pkg, id]);
};
var GET_LAST_MODIFIED_MODULE_SQL = multiline(function () {;/*
SELECT
id, gmt_modified
@@ -668,9 +656,23 @@ var GET_LAST_MODIFIED_MODULE_SQL = multiline(function () {;/*
WHERE
name=?
ORDER BY
gmt_modified DESC;
gmt_modified DESC
LIMIT 1;
*/});
exports.getLastModified = function *(name) {
exports.getLastModified = function* (name) {
var row = yield mysql.queryOne(GET_LAST_MODIFIED_MODULE_SQL, [name]);
return row && row.gmt_modified;
};
var UPDATE_LAST_MODIFIED_SQL = 'UPDATE module SET gmt_modified=now() WHERE id=?;';
exports.updateLastModified = function* (name) {
var row = yield mysql.queryOne(GET_LAST_MODIFIED_MODULE_SQL, [name]);
if (row) {
yield mysql.query(UPDATE_LAST_MODIFIED_SQL, [row.id]);
}
};
var DELETE_TAGS_BY_NAMES_SQL = 'DELETE FROM tag WHERE name=? AND tag IN (?);';
exports.removeTagsByNames = function* (moduleName, tagNames) {
return yield mysql.query(DELETE_TAGS_BY_NAMES_SQL, [moduleName, tagNames]);
};

View File

@@ -16,16 +16,18 @@
var thunkify = require('thunkify-wrap');
var mysql = require('../common/mysql');
var multiline = require('multiline');
var INSERT_LOG_SQL = multiline(function () {;/*
INSERT INTO
module_log(gmt_create, gmt_modified, name, username, log)
VALUES
(now(), now(), ?, ?, "");
*/});
var INSERT_LOG_SQL = 'INSERT INTO module_log SET ?';
exports.create = function (data, callback) {
mysql.query(INSERT_LOG_SQL, [data.name, data.username], function (err, result) {
var now = new Date();
var args = {
gmt_create: now,
gmt_modified: now,
name: data.name,
username: data.username,
log: ''
}
mysql.query(INSERT_LOG_SQL, [args], function (err, result) {
if (err) {
return callback(err);
}
@@ -33,15 +35,7 @@ exports.create = function (data, callback) {
});
};
var APPEND_SQL = multiline(function () {;/*
UPDATE
module_log
SET
log=CONCAT(log, ?),
gmt_modified=now()
WHERE
id=?;
*/});
var APPEND_SQL = 'UPDATE module_log SET log=CONCAT(log, ?), gmt_modified=now() WHERE id=?';
exports.append = function (id, log, callback) {
log = '\n' + log;
mysql.query(APPEND_SQL, [log, id], function (err) {
@@ -49,14 +43,7 @@ exports.append = function (id, log, callback) {
});
};
var SELECT_SQL = multiline(function () {;/*
SELECT
*
FROM
module_log
WHERE
id=?;
*/});
var SELECT_SQL = 'SELECT * FROM module_log WHERE id=?';
exports.get = function (id, callback) {
mysql.queryOne(SELECT_SQL, [id], callback);
};

View File

@@ -0,0 +1,81 @@
/**!
* cnpmjs.org - proxy/module_maintainer.js
*
* Copyright(c) fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
'use strict';
/**
* Module dependencies.
*/
var debug = require('debug')('cnpmjs.org:proxy:module_maintainer');
var mysql = require('../common/mysql');
var Module = require('./module');
var GET_MAINTANINERS_SQL = 'SELECT user FROM module_maintainer WHERE name = ?;';
exports.get = function* (name) {
var users = yield mysql.query(GET_MAINTANINERS_SQL, [name]);
return users.map(function (row) {
return row.user;
});
};
var ADD_SQL = 'INSERT INTO module_maintainer(name, user, gmt_create) \
VALUES (?, ?, now());';
function* add(name, username) {
try {
yield mysql.query(ADD_SQL, [name, username]);
} catch (err) {
if (err.code !== 'ER_DUP_ENTRY') {
throw err;
}
}
}
var REMOVE_SQL = 'DELETE FROM module_maintainer WHERE name = ? AND user IN (?);';
function* remove(name, usernames) {
return yield mysql.query(REMOVE_SQL, [name, usernames]);
}
exports.update = function* (name, maintainers) {
// maintainers should be [name1, name2, ...] format
// find out the exists maintainers then remove the deletes and add the left
if (maintainers.length === 0) {
return {
add: [],
remove: []
};
}
var exists = yield* exports.get(name);
var addUsers = maintainers;
var removeUsers = [];
if (exists.length > 0) {
for (var i = 0; i < exists.length; i++) {
var username = exists[i];
if (addUsers.indexOf(username) === -1) {
removeUsers.push(username);
}
}
}
var tasks = [];
for (var i = 0; i < addUsers.length; i++) {
tasks.push(add(name, addUsers[i]));
}
yield tasks;
// make sure all add users success then remove users
if (removeUsers.length > 0) {
yield* remove(name, removeUsers);
}
debug('add %d users, remove %d users', addUsers.length, removeUsers.length);
return {
add: addUsers,
remove: removeUsers
};
};

View File

@@ -18,10 +18,10 @@ var mysql = require('../common/mysql');
var multiline = require('multiline');
var ADD_SQL = multiline(function () {;/*
INSERT iNTO
module_star(name, user)
INSERT INTO
module_star(name, user, gmt_create)
VALUES
(?, ?);
(?, ?, now());
*/});
exports.add = function *add(name, user) {
try {

View File

@@ -0,0 +1,43 @@
/**!
* cnpmjs.org - proxy/module_unpublished.js
*
* Copyright(c) fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
'use strict';
/**
* Module dependencies.
*/
var multiline = require('multiline');
var mysql = require('../common/mysql');
var SAVE_SQL = multiline(function () {;/*
INSERT INTO
module_unpublished(gmt_create, gmt_modified, name, package)
VALUES
(now(), now(), ?, ?)
ON DUPLICATE KEY UPDATE
gmt_modified=now(),
name=VALUES(name),
package=VALUES(package);
*/});
exports.add = function* (name, pkg) {
return yield mysql.query(SAVE_SQL, [name, JSON.stringify(pkg)]);
};
var GET_SQL = 'SELECT gmt_modified, name, package FROM module_unpublished WHERE name=?;';
exports.get = function* (name) {
var row = yield mysql.queryOne(GET_SQL, [name]);
if (row) {
row.package = JSON.parse(row.package);
}
return row;
};

View File

@@ -19,7 +19,7 @@ var config = require('../config');
var USER_AGENT = 'cnpmjs.org/' + config.version + ' ' + urllib.USER_AGENT;
function *request(url, options) {
function* request(url, options) {
options = options || {};
options.dataType = options.dataType || 'json';
options.timeout = options.timeout || 120000;
@@ -30,7 +30,7 @@ function *request(url, options) {
url = registry + url;
var r;
try {
r = yield *urllib.request(url, options);
r = yield* urllib.request(url, options);
} catch (err) {
var statusCode = err.status || -1;
var data = err.data || '[empty]';
@@ -43,6 +43,8 @@ function *request(url, options) {
return r;
}
exports.request = request;
exports.getUser = function *(name) {
var url = '/-/user/org.couchdb.user:' + name;
var r = yield *request(url);

View File

@@ -28,6 +28,7 @@ var crypto = require('crypto');
var urllib = require('co-urllib');
var utility = require('utility');
var ms = require('ms');
var urlparse = require('url').parse;
var nfs = require('../common/nfs');
var npm = require('./npm');
var common = require('../lib/common');
@@ -37,6 +38,7 @@ var Log = require('./module_log');
var config = require('../config');
var ModuleStar = require('./module_star');
var User = require('./user');
var ModuleUnpublished = require('./module_unpublished');
var USER_AGENT = 'sync.cnpmjs.org/' + config.version + ' ' + urllib.USER_AGENT;
@@ -121,7 +123,7 @@ SyncModuleWorker.prototype.add = function (name) {
this.log(' add dependencies: %s', name);
};
SyncModuleWorker.prototype._doneOne = function *(concurrencyId, name, success) {
SyncModuleWorker.prototype._doneOne = function* (concurrencyId, name, success) {
if (success) {
this.pushSuccess(name);
} else {
@@ -144,39 +146,66 @@ SyncModuleWorker.prototype.next = function *(concurrencyId) {
var that = this;
that.syncingNames[name] = true;
var pkg;
var pkg = null;
var status = 0;
// get from npm
try {
pkg = yield npm.get(name);
var result = yield npm.request('/' + name);
pkg = result.data;
status = result.status;
} catch (err) {
// if 404
if (!err.res || err.res.statusCode !== 404) {
var errMessage = err.name + ': ' + err.message;
that.log('[c#%s] [error] [%s] get package error: %s', concurrencyId, name, errMessage);
that.log('[c#%s] [error] [%s] get package error: %s, status: %s',
concurrencyId, name, errMessage, status);
yield *that._doneOne(concurrencyId, name, false);
return;
}
}
var unpublishedInfo = null;
if (status === 404) {
// check if it's unpublished
if (pkg.time && pkg.time.unpublished && pkg.time.unpublished.time) {
unpublishedInfo = pkg.time.unpublished;
} else {
pkg = null;
}
}
if (!pkg) {
that.log('[c#%s] [error] [%s] get package error: package not exists', concurrencyId, name);
yield *that._doneOne(concurrencyId, name, true);
that.log('[c#%s] [error] [%s] get package error: package not exists, status: %s',
concurrencyId, name, status);
yield* that._doneOne(concurrencyId, name, true);
return;
}
that.log('[c#%d] [%s] start...', concurrencyId, name);
that.log('[c#%d] [%s] pkg status: %d, start...', concurrencyId, name, status);
if (unpublishedInfo) {
try {
yield* that._unpublished(name, unpublishedInfo);
} catch (err) {
that.log('[c#%s] [error] [%s] sync error: %s', concurrencyId, name, err.stack);
yield* that._doneOne(concurrencyId, name, false);
return;
}
return yield* that._doneOne(concurrencyId, name, true);
}
var versions;
try {
versions = yield that._sync(name, pkg);
versions = yield* that._sync(name, pkg);
} catch (err) {
that.log('[c#%s] [error] [%s] sync error: %s', concurrencyId, name, err.stack);
yield *that._doneOne(concurrencyId, name, false);
yield* that._doneOne(concurrencyId, name, false);
return;
}
that.log('[c#%d] [%s] synced success, %d versions: %s',
concurrencyId, name, versions.length, versions.join(', '));
yield *that._doneOne(concurrencyId, name, true);
yield* that._doneOne(concurrencyId, name, true);
};
function *_listStarUsers(modName) {
@@ -200,7 +229,61 @@ function *_saveNpmUser(username) {
yield User.saveNpmUser(user);
}
SyncModuleWorker.prototype._sync = function *(name, pkg) {
SyncModuleWorker.prototype._unpublished = function* (name, unpublishedInfo) {
var mods = yield Module.listByName(name);
this.log(' [%s] start unpublished %d versions from local cnpm registry',
name, mods.length);
if (this._isLocalModule(mods)) {
// publish on cnpm, dont sync this version package
this.log(' [%s] publish on local cnpm registry, don\'t sync', name);
return [];
}
var r = yield* ModuleUnpublished.add(name, unpublishedInfo);
this.log(' [%s] save unpublished info: %j to row#%s',
name, unpublishedInfo, r.insertId);
if (mods.length === 0) {
return;
}
yield [Module.removeByName(name), Module.removeTags(name)];
var keys = [];
for (var i = 0; i < mods.length; i++) {
var row = mods[i];
var dist = row.package.dist;
var key = dist.key;
if (!key) {
key = urlparse(dist.tarball).pathname;
}
key && keys.push(key);
}
if (keys.length === 0) {
return;
}
try {
yield keys.map(function (key) {
return nfs.remove(key);
});
} catch (err) {
// ignore error here
this.log(' [%s] delete nfs files: %j error: %s: %s',
name, keys, err.name, err.message);
}
this.log(' [%s] delete nfs files: %j success', name, keys);
};
SyncModuleWorker.prototype._isLocalModule = function (mods) {
for (var i = 0; i < mods.length; i++) {
var r = mods[i];
if (r.package && r.package._publish_on_cnpm) {
return true;
}
}
return false;
};
SyncModuleWorker.prototype._sync = function* (name, pkg) {
var username = this.username;
var that = this;
@@ -214,8 +297,15 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
var tagRows = result[1];
var existsStarUsers = result[2];
if (that._isLocalModule(moduleRows)) {
// publish on cnpm, dont sync this version package
that.log(' [%s] publish on local cnpm registry, don\'t sync', name);
return [];
}
hasModules = moduleRows.length > 0;
var map = {};
var localVersionNames = [];
for (var i = 0; i < moduleRows.length; i++) {
var r = moduleRows[i];
if (!r.package || !r.package.dist) {
@@ -223,12 +313,6 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
continue;
}
if (r.package && r.package._publish_on_cnpm) {
// publish on cnpm, dont sync this version package
that.log(' [%s] publish on local cnpm, don\'t sync', name);
return [];
}
if (r.version === 'next') {
continue;
}
@@ -236,6 +320,7 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
map.latest = r;
}
map[r.version] = r;
localVersionNames.push(r.version);
}
var tags = {};
@@ -285,11 +370,8 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
var times = pkg.time || {};
pkg.versions = pkg.versions || {};
var versionNames = Object.keys(times);
if (versionNames.length === 0) {
versionNames = Object.keys(pkg.versions);
}
if (versionNames.length === 0) {
var remoteVersionNames = Object.keys(pkg.versions);
if (remoteVersionNames.length === 0) {
that.log(' [%s] no times and no versions, hasModules: %s', name, hasModules);
if (!hasModules) {
// save a next module
@@ -323,9 +405,12 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
}
}
var versions = [];
for (var i = 0; i < versionNames.length; i++) {
var v = versionNames[i];
var remoteVersionNameMap = {};
// find out missing versions
for (var i = 0; i < remoteVersionNames.length; i++) {
var v = remoteVersionNames[i];
remoteVersionNameMap[v] = v;
var exists = map[v] || {};
var version = pkg.versions[v];
if (!version || !version.dist || !version.dist.tarball) {
@@ -370,9 +455,19 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
continue;
}
}
versions.push(version);
missingVersions.push(version);
}
// find out deleted versions
var deletedVersionNames = [];
for (var i = 0; i < localVersionNames.length; i++) {
var v = localVersionNames[i];
if (!remoteVersionNameMap[v]) {
deletedVersionNames.push(v);
}
}
// find out missing tags
var sourceTags = pkg['dist-tags'] || {};
for (var t in sourceTags) {
var sourceTagVersion = sourceTags[t];
@@ -380,18 +475,25 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
missingTags.push([t, sourceTagVersion]);
}
}
if (versions.length === 0) {
that.log(' [%s] all versions are exists', name);
} else {
versions.sort(function (a, b) {
return a.publish_time - b.publish_time;
});
that.log(' [%s] %d versions need to sync', name, versions.length);
// find out deleted tags
var deletedTags = [];
for (var t in tags) {
if (!sourceTags[t]) {
// not in remote tags, delete it from local registry
deletedTags.push(t);
}
}
missingVersions = versions;
var versionNames = [];
if (missingVersions.length === 0) {
that.log(' [%s] all versions are exists', name);
} else {
missingVersions.sort(function (a, b) {
return a.publish_time - b.publish_time;
});
that.log(' [%s] %d versions need to sync', name, missingVersions.length);
}
var syncedVersionNames = [];
var syncIndex = 0;
// sync missing versions
@@ -403,13 +505,27 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
}
try {
var result = yield that._syncOneVersion(index, syncModule);
versionNames.push(syncModule.version);
syncedVersionNames.push(syncModule.version);
} catch (err) {
that.log(' [%s:%d] error, version: %s, %s: %s',
that.log(' [%s:%d] sync error, version: %s, %s: %s',
syncModule.name, index, syncModule.version, err.name, err.message);
}
}
if (deletedVersionNames.length === 0) {
that.log(' [%s] no versions need to deleted', name);
} else {
that.log(' [%s] %d versions: %j need to deleted',
name, deletedVersionNames.length, deletedVersionNames);
try {
yield Module.removeByNameAndVersions(name, deletedVersionNames);
} catch (err) {
that.log(' [%s] delete error, %s: %s', name, err.name, err.message);
}
}
// sync missing descriptions
function *syncDes() {
if (missingDescriptions.length === 0) {
@@ -434,7 +550,13 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
}
// sync missing tags
function *syncTag() {
function* syncTag() {
if (deletedTags.length > 0) {
yield* Module.removeTagsByNames(name, deletedTags);
that.log(' [%s] deleted %d tags: %j',
name, deletedTags.length, deletedTags);
}
if (missingTags.length === 0) {
return;
}
@@ -537,7 +659,7 @@ SyncModuleWorker.prototype._sync = function *(name, pkg) {
}
yield [syncDes(), syncTag(), syncReadme(), syncMissingStarUsers(), syncMissingUsers()];
return versionNames;
return syncedVersionNames;
};
SyncModuleWorker.prototype._syncOneVersion = function *(versionIndex, sourcePackage) {
@@ -705,16 +827,23 @@ SyncModuleWorker.prototype._syncOneVersion = function *(versionIndex, sourcePack
}
};
SyncModuleWorker.sync = function *(name, username, options) {
SyncModuleWorker.sync = function* (name, username, options) {
options = options || {};
var pkg = yield npm.get(name);
if (!pkg || !pkg._rev) {
var result = yield npm.request('/' + name);
var pkg = result.data;
if (result.status === 404 &&
(!pkg.time || !pkg.time.unpublished || !pkg.time.unpublished.time)) {
pkg = null;
}
if (!pkg || !pkg._id) {
return {
ok: false,
pkg: pkg,
statusCode: 404
};
}
var result = yield Log.create({name: name, username: username});
var worker = new SyncModuleWorker({
logId: result.id,

View File

@@ -149,22 +149,21 @@ var UPDATE_SYNC_NUM_SQL = multiline(function () {;/*
UPDATE
total
SET
sync_status = ?,
need_sync_num = ?,
success_sync_num = ?,
fail_sync_num = ?,
left_sync_num = ?,
last_sync_module = ?
?
WHERE
name="total";
*/});
exports.updateSyncNum = function (params, callback) {
var query = [
params.syncStatus, params.need || 0,
params.success || 0, params.fail || 0, params.left || 0,
params.lastSyncModule,
];
mysql.query(UPDATE_SYNC_NUM_SQL, query, callback);
var arg = {
sync_status: params.syncStatus,
need_sync_num: params.need || 0,
success_sync_num: params.success || 0,
fail_sync_num: params.fail || 0,
left_sync_num: params.left || 0,
last_sync_module: params.lastSyncModule
};
mysql.query(UPDATE_SYNC_NUM_SQL, [arg], callback);
};
thunkify(exports);

View File

@@ -66,13 +66,7 @@ exports.auth = function (name, password, callback) {
};
var INSERT_USER_SQL = multiline(function () {;/*
INSERT INTO
user(rev, name, email, salt, password_sha,
ip, roles, gmt_create, gmt_modified)
VALUES
(?, ?, ?, ?, ?, ?, ?, now(), now());
*/});
var INSERT_USER_SQL = 'INSERT INTO user SET ?';
exports.add = function (user, callback) {
var roles = user.roles || [];
try {
@@ -81,8 +75,22 @@ exports.add = function (user, callback) {
roles = '[]';
}
var rev = '1-' + utility.md5(JSON.stringify(user));
var values = [rev, user.name, user.email, user.salt, user.password_sha, user.ip, roles];
mysql.query(INSERT_USER_SQL, values, function (err) {
var now = new Date();
var arg = {
rev: rev,
name: user.name,
email: user.email,
salt: user.salt,
password_sha: user.password_sha,
ip: user.ip,
roles: roles,
gmt_create: now,
gmt_modified: now
};
mysql.query(INSERT_USER_SQL, [arg], function (err) {
callback(err, {rev: rev});
});
};
@@ -91,13 +99,7 @@ var UPDATE_USER_SQL = multiline(function () {;/*
UPDATE
user
SET
rev=?,
email=?,
salt=?,
password_sha=?,
ip=?,
roles=?,
gmt_modified=now()
?
WHERE
name=? AND rev=?;
*/});
@@ -119,8 +121,17 @@ exports.update = function (user, callback) {
roles = '[]';
}
var values = [newRev, user.email, user.salt, user.password_sha, user.ip, roles, user.name, rev];
mysql.query(UPDATE_USER_SQL, values, function (err, data) {
var arg = {
rev: newRev,
email: user.email,
salt: user.salt,
password_sha: user.password_sha,
ip: user.ip,
roles: roles,
gmt_modified: new Date()
};
mysql.query(UPDATE_USER_SQL, [arg, user.name, rev], function (err, data) {
if (err) {
return callback(err);
}

View File

@@ -15,7 +15,6 @@
* Module dependencies.
*/
var middlewares = require('koa-middlewares');
var limit = require('../middleware/limit');
var login = require('../middleware/login');
var publishable = require('../middleware/publishable');
@@ -26,7 +25,15 @@ var user = require('../controllers/registry/user');
var sync = require('../controllers/sync');
function routes(app) {
app.get('/', middlewares.jsonp(), total.show);
function* jsonp(next) {
yield* next;
if (this.body) {
this.jsonp = this.body;
}
}
app.get('/', jsonp, total.show);
//before /:name/:version
//get all modules, for npm search
@@ -36,10 +43,15 @@ function routes(app) {
app.get('/-/short', mod.listAllModuleNames);
// module
// scope package: params: [$name]
app.get(/\/(@[\w\-\.]+\/[\w\-\.]+)$/, syncByInstall, mod.show);
// scope package: params: [$name, $version]
app.get(/\/(@[\w\-\.]+\/[\w\-\.]+)\/([\w\.\-]+)$/, syncByInstall, mod.get);
app.get('/:name', syncByInstall, mod.show);
app.get('/:name/:version', syncByInstall, mod.get);
// try to add module
app.put('/:name', login, publishable, mod.add);
app.put('/:name', login, publishable, mod.addPackageAndDist);
// sync from source npm
app.put('/:name/sync', sync.sync);
@@ -50,15 +62,9 @@ function routes(app) {
// need limit by ip
app.get('/:name/download/:filename', limit, mod.download);
// put tarball
// https://registry.npmjs.org/cnpmjs.org/-/cnpmjs.org-0.0.0.tgz/-rev/1-c85bc65e8d2470cc4d82b8f40da65b8e
app.put('/:name/-/:filename/-rev/:rev', login, publishable, mod.upload);
// delete tarball
app.delete('/:name/download/:filename/-rev/:rev', login, publishable, mod.removeTar);
// put package.json to module
app.put('/:name/:version/-tag/latest', login, publishable, mod.updateLatest);
// update module, unpublish will PUT this
app.put('/:name/-rev/:rev', login, publishable, mod.updateOrRemove);
app.delete('/:name/-rev/:rev', login, publishable, mod.removeAll);

View File

@@ -23,15 +23,29 @@ var dist = require('../controllers/web/dist');
function routes(app) {
app.get('/total', total.show);
// scope package without version
app.get(/\/package\/(@[\w\-\.]+\/[\w\-\.]+)$/, pkg.display);
// scope package with version
app.get(/\/package\/(@[\w\-\.]+\/[\w\-\.]+)\/([\w\d\.]+)$/, pkg.display);
app.get('/package/:name', pkg.display);
app.get('/package/:name/:version', pkg.display);
app.get(/\/browse\/keyword\/(@[\w\-\.]+\/[\w\-\.]+)$/, pkg.search);
app.get('/browse/keyword/:word', pkg.search);
app.get('/~:name', user.display);
app.get(/\/sync\/(@[\w\-\.]+\/[\w\-\.]+)$/, pkg.displaySync);
app.get('/sync/:name', pkg.displaySync);
app.put(/\/sync\/(@[\w\-\.]+\/[\w\-\.]+)$/, sync.sync);
app.put('/sync/:name', sync.sync);
// params: [$name, $id]
app.get(/\/sync\/(@[\w\-\.]+\/[\w\-\.]+)\/log\/(\d+)$/, sync.getSyncLog);
app.get('/sync/:name/log/:id', sync.getSyncLog);
app.get('/sync', pkg.displaySync);
app.get('/_list/search/search', pkg.rangeSearch);

View File

@@ -27,6 +27,7 @@ var auth = require('../middleware/auth');
var staticCache = require('../middleware/static');
var notFound = require('../middleware/registry_not_found');
middlewares.jsonp(app);
app.use(middlewares.rt({headerName: 'X-ReadTime'}));
app.use(middlewares.rewrite('/favicon.ico', '/favicon.png'));
app.use(staticCache);

60
services/package.js Normal file
View File

@@ -0,0 +1,60 @@
/**!
* cnpmjs.org - services/package.js
*
* Copyright(c) fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
'use strict';
/**
* Module dependencies.
*/
var Module = require('../proxy/module');
var ModuleMaintainer = require('../proxy/module_maintainer');
var User = require('../proxy/user');
exports.listMaintainers = function* (name) {
var names = yield* ModuleMaintainer.get(name);
if (names.length === 0) {
return names;
}
var users = yield* User.listByNames(names);
return users.map(function (user) {
return {
name: user.name,
email: user.email
};
});
};
exports.updateMaintainers = function* (name, usernames) {
var rs = yield [
ModuleMaintainer.update(name, usernames),
Module.updateLastModified(name),
];
return rs[0];
};
exports.isMaintainer = function* (name, username) {
var maintainers = yield* ModuleMaintainer.get(name);
if (maintainers.length === 0) {
// if not found maintainers, try to get from latest module package info
var latestMod = yield Module.getLatest(name);
var ms = latestMod && latestMod.package && latestMod.package.maintainers;
if (ms && ms.length > 0) {
maintainers = ms.map(function (user) {
return user.name;
});
}
}
if (maintainers.length === 0) {
// no maintainers, meaning this module is free for everyone
return true;
}
return maintainers.indexOf(username) >= 0;
};

View File

@@ -61,11 +61,11 @@ var handleSync = co(function *() {
var data = yield *sync();
} catch (err) {
error = err;
error.message += ' (sync package error)';
logger.syncError(error);
}
if (config.debug) {
error && console.error(error.stack);
data && console.log(data);
} else {
data && logger.syncInfo(data);
if (!config.debug) {
sendMailToAdmin(error, data, new Date());
}
syncing = false;
@@ -83,14 +83,16 @@ var syncDist = co(function* syncDist() {
return;
}
syncingDist = true;
logger.info('Start syncing dist...');
logger.syncInfo('Start syncing dist...');
try {
yield* syncDistWorker();
yield* syncDistWorker.syncPhantomjsDir();
} catch (err) {
err.message += ' (sync dist error)';
logger.warn('Sync dist error: %s: %s\n%s', err.name, err.message, err.stack);
sendMailToAdmin(err, null, new Date());
logger.syncError(err);
if (config.noticeSyncDistError) {
sendMailToAdmin(err, null, new Date());
}
}
syncingDist = false;
});
@@ -99,7 +101,7 @@ if (config.syncDist) {
syncDist();
setInterval(syncDist, ms(config.syncInterval));
} else {
logger.info('sync dist disable');
logger.syncInfo('sync dist disable');
}
function sendMailToAdmin(err, result, syncTime) {
@@ -133,10 +135,10 @@ function sendMailToAdmin(err, result, syncTime) {
syncTime, result.successes.length, result.successes.slice(0, 10));
}
debug('send email with type: %s, subject: %s, html: %s', type, subject, html);
logger.syncInfo('send email with type: %s, subject: %s, html: %s', type, subject, html);
if (type && type !== 'log') {
mail[type](to, subject, html, function (err) {
if (err) {
logger.info('send email with type: %s, subject: %s, html: %s', type, subject, html);
logger.error(err);
}
});

View File

@@ -54,7 +54,7 @@ function* syncDir(fullname, info) {
}
}
logger.info('sync remote:%s got %d new items, %d dirs, %d files to sync',
logger.syncInfo('sync remote:%s got %d new items, %d dirs, %d files to sync',
fullname, news.length, dirs.length, files.length);
for (var i = 0; i < files.length; i++) {
@@ -67,11 +67,11 @@ function* syncDir(fullname, info) {
}
if (info) {
logger.info('Save dir:%s %j to database', fullname, info);
logger.syncInfo('Save dir:%s %j to database', fullname, info);
yield* Dist.savedir(info);
}
logger.info('Sync %s finished, %d dirs, %d files',
logger.syncInfo('Sync %s finished, %d dirs, %d files',
fullname, dirs.length, files.length);
}
@@ -97,12 +97,12 @@ function* syncFile(info) {
};
try {
logger.info('downloading %s %s to %s, isPhantomjsURL: %s',
logger.syncInfo('downloading %s %s to %s, isPhantomjsURL: %s',
bytes(info.size), downurl, filepath, isPhantomjsURL);
// get tarball
var r = yield *urllib.request(downurl, options);
var statusCode = r.status || -1;
logger.info('download %s got status %s, headers: %j',
logger.syncInfo('download %s got status %s, headers: %j',
downurl, statusCode, r.headers);
if (statusCode !== 200) {
var err = new Error('Download ' + downurl + ' fail, status: ' + statusCode);
@@ -151,19 +151,19 @@ function* syncFile(info) {
};
// upload to NFS
logger.info('uploading %s to nfs:%s', filepath, args.key);
logger.syncInfo('uploading %s to nfs:%s', filepath, args.key);
var result = yield nfs.upload(filepath, args);
info.url = result.url || result.key;
info.sha1 = shasum;
logger.info('upload %s to nfs:%s with size:%d, sha1:%s',
logger.syncInfo('upload %s to nfs:%s with size:%d, sha1:%s',
args.key, info.url, info.size, info.sha1);
} finally {
// remove tmp file whatever
fs.unlink(filepath, utility.noop);
}
logger.info('Sync dist file: %j done', info);
logger.syncInfo('Sync dist file: %j done', info);
yield* Dist.savefile(info);
}
@@ -241,14 +241,14 @@ function* syncPhantomjsDir() {
var fullname = '/phantomjs/';
var files = yield* sync.listPhantomjsDiff(fullname);
logger.info('sync remote:%s got %d files to sync',
logger.syncInfo('sync remote:%s got %d files to sync',
fullname, files.length);
for (var i = 0; i < files.length; i++) {
yield* syncFile(files[i]);
}
logger.info('SyncPhantomjsDir %s finished, %d files',
logger.syncInfo('SyncPhantomjsDir %s finished, %d files',
fullname, files.length);
}
sync.syncPhantomjsDir = syncPhantomjsDir;

View File

@@ -21,25 +21,46 @@ var thunkify = require('thunkify-wrap');
var should = require('should');
var request = require('supertest');
var mm = require('mm');
var pedding = require('pedding');
var config = require('../../../config');
var app = require('../../../servers/registry');
var Module = require('../../../proxy/module');
var Npm = require('../../../proxy/npm');
var controller = require('../../../controllers/registry/module');
var ModuleDeps = require('../../../proxy/module_deps');
var SyncModuleWorker = require('../../../proxy/sync_module_worker');
var utils = require('../../utils');
var fixtures = path.join(path.dirname(path.dirname(__dirname)), 'fixtures');
describe('controllers/registry/module.test.js', function () {
var baseauth = 'Basic ' + new Buffer('cnpmjstest10:cnpmjstest10').toString('base64');
var baseauthOther = 'Basic ' + new Buffer('cnpmjstest101:cnpmjstest101').toString('base64');
before(function (done) {
app.listen(0, function () {
var pkg = require(path.join(fixtures, 'package_and_tgz.json'));
app = app.listen(0, function () {
done = pedding(2, done);
// name: mk2testmodule
var pkg = utils.getPackage('mk2testmodule', '0.0.1', utils.admin);
request(app)
.put('/' + pkg.name)
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err) {
should.not.exist(err);
pkg = utils.getPackage('mk2testmodule', '0.0.2', utils.admin);
// publish 0.0.2
request(app)
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
// testputmodule@0.1.9
var testpkg = utils.getPackage('testputmodule', '0.1.9', utils.admin);
request(app)
.put('/' + testpkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
@@ -53,7 +74,7 @@ describe('controllers/registry/module.test.js', function () {
mm.data(Npm, 'get', require(path.join(fixtures, 'utility.json')));
request(app)
.put('/utility/sync')
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.end(function (err, res) {
should.not.exist(err);
res.body.should.have.keys('ok', 'logId');
@@ -73,6 +94,77 @@ describe('controllers/registry/module.test.js', function () {
});
});
describe('GET /:name unpublished', function () {
before(function (done) {
var worker = new SyncModuleWorker({
name: ['tnpm'],
username: 'fengmk2'
});
worker.start();
worker.on('end', function () {
var names = worker.successes.concat(worker.fails);
names.sort();
names.should.eql(['tnpm']);
done();
});
});
it('should show unpublished info', function (done) {
request(app)
.get('/tnpm')
.expect('content-type', 'application/json; charset=utf-8')
.expect(404, function (err, res) {
should.not.exist(err);
res.body.should.eql({
_id: 'tnpm',
name: 'tnpm',
time: {
modified: '2014-06-05T01:33:59.668Z',
unpublished:
{ name: 'fengmk2',
time: '2014-06-05T01:33:59.668Z',
tags: { latest: '0.3.10' },
maintainers:
[ { name: 'fengmk2', email: 'fengmk2@gmail.com' },
{ name: 'dead_horse', email: 'dead_horse@qq.com' } ],
description: 'npm client for alibaba private npm registry',
versions:
[ '0.0.1',
'0.0.2',
'0.0.3',
'0.0.4',
'0.1.0',
'0.1.1',
'0.1.2',
'0.1.3',
'0.1.4',
'0.1.5',
'0.1.8',
'0.1.9',
'0.2.0',
'0.2.1',
'0.2.2',
'0.2.3',
'0.2.4',
'0.3.0',
'0.3.1',
'0.3.2',
'0.3.3',
'0.3.4',
'0.3.5',
'0.3.6',
'0.3.7',
'0.3.8',
'0.3.9',
'0.3.10' ] } },
_attachments: {}
});
done();
});
});
});
describe('GET /:name get module package info', function () {
var etag;
@@ -170,15 +262,15 @@ describe('controllers/registry/module.test.js', function () {
should.not.exist(err);
var body = res.body;
body.name.should.equal('mk2testmodule');
body.version.should.equal('0.0.1');
body._id.should.equal('mk2testmodule@0.0.1');
body.dist.tarball.should.containEql('/mk2testmodule/download/mk2testmodule-0.0.1.tgz');
body.version.should.equal('0.0.2');
body._id.should.equal('mk2testmodule@0.0.2');
body.dist.tarball.should.containEql('/mk2testmodule/download/mk2testmodule-0.0.2.tgz');
done();
});
});
});
describe('PUT /:name/-rev/id update maintainers', function () {
describe('PUT /:name/-rev/id updateMaintainers()', function () {
before(function (done) {
request(app)
.put('/mk2testmodule/-rev/1')
@@ -188,8 +280,8 @@ describe('controllers/registry/module.test.js', function () {
email: 'cnpmjstest10@cnpmjs.org'
}]
})
.set('authorization', baseauth)
.expect('content-type', 'application/json; charset=utf-8', done);
.set('authorization', utils.adminAuth)
.expect({"ok":true,"id":"mk2testmodule","rev":"1"}, done);
});
it('should add new maintainers', function (done) {
@@ -200,13 +292,52 @@ describe('controllers/registry/module.test.js', function () {
name: 'cnpmjstest10',
email: 'cnpmjstest10@cnpmjs.org'
}, {
name: 'fengmk2',
name: 'cnpmjstest101',
email: 'fengmk2@cnpmjs.org'
}]
})
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.expect(201)
.expect('content-type', 'application/json; charset=utf-8', done);
.expect({
ok: true, id: 'mk2testmodule', rev: '1'
}, function (err) {
should.not.exist(err);
done = pedding(2, done);
// check maintainers update
request(app)
.get('/mk2testmodule')
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg.maintainers.should.length(2);
pkg.maintainers.should.eql(pkg.versions['0.0.1'].maintainers);
pkg.maintainers.sort(function (a, b) {
return a.name > b.name ? 1 : -1;
});
pkg.maintainers.should.eql([
{ name: 'cnpmjstest10', email: 'fengmk2@gmail.com' },
{ name: 'cnpmjstest101', email: 'fengmk2@gmail.com' },
]);
done();
});
// /pkg/0.0.1
request(app)
.get('/mk2testmodule/0.0.1')
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg.maintainers.should.length(2);
pkg.maintainers.sort(function (a, b) {
return a.name > b.name ? 1 : -1;
});
pkg.maintainers.should.eql([
{ name: 'cnpmjstest10', email: 'fengmk2@gmail.com' },
{ name: 'cnpmjstest101', email: 'fengmk2@gmail.com' },
]);
done();
});
});
});
it('should add again new maintainers', function (done) {
@@ -221,7 +352,7 @@ describe('controllers/registry/module.test.js', function () {
email: 'fengmk2@cnpmjs.org'
}]
})
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.expect(201)
.expect('content-type', 'application/json; charset=utf-8', done);
});
@@ -235,7 +366,7 @@ describe('controllers/registry/module.test.js', function () {
email: 'cnpmjstest10@cnpmjs.org'
}]
})
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.expect(201)
.expect('content-type', 'application/json; charset=utf-8', done);
});
@@ -249,13 +380,64 @@ describe('controllers/registry/module.test.js', function () {
email: 'cnpmjstest10@cnpmjs.org'
}]
})
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.expect(201)
.expect({
id: 'mk2testmodule',
rev: '1',
ok: true
}, done);
});
it('should rm all maintainers forbidden 403', function (done) {
request(app)
.put('/mk2testmodule/-rev/1')
.send({
maintainers: []
})
.set('authorization', utils.adminAuth)
.expect(403)
.expect({error: 'invalid operation', reason: 'Can not remove all maintainers'})
.expect('content-type', 'application/json; charset=utf-8', done);
});
it('should 403 when not maintainer update in private mode', function (done) {
request(app)
.put('/mk2testmodule/-rev/1')
.send({
maintainers: [{
name: 'cnpmjstest10',
email: 'cnpmjstest10@cnpmjs.org'
}]
})
.set('authorization', utils.otherUserAuth)
.expect(403)
.expect({
error: 'no_perms',
reason: 'Private mode enable, only admin can publish this module'
}, done);
});
it('should 403 when not maintainer update in public mode', function (done) {
mm(config, 'enablePrivate', false);
request(app)
.put('/mk2testmodule/-rev/1')
.send({
maintainers: [{
name: 'cnpmjstest10',
email: 'cnpmjstest10@cnpmjs.org'
}]
})
.set('authorization', utils.otherUserAuth)
.expect(403)
.expect({
error: 'forbidden user',
reason: 'cnpmjstest101 not authorized to modify mk2testmodule'
}, done);
});
});
describe('PUT /:name', function () {
describe('PUT /:name old publish flow (stop support)', function () {
var pkg = {
name: 'testputmodule',
description: 'test put module',
@@ -286,63 +468,44 @@ describe('controllers/registry/module.test.js', function () {
});
});
it('should try to add not exists module return 201', function (done) {
it('should publish return 400', function (done) {
request(app)
.put('/' + pkg.name)
.set('authorization', baseauth)
.send(pkg)
.expect(201, function (err, res) {
.expect(400, function (err, res) {
should.not.exist(err);
res.body.should.have.keys('ok', 'id', 'rev');
res.body.ok.should.equal(true);
res.body.id.should.equal(pkg.name);
res.body.rev.should.be.a.String;
res.body.reason.should.containEql('filename or version not found');
done();
});
});
it('should try to add return 409 when only next module exists', function (done) {
it('should publish exists return 400', function (done) {
request(app)
.put('/' + pkg.name)
.set('authorization', baseauth)
.send(pkg)
.expect(201, done);
.expect(400, done);
});
it.skip('should try to add return 403 when not module user and only next module exists',
it('should try to add return 400 when not module user and only next module exists',
function (done) {
mm(config, 'enablePrivate', false);
request(app)
.put('/' + pkg.name)
.set('authorization', baseauthOther)
.send(pkg)
.expect(403, function (err, res) {
.expect(400, function (err, res) {
should.not.exist(err);
res.body.should.eql({
error: 'no_perms',
reason: 'Current user can not publish this module'
error: 'version_error',
reason: 'filename or version not found, filename: undefined, version: undefined'
});
done();
});
});
it('should get versions empty when only next module exists', function (done) {
request(app)
.get('/' + pkg.name)
.expect(200, function (err, res) {
should.not.exist(err);
res.body.should.have.keys('_id', '_rev', 'name', 'description', 'versions', 'dist-tags',
'readme', 'maintainers', 'time', '_attachments', 'users');
res.body.versions.should.eql({});
res.body.time.should.eql({});
res.body['dist-tags'].should.eql({});
lastRev = res.body._rev;
// console.log('lastRev: %s', lastRev);
done();
});
});
it('should upload tarball success: /:name/-/:filename/-rev/:rev', function (done) {
it('should upload 404', function (done) {
var body = fs.readFileSync(path.join(fixtures, 'testputmodule-0.1.9.tgz'));
request(app)
.put('/' + pkg.name + '/-/' + pkg.name + '-0.1.9.tgz/-rev/' + lastRev)
@@ -350,96 +513,10 @@ describe('controllers/registry/module.test.js', function () {
.set('content-type', 'application/octet-stream')
.set('content-length', '' + body.length)
.send(body)
.expect(201, function (err, res) {
should.not.exist(err);
res.body.should.eql({
ok: true,
rev: lastRev,
});
done();
});
.expect(404, done);
});
it('should upload tarball success again: /:name/-/:filename/-rev/:rev', function (done) {
var body = fs.readFileSync(path.join(fixtures, 'testputmodule-0.1.9.tgz'));
request(app)
.put('/' + pkg.name + '/-/' + pkg.name + '-0.1.9.tgz/-rev/' + lastRev)
.set('authorization', baseauth)
.set('content-type', 'application/octet-stream')
.set('content-length', '' + body.length)
.send(body)
.expect(201, function (err, res) {
should.not.exist(err);
res.body.should.eql({
ok: true,
rev: lastRev,
});
done();
});
});
it('should upload tarball fail 403 when user not admin', function (done) {
var body = fs.readFileSync(path.join(fixtures, 'testputmodule-0.1.9.tgz'));
request(app)
.put('/' + pkg.name + '/-/' + pkg.name + '-0.1.9.tgz/-rev/25')
.set('authorization', baseauthOther)
.set('content-type', 'application/octet-stream')
.set('content-length', '' + body.length)
.send(body)
.expect(403, function (err, res) {
should.not.exist(err);
res.body.should.eql({
error: 'no_perms',
reason: 'Private mode enable, only admin can publish this module'
});
done();
});
});
it('should upload tarball fail 404 when rev wrong', function (done) {
var body = fs.readFileSync(path.join(fixtures, 'testputmodule-0.1.9.tgz'));
request(app)
.put('/' + pkg.name + '/-/' + pkg.name + '-0.1.9.tgz/-rev/' + '1231231')
.set('authorization', baseauth)
.set('content-type', 'application/octet-stream')
.set('content-length', '' + body.length)
.send(body)
.expect(404, function (err, res) {
should.not.exist(err);
res.body.should.eql({
error: 'not_found',
reason: 'document not found'
});
done();
});
});
it('should update package.json info success: /:name/:version/-tag/latest', function (done) {
var pkg = require(path.join(fixtures, 'testputmodule.json')).versions['0.1.8'];
pkg.name = 'testputmodule';
pkg.version = '0.1.9';
pkg.dependencies['foo-testputmodule'] = '*';
request(app)
.put('/' + pkg.name + '/' + pkg.version + '/-tag/latest')
.set('authorization', baseauth)
.send(pkg)
.expect(201, function (err, res) {
should.not.exist(err);
res.body.should.have.keys('ok', 'rev');
// should get deps foo-testputmodule contains 'testputmodule'
ModuleDeps.list('foo-testputmodule', function (err, rows) {
should.not.exist(err);
var exists = rows.filter(function (r) {
return r.deps === 'testputmodule';
});
exists.should.length(1);
exists[0].deps.should.equal('testputmodule');
done();
});
});
});
it('should update package.json info version invalid: /:name/:version/-tag/latest', function (done) {
it('should update 404 package.json info version invalid: /:name/:version/-tag/latest', function (done) {
var pkg = require(path.join(fixtures, 'testputmodule.json')).versions['0.1.8'];
pkg.name = 'testputmodule';
pkg.version = '0.1.9.alpha';
@@ -447,86 +524,44 @@ describe('controllers/registry/module.test.js', function () {
.put('/' + pkg.name + '/' + pkg.version + '/-tag/latest')
.set('authorization', baseauth)
.send(pkg)
.expect(400)
.expect({
error: 'Params Invalid',
reason: 'Invalid version: ' + pkg.version
}, done);
});
it('should update package.json info again fail 403: /:name/:version/-tag/latest', function (done) {
var pkg = require(path.join(fixtures, 'testputmodule.json')).versions['0.1.8'];
pkg.name = 'testputmodule';
pkg.version = '0.1.10';
request(app)
.put('/' + pkg.name + '/' + pkg.version + '/-tag/latest')
.set('authorization', baseauth)
.send(pkg)
.expect(403, function (err, res) {
should.not.exist(err);
res.body.should.eql({
error: 'version_wrong',
reason: 'version not match'
});
done();
});
});
it('should get new package info', function (done) {
request(app)
.get('/testputmodule/0.1.9')
.expect(200, function (err, res) {
should.not.exist(err);
res.body.name.should.equal('testputmodule');
res.body.version.should.equal('0.1.9');
res.body.dist.tarball.should.containEql('/testputmodule/download/testputmodule-0.1.9.tgz');
done();
});
.expect(404, done);
});
});
describe('PUT /:name publish new flow addPackageAndDist()', function () {
it('should publish with tgz base64, addPackageAndDist()', function (done) {
var pkg = require(path.join(fixtures, 'package_and_tgz.json'));
// delete first
var pkg = utils.getPackage('testpublishmodule', '0.0.2');
request(app)
.del('/' + pkg.name + '/-rev/1')
.set('authorization', baseauth)
.expect({ok: true})
.expect(200, function (err, res) {
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err, res) {
should.not.exist(err);
res.body.should.have.keys('ok', 'rev');
res.body.ok.should.equal(true);
// upload again should 403
request(app)
.put('/' + pkg.name)
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err, res) {
.expect(403, function (err, res) {
should.not.exist(err);
res.body.should.have.keys('ok', 'rev');
res.body.ok.should.equal(true);
// upload again should 409
request(app)
.put('/' + pkg.name)
.set('authorization', baseauth)
.send(pkg)
.expect(409, function (err, res) {
should.not.exist(err);
res.body.should.eql({
error: 'conflict',
reason: 'Document update conflict.'
});
done();
res.body.should.eql({
error: 'forbidden',
reason: 'cannot modify pre-existing version: 0.0.2'
});
done();
});
});
});
it('should version_error when versions missing', function (done) {
var pkg = require(path.join(fixtures, 'package_and_tgz.json'));
var pkg = utils.getPackage('version_missing_module');
delete pkg.versions;
request(app)
.put('/' + pkg.name)
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(400, function (err, res) {
should.not.exist(err);
@@ -537,6 +572,86 @@ describe('controllers/registry/module.test.js', function () {
done();
});
});
it('should 400 when dist-tags empty', function (done) {
var pkg = utils.getPackage('dist-tags-empty');
pkg['dist-tags'] = {};
request(app)
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(400, function (err, res) {
should.not.exist(err);
res.body.should.eql({
error: 'invalid',
reason: 'dist-tags should not be empty'
});
done();
});
});
it('should publish with beta tag addPackageAndDist()', function (done) {
var version = '0.1.1';
var pkg = utils.getPackage('publish-with-beta-tag', version);
pkg['dist-tags'] = {
beta: version
};
request(app)
.del('/' + pkg.name + '/-rev/1')
.set('authorization', utils.adminAuth)
.end(function (err, res) {
should.not.exist(err);
request(app)
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err, res) {
should.not.exist(err);
res.body.should.have.keys('ok', 'rev');
res.body.ok.should.equal(true);
// should auto set latest
request(app)
.get('/' + pkg.name)
.expect(200, function (err, res) {
should.not.exist(err);
res.body['dist-tags'].should.eql({
beta: version,
latest: version
});
// update new beta
pkg['dist-tags'] = {
beta: '10.10.1'
};
pkg.versions = {
'10.10.1': pkg.versions[version]
};
request(app)
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err, res) {
should.not.exist(err);
res.body.should.have.keys('ok', 'rev');
res.body.ok.should.equal(true);
// should auto set latest
request(app)
.get('/' + pkg.name)
.expect(200, function (err, res) {
should.not.exist(err);
res.body['dist-tags'].should.eql({
beta: '10.10.1',
latest: version
});
done();
});
});
});
});
});
});
});
describe('GET /-/all', function () {
@@ -595,13 +710,15 @@ describe('controllers/registry/module.test.js', function () {
});
});
describe('PUT /:name/-rev/:rev', function () {
describe('PUT /:name/-rev/:rev removeWithVersions', function () {
var pkg = require(path.join(fixtures, 'package_and_tgz.json'));
var pkgname = pkg.name;
var baseauth = 'Basic ' + new Buffer('cnpmjstest10:cnpmjstest10').toString('base64');
var baseauthOther = 'Basic ' + new Buffer('cnpmjstest101:cnpmjstest101').toString('base64');
var lastRev;
before(function (done) {
request(app)
.get('/testputmodule')
.get('/' + pkgname)
.end(function (err, res) {
lastRev = res.body._rev;
done(err);
@@ -610,39 +727,39 @@ describe('controllers/registry/module.test.js', function () {
it('should update 401 when no auth', function (done) {
request(app)
.put('/testputmodule/-rev/123')
.put('/' + pkgname + '/-rev/123')
.expect(401, done);
});
it('should update 403 when auth error', function (done) {
request(app)
.put('/testputmodule/-rev/123')
.put('/' + pkgname + '/-rev/123')
.set('authorization', baseauthOther)
.expect(403, done);
});
it('should remove nothing removed ok', function (done) {
request(app)
.put('/testputmodule/-rev/' + lastRev)
.put('/' + pkgname + '/-rev/' + lastRev)
.set('authorization', baseauth)
.send({
versions: {
'0.1.9': {}
'0.0.1': {},
'0.0.2': {}
}
})
.expect(201, done);
});
it('should remove version ok', function (done) {
it('should remove all version ok', function (done) {
//do not really remove it here
mm.empty(Module, 'removeByNameAndVersions');
mm.empty(Module, 'removeTagsByIds');
request(app)
.put('/testputmodule/-rev/' + lastRev)
.put('/' + pkgname + '/-rev/' + lastRev)
.set('authorization', baseauth)
.send({
versions: {
}
versions: {}
})
.expect(201, done);
});
@@ -660,91 +777,109 @@ describe('controllers/registry/module.test.js', function () {
describe('DELETE /:name/download/:filename/-rev/:rev', function () {
var lastRev;
before(function (done) {
var pkg = utils.getPackage('test-delete-download-module', '0.1.9');
request(app)
.get('/testputmodule')
.end(function (err, res) {
lastRev = res.body._rev;
done(err);
.put('/' + pkg.name)
.set('content-type', 'application/json')
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err, res) {
should.not.exist(err);
lastRev = res.body.rev;
done();
});
});
it('should delete 401 when no auth', function (done) {
request(app)
.del('/testputmodule/download/testputmodule-0.1.9.tgz/-rev/' + lastRev)
.del('/test-delete-download-module/download/test-delete-download-module-0.1.9.tgz/-rev/' + lastRev)
.expect(401, done);
});
it('should delete 403 when auth error', function (done) {
request(app)
.del('/testputmodule/download/testputmodule-0.1.9.tgz/-rev/' + lastRev)
.set('authorization', baseauthOther)
.del('/test-delete-download-module/download/test-delete-download-module-0.1.9.tgz/-rev/' + lastRev)
.set('authorization', utils.otherUserAuth)
.expect(403, done);
});
it('should delete file ok', function (done) {
request(app)
.del('/testputmodule/download/testputmodule-0.1.9.tgz/-rev/' + lastRev)
.set('authorization', baseauth)
.del('/test-delete-download-module/download/test-delete-download-module-0.1.9.tgz/-rev/' + lastRev)
.set('authorization', utils.adminAuth)
.expect(200, done);
});
});
describe('PUT /:name/:tag', function () {
describe('PUT /:name/:tag updateTag()', function () {
it('should create new tag ok', function (done) {
request(app)
.put('/testputmodule/newtag')
.put('/mk2testmodule/newtag')
.set('content-type', 'application/json')
.set('authorization', baseauth)
.send('"0.1.9"')
.expect(201, done);
.set('authorization', utils.adminAuth)
.send('"0.0.1"')
.expect(201)
.expect({"ok":true}, done);
});
it('should override exist tag ok', function (done) {
request(app)
.put('/testputmodule/newtag')
.put('/mk2testmodule/newtag')
.set('content-type', 'application/json')
.set('authorization', baseauth)
.send('"0.1.9"')
.set('authorization', utils.adminAuth)
.send('"0.0.1"')
.expect(201, done);
});
it('should tag invalid version 403', function (done) {
request(app)
.put('/testputmodule/newtag')
.put('/mk2testmodule/newtag')
.set('content-type', 'application/json')
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.send('"hello"')
.expect(403)
.expect({
error: 'forbidden',
reason: 'setting tag newtag to invalid version: hello: testputmodule/newtag'
reason: 'setting tag newtag to invalid version: hello: mk2testmodule/newtag'
}, done);
});
it('should tag not eixst version 403', function (done) {
request(app)
.put('/testputmodule/newtag')
.put('/mk2testmodule/newtag')
.set('content-type', 'application/json')
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.send('"5.0.0"')
.expect(403)
.expect({
error: 'forbidden',
reason: 'setting tag newtag to unknown version: 5.0.0: testputmodule/newtag'
reason: 'setting tag newtag to unknown version: 5.0.0: mk2testmodule/newtag'
}, done);
});
it('should tag permission 403', function (done) {
request(app)
.put('/testputmodule/newtag')
.set('content-type', 'application/json')
.set('authorization', baseauthOther)
.send('"0.1.9"')
.expect(403)
.expect({
error: 'forbidden',
reason: 'no permission to modify testputmodule'
}, done);
describe('update tag not maintainer', function () {
before(function (done) {
var pkg = utils.getPackage('update-tag-not-maintainer', '1.0.0');
request(app)
.put('/' + pkg.name)
.set('content-type', 'application/json')
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
it('should not maintainer update tag return no permission 403', function (done) {
request(app)
.put('/update-tag-not-maintainer/newtag')
.set('content-type', 'application/json')
.set('authorization', utils.otherUserAuth)
.send('"1.0.0"')
.expect(403)
.expect({
error: 'forbidden user',
reason: 'cnpmjstest101 not authorized to modify update-tag-not-maintainer'
}, done);
});
});
});
@@ -770,20 +905,44 @@ describe('controllers/registry/module.test.js', function () {
it('should delete 403 when auth error', function (done) {
request(app)
.del('/testputmodule/-rev/' + lastRev)
.set('authorization', baseauthOther)
.set('authorization', utils.otherUserAuth)
.expect(403, done);
});
it('shold remove all the module ok', function (done) {
//do not really remove
mm.empty(Module, 'removeByName');
request(app)
.del('/testputmodule/-rev/' + lastRev)
.set('authorization', baseauth)
.expect(200, function (err, res) {
should.not.exist(err);
should.not.exist(res.headers['set-cookie']);
done();
describe('remove all modules by name', function () {
before(function (done) {
var pkg = utils.getPackage('remove-all-module');
request(app)
.put('/remove-all-module')
.set('content-type', 'application/json')
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
it('shold fail when user not maintainer', function (done) {
request(app)
.del('/remove-all-module/-rev/1')
.set('authorization', utils.otherUserAuth)
.expect(403, function (err, res) {
should.not.exist(err);
res.body.should.eql({
error: 'no_perms',
reason: 'Private mode enable, only admin can publish this module'
});
done();
});
});
it('shold ok', function (done) {
request(app)
.del('/remove-all-module/-rev/1')
.set('authorization', utils.adminAuth)
.expect(200, function (err, res) {
should.not.exist(err);
should.not.exist(res.headers['set-cookie']);
done();
});
});
});
});

View File

@@ -0,0 +1,184 @@
/**!
* cnpmjs.org - test/controllers/registry/module/scope_package.test.js
*
* Copyright(c) fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
'use strict';
/**
* Module dependencies.
*/
var should = require('should');
var request = require('supertest');
var mm = require('mm');
var config = require('../../../../config');
var app = require('../../../../servers/registry');
var utils = require('../../../utils');
describe('controllers/registry/module/scope_package.test.js', function () {
var pkgname = '@cnpm/test-scope-package';
var pkgURL = '/@' + encodeURIComponent(pkgname.substring(1));
before(function (done) {
app = app.listen(0, function () {
// add scope package
var pkg = utils.getPackage(pkgname, '0.0.1', utils.admin);
request(app)
.put(pkgURL)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err) {
should.not.exist(err);
pkg = utils.getPackage(pkgname, '0.0.2', utils.admin);
// publish 0.0.2
request(app.listen())
.put(pkgURL)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
});
});
afterEach(mm.restore);
it('should get scope package info: /@scope%2Fname', function (done) {
request(app)
.get(pkgURL)
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg.name.should.equal(pkgname);
pkg.versions.should.have.keys('0.0.1', '0.0.2');
pkg['dist-tags'].latest.should.equal('0.0.2');
pkg.versions['0.0.1'].name.should.equal(pkgname);
pkg.versions['0.0.1'].dist.tarball
.should.containEql('/@cnpm/test-scope-package/download/@cnpm/test-scope-package-0.0.1.tgz');
done();
});
});
it('should get scope package info: /@scope/name', function (done) {
request(app.listen())
.get('/' + pkgname)
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg.name.should.equal(pkgname);
pkg.versions.should.have.keys('0.0.1', '0.0.2');
pkg['dist-tags'].latest.should.equal('0.0.2');
pkg.versions['0.0.1'].name.should.equal(pkgname);
pkg.versions['0.0.1'].dist.tarball
.should.containEql('/@cnpm/test-scope-package/download/@cnpm/test-scope-package-0.0.1.tgz');
done();
});
});
it('should get scope package info: /%40scope%2Fname', function (done) {
request(app)
.get('/' + encodeURIComponent(pkgname))
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg.name.should.equal(pkgname);
pkg.versions.should.have.keys('0.0.1', '0.0.2');
pkg['dist-tags'].latest.should.equal('0.0.2');
pkg.versions['0.0.1'].name.should.equal(pkgname);
pkg.versions['0.0.1'].dist.tarball
.should.containEql('/@cnpm/test-scope-package/download/@cnpm/test-scope-package-0.0.1.tgz');
done();
});
});
it('should get scope package with version', function (done) {
request(app)
.get('/' + pkgname + '/0.0.1')
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg.name.should.equal(pkgname);
pkg.version.should.equal('0.0.1');
pkg.dist.tarball
.should.containEql('/@cnpm/test-scope-package/download/@cnpm/test-scope-package-0.0.1.tgz');
done();
});
});
it('should get scope package with tag', function (done) {
request(app)
.get('/' + pkgname + '/latest')
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg.name.should.equal(pkgname);
pkg.version.should.equal('0.0.2');
pkg.dist.tarball
.should.containEql('/@cnpm/test-scope-package/download/@cnpm/test-scope-package-0.0.2.tgz');
done();
});
});
describe('support defaultScope', function () {
before(function (done) {
var pkg = utils.getPackage('test-default-scope-package', '0.0.1', utils.admin);
request(app)
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
it('should adapt /@cnpm/test-default-scope-package => /test-default-scope-package', function (done) {
mm(config, 'defaultScope', '@cnpm');
request(app)
.get('/@cnpm/test-default-scope-package')
.expect(200, function (err, res) {
should.not.exist(err);
var pkg = res.body;
pkg._id.should.equal('@cnpm/test-default-scope-package');
pkg.name.should.equal('@cnpm/test-default-scope-package');
pkg.versions.should.have.keys('0.0.1');
pkg['dist-tags'].latest.should.equal('0.0.1');
pkg.versions['0.0.1'].name.should.equal('@cnpm/test-default-scope-package');
pkg.versions['0.0.1']._id.should.equal('@cnpm/test-default-scope-package@0.0.1');
pkg.versions['0.0.1'].dist.tarball
.should.containEql('/test-default-scope-package/download/test-default-scope-package-0.0.1.tgz');
done();
});
});
it('should not adapt /@cnpm123/test-default-scope-package', function (done) {
mm(config, 'defaultScope', '@cnpm');
request(app)
.get('/@cnpm123/test-default-scope-package')
.expect(404, done);
});
it('should not adapt when defaultScope is empty', function (done) {
mm(config, 'defaultScope', '');
request(app)
.get('/@cnpm/test-default-scope-package')
.expect(404, done);
});
it('should 404 when pkg not exists', function (done) {
mm(config, 'defaultScope', '@cnpm');
request(app)
.get('/@cnpm/test-default-scope-package-not-exists')
.expect(404, done);
});
it('should 404 when scope not match', function (done) {
mm(config, 'defaultScope', '@cnpm123');
request(app)
.get('/@cnpm/test-default-scope-package')
.expect(404, done);
});
});
});

View File

@@ -23,11 +23,7 @@ var mysql = require('../../../common/mysql');
describe('controllers/registry/user.test.js', function () {
before(function (done) {
app.listen(0, done);
});
after(function (done) {
app.close(done);
app = app.listen(0, done);
});
afterEach(mm.restore);

View File

@@ -27,8 +27,8 @@ var webApp = require('../../servers/web');
describe('controllers/sync.test.js', function () {
before(function (done) {
done = pedding(2, done);
registryApp.listen(0, done);
webApp.listen(0, done);
registryApp = registryApp.listen(0, done);
webApp = webApp.listen(0, done);
});
afterEach(mm.restore);
@@ -116,4 +116,16 @@ describe('controllers/sync.test.js', function () {
});
});
});
describe('scope package', function () {
it('should sync scope package not found', function (done) {
request(webApp)
.put('/sync/@cnpm/not-exists-package')
.expect({
"ok":false,
"reason":"can not found @cnpm/not-exists-package in the source registry"
})
.expect(404, done);
});
});
});

View File

@@ -27,11 +27,6 @@ describe('controllers/total.test.js', function () {
registryApp.listen(0, done);
webApp.listen(0, done);
});
after(function (done) {
done = pedding(2, done);
registryApp.close(done);
webApp.close(done);
});
describe('GET / in registry', function () {
it('should return total info', function (done) {

View File

@@ -23,11 +23,7 @@ var Dist = require('../../../proxy/dist');
describe('controllers/web/dist.test.js', function () {
before(function (done) {
app.listen(0, done);
});
after(function (done) {
app.close(done);
app = app.listen(0, done);
});
afterEach(mm.restore);

View File

@@ -18,31 +18,29 @@ var should = require('should');
var request = require('supertest');
var mm = require('mm');
var path = require('path');
var pedding = require('pedding');
var mysql = require('../../../common/mysql');
var app = require('../../../servers/web');
var registry = require('../../../servers/registry');
var pkg = require('../../../controllers/web/package');
var utils = require('../../utils');
var fixtures = path.join(path.dirname(path.dirname(__dirname)), 'fixtures');
describe('controllers/web/package.test.js', function () {
var baseauth = 'Basic ' + new Buffer('cnpmjstest10:cnpmjstest10').toString('base64');
before(function (done) {
registry.listen(0, function () {
var pkg = require(path.join(fixtures, 'package_and_tgz.json'));
done = pedding(2, done);
registry = registry.listen(0, function () {
// name: mk2testmodule
var pkg = utils.getPackage('mk2testmodule', '0.0.1', utils.admin);
request(registry)
.put('/' + pkg.name)
.set('authorization', baseauth)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function () {
app.listen(0, done);
});
.end(done);
});
});
after(function (done) {
app.close(done);
app = app.listen(0, done);
});
afterEach(mm.restore);

View File

@@ -0,0 +1,166 @@
/**!
* cnpmjs.org - test/controllers/web/package/scope_package.test.js
*
* Copyright(c) fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
'use strict';
/**
* Module dependencies.
*/
var should = require('should');
var request = require('supertest');
var pedding = require('pedding');
var mm = require('mm');
var config = require('../../../../config');
var registry = require('../../../../servers/registry');
var web = require('../../../../servers/web');
var utils = require('../../../utils');
describe('controllers/web/package/scope_package.test.js', function () {
var pkgname = '@cnpm/test-web-scope-package';
var pkgURL = '/@' + encodeURIComponent(pkgname.substring(1));
before(function (done) {
done = pedding(2, done);
registry = registry.listen(0, function () {
// add scope package
var pkg = utils.getPackage(pkgname, '0.0.1', utils.admin);
request(registry)
.put(pkgURL)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, function (err) {
should.not.exist(err);
pkg = utils.getPackage(pkgname, '0.0.2', utils.admin);
// publish 0.0.2
request(registry)
.put(pkgURL)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
});
web = web.listen(0, done);
});
afterEach(mm.restore);
it('should show scope package info page: /@scope%2Fname', function (done) {
request(web)
.get('/package' + pkgURL)
.expect(200, function (err, res) {
should.not.exist(err);
var body = res.text;
body.should.containEql('$ cnpm install @cnpm/test-web-scope-package');
body.should.containEql('/@cnpm/test-web-scope-package/download/@cnpm/test-web-scope-package-0.0.2.tgz');
done();
});
});
it('should show scope package info page: encodeURIComponent("/@scope/name")', function (done) {
request(web)
.get('/package/' + encodeURIComponent(pkgname))
.expect(200, function (err, res) {
should.not.exist(err);
var body = res.text;
body.should.containEql('$ cnpm install @cnpm/test-web-scope-package');
body.should.containEql('/@cnpm/test-web-scope-package/download/@cnpm/test-web-scope-package-0.0.2.tgz');
done();
});
});
it('should show scope package info page: /@scope/name', function (done) {
request(web)
.get('/package/' + pkgname)
.expect(200, function (err, res) {
should.not.exist(err);
var body = res.text;
body.should.containEql('$ cnpm install @cnpm/test-web-scope-package');
body.should.containEql('/@cnpm/test-web-scope-package/download/@cnpm/test-web-scope-package-0.0.2.tgz');
done();
});
});
it('should /package/@scope/name/ 404', function (done) {
request(web)
.get('/package/' + pkgname + '/')
.expect(404, done);
});
it('should show scope package with version: /@scope/name/0.0.2', function (done) {
request(web)
.get('/package/' + pkgname + '/0.0.2')
.expect(200, function (err, res) {
should.not.exist(err);
var body = res.text;
body.should.containEql('$ cnpm install @cnpm/test-web-scope-package');
body.should.containEql('/@cnpm/test-web-scope-package/download/@cnpm/test-web-scope-package-0.0.2.tgz');
done();
});
});
it('should /@scope/name redirect to /package/@scope/name', function (done) {
request(web)
.get('/' + pkgname)
.expect('Location', '/package/' + pkgname)
.expect(302, done);
});
describe('support default scope', function () {
before(function (done) {
var pkg = utils.getPackage('test-default-web-scope-package', '0.0.1', utils.admin);
request(registry)
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.expect(201, done);
});
it('should adapt /@cnpm/test-default-web-scope-package => /test-default-web-scope-package', function (done) {
mm(config, 'defaultScope', '@cnpm');
request(web)
.get('/package/@cnpm/test-default-web-scope-package')
.expect(200, function (err, res) {
should.not.exist(err);
var body = res.text;
body.should.containEql('@cnpm/test-default-web-scope-package');
body.should.containEql('/test-default-web-scope-package/download/test-default-web-scope-package-0.0.1.tgz');
done();
});
});
it('should not adapt /@cnpm123/test-default-web-scope-package', function (done) {
mm(config, 'defaultScope', '@cnpm');
request(web)
.get('/package/@cnpm123/test-default-web-scope-package')
.expect(404, done);
});
it('should not adapt', function (done) {
mm(config, 'defaultScope', '');
request(web)
.get('/package/@cnpm/test-default-web-scope-package')
.expect(404, done);
});
it('should 404 when scope not match', function (done) {
mm(config, 'defaultScope', '@cnpm123');
request(web)
.get('/package/@cnpm/test-default-web-scope-package')
.expect(404, done);
});
it('should 404 when pkg not exists', function (done) {
mm(config, 'defaultScope', '@cnpm');
request(web)
.get('/package/@cnpm/test-default-web-scope-package-not-exists')
.expect(404, done);
});
});
});

View File

@@ -20,10 +20,7 @@ var app = require('../../../servers/web');
describe('controllers/web/user.test.js', function () {
before(function (done) {
app.listen(0, done);
});
after(function (done) {
app.close(done);
app = app.listen(0, done);
});
describe('GET /~:name', function (done) {

16
test/fixtures/scope-package/.gitignore vendored Normal file
View File

@@ -0,0 +1,16 @@
coverage.html
*.seed
*.log
*.csv
*.dat
*.out
*.pid
*.gz
pids
logs
results
node_modules
npm-debug.log
coverage/

View File

@@ -0,0 +1,4 @@
node_modules/
coverage/
.tmp/
.git/

95
test/fixtures/scope-package/.jshintrc vendored Normal file
View File

@@ -0,0 +1,95 @@
{
// JSHint Default Configuration File (as on JSHint website)
// See http://jshint.com/docs/ for more details
"maxerr" : 50, // {int} Maximum error before stopping
// Enforcing
"bitwise" : true, // true: Prohibit bitwise operators (&, |, ^, etc.)
"camelcase" : false, // true: Identifiers must be in camelCase
"curly" : true, // true: Require {} for every new block or scope
"eqeqeq" : true, // true: Require triple equals (===) for comparison
"forin" : false, // true: Require filtering for..in loops with obj.hasOwnProperty()
"immed" : false, // true: Require immediate invocations to be wrapped in parens e.g. `(function () { } ());`
"indent" : false, // {int} Number of spaces to use for indentation
"latedef" : false, // true: Require variables/functions to be defined before being used
"newcap" : false, // true: Require capitalization of all constructor functions e.g. `new F()`
"noarg" : true, // true: Prohibit use of `arguments.caller` and `arguments.callee`
"noempty" : true, // true: Prohibit use of empty blocks
"nonew" : false, // true: Prohibit use of constructors for side-effects (without assignment)
"plusplus" : false, // true: Prohibit use of `++` & `--`
"quotmark" : false, // Quotation mark consistency:
// false : do nothing (default)
// true : ensure whatever is used is consistent
// "single" : require single quotes
// "double" : require double quotes
"undef" : true, // true: Require all non-global variables to be declared (prevents global leaks)
"unused" : false, // true: Require all defined variables be used
"strict" : true, // true: Requires all functions run in ES5 Strict Mode
"trailing" : false, // true: Prohibit trailing whitespaces
"maxparams" : false, // {int} Max number of formal params allowed per function
"maxdepth" : false, // {int} Max depth of nested blocks (within functions)
"maxstatements" : false, // {int} Max number statements per function
"maxcomplexity" : false, // {int} Max cyclomatic complexity per function
"maxlen" : false, // {int} Max number of characters per line
// Relaxing
"asi" : false, // true: Tolerate Automatic Semicolon Insertion (no semicolons)
"boss" : true, // true: Tolerate assignments where comparisons would be expected
"debug" : false, // true: Allow debugger statements e.g. browser breakpoints.
"eqnull" : false, // true: Tolerate use of `== null`
"es5" : false, // true: Allow ES5 syntax (ex: getters and setters)
"esnext" : true, // true: Allow ES.next (ES6) syntax (ex: `const`)
"moz" : false, // true: Allow Mozilla specific syntax (extends and overrides esnext features)
// (ex: `for each`, multiple try/catch, function expression…)
"evil" : false, // true: Tolerate use of `eval` and `new Function()`
"expr" : true, // true: Tolerate `ExpressionStatement` as Programs
"funcscope" : false, // true: Tolerate defining variables inside control statements"
"globalstrict" : false, // true: Allow global "use strict" (also enables 'strict')
"iterator" : false, // true: Tolerate using the `__iterator__` property
"lastsemic" : false, // true: Tolerate omitting a semicolon for the last statement of a 1-line block
"laxbreak" : true, // true: Tolerate possibly unsafe line breakings
"laxcomma" : false, // true: Tolerate comma-first style coding
"loopfunc" : false, // true: Tolerate functions being defined in loops
"multistr" : true, // true: Tolerate multi-line strings
"proto" : false, // true: Tolerate using the `__proto__` property
"scripturl" : false, // true: Tolerate script-targeted URLs
"smarttabs" : false, // true: Tolerate mixed tabs/spaces when used for alignment
"shadow" : true, // true: Allows re-define variables later in code e.g. `var x=1; x=2;`
"sub" : false, // true: Tolerate using `[]` notation when it can still be expressed in dot notation
"supernew" : false, // true: Tolerate `new function () { ... };` and `new Object;`
"validthis" : false, // true: Tolerate using this in a non-constructor function
// Environments
"browser" : true, // Web Browser (window, document, etc)
"couch" : false, // CouchDB
"devel" : true, // Development/debugging (alert, confirm, etc)
"dojo" : false, // Dojo Toolkit
"jquery" : false, // jQuery
"mootools" : false, // MooTools
"node" : true, // Node.js
"nonstandard" : false, // Widely adopted globals (escape, unescape, etc)
"prototypejs" : false, // Prototype and Scriptaculous
"rhino" : false, // Rhino
"worker" : false, // Web Workers
"wsh" : false, // Windows Scripting Host
"yui" : false, // Yahoo User Interface
"noyield" : true, // allow generators without a yield
// Legacy
"nomen" : false, // true: Prohibit dangling `_` in variables
"onevar" : false, // true: Allow only one `var` statement per function
"passfail" : false, // true: Stop on first error
"white" : false, // true: Check against strict whitespace and indentation rules
// Custom Globals
"globals" : { // additional predefined global variables
// mocha
"describe": true,
"it": true,
"before": true,
"afterEach": true,
"beforeEach": true,
"after": true
}
}

View File

@@ -0,0 +1,8 @@
test/
Makefile
.travis.yml
logo.png
.jshintignore
.jshintrc
.gitingore
coverage/

View File

@@ -0,0 +1,6 @@
language: node_js
node_js:
- '0.11'
- '0.10'
script: "make test-travis"
after_script: "npm install coveralls@2 && cat ./coverage/lcov.info | coveralls"

0
test/fixtures/scope-package/AUTHORS vendored Normal file
View File

21
test/fixtures/scope-package/LICENSE.txt vendored Normal file
View File

@@ -0,0 +1,21 @@
This software is licensed under the MIT License.
Copyright (c) 2014 fengmk2 <fengmk2@gmail.com> and other contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

51
test/fixtures/scope-package/Makefile vendored Normal file
View File

@@ -0,0 +1,51 @@
TESTS = test/*.test.js
REPORTER = spec
TIMEOUT = 1000
MOCHA_OPTS =
install:
@npm install --registry=https://registry.npm.taobao.org --disturl=https://npm.taobao.org/dist
jshint: install
@./node_modules/.bin/jshint .
test: install
@NODE_ENV=test ./node_modules/.bin/mocha \
--harmony \
--reporter $(REPORTER) \
--timeout $(TIMEOUT) \
$(MOCHA_OPTS) \
$(TESTS)
test-cov cov: install
@NODE_ENV=test node --harmony \
node_modules/.bin/istanbul cover --preserve-comments \
./node_modules/.bin/_mocha \
-- \
--reporter $(REPORTER) \
--timeout $(TIMEOUT) \
$(MOCHA_OPTS) \
$(TESTS)
@./node_modules/.bin/cov coverage
test-travis: install
@NODE_ENV=test node --harmony \
node_modules/.bin/istanbul cover --preserve-comments \
./node_modules/.bin/_mocha \
--report lcovonly \
-- \
--reporter dot \
--timeout $(TIMEOUT) \
$(MOCHA_OPTS) \
$(TESTS)
test-all: install jshint test cov
autod: install
@./node_modules/.bin/autod -w --prefix "~"
@$(MAKE) install
contributors: install
@./node_modules/.bin/contributors -f plain -o AUTHORS
.PHONY: test

64
test/fixtures/scope-package/README.md vendored Normal file
View File

@@ -0,0 +1,64 @@
scope-package
=======
[![NPM version][npm-image]][npm-url]
[![build status][travis-image]][travis-url]
[![Test coverage][coveralls-image]][coveralls-url]
[![Gittip][gittip-image]][gittip-url]
[![David deps][david-image]][david-url]
[npm-image]: https://img.shields.io/npm/v/scope-package.svg?style=flat
[npm-url]: https://npmjs.org/package/scope-package
[travis-image]: https://img.shields.io/travis/node-modules/scope-package.svg?style=flat
[travis-url]: https://travis-ci.org/node-modules/scope-package
[coveralls-image]: https://img.shields.io/coveralls/node-modules/scope-package.svg?style=flat
[coveralls-url]: https://coveralls.io/r/node-modules/scope-package?branch=master
[gittip-image]: https://img.shields.io/gittip/fengmk2.svg?style=flat
[gittip-url]: https://www.gittip.com/fengmk2/
[david-image]: https://img.shields.io/david/node-modules/scope-package.svg?style=flat
[david-url]: https://david-dm.org/node-modules/scope-package
![logo](https://raw.github.com/node-modules/scope-package/master/logo.png)
scope-package desc
## Install
```bash
$ npm install scope-package
```
## Usage
```js
var scope-package = require('scope-package');
scope-package.foo(function (err) {
});
```
## License
(The MIT License)
Copyright (c) 2014 fengmk2 &lt;fengmk2@gmail.com&gt; and other contributors
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

1
test/fixtures/scope-package/index.js vendored Normal file
View File

@@ -0,0 +1 @@
module.exports = require('./lib/scope-package');

View File

@@ -0,0 +1,17 @@
/**!
* scope-package - lib/scope-package.js
*
* Copyright(c) 2014 fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
"use strict";
/**
* Module dependencies.
*/
var varname = require('modulename');

View File

@@ -0,0 +1,44 @@
{
"name": "@cnpm/scope-package",
"version": "0.0.1",
"description": "scope-package",
"main": "index.js",
"scripts": {
"test": "make test-all"
},
"config": {
"cov": {
"threshold": 100
}
},
"dependencies": {
},
"devDependencies": {
"autod": "*",
"contributors": "*",
"should": "*",
"jshint": "*",
"cov": "*",
"istanbul-harmony": "*",
"mocha": "*"
},
"homepage": "https://github.com/node-modules/scope-package",
"repository": {
"type": "git",
"url": "git://github.com/node-modules/scope-package.git",
"web": "https://github.com/node-modules/scope-package"
},
"bugs": {
"url": "https://github.com/node-modules/scope-package/issues",
"email": "fengmk2@gmail.com"
},
"keywords": [
"scope-package"
],
"engines": {
"node": ">= 0.10.0"
},
"author": "fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)",
"license": "MIT"
}

View File

@@ -16,15 +16,22 @@
var should = require('should');
var request = require('supertest');
var utils = require('../utils');
var app = require('../../servers/web');
var registry = require('../../servers/registry');
describe('middleware/web_not_found.test.js', function () {
before(function (done) {
app.listen(0, done);
});
// make sure mk2testmodule exists
var baseauth = 'Basic ' + new Buffer('cnpmjstest10:cnpmjstest10').toString('base64');
// name: mk2testmodule
var pkg = utils.getPackage('mk2testmodule');
after(function (done) {
app.close(done);
request(registry)
.put('/' + pkg.name)
.set('authorization', utils.adminAuth)
.send(pkg)
.end(done);
});
describe('web_not_found()', function () {

View File

@@ -167,4 +167,10 @@ describe('proxy/module.test.js', function () {
});
});
});
describe('removeTagsByNames()', function () {
it('should work', function* () {
yield* Module.removeTagsByNames('foo', ['latest', '1.0']);
});
});
});

View File

@@ -0,0 +1,69 @@
/**!
* cnpmjs.org - test/proxy/module_maintainer.test.js
*
* Copyright(c) fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
'use strict';
/**
* Module dependencies.
*/
var should = require('should');
var ModuleMaintainer = require('../../proxy/module_maintainer');
describe('proxy/module_maintainer.test.js', function () {
describe('update()', function () {
it('should update one maintainer', function* () {
var rs = yield* ModuleMaintainer.update('testfoo', ['fengmk2']);
rs.should.eql({
add: ['fengmk2'],
remove: []
});
// again should be fine
var rs = yield* ModuleMaintainer.update('testfoo', ['fengmk2']);
rs.should.eql({
add: ['fengmk2'],
remove: []
});
// remove the exists
var rs = yield* ModuleMaintainer.update('testfoo', ['fengmk2-1', 'foobar']);
rs.should.eql({
add: ['fengmk2-1', 'foobar'],
remove: ['fengmk2']
});
});
it('should update multi maintainers', function* () {
var rs = yield* ModuleMaintainer.update('testfoo2', ['fengmk2', 'ok', 'foobar']);
rs.should.eql({
add: ['fengmk2', 'ok', 'foobar'],
remove: []
});
// remove exists
var rs = yield* ModuleMaintainer.update('testfoo2', ['fengmk2']);
rs.should.eql({
add: ['fengmk2'],
remove: ['ok', 'foobar']
});
var rs = yield* ModuleMaintainer.update('testfoo3', ['fengmk2', 'ok', 'foobar']);
rs.should.eql({
add: ['fengmk2', 'ok', 'foobar'],
remove: []
});
});
it('should add empty maintainers do nothing', function* () {
var rs = yield* ModuleMaintainer.update('tesfoobar', []);
rs.should.eql({
add: [],
remove: []
});
});
});
});

View File

@@ -63,4 +63,31 @@ describe('proxy/sync_module_worker.test.js', function () {
worker.start();
worker.on('end', done);
});
it('should sync unpublished module by name', function* () {
var result = yield* SyncModuleWorker.sync('tnpm', 'fengmk2');
result.ok.should.equal(true);
result.should.have.property('logId');
});
it('should not sync not exists module', function* () {
var result = yield* SyncModuleWorker.sync('tnpm-not-exists', 'fengmk2');
result.ok.should.equal(false);
result.should.not.have.property('logId');
});
it('should sync unpublished info', function (done) {
var worker = new SyncModuleWorker({
name: ['tnpm'],
username: 'fengmk2'
});
worker.start();
worker.on('end', function () {
var names = worker.successes.concat(worker.fails);
names.sort();
names.should.eql(['tnpm']);
done();
});
});
});

View File

@@ -14,12 +14,14 @@
* Module dependencies.
*/
var debug = require('debug');
debug.enable('cnpmjs.org*');
var SyncModuleWorker = require('../proxy/sync_module_worker');
var mysql = require('../common/mysql');
var Log = require('../proxy/module_log');
var config = require('../config');
config.sourceNpmRegistry = 'http://r.cnpmjs.org';
config.sourceNpmRegistry = 'http://registry.npmjs.org';
var names = process.argv[2] || 'byte';
names = names.split(',');

48
test/utils.js Normal file
View File

@@ -0,0 +1,48 @@
/**!
* cnpmjs.org - test/utils.js
*
* Copyright(c) fengmk2 and other contributors.
* MIT Licensed
*
* Authors:
* fengmk2 <fengmk2@gmail.com> (http://fengmk2.github.com)
*/
'use strict';
/**
* Module dependencies.
*/
var path = require('path');
var fs = require('fs');
var fixtures = path.join(__dirname, 'fixtures');
var admin = exports.admin = 'cnpmjstest10';
exports.adminAuth = 'Basic ' + new Buffer(admin + ':' + admin).toString('base64');
var otherUser = exports.otherUser = 'cnpmjstest101';
exports.otherUserAuth = 'Basic ' + new Buffer(otherUser + ':' + otherUser).toString('base64');
var _pkg = fs.readFileSync(path.join(fixtures, 'package_and_tgz.json'));
exports.getPackage = function (name, version, user) {
// name: mk2testmodule
name = name || 'mk2testmodule';
version = version || '0.0.1';
user = user || admin;
var pkg = JSON.parse(_pkg);
var versions = pkg.versions;
pkg.versions = {};
pkg.versions[version] = versions[Object.keys(versions)[0]];
pkg.maintainers[0].name = user;
pkg.versions[version].maintainers[0].name = user;
pkg.versions[version].name = name;
pkg.versions[version].version = version;
pkg.versions[version]._id = name + '@' + version;
pkg.name = name;
pkg['dist-tags'] = {latest: version};
return pkg;
};

View File

@@ -6,10 +6,10 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="shortcut icon" href="/favicon.png">
<!-- Bootstrap -->
<link href="http://cdn.staticfile.org/twitter-bootstrap/3.0.0-rc2/css/bootstrap.min.css" rel="stylesheet" media="screen">
<link href="http://cdn.staticfile.org/prettify/r298/prettify.min.css" rel="stylesheet" media="screen">
<link href="//dn-staticfile.qbox.me/twitter-bootstrap/3.0.0-rc2/css/bootstrap.min.css" rel="stylesheet" media="screen">
<link href="//dn-staticfile.qbox.me/prettify/r298/prettify.min.css" rel="stylesheet" media="screen">
<!-- JavaScript plugins (requires jQuery) -->
<script src="http://cdn.staticfile.org/jquery/2.0.3/jquery.min.js"></script>
<script src="//dn-staticfile.qbox.me/jquery/2.0.3/jquery.min.js"></script>
<link rel="search" type="application/opensearchdescription+xml" href="/opensearch.xml" title="CNPM" />
<style>
a{color:#09f;}
@@ -72,17 +72,17 @@
{{footer}}
</p>
<a href="https://github.com/cnpm/cnpmjs.org" id="fork" target="_blank">
<img alt="Fork me on GitHub" src="http://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png">
<img alt="Fork me on GitHub" src="//s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png">
</a>
</div>
</div>
<!-- Include all compiled plugins (below), or include individual files as needed -->
<script src="http://cdn.staticfile.org/twitter-bootstrap/3.0.0-rc2/js/bootstrap.min.js"></script>
<script src="//dn-staticfile.qbox.me/twitter-bootstrap/3.0.0-rc2/js/bootstrap.min.js"></script>
<!-- Enable responsive features in IE8 with Respond.js (https://github.com/scottjehl/Respond) -->
<script src="http://cdn.staticfile.org/respond.js/1.2.0/respond.min.js"></script>
<script src="//dn-staticfile.qbox.me/respond.js/1.2.0/respond.min.js"></script>
<script src="http://cdn.staticfile.org/prettify/r298/prettify.min.js"></script>
<script src="//dn-staticfile.qbox.me/prettify/r298/prettify.min.js"></script>
<!-- Specific to this page -->
<script>