feat: support PostgreSQL (#733)

closes https://github.com/cnpm/cnpmcore/issues/731

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Added support for PostgreSQL alongside MySQL, including a new database
setup script and comprehensive documentation for local development.
- Introduced a new CI job for PostgreSQL testing in the GitHub Actions
workflow.
- Enhanced the `README.md` and `DEVELOPER.md` files to provide clearer
instructions for using both database systems.
- Added new environment variable configurations for PostgreSQL in the
Docker deployment documentation.

- **Bug Fixes**
- Improved error handling in tests for duplicate entries to accommodate
both MySQL and PostgreSQL error messages.

- **Documentation**
- Updated setup instructions for PostgreSQL and clarified MySQL setup in
the documentation.
	- Enhanced contributor information in the README.
- Expanded instructions for setting up Elasticsearch and Kibana,
including environment variable configurations.

- **Chores**
- Updated package dependencies to include PostgreSQL client libraries
and modified scripts to support both databases.
	- Changed the base image in the Dockerfile to a newer Node.js version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
fengmk2
2024-11-30 21:56:45 +08:00
committed by GitHub
parent dd15b08fa2
commit f240799fa2
37 changed files with 2468 additions and 336 deletions

View File

@@ -10,6 +10,67 @@ on:
branches: [ master ]
jobs:
test-postgresql-fs-nfs:
runs-on: ${{ matrix.os }}
services:
# https://docs.github.com/en/actions/use-cases-and-examples/using-containerized-services/creating-postgresql-service-containers
# Label used to access the service container
postgres:
# Docker Hub image
image: postgres
# Provide the password for postgres
env:
POSTGRES_PASSWORD: postgres
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
# Maps tcp port 5432 on service container to the host
- 5432:5432
redis:
# https://docs.github.com/en/actions/using-containerized-services/about-service-containers#example-mapping-redis-ports
image: redis
ports:
# Opens tcp port 6379 on the host and service container
- 6379:6379
strategy:
fail-fast: false
matrix:
node-version: [18.20.0, 18, 20, 22]
os: [ubuntu-latest]
steps:
- name: Checkout Git Source
uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- name: Install Dependencies
run: npm i -g npminstall && npminstall
- name: Continuous Integration
run: npm run ci:postgresql
env:
# The hostname used to communicate with the PostgreSQL service container
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
# The default PostgreSQL port
POSTGRES_PORT: 5432
- name: Code Coverage
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
test-mysql57-fs-nfs:
runs-on: ${{ matrix.os }}
@@ -37,10 +98,10 @@ jobs:
steps:
- name: Checkout Git Source
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
@@ -88,10 +149,10 @@ jobs:
steps:
- name: Checkout Git Source
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}

View File

@@ -2,7 +2,7 @@
## 环境初始化
本项目的外部服务依赖有MySQL 数据服务、Redis 缓存服务。
本项目的外部服务依赖有MySQL 数据库或 PostgreSQL 数据库、Redis 缓存服务。
可以通过 Docker 来快速启动本地开发环境:
@@ -14,7 +14,7 @@ docker-compose up -d
docker-compose down
```
> 手动初始化依赖服务参见[文档](./docs/setup.md)
> 手动初始化依赖服务参见[本地开发环境 - MySQL](./docs/setup.md) 或 [本地开发环境 - PostgreSQL](./docs/setup-with-postgresql.md)
## 本地开发
@@ -24,11 +24,11 @@ docker-compose down
npm install
```
### 开发运行
### 开发运行 - MySQL
```bash
# 初始化数据库
MYSQL_DATABASE=cnpmcore bash ./prepare-database.sh
CNPMCORE_DATABASE_NAME=cnpmcore bash ./prepare-database-mysql.sh
# 启动 Web 服务
npm run dev
@@ -37,12 +37,33 @@ npm run dev
curl -v http://127.0.0.1:7001
```
### 开发运行 - PostgreSQL
```bash
# 初始化数据库
CNPMCORE_DATABASE_NAME=cnpmcore bash ./prepare-database-postgresql.sh
# 启动 Web 服务
npm run dev:postgresql
# 访问
curl -v http://127.0.0.1:7001
```
### 单元测试
MySQL
```bash
npm run test
```
PostgreSQL
```bash
npm run test:postgresql
```
## 项目结构
```txt
@@ -268,9 +289,9 @@ Repository 依赖 Model然后被 Service 和 Controller 依赖
可能需要涉及3个地方的修改
1. sql/*.sql
2. repository/model/*.ts
3. core/entity/*.ts
1. `sql/mysql/*.sql`, `sql/postgresql/*.sql`
2. `repository/model/*.ts`
3. `core/entity/*.ts`
目前还不会做 Model 到 SQL 的自动转换生成,核心原因有:

View File

@@ -1,4 +1,4 @@
FROM node:18
FROM node:20
# Create app directory
WORKDIR /usr/src/app

View File

@@ -5,7 +5,7 @@
[![CodeQL](https://github.com/cnpm/cnpmcore/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/cnpm/cnpmcore/actions/workflows/codeql-analysis.yml)
[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fcnpm%2Fcnpmcore.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Fcnpm%2Fcnpmcore?ref=badge_shield)
Reimplementation based on [cnpmjs.org](https://github.com/cnpm/cnpmjs.org) with TypeScript.
Reimplement based on [cnpmjs.org](https://github.com/cnpm/cnpmjs.org) with TypeScript.
## Registry HTTP API
@@ -23,19 +23,10 @@ See [INTEGRATE.md](INTEGRATE.md)
[MIT](LICENSE)
<!-- GITCONTRIBUTOR_START -->
## Contributors
|[<img src="https://avatars.githubusercontent.com/u/156269?v=4" width="100px;"/><br/><sub><b>fengmk2</b></sub>](https://github.com/fengmk2)<br/>|[<img src="https://avatars.githubusercontent.com/u/32174276?v=4" width="100px;"/><br/><sub><b>semantic-release-bot</b></sub>](https://github.com/semantic-release-bot)<br/>|[<img src="https://avatars.githubusercontent.com/u/5574625?v=4" width="100px;"/><br/><sub><b>elrrrrrrr</b></sub>](https://github.com/elrrrrrrr)<br/>|[<img src="https://avatars.githubusercontent.com/u/6897780?v=4" width="100px;"/><br/><sub><b>killagu</b></sub>](https://github.com/killagu)<br/>|[<img src="https://avatars.githubusercontent.com/u/35598090?v=4" width="100px;"/><br/><sub><b>hezhengxu2018</b></sub>](https://github.com/hezhengxu2018)<br/>|[<img src="https://avatars.githubusercontent.com/u/13284978?v=4" width="100px;"/><br/><sub><b>Beace</b></sub>](https://github.com/Beace)<br/>|
| :---: | :---: | :---: | :---: | :---: | :---: |
|[<img src="https://avatars.githubusercontent.com/u/4635838?v=4" width="100px;"/><br/><sub><b>gemwuu</b></sub>](https://github.com/gemwuu)<br/>|[<img src="https://avatars.githubusercontent.com/u/26033663?v=4" width="100px;"/><br/><sub><b>Zian502</b></sub>](https://github.com/Zian502)<br/>|[<img src="https://avatars.githubusercontent.com/u/17879221?v=4" width="100px;"/><br/><sub><b>laibao101</b></sub>](https://github.com/laibao101)<br/>|[<img src="https://avatars.githubusercontent.com/u/3478550?v=4" width="100px;"/><br/><sub><b>coolyuantao</b></sub>](https://github.com/coolyuantao)<br/>|[<img src="https://avatars.githubusercontent.com/u/10163680?v=4" width="100px;"/><br/><sub><b>Wellaiyo</b></sub>](https://github.com/Wellaiyo)<br/>|[<img src="https://avatars.githubusercontent.com/u/227713?v=4" width="100px;"/><br/><sub><b>atian25</b></sub>](https://github.com/atian25)<br/>|
|[<img src="https://avatars.githubusercontent.com/u/33210001?v=4" width="100px;"/><br/><sub><b>hljwkwm</b></sub>](https://github.com/hljwkwm)<br/>|[<img src="https://avatars.githubusercontent.com/u/8198408?v=4" width="100px;"/><br/><sub><b>BlackHole1</b></sub>](https://github.com/BlackHole1)<br/>|[<img src="https://avatars.githubusercontent.com/u/1814071?v=4" width="100px;"/><br/><sub><b>xiekw2010</b></sub>](https://github.com/xiekw2010)<br/>|[<img src="https://avatars.githubusercontent.com/u/7054676?v=4" width="100px;"/><br/><sub><b>Zheaoli</b></sub>](https://github.com/Zheaoli)<br/>|[<img src="https://avatars.githubusercontent.com/u/13471233?v=4" width="100px;"/><br/><sub><b>OpportunityLiu</b></sub>](https://github.com/OpportunityLiu)<br/>|[<img src="https://avatars.githubusercontent.com/u/958063?v=4" width="100px;"/><br/><sub><b>thonatos</b></sub>](https://github.com/thonatos)<br/>|
|[<img src="https://avatars.githubusercontent.com/u/26962197?v=4" width="100px;"/><br/><sub><b>chilingling</b></sub>](https://github.com/chilingling)<br/>|[<img src="https://avatars.githubusercontent.com/u/11039003?v=4" width="100px;"/><br/><sub><b>chenpx976</b></sub>](https://github.com/chenpx976)<br/>|[<img src="https://avatars.githubusercontent.com/u/29791463?v=4" width="100px;"/><br/><sub><b>fossabot</b></sub>](https://github.com/fossabot)<br/>|[<img src="https://avatars.githubusercontent.com/u/1119126?v=4" width="100px;"/><br/><sub><b>looksgood</b></sub>](https://github.com/looksgood)<br/>|[<img src="https://avatars.githubusercontent.com/u/23701019?v=4" width="100px;"/><br/><sub><b>laoboxie</b></sub>](https://github.com/laoboxie)<br/>|[<img src="https://avatars.githubusercontent.com/u/5772358?v=4" width="100px;"/><br/><sub><b>unbyte</b></sub>](https://github.com/unbyte)<br/>|
[<img src="https://avatars.githubusercontent.com/u/5799374?v=4" width="100px;"/><br/><sub><b>wandergis</b></sub>](https://github.com/wandergis)<br/>|[<img src="https://avatars.githubusercontent.com/u/13448833?v=4" width="100px;"/><br/><sub><b>windhc</b></sub>](https://github.com/windhc)<br/>|[<img src="https://avatars.githubusercontent.com/u/2784308?v=4" width="100px;"/><br/><sub><b>yisibl</b></sub>](https://github.com/yisibl)<br/>|[<img src="https://avatars.githubusercontent.com/u/13127586?v=4" width="100px;"/><br/><sub><b>vimplus</b></sub>](https://github.com/vimplus)<br/>|[<img src="https://avatars.githubusercontent.com/u/5550931?v=4" width="100px;"/><br/><sub><b>feichao93</b></sub>](https://github.com/feichao93)<br/>
[![Contributors](https://contrib.rocks/image?repo=cnpm/cnpmcore)](https://github.com/cnpm/cnpmcore/graphs/contributors)
This project follows the git-contributor [spec](https://github.com/xudafeng/git-contributor), auto updated at `Fri May 17 2024 22:31:22 GMT+0800`.
<!-- GITCONTRIBUTOR_END -->
Made with [contributors-img](https://contrib.rocks).
[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fcnpm%2Fcnpmcore.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fcnpm%2Fcnpmcore?ref=badge_large)

View File

@@ -10,6 +10,7 @@ import { BadRequestError, ForbiddenError, NotFoundError } from 'egg-errors';
import { RequireAtLeastOne } from 'type-fest';
import npa from 'npm-package-arg';
import semver from 'semver';
import pMap from 'p-map';
import {
calculateIntegrity,
detectInstallScript,
@@ -23,6 +24,7 @@ import { AbbreviatedPackageJSONType, AbbreviatedPackageManifestType, PackageJSON
import { PackageVersionBlockRepository } from '../../repository/PackageVersionBlockRepository';
import { PackageVersionDownloadRepository } from '../../repository/PackageVersionDownloadRepository';
import { DistRepository } from '../../repository/DistRepository';
import { isDuplicateKeyError } from '../../repository/util/ErrorUtil';
import { Package } from '../entity/Package';
import { PackageVersion } from '../entity/PackageVersion';
import { PackageVersionBlock } from '../entity/PackageVersionBlock';
@@ -47,7 +49,6 @@ import { BugVersion } from '../entity/BugVersion';
import { RegistryManagerService } from './RegistryManagerService';
import { Registry } from '../entity/Registry';
import { PackageVersionService } from './PackageVersionService';
import pMap from 'p-map';
export interface PublishPackageCmd {
// maintainer: Maintainer;
@@ -271,7 +272,7 @@ export class PackageManagerService extends AbstractService {
try {
await this.packageRepository.createPackageVersion(pkgVersion);
} catch (e) {
if (e.code === 'ER_DUP_ENTRY') {
if (isDuplicateKeyError(e)) {
throw new ForbiddenError(`Can't modify pre-existing version: ${pkg.fullname}@${cmd.version}`);
}
throw e;

View File

@@ -21,6 +21,7 @@ import {
import { PackageVersionFileRepository } from '../../repository/PackageVersionFileRepository';
import { PackageVersionRepository } from '../../repository/PackageVersionRepository';
import { DistRepository } from '../../repository/DistRepository';
import { isDuplicateKeyError } from '../../repository/util/ErrorUtil';
import { PackageVersionFile } from '../entity/PackageVersionFile';
import { PackageVersion } from '../entity/PackageVersion';
import { Package } from '../entity/Package';
@@ -272,7 +273,9 @@ export class PackageVersionFileService extends AbstractService {
file.packageVersionFileId, dist.size, file.path);
} catch (err) {
// ignore Duplicate entry
if (err.code === 'ER_DUP_ENTRY') return file;
if (isDuplicateKeyError(err)) {
return file;
}
throw err;
}
return file;

View File

@@ -1,4 +1,5 @@
import { SyncDeleteMode, SyncMode, ChangesStreamMode } from '../common/constants';
import { DATABASE_TYPE } from '../../config/database';
export { cnpmcoreConfig } from '../../config/config.default';
@@ -94,7 +95,7 @@ export type CnpmcoreConfig = {
/**
* white scope list
*/
allowScopes: string [],
allowScopes: string[],
/**
* allow publish non-scope package, disable by default
*/
@@ -175,4 +176,11 @@ export type CnpmcoreConfig = {
* strictly enforces/validates dependencies version when publish or sync
*/
strictValidatePackageDeps?: boolean,
/**
* database config
*/
database: {
type: DATABASE_TYPE | string,
},
};

View File

@@ -18,6 +18,7 @@ import type { User as UserModel } from './model/User';
import { User as UserEntity } from '../core/entity/User';
import { AbstractRepository } from './AbstractRepository';
import { BugVersionPackages } from '../core/entity/BugVersion';
import { DATABASE_TYPE } from '../../config/database';
export type PackageManifestType = Pick<PackageJSONType, PackageJSONPickKey> & {
_id: string;
@@ -406,18 +407,25 @@ export class PackageRepository extends AbstractRepository {
return ModelConvertor.convertModelToEntity(model, this.PackageVersionManifest);
}
private getCountSql(model: typeof Bone):string {
const { database } = this.config.orm;
const sql = `
SELECT
TABLE_ROWS
FROM
information_schema.tables
WHERE
table_schema = '${database}'
AND table_name = '${model.table}'
`;
return sql;
private async getTotalCountByModel(model: typeof Bone): Promise<number> {
if (this.config.cnpmcore.database.type === DATABASE_TYPE.MySQL) {
const { database } = this.config.orm as { database: string };
const sql = `
SELECT
TABLE_ROWS as table_rows
FROM
information_schema.tables
WHERE
table_schema = '${database}'
AND table_name = '${model.table}';
`;
const result = await this.orm.client.query(sql);
return result.rows?.[0].table_rows as number;
}
const sql = `SELECT count(id) as total FROM ${model.table};`;
const result = await this.orm.client.query(sql);
const total = Number(result.rows?.[0].total);
return total;
}
public async queryTotal() {
@@ -432,8 +440,7 @@ export class PackageRepository extends AbstractRepository {
lastPackage = lastPkg.scope ? `${lastPkg.scope}/${lastPkg.name}` : lastPkg.name;
// FIXME: id will be out of range number
// 可能存在 id 增长不连续的情况,通过 count 查询
const queryRes = await this.orm.client.query(this.getCountSql(PackageModel));
packageCount = queryRes.rows?.[0].TABLE_ROWS as number;
packageCount = await this.getTotalCountByModel(PackageModel);
}
if (lastVersion) {
@@ -442,8 +449,7 @@ export class PackageRepository extends AbstractRepository {
const fullname = pkg.scope ? `${pkg.scope}/${pkg.name}` : pkg.name;
lastPackageVersion = `${fullname}@${lastVersion.version}`;
}
const queryRes = await this.orm.client.query(this.getCountSql(PackageVersionModel));
packageVersionCount = queryRes.rows?.[0].TABLE_ROWS as number;
packageVersionCount = await this.getTotalCountByModel(PackageVersionModel);
}
return {
packageCount,

View File

@@ -1,12 +1,13 @@
import { strict as assert } from 'node:assert';
import { uniq } from 'lodash';
import { AccessLevel, SingletonProto, Inject } from '@eggjs/tegg';
import { ModelConvertor } from './util/ModelConvertor';
import { isDuplicateKeyError } from './util/ErrorUtil';
import type { Task as TaskModel } from './model/Task';
import type { HistoryTask as HistoryTaskModel } from './model/HistoryTask';
import { Task as TaskEntity, TaskUpdateCondition } from '../core/entity/Task';
import { AbstractRepository } from './AbstractRepository';
import { TaskType, TaskState } from '../../app/common/enum/Task';
import { uniq } from 'lodash';
import { Task as TaskEntity, TaskUpdateCondition } from '../core/entity/Task';
@SingletonProto({
accessLevel: AccessLevel.PUBLIC,
@@ -28,7 +29,7 @@ export class TaskRepository extends AbstractRepository {
await ModelConvertor.convertEntityToModel(task, this.Task);
} catch (e) {
e.message = '[TaskRepository] insert Task failed: ' + e.message;
if (e.code === 'ER_DUP_ENTRY') {
if (isDuplicateKeyError(e)) {
this.logger.warn(e);
const taskModel = await this.Task.findOne({ bizId: task.bizId });
// 覆盖 bizId 相同的 id 和 taskId

View File

@@ -0,0 +1,10 @@
export function isDuplicateKeyError(err: any) {
if (err.code === 'ER_DUP_ENTRY') {
return true;
}
if (err.message.includes('duplicate key value violates unique constraint')) {
// pg: duplicate key value violates unique constraint "tasks_uk_task_id"
// code: '23505'
return true;
}
}

View File

@@ -5,7 +5,8 @@ import { EggAppConfig, PowerPartial } from 'egg';
import OSSClient from 'oss-cnpm';
import { patchAjv } from '../app/port/typebox';
import { ChangesStreamMode, NOT_IMPLEMENTED_PATH, SyncDeleteMode, SyncMode } from '../app/common/constants';
import { CnpmcoreConfig } from '../app/port/config';
import type { CnpmcoreConfig } from '../app/port/config';
import { database } from './database';
export const cnpmcoreConfig: CnpmcoreConfig = {
name: 'cnpm',
@@ -60,6 +61,9 @@ export const cnpmcoreConfig: CnpmcoreConfig = {
elasticsearchIndex: 'cnpmcore_packages',
strictValidateTarballPkg: false,
strictValidatePackageDeps: false,
database: {
type: database.type,
},
};
export default (appInfo: EggAppConfig) => {
@@ -70,19 +74,17 @@ export default (appInfo: EggAppConfig) => {
// override config from framework / plugin
config.dataDir = process.env.CNPMCORE_DATA_DIR || join(appInfo.root, '.cnpmcore');
config.orm = {
client: 'mysql2',
database: process.env.CNPMCORE_MYSQL_DATABASE || process.env.MYSQL_DATABASE || 'cnpmcore',
host: process.env.CNPMCORE_MYSQL_HOST || process.env.MYSQL_HOST || '127.0.0.1',
port: process.env.CNPMCORE_MYSQL_PORT || process.env.MYSQL_PORT || 3306,
user: process.env.CNPMCORE_MYSQL_USER || process.env.MYSQL_USER || 'root',
password: process.env.CNPMCORE_MYSQL_PASSWORD || process.env.MYSQL_PASSWORD,
...database,
database: database.name ?? 'cnpmcore',
charset: 'utf8mb4',
logger: {
// https://github.com/cyjake/leoric/blob/master/docs/zh/logging.md#logqueryerror
// ignore query error
logQueryError() {},
// logQueryError(...args: any[]) {
// console.log(args);
// },
},
};

View File

@@ -1,6 +1,7 @@
import { join } from 'path';
import { EggAppConfig, PowerPartial } from 'egg';
import Mock from '@elastic/elasticsearch-mock';
import { database } from './database';
export const mockES = new Mock();
@@ -9,7 +10,7 @@ export default (appInfo: EggAppConfig) => {
config.dataDir = join(appInfo.root, '.cnpmcore_unittest');
config.orm = {
database: process.env.MYSQL_DATABASE || 'cnpmcore_unittest',
database: database.name ?? 'cnpmcore_unittest',
};
config.nfs = {

44
config/database.ts Normal file
View File

@@ -0,0 +1,44 @@
export enum DATABASE_TYPE {
MySQL = 'MySQL',
PostgreSQL = 'PostgreSQL',
SQLite = 'SQLite',
}
const dbType = process.env.CNPMCORE_DATABASE_TYPE ?? DATABASE_TYPE.MySQL;
let dbName = process.env.CNPMCORE_DATABASE_NAME;
let dbHost = process.env.CNPMCORE_DATABASE_HOST;
let dbPort = process.env.CNPMCORE_DATABASE_PORT;
let dbUser = process.env.CNPMCORE_DATABASE_USER;
let dbPassword = process.env.CNPMCORE_DATABASE_PASSWORD;
let dialect = 'mysql';
let dbClient = 'mysql2';
if (dbType === DATABASE_TYPE.MySQL) {
// Compatible mysql configurations
dbName = dbName ?? process.env.CNPMCORE_MYSQL_DATABASE ?? process.env.MYSQL_DATABASE;
dbHost = dbHost ?? process.env.CNPMCORE_MYSQL_HOST ?? process.env.MYSQL_HOST ?? '127.0.0.1';
dbPort = dbPort ?? process.env.CNPMCORE_MYSQL_PORT ?? process.env.MYSQL_PORT ?? '3306';
dbUser = dbUser ?? process.env.CNPMCORE_MYSQL_USER ?? process.env.MYSQL_USER ?? 'root';
dbPassword = dbPassword ?? process.env.CNPMCORE_MYSQL_PASSWORD ?? process.env.MYSQL_PASSWORD;
} else if (dbType === DATABASE_TYPE.PostgreSQL) {
dbClient = 'pg';
dialect = 'postgres';
dbHost = dbHost ?? process.env.CNPMCORE_POSTGRES_HOST ?? process.env.POSTGRES_HOST;
dbPort = dbPort ?? process.env.CNPMCORE_POSTGRES_PORT ?? process.env.POSTGRES_PORT ?? '5432';
dbUser = dbUser ?? process.env.CNPMCORE_POSTGRES_USER ?? process.env.POSTGRES_USER;
dbPassword = dbPassword ?? process.env.CNPMCORE_POSTGRES_PASSWORD ?? process.env.POSTGRES_PASSWORD;
} else if (dbType === DATABASE_TYPE.SQLite) {
// TODO
dbClient = 'sqlite';
dialect = 'sqlite';
}
export const database = {
type: dbType,
dialect,
client: dbClient,
name: dbName,
host: dbHost,
port: dbPort,
user: dbUser,
password: dbPassword,
};

View File

@@ -13,11 +13,23 @@ docker build -t cnpmcore .
### MySQL
```bash
CNPMCORE_MYSQL_DATABASE=cnpmcore
CNPMCORE_MYSQL_HOST=127.0.0.1
CNPMCORE_MYSQL_PORT=3306
CNPMCORE_MYSQL_USER=your-db-user-name
CNPMCORE_MYSQL_PASSWORD=your-db-user-password
CNPMCORE_DATABASE_TYPE=MySQL
CNPMCORE_DATABASE_NAME=cnpmcore
CNPMCORE_DATABASE_HOST=127.0.0.1
CNPMCORE_DATABASE_PORT=3306
CNPMCORE_DATABASE_USER=your-db-user-name
CNPMCORE_DATABASE_PASSWORD=your-db-user-password
```
### PostgreSQL
```bash
CNPMCORE_DATABASE_TYPE=PostgreSQL
CNPMCORE_DATABASE_NAME=cnpmcore
CNPMCORE_DATABASE_HOST=127.0.0.1
CNPMCORE_DATABASE_PORT=5432
CNPMCORE_DATABASE_USER=your-db-user-name
CNPMCORE_DATABASE_PASSWORD=your-db-user-password
```
### Redis
@@ -134,14 +146,43 @@ docker run -p 7001:7001 -it --rm \
## 运行容器
### 基于 MySQL 运行
```bash
docker run -p 7001:7001 -it --rm \
-e CNPMCORE_CONFIG_REGISTRY=https://your-registry.com \
-e CNPMCORE_MYSQL_DATABASE=cnpmcore \
-e CNPMCORE_MYSQL_HOST=127.0.0.1 \
-e CNPMCORE_MYSQL_PORT=3306 \
-e CNPMCORE_MYSQL_USER=your-db-user-name \
-e CNPMCORE_MYSQL_PASSWORD=your-db-user-password \
-e CNPMCORE_DATABASE_TYPE=MySQL \
-e CNPMCORE_DATABASE_NAME=cnpmcore \
-e CNPMCORE_DATABASE_HOST=127.0.0.1 \
-e CNPMCORE_DATABASE_PORT=3306 \
-e CNPMCORE_DATABASE_USER=your-db-user-name \
-e CNPMCORE_DATABASE_PASSWORD=your-db-user-password \
-e CNPMCORE_NFS_TYPE=s3 \
-e CNPMCORE_NFS_S3_CLIENT_ENDPOINT=https://your-s3-endpoint \
-e CNPMCORE_NFS_S3_CLIENT_BUCKET=your-bucket-name \
-e CNPMCORE_NFS_S3_CLIENT_ID=s3-ak \
-e CNPMCORE_NFS_S3_CLIENT_SECRET=s3-sk \
-e CNPMCORE_NFS_S3_CLIENT_FORCE_PATH_STYLE=true \
-e CNPMCORE_NFS_S3_CLIENT_DISABLE_URL=true \
-e CNPMCORE_REDIS_HOST=127.0.0.1 \
-e CNPMCORE_REDIS_PORT=6379 \
-e CNPMCORE_REDIS_PASSWORD=your-redis-password \
-e CNPMCORE_REDIS_DB=1 \
-e TZ=Asia/Shanghai \
--name cnpmcore-prod cnpmcore
```
### 基于 PostgreSQL 运行
```bash
docker run -p 7001:7001 -it --rm \
-e CNPMCORE_CONFIG_REGISTRY=https://your-registry.com \
-e CNPMCORE_DATABASE_TYPE=PostgreSQL \
-e CNPMCORE_DATABASE_NAME=cnpmcore \
-e CNPMCORE_DATABASE_HOST=127.0.0.1 \
-e CNPMCORE_DATABASE_PORT=5432 \
-e CNPMCORE_DATABASE_USER=your-db-user-name \
-e CNPMCORE_DATABASE_PASSWORD=your-db-user-password \
-e CNPMCORE_NFS_TYPE=s3 \
-e CNPMCORE_NFS_S3_CLIENT_ENDPOINT=https://your-s3-endpoint \
-e CNPMCORE_NFS_S3_CLIENT_BUCKET=your-bucket-name \
@@ -161,7 +202,7 @@ docker run -p 7001:7001 -it --rm \
https://registry-demo.fengmk2.com:9443
管理员账号cnpmcore_admin/12345678
管理员账号:`cnpmcore_admin/12345678`
通过 npm login 可以登录

View File

@@ -1,6 +1,7 @@
# 本地搭建 ES 搜索环境
## 单机搭建
### 下载安装 ES
首先我们进入 [ES 下载的官方网站](https://www.elastic.co/cn/downloads/elasticsearch) ,选择合适的操作系统版本并下载。下载完成后再适当位置解压并运行
@@ -50,7 +51,6 @@ path.logs: ./logs
此时,访问 http://localhost:5601 ,即可看到 Kibana 引导页面。
我们仅仅将其作为一个可视化的操作 API 的可视化工具,可以跳过其引导,访问 `/app/dev_tools#/console` 进入 devtool 页面。
## docker compose
@@ -247,14 +247,13 @@ ENCRYPTION_KEY=c34d38b3a14956121ff2170e5030b471551370178f43e5626eec58b04a30fae2
执行如下命令,启动服务
```bash
$ docker compose up
docker compose up
```
### 访问 Elastic
浏览器打开 http://localhost:5601/app/dev_tools#/console,默认账号为 `elastic` 密码为 .env 文件中定义的 `abcdef`
## 创建索引
ES 可以通过 Kibana devtool 进行数据的写入和查询操作。下面创建一个索引,`cnpmcore_packages` 为索引名称。
@@ -982,8 +981,9 @@ $ curl -X PUT https://r.cnpmjs.org/-/v1/search/sync/${pkgName}
### 删除一条数据
注意需要添加管理员 token管理员在本地进行登录后可通过查询 `~/.npmrc` 查看
```bash
$ curl -X DELETE -H 'Authorization: Bearer ${token}' http://localhost:7001/-/v1/search/${pkgName}
curl -X DELETE -H 'Authorization: Bearer ${token}' http://localhost:7001/-/v1/search/${pkgName}
```
### 修改数据
@@ -993,5 +993,5 @@ $ curl -X DELETE -H 'Authorization: Bearer ${token}' http://localhost:7001/-/v1/
### 查询
```bash
$ npm search colors --registry=http://localhost:7001
npm search colors --registry=http://localhost:7001
```

View File

@@ -0,0 +1,36 @@
# 本地开发环境 - PostgreSQL
## 依赖列表
本项目的外部服务依赖有:
- 数据库PostgreSQL
- 缓存Redis
## 手动初始化
假设大家使用 macOS 开发Linux 和 Windows 环境自行参考。
### PostgreSQL 17
> https://wiki.postgresql.org/wiki/Homebrew
```bash
brew install postgresql@17
brew services start postgresql@17
echo 'export PATH="/opt/homebrew/opt/postgresql@17/bin:$PATH"' >> ~/.zshrc
```
验证是否安装成功
```bash
psql postgres
```
### Redis
```bash
brew install redis
brew services start redis
```

View File

@@ -1,4 +1,4 @@
# 本地开发环境
# 本地开发环境 - MySQL
## 依赖列表

View File

@@ -1,7 +1,7 @@
{
"name": "cnpmcore",
"version": "3.69.0",
"description": "npm core",
"description": "Private NPM Registry for Enterprise",
"files": [
"dist/**/*"
],
@@ -35,18 +35,24 @@
}
},
"scripts": {
"contributor": "git-contributor",
"dev": "egg-bin dev",
"dev:postgresql": "CNPMCORE_DATABASE_TYPE=PostgreSQL egg-bin dev",
"lint": "eslint --cache --ext .ts .",
"lint:fix": "eslint --cache --ext .ts --fix .",
"test": "npm run lint:fix && npm run test-local",
"pretest-local": "bash ./prepare-database.sh",
"test-local": "egg-bin test",
"pret": "bash ./prepare-database.sh",
"test:postgresql": "npm run lint:fix && npm run test:local:postgresql",
"pretest:local:postgresql": "bash prepare-database-postgresql.sh",
"test:local:postgresql": "CNPMCORE_DATABASE_TYPE=PostgreSQL egg-bin test",
"test": "npm run lint:fix && npm run test:local",
"pretest:local": "bash prepare-database-mysql.sh",
"test:local": "egg-bin test",
"pret": "bash prepare-database-mysql.sh",
"t": "npm run lint:fix && egg-bin test --changed",
"precov": "bash ./prepare-database.sh",
"precov": "bash prepare-database-mysql.sh",
"cov": "egg-bin cov",
"precov:postgresql": "bash prepare-database-postgresql.sh",
"cov:postgresql": "CNPMCORE_DATABASE_TYPE=PostgreSQL egg-bin cov",
"ci": "npm run lint && npm run cov && npm run tsc:prod",
"ci:postgresql": "npm run lint && npm run cov:postgresql && npm run tsc:prod",
"clean": "tsc -b --clean && rm -rf dist",
"tsc": "npm run clean && tsc -p ./tsconfig.json",
"tsc:prod": "npm run clean && tsc -p ./tsconfig.prod.json",
@@ -87,7 +93,7 @@
"base64url": "^3.0.1",
"bson-objectid": "^2.0.1",
"dayjs": "^1.10.7",
"egg": "^3.9.2",
"egg": "^3.29.0",
"egg-cors": "^3.0.0",
"egg-errors": "^2.3.0",
"egg-redis": "^2.4.0",
@@ -107,6 +113,7 @@
"npm-package-arg": "^10.1.0",
"oss-cnpm": "^5.0.1",
"p-map": "^4.0.0",
"pg": "^8.13.1",
"s3-cnpmcore": "^1.1.2",
"semver": "^7.3.5",
"ssri": "^8.0.1",
@@ -127,6 +134,7 @@
"@types/mysql": "^2.15.21",
"@types/node-rsa": "^1.1.4",
"@types/npm-package-arg": "^6.1.1",
"@types/pg": "^8.11.10",
"@types/semver": "^7.3.12",
"@types/tar": "^6.1.4",
"@types/ua-parser-js": "^0.7.36",
@@ -136,7 +144,6 @@
"egg-mock": "^5.10.4",
"eslint": "^8.29.0",
"eslint-config-egg": "^13.0.0",
"git-contributor": "2",
"typescript": "5.2.2"
},
"author": "killagu",

View File

@@ -1,13 +1,13 @@
#!/bin/bash
# read variables from environment
db_host=${MYSQL_HOST:-127.0.0.1}
db_port=${MYSQL_PORT:-3306}
db_username=${MYSQL_USER:-root}
db_password=${MYSQL_PASSWORD:-} # default to empty password
db_name=${MYSQL_DATABASE:-cnpmcore_unittest}
db_host=${CNPMCORE_DATABASE_HOST:-127.0.0.1}
db_port=${CNPMCORE_DATABASE_PORT:-3306}
db_username=${CNPMCORE_DATABASE_USER:-root}
db_password=${CNPMCORE_DATABASE_PASSWORD:-} # default to empty password
db_name=${CNPMCORE_DATABASE_NAME:-cnpmcore_unittest}
# preapre mysql param
# prepare MySQL param
param="-h $db_host -P $db_port -u $db_username"
if [ -n "$db_password" ]; then
param="$param -p$db_password"
@@ -17,14 +17,14 @@ if [ "$CI" ]; then
echo "⛷️ Skipping database creation in CI environment."
else
# reset database
echo "️😈 Reset database in local"
echo "️😈 Reset database $db_name in local"
mysql $param -e "DROP DATABASE IF EXISTS $db_name"
mysql $param -e "CREATE DATABASE $db_name CHARACTER SET utf8"
fi
# find all sql files and sort
sql_files=$(ls sql/*.sql | sort)
sql_files=$(ls sql/mysql/*.sql | sort)
echo "🤖 Running the following SQL files:"
# execute sql files
@@ -33,5 +33,5 @@ for file in $sql_files; do
mysql $param $db_name < "$file"
done
echo "🎉 prepare database done"
echo "🎉 prepare database $db_name done"
mysql $param -e "USE $db_name; SHOW TABLES;"

View File

@@ -0,0 +1,47 @@
#!/bin/bash
# set -ex
# read variables from environment
db_host=${POSTGRES_HOST:-}
db_port=${POSTGRES_PORT:-5432}
db_username=${POSTGRES_USER:-}
db_password=${POSTGRES_PASSWORD:-} # default to empty password
db_name=${CNPMCORE_DATABASE_NAME:-cnpmcore_unittest}
# prepare PostgreSQL param
param=""
if [ -n "$db_host" ]; then
param="$param --host=$db_host"
fi
if [ -n "$db_port" ]; then
param="$param --port=$db_port"
fi
if [ -n "$db_username" ]; then
param="$param --username=$db_username"
fi
if [ -n "$db_password" ]; then
# https://stackoverflow.com/questions/6405127/how-do-i-specify-a-password-to-psql-non-interactively
export PGPASSWORD=$db_password
fi
# reset database
echo "️😈 Reset database $db_name in local"
dropdb $param $db_name || echo "ignore database not exists"
# http://www.postgres.cn/docs/15/app-createdb.html
createdb $param --echo --encoding=UTF8 $db_name
# find all sql files and sort
sql_files=$(ls sql/postgresql/*.sql | sort)
echo "🤖 Running the following SQL files:"
# execute sql files
for file in $sql_files; do
echo "🔖 Running $file..."
# psql $param --dbname=$db_name --file=$file --echo-all
psql $param --dbname=$db_name --file=$file --quiet
done
echo "🎉 prepare database $db_name done"
# psql $param --dbname=$db_name -c "SELECT * FROM pg_catalog.pg_tables where schemaname = 'public';"
psql $param --dbname=$db_name -c "\dt"

434
sql/ddl_mysql.sql Normal file
View File

@@ -0,0 +1,434 @@
CREATE TABLE `binaries` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`binary_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'binary id',
`category` varchar(50) CHARACTER SET ascii COLLATE ascii_general_ci NOT NULL COMMENT 'binary category, e.g.: node, sass',
`parent` varchar(500) CHARACTER SET ascii COLLATE ascii_general_ci NOT NULL COMMENT 'binary parent name, e.g.: /, /v1.0.0/, /v1.0.0/docs/',
`name` varchar(200) CHARACTER SET ascii COLLATE ascii_general_ci NOT NULL COMMENT 'binary name, dir should ends with /',
`is_dir` tinyint NOT NULL DEFAULT '0' COMMENT 'is dir or not, 1: true, other: false',
`size` int unsigned NOT NULL COMMENT 'file size',
`date` varchar(100) COLLATE utf8_unicode_ci NOT NULL COMMENT 'date display string',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_binary_id` (`binary_id`),
UNIQUE KEY `uk_category_parent_name` (`category`,`parent`,`name`),
KEY `idx_category_parent` (`category`,`parent`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='binary info'
;
CREATE TABLE `changes` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`change_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'change id',
`type` varchar(50) COLLATE utf8_unicode_ci NOT NULL COMMENT 'change type',
`target_name` varchar(214) COLLATE utf8_unicode_ci NOT NULL COMMENT 'target name, like package name / user name',
`data` json DEFAULT NULL COMMENT 'change params',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_change_id` (`change_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='change info'
;
CREATE TABLE `dists` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`dist_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'dist id',
`name` varchar(428) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL COMMENT 'dist name, 2x size of package name',
`path` varchar(767) CHARACTER SET ascii COLLATE ascii_general_ci NOT NULL COMMENT 'access path',
`size` int unsigned NOT NULL COMMENT 'file size',
`shasum` varchar(512) COLLATE utf8_unicode_ci NOT NULL COMMENT 'dist shasum',
`integrity` varchar(512) COLLATE utf8_unicode_ci NOT NULL COMMENT 'dist integrity',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_dist_id` (`dist_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='dist info'
;
CREATE TABLE `history_tasks` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`task_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'task id',
`type` varchar(20) COLLATE utf8_unicode_ci NOT NULL COMMENT 'task type',
`state` varchar(20) COLLATE utf8_unicode_ci NOT NULL COMMENT 'task state',
`target_name` varchar(214) COLLATE utf8_unicode_ci NOT NULL COMMENT 'target name, like package name / user name',
`author_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'create task user id',
`author_ip` varchar(100) COLLATE utf8_unicode_ci NOT NULL COMMENT 'create task user request ip',
`data` json DEFAULT NULL COMMENT 'task params',
`log_path` varchar(512) COLLATE utf8_unicode_ci NOT NULL COMMENT 'access path',
`log_store_position` varchar(10) COLLATE utf8_unicode_ci NOT NULL COMMENT 'cloud store disk position',
`attempts` int unsigned DEFAULT '0' COMMENT 'task execute attempts times',
`error` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci COMMENT 'error description',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_task_id` (`task_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='history task info'
;
CREATE TABLE `hooks` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`hook_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'hook id',
`type` varchar(20) COLLATE utf8_unicode_ci NOT NULL COMMENT 'hook type, scope, name, owner',
`name` varchar(428) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'hook name',
`owner_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'hook owner id',
`endpoint` varchar(2048) COLLATE utf8_unicode_ci NOT NULL COMMENT 'hook url',
`secret` varchar(200) COLLATE utf8_unicode_ci NOT NULL COMMENT 'sign secret',
`latest_task_id` varchar(24) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'latest task id',
`enable` tinyint NOT NULL DEFAULT '0' COMMENT 'hook is enable not, 1: true, other: false',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_type_name_owner_id` (`type`,`name`,`owner_id`),
KEY `idx_type_name_id` (`type`,`name`,`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='task info'
;
CREATE TABLE `maintainers` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id',
`user_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'user id',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_id_user_id` (`package_id`,`user_id`),
KEY `idx_package_id` (`package_id`),
KEY `idx_user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package maintainers'
;
CREATE TABLE `package_deps` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_version_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version id',
`package_dep_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package dep id',
`scope` varchar(214) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package scope',
`name` varchar(214) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'package name',
`spec` varchar(100) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package dep spec',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_dep_id` (`package_dep_id`),
UNIQUE KEY `uk_package_version_id_scope_name` (`package_version_id`,`scope`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package dependency info'
;
CREATE TABLE `package_tags` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id',
`package_tag_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package tag id',
`tag` varchar(214) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package tag',
`version` varchar(256) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_tag_id` (`package_tag_id`),
UNIQUE KEY `uk_package_tag` (`package_id`,`tag`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package tag info'
;
CREATE TABLE `package_version_blocks` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_version_block_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version block id',
`package_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id',
`version` varchar(256) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version, "*" meaning all versions',
`reason` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL COMMENT 'block reason',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_version_block_id` (`package_version_block_id`),
UNIQUE KEY `uk_name_version` (`package_id`,`version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='blocklist package versions'
;
CREATE TABLE `package_version_downloads` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`year_month` int unsigned NOT NULL COMMENT 'YYYYMM format',
`package_id` varchar(214) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id, maybe scope name',
`version` varchar(256) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version',
`d01` int unsigned NOT NULL DEFAULT '0' COMMENT '01 download count',
`d02` int unsigned NOT NULL DEFAULT '0' COMMENT '02 download count',
`d03` int unsigned NOT NULL DEFAULT '0' COMMENT '03 download count',
`d04` int unsigned NOT NULL DEFAULT '0' COMMENT '04 download count',
`d05` int unsigned NOT NULL DEFAULT '0' COMMENT '05 download count',
`d06` int unsigned NOT NULL DEFAULT '0' COMMENT '06 download count',
`d07` int unsigned NOT NULL DEFAULT '0' COMMENT '07 download count',
`d08` int unsigned NOT NULL DEFAULT '0' COMMENT '08 download count',
`d09` int unsigned NOT NULL DEFAULT '0' COMMENT '09 download count',
`d10` int unsigned NOT NULL DEFAULT '0' COMMENT '10 download count',
`d11` int unsigned NOT NULL DEFAULT '0' COMMENT '11 download count',
`d12` int unsigned NOT NULL DEFAULT '0' COMMENT '12 download count',
`d13` int unsigned NOT NULL DEFAULT '0' COMMENT '13 download count',
`d14` int unsigned NOT NULL DEFAULT '0' COMMENT '14 download count',
`d15` int unsigned NOT NULL DEFAULT '0' COMMENT '15 download count',
`d16` int unsigned NOT NULL DEFAULT '0' COMMENT '16 download count',
`d17` int unsigned NOT NULL DEFAULT '0' COMMENT '17 download count',
`d18` int unsigned NOT NULL DEFAULT '0' COMMENT '18 download count',
`d19` int unsigned NOT NULL DEFAULT '0' COMMENT '19 download count',
`d20` int unsigned NOT NULL DEFAULT '0' COMMENT '20 download count',
`d21` int unsigned NOT NULL DEFAULT '0' COMMENT '21 download count',
`d22` int unsigned NOT NULL DEFAULT '0' COMMENT '22 download count',
`d23` int unsigned NOT NULL DEFAULT '0' COMMENT '23 download count',
`d24` int unsigned NOT NULL DEFAULT '0' COMMENT '24 download count',
`d25` int unsigned NOT NULL DEFAULT '0' COMMENT '25 download count',
`d26` int unsigned NOT NULL DEFAULT '0' COMMENT '26 download count',
`d27` int unsigned NOT NULL DEFAULT '0' COMMENT '27 download count',
`d28` int unsigned NOT NULL DEFAULT '0' COMMENT '28 download count',
`d29` int unsigned NOT NULL DEFAULT '0' COMMENT '29 download count',
`d30` int unsigned NOT NULL DEFAULT '0' COMMENT '30 download count',
`d31` int unsigned NOT NULL DEFAULT '0' COMMENT '31 download count',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_year_month_package_id_version` (`year_month`,`package_id`,`version`),
KEY `idx_year_month` (`year_month`),
KEY `idx_packageid_yearmonth` (`package_id`,`year_month`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package version download total info'
;
CREATE TABLE `package_version_files` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_version_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version id',
`package_version_file_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version file id',
`dist_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'file dist id',
`directory` varchar(500) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL COMMENT 'directory path, e.g.: /bin',
`name` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL COMMENT 'file name, e.g.: index.js',
`content_type` varchar(200) COLLATE utf8_unicode_ci NOT NULL COMMENT 'file content type, e.g.: application/javascript',
`mtime` datetime(3) NOT NULL COMMENT 'file modified time',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_version_file_id` (`package_version_file_id`),
UNIQUE KEY `ux_package_version_id_directory_name` (`package_version_id`,`directory`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package version file'
;
CREATE TABLE `package_version_manifests` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id',
`package_version_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version id',
`package_version_manifest_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version manifest id',
`manifest` json NOT NULL COMMENT 'manifest JSON, including README text',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_version_manifest_id` (`package_version_manifest_id`),
UNIQUE KEY `uk_package_version_id` (`package_version_id`),
KEY `idx_package_id` (`package_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package version manifest'
;
CREATE TABLE `package_versions` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id',
`package_version_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version id',
`version` varchar(256) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package version',
`abbreviated_dist_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'abbreviated manifest dist id',
`manifest_dist_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'manifest dist id',
`tar_dist_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'tar dist id',
`readme_dist_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'readme dist id',
`publish_time` datetime(3) NOT NULL COMMENT 'publish time',
`padding_version` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'token name',
`is_pre_release` tinyint DEFAULT NULL COMMENT '是否是先行版本',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_version_id` (`package_version_id`),
UNIQUE KEY `uk_package_id_version` (`package_id`,`version`),
KEY `idx_pkg_id_is_pre_release_padding_version` (`package_id`,`padding_version`,`is_pre_release`,`version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package version info'
;
CREATE TABLE `packages` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id',
`is_private` tinyint NOT NULL DEFAULT '0' COMMENT 'private pkg or not, 1: true, other: false',
`name` varchar(214) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'package name',
`scope` varchar(214) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package name, empty string meaning no scope',
`description` varchar(10240) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL COMMENT 'package description',
`abbreviateds_dist_id` varchar(24) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'all abbreviated manifests dist id',
`manifests_dist_id` varchar(24) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'all full manifests dist id',
`registry_id` varchar(24) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'source registry',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_id` (`package_id`),
UNIQUE KEY `uk_scope_name` (`scope`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='package info'
;
CREATE TABLE `proxy_caches` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modify time',
`fullname` varchar(214) COLLATE utf8_unicode_ci NOT NULL DEFAULT '' COMMENT '@scope/package name',
`version` varchar(214) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'package version',
`file_type` varchar(30) COLLATE utf8_unicode_ci NOT NULL DEFAULT '' COMMENT 'file type',
`file_path` varchar(512) COLLATE utf8_unicode_ci NOT NULL DEFAULT '' COMMENT 'nfs file path',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_package_version_path_name` (`file_path`),
UNIQUE KEY `ux_package_version_file_name` (`fullname`,`file_type`,`version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='proxy mode cached files index'
;
CREATE TABLE `registries` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`registry_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'registry id',
`name` varchar(256) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'registry name',
`host` varchar(4096) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'registry host',
`change_stream` varchar(4096) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'change stream url',
`type` varchar(256) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'registry type cnpmjsorg/cnpmcore/npm ',
`user_prefix` varchar(256) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'user prefix',
`auth_token` varchar(256) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'registry auth token',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='registry info'
;
CREATE TABLE `scopes` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`scope_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'scope id',
`name` varchar(214) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'scope name',
`registry_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'registry id',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='scope info'
;
CREATE TABLE `tasks` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`task_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'task id',
`type` varchar(20) COLLATE utf8_unicode_ci NOT NULL COMMENT 'task type',
`state` varchar(20) COLLATE utf8_unicode_ci NOT NULL COMMENT 'task state',
`target_name` varchar(214) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'target name, like package name / user name',
`author_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'create task user id',
`author_ip` varchar(100) COLLATE utf8_unicode_ci NOT NULL COMMENT 'create task user request ip',
`data` json DEFAULT NULL COMMENT 'task params',
`log_path` varchar(512) COLLATE utf8_unicode_ci NOT NULL COMMENT 'access path',
`log_store_position` varchar(10) COLLATE utf8_unicode_ci NOT NULL COMMENT 'cloud store disk position',
`attempts` int unsigned DEFAULT '0' COMMENT 'task execute attempts times',
`error` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci COMMENT 'error description',
`biz_id` varchar(100) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'unique biz id to keep task unique',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_task_id` (`task_id`),
UNIQUE KEY `uk_biz_id` (`biz_id`),
KEY `idx_type_state_target_name` (`target_name`,`type`,`state`),
KEY `idx_type_state_gmt_modified` (`type`,`state`,`gmt_modified`),
KEY `idx_gmt_modified` (`gmt_modified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='task info'
;
CREATE TABLE `token_packages` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`token_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'token id',
`package_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'package id',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_token_id_package_id` (`token_id`,`package_id`),
KEY `idx_token_id` (`token_id`),
KEY `idx_package_id` (`package_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='token allowed packages'
;
CREATE TABLE `tokens` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`token_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'token id',
`token_mark` varchar(20) COLLATE utf8_unicode_ci NOT NULL COMMENT 'token mark value',
`token_key` varchar(200) COLLATE utf8_unicode_ci NOT NULL COMMENT 'token value sha512 hex',
`is_readonly` tinyint NOT NULL DEFAULT '0' COMMENT 'readonly token or not, 1: true, other: false',
`is_automation` tinyint NOT NULL DEFAULT '0' COMMENT 'automation token or not, 1: true, other: false',
`cidr_whitelist` json NOT NULL COMMENT 'ip list, ["127.0.0.1"]',
`user_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'user id',
`name` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'token name',
`type` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'token type, granular or legacy',
`description` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'token description',
`allowed_scopes` text COLLATE utf8_unicode_ci COMMENT 'scope allowed list',
`expired_at` datetime(3) DEFAULT NULL COMMENT 'token expiration time',
`last_used_at` datetime(3) DEFAULT NULL COMMENT 'token last used time',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_token_id` (`token_id`),
UNIQUE KEY `uk_token_key` (`token_key`),
UNIQUE KEY `uk_user_id_name` (`user_id`,`name`),
KEY `idx_user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='token info'
;
CREATE TABLE `total` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`total_id` varchar(24) NOT NULL COMMENT 'total id, should set it to "global"',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`package_count` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'package count',
`package_file_size` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'package all files total size',
`package_version_count` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'package version count',
`package_version_delete_count` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'package version delete count',
`private_package_count` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'private package count',
`private_package_file_size` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'private package all files total size',
`private_package_version_count` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'private package version count',
`private_package_version_delete_count` bigint unsigned NOT NULL DEFAULT '0' COMMENT 'private package version delete count',
`change_stream_seq` varchar(100) DEFAULT NULL COMMENT 'npm change stream sync data seq id',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_total_id` (`total_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COMMENT='total info'
;
CREATE TABLE `users` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`user_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'user id',
`name` varchar(100) COLLATE utf8_unicode_ci NOT NULL COMMENT 'user name',
`email` varchar(400) COLLATE utf8_unicode_ci NOT NULL COMMENT 'user email',
`password_salt` varchar(100) COLLATE utf8_unicode_ci NOT NULL COMMENT 'password salt',
`password_integrity` varchar(512) COLLATE utf8_unicode_ci NOT NULL COMMENT 'password integrity',
`ip` varchar(100) COLLATE utf8_unicode_ci NOT NULL COMMENT 'user login request ip',
`is_private` tinyint NOT NULL DEFAULT '1' COMMENT 'private user or not, 1: true, other: false',
`scopes` json DEFAULT NULL COMMENT 'white scope list, ["@cnpm", "@foo"]',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_user_id` (`user_id`),
UNIQUE KEY `uk_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='user info'
;
CREATE TABLE `webauthn_credentials` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'primary key',
`gmt_create` datetime(3) NOT NULL COMMENT 'create time',
`gmt_modified` datetime(3) NOT NULL COMMENT 'modified time',
`wanc_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'webauthn credential id',
`user_id` varchar(24) COLLATE utf8_unicode_ci NOT NULL COMMENT 'user id',
`credential_id` varchar(200) COLLATE utf8_unicode_ci NOT NULL COMMENT 'webauthn credential id',
`public_key` varchar(512) COLLATE utf8_unicode_ci NOT NULL COMMENT 'webauthn credential publick key',
`browser_type` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT 'user browser name',
PRIMARY KEY (`id`),
UNIQUE KEY `uk_wanc_id` (`wanc_id`),
KEY `idx_user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8_unicode_ci COMMENT='webauthn credential info'
;

704
sql/ddl_postgresql.sql Normal file
View File

@@ -0,0 +1,704 @@
CREATE TABLE binaries (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
binary_id varchar(24) NOT NULL,
category varchar(50) NOT NULL,
parent varchar(500) NOT NULL,
name varchar(200) NOT NULL,
is_dir boolean NOT NULL DEFAULT false,
size integer NOT NULL,
date varchar(100) NOT NULL
);
CREATE UNIQUE INDEX binaries_uk_binary_id ON binaries (binary_id);
CREATE UNIQUE INDEX binaries_uk_category_parent_name ON binaries (category, parent, name);
CREATE INDEX binaries_idx_category_parent ON binaries (category, parent);
COMMENT ON TABLE binaries IS 'binary info';
COMMENT ON COLUMN binaries.id IS 'primary key';
COMMENT ON COLUMN binaries.gmt_create IS 'create time';
COMMENT ON COLUMN binaries.gmt_modified IS 'modified time';
COMMENT ON COLUMN binaries.binary_id IS 'binary id';
COMMENT ON COLUMN binaries.category IS 'binary category, e.g.: node, sass';
COMMENT ON COLUMN binaries.parent IS 'binary parent name, e.g.: /, /v1.0.0/, /v1.0.0/docs/';
COMMENT ON COLUMN binaries.name IS 'binary name, dir should ends with /';
COMMENT ON COLUMN binaries.is_dir IS 'is dir or not, 1: true, other: false';
COMMENT ON COLUMN binaries.size IS 'file size';
COMMENT ON COLUMN binaries.date IS 'date display string';
CREATE TABLE changes (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
change_id varchar(24) NOT NULL,
type varchar(50) NOT NULL,
target_name varchar(214) NOT NULL,
data json DEFAULT NULL
);
CREATE UNIQUE INDEX changes_uk_change_id ON changes (change_id);
COMMENT ON TABLE changes IS 'change info';
COMMENT ON COLUMN changes.id IS 'primary key';
COMMENT ON COLUMN changes.gmt_create IS 'create time';
COMMENT ON COLUMN changes.gmt_modified IS 'modified time';
COMMENT ON COLUMN changes.change_id IS 'change id';
COMMENT ON COLUMN changes.type IS 'change type';
COMMENT ON COLUMN changes.target_name IS 'target name, like package name / user name';
COMMENT ON COLUMN changes.data IS 'change params';
CREATE TABLE dists (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
dist_id varchar(24) NOT NULL,
name varchar(428) NOT NULL,
path varchar(767) NOT NULL,
size integer NOT NULL,
shasum varchar(512) NOT NULL,
integrity varchar(512) NOT NULL
);
CREATE UNIQUE INDEX dists_uk_dist_id ON dists (dist_id);
COMMENT ON TABLE dists IS 'dist info';
COMMENT ON COLUMN dists.id IS 'primary key';
COMMENT ON COLUMN dists.gmt_create IS 'create time';
COMMENT ON COLUMN dists.gmt_modified IS 'modified time';
COMMENT ON COLUMN dists.dist_id IS 'dist id';
COMMENT ON COLUMN dists.name IS 'dist name, 2x size of package name';
COMMENT ON COLUMN dists.path IS 'access path';
COMMENT ON COLUMN dists.size IS 'file size';
COMMENT ON COLUMN dists.shasum IS 'dist shasum';
COMMENT ON COLUMN dists.integrity IS 'dist integrity';
CREATE TABLE history_tasks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
task_id varchar(24) NOT NULL,
type varchar(20) NOT NULL,
state varchar(20) NOT NULL,
target_name varchar(214) NOT NULL,
author_id varchar(24) NOT NULL,
author_ip varchar(100) NOT NULL,
data json DEFAULT NULL,
log_path varchar(512) NOT NULL,
log_store_position varchar(10) NOT NULL,
attempts integer DEFAULT 0,
error text
);
CREATE UNIQUE INDEX history_tasks_uk_task_id ON history_tasks (task_id);
COMMENT ON TABLE history_tasks IS 'history task info';
COMMENT ON COLUMN history_tasks.id IS 'primary key';
COMMENT ON COLUMN history_tasks.gmt_create IS 'create time';
COMMENT ON COLUMN history_tasks.gmt_modified IS 'modified time';
COMMENT ON COLUMN history_tasks.task_id IS 'task id';
COMMENT ON COLUMN history_tasks.type IS 'task type';
COMMENT ON COLUMN history_tasks.state IS 'task state';
COMMENT ON COLUMN history_tasks.target_name IS 'target name, like package name / user name';
COMMENT ON COLUMN history_tasks.author_id IS 'create task user id';
COMMENT ON COLUMN history_tasks.author_ip IS 'create task user request ip';
COMMENT ON COLUMN history_tasks.data IS 'task params';
COMMENT ON COLUMN history_tasks.log_path IS 'access path';
COMMENT ON COLUMN history_tasks.log_store_position IS 'cloud store disk position';
COMMENT ON COLUMN history_tasks.attempts IS 'task execute attempts times';
COMMENT ON COLUMN history_tasks.error IS 'error description';
CREATE TABLE hooks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
hook_id varchar(24) NOT NULL,
type varchar(20) NOT NULL,
name varchar(428) NOT NULL,
owner_id varchar(24) NOT NULL,
endpoint varchar(2048) NOT NULL,
secret varchar(200) NOT NULL,
latest_task_id varchar(24) DEFAULT NULL,
enable boolean NOT NULL DEFAULT false
);
CREATE UNIQUE INDEX hooks_uk_type_name_owner_id ON hooks (type, name, owner_id);
CREATE INDEX hooks_idx_type_name_id ON hooks (type, name, id);
COMMENT ON TABLE hooks IS 'task info';
COMMENT ON COLUMN hooks.id IS 'primary key';
COMMENT ON COLUMN hooks.gmt_create IS 'create time';
COMMENT ON COLUMN hooks.gmt_modified IS 'modified time';
COMMENT ON COLUMN hooks.hook_id IS 'hook id';
COMMENT ON COLUMN hooks.type IS 'hook type, scope, name, owner';
COMMENT ON COLUMN hooks.name IS 'hook name';
COMMENT ON COLUMN hooks.owner_id IS 'hook owner id';
COMMENT ON COLUMN hooks.endpoint IS 'hook url';
COMMENT ON COLUMN hooks.secret IS 'sign secret';
COMMENT ON COLUMN hooks.latest_task_id IS 'latest task id';
COMMENT ON COLUMN hooks.enable IS 'hook is enable not, 1: true, other: false';
CREATE TABLE maintainers (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
user_id varchar(24) NOT NULL
);
CREATE UNIQUE INDEX maintainers_uk_package_id_user_id ON maintainers (package_id, user_id);
CREATE INDEX maintainers_idx_package_id ON maintainers (package_id);
CREATE INDEX maintainers_idx_user_id ON maintainers (user_id);
COMMENT ON TABLE maintainers IS 'package maintainers';
COMMENT ON COLUMN maintainers.id IS 'primary key';
COMMENT ON COLUMN maintainers.gmt_create IS 'create time';
COMMENT ON COLUMN maintainers.gmt_modified IS 'modified time';
COMMENT ON COLUMN maintainers.package_id IS 'package id';
COMMENT ON COLUMN maintainers.user_id IS 'user id';
CREATE TABLE package_deps (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_version_id varchar(24) NOT NULL,
package_dep_id varchar(24) NOT NULL,
scope varchar(214) NOT NULL,
name varchar(214) NOT NULL,
spec varchar(100) NOT NULL
);
CREATE UNIQUE INDEX package_deps_uk_package_dep_id ON package_deps (package_dep_id);
CREATE UNIQUE INDEX package_deps_uk_package_version_id_scope_name ON package_deps (package_version_id, scope, name);
COMMENT ON TABLE package_deps IS 'package dependency info';
COMMENT ON COLUMN package_deps.id IS 'primary key';
COMMENT ON COLUMN package_deps.gmt_create IS 'create time';
COMMENT ON COLUMN package_deps.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_deps.package_version_id IS 'package version id';
COMMENT ON COLUMN package_deps.package_dep_id IS 'package dep id';
COMMENT ON COLUMN package_deps.scope IS 'package scope';
COMMENT ON COLUMN package_deps.name IS 'package name';
COMMENT ON COLUMN package_deps.spec IS 'package dep spec';
CREATE TABLE package_tags (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
package_tag_id varchar(24) NOT NULL,
tag varchar(214) NOT NULL,
version varchar(256) NOT NULL
);
CREATE UNIQUE INDEX package_tags_uk_package_tag_id ON package_tags (package_tag_id);
CREATE UNIQUE INDEX package_tags_uk_package_tag ON package_tags (package_id, tag);
COMMENT ON TABLE package_tags IS 'package tag info';
COMMENT ON COLUMN package_tags.id IS 'primary key';
COMMENT ON COLUMN package_tags.gmt_create IS 'create time';
COMMENT ON COLUMN package_tags.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_tags.package_id IS 'package id';
COMMENT ON COLUMN package_tags.package_tag_id IS 'package tag id';
COMMENT ON COLUMN package_tags.tag IS 'package tag';
COMMENT ON COLUMN package_tags.version IS 'package version';
CREATE TABLE package_version_blocks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_version_block_id varchar(24) NOT NULL,
package_id varchar(24) NOT NULL,
version varchar(256) NOT NULL,
reason text NOT NULL
);
CREATE UNIQUE INDEX package_version_blocks_uk_package_version_block_id ON package_version_blocks (package_version_block_id);
CREATE UNIQUE INDEX package_version_blocks_uk_name_version ON package_version_blocks (package_id, version);
COMMENT ON TABLE package_version_blocks IS 'blocklist package versions';
COMMENT ON COLUMN package_version_blocks.id IS 'primary key';
COMMENT ON COLUMN package_version_blocks.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_blocks.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_blocks.package_version_block_id IS 'package version block id';
COMMENT ON COLUMN package_version_blocks.package_id IS 'package id';
COMMENT ON COLUMN package_version_blocks.version IS 'package version, "*" meaning all versions';
COMMENT ON COLUMN package_version_blocks.reason IS 'block reason';
CREATE TABLE package_version_downloads (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
year_month integer NOT NULL,
package_id varchar(214) NOT NULL,
version varchar(256) NOT NULL,
d01 integer NOT NULL DEFAULT 0,
d02 integer NOT NULL DEFAULT 0,
d03 integer NOT NULL DEFAULT 0,
d04 integer NOT NULL DEFAULT 0,
d05 integer NOT NULL DEFAULT 0,
d06 integer NOT NULL DEFAULT 0,
d07 integer NOT NULL DEFAULT 0,
d08 integer NOT NULL DEFAULT 0,
d09 integer NOT NULL DEFAULT 0,
d10 integer NOT NULL DEFAULT 0,
d11 integer NOT NULL DEFAULT 0,
d12 integer NOT NULL DEFAULT 0,
d13 integer NOT NULL DEFAULT 0,
d14 integer NOT NULL DEFAULT 0,
d15 integer NOT NULL DEFAULT 0,
d16 integer NOT NULL DEFAULT 0,
d17 integer NOT NULL DEFAULT 0,
d18 integer NOT NULL DEFAULT 0,
d19 integer NOT NULL DEFAULT 0,
d20 integer NOT NULL DEFAULT 0,
d21 integer NOT NULL DEFAULT 0,
d22 integer NOT NULL DEFAULT 0,
d23 integer NOT NULL DEFAULT 0,
d24 integer NOT NULL DEFAULT 0,
d25 integer NOT NULL DEFAULT 0,
d26 integer NOT NULL DEFAULT 0,
d27 integer NOT NULL DEFAULT 0,
d28 integer NOT NULL DEFAULT 0,
d29 integer NOT NULL DEFAULT 0,
d30 integer NOT NULL DEFAULT 0,
d31 integer NOT NULL DEFAULT 0
);
CREATE UNIQUE INDEX package_version_downloads_uk_year_month_package_id_version ON package_version_downloads (year_month, package_id, version);
CREATE INDEX package_version_downloads_idx_year_month ON package_version_downloads (year_month);
CREATE INDEX package_version_downloads_idx_packageid_yearmonth ON package_version_downloads (package_id, year_month);
COMMENT ON TABLE package_version_downloads IS 'package version download total info';
COMMENT ON COLUMN package_version_downloads.id IS 'primary key';
COMMENT ON COLUMN package_version_downloads.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_downloads.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_downloads.year_month IS 'YYYYMM format';
COMMENT ON COLUMN package_version_downloads.package_id IS 'package id, maybe scope name';
COMMENT ON COLUMN package_version_downloads.version IS 'package version';
COMMENT ON COLUMN package_version_downloads.d01 IS '01 download count';
COMMENT ON COLUMN package_version_downloads.d02 IS '02 download count';
COMMENT ON COLUMN package_version_downloads.d03 IS '03 download count';
COMMENT ON COLUMN package_version_downloads.d04 IS '04 download count';
COMMENT ON COLUMN package_version_downloads.d05 IS '05 download count';
COMMENT ON COLUMN package_version_downloads.d06 IS '06 download count';
COMMENT ON COLUMN package_version_downloads.d07 IS '07 download count';
COMMENT ON COLUMN package_version_downloads.d08 IS '08 download count';
COMMENT ON COLUMN package_version_downloads.d09 IS '09 download count';
COMMENT ON COLUMN package_version_downloads.d10 IS '10 download count';
COMMENT ON COLUMN package_version_downloads.d11 IS '11 download count';
COMMENT ON COLUMN package_version_downloads.d12 IS '12 download count';
COMMENT ON COLUMN package_version_downloads.d13 IS '13 download count';
COMMENT ON COLUMN package_version_downloads.d14 IS '14 download count';
COMMENT ON COLUMN package_version_downloads.d15 IS '15 download count';
COMMENT ON COLUMN package_version_downloads.d16 IS '16 download count';
COMMENT ON COLUMN package_version_downloads.d17 IS '17 download count';
COMMENT ON COLUMN package_version_downloads.d18 IS '18 download count';
COMMENT ON COLUMN package_version_downloads.d19 IS '19 download count';
COMMENT ON COLUMN package_version_downloads.d20 IS '20 download count';
COMMENT ON COLUMN package_version_downloads.d21 IS '21 download count';
COMMENT ON COLUMN package_version_downloads.d22 IS '22 download count';
COMMENT ON COLUMN package_version_downloads.d23 IS '23 download count';
COMMENT ON COLUMN package_version_downloads.d24 IS '24 download count';
COMMENT ON COLUMN package_version_downloads.d25 IS '25 download count';
COMMENT ON COLUMN package_version_downloads.d26 IS '26 download count';
COMMENT ON COLUMN package_version_downloads.d27 IS '27 download count';
COMMENT ON COLUMN package_version_downloads.d28 IS '28 download count';
COMMENT ON COLUMN package_version_downloads.d29 IS '29 download count';
COMMENT ON COLUMN package_version_downloads.d30 IS '30 download count';
COMMENT ON COLUMN package_version_downloads.d31 IS '31 download count';
CREATE TABLE package_version_files (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_version_id varchar(24) NOT NULL,
package_version_file_id varchar(24) NOT NULL,
dist_id varchar(24) NOT NULL,
directory varchar(500) NOT NULL,
name varchar(200) NOT NULL,
content_type varchar(200) NOT NULL,
mtime timestamp(3) NOT NULL
);
CREATE UNIQUE INDEX package_version_files_uk_package_version_file_id ON package_version_files (package_version_file_id);
CREATE UNIQUE INDEX package_version_files_ux_package_version_id_directory_name ON package_version_files (package_version_id, directory, name);
COMMENT ON TABLE package_version_files IS 'package version file';
COMMENT ON COLUMN package_version_files.id IS 'primary key';
COMMENT ON COLUMN package_version_files.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_files.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_files.package_version_id IS 'package version id';
COMMENT ON COLUMN package_version_files.package_version_file_id IS 'package version file id';
COMMENT ON COLUMN package_version_files.dist_id IS 'file dist id';
COMMENT ON COLUMN package_version_files.directory IS 'directory path, e.g.: /bin';
COMMENT ON COLUMN package_version_files.name IS 'file name, e.g.: index.js';
COMMENT ON COLUMN package_version_files.content_type IS 'file content type, e.g.: application/javascript';
COMMENT ON COLUMN package_version_files.mtime IS 'file modified time';
CREATE TABLE package_version_manifests (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
package_version_id varchar(24) NOT NULL,
package_version_manifest_id varchar(24) NOT NULL,
manifest json NOT NULL
);
CREATE UNIQUE INDEX package_version_manifests_uk_package_version_manifest_id ON package_version_manifests (package_version_manifest_id);
CREATE UNIQUE INDEX package_version_manifests_uk_package_version_id ON package_version_manifests (package_version_id);
CREATE INDEX package_version_manifests_idx_package_id ON package_version_manifests (package_id);
COMMENT ON TABLE package_version_manifests IS 'package version manifest';
COMMENT ON COLUMN package_version_manifests.id IS 'primary key';
COMMENT ON COLUMN package_version_manifests.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_manifests.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_manifests.package_id IS 'package id';
COMMENT ON COLUMN package_version_manifests.package_version_id IS 'package version id';
COMMENT ON COLUMN package_version_manifests.package_version_manifest_id IS 'package version manifest id';
COMMENT ON COLUMN package_version_manifests.manifest IS 'manifest JSON, including README text';
CREATE TABLE package_versions (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
package_version_id varchar(24) NOT NULL,
version varchar(256) NOT NULL,
abbreviated_dist_id varchar(24) NOT NULL,
manifest_dist_id varchar(24) NOT NULL,
tar_dist_id varchar(24) NOT NULL,
readme_dist_id varchar(24) NOT NULL,
publish_time timestamp(3) NOT NULL,
padding_version varchar(255) DEFAULT NULL,
is_pre_release boolean DEFAULT NULL
);
CREATE UNIQUE INDEX package_versions_uk_package_version_id ON package_versions (package_version_id);
CREATE UNIQUE INDEX package_versions_uk_package_id_version ON package_versions (package_id, version);
CREATE INDEX package_versions_idx_pkg_id_is_pre_release_padding_version ON package_versions (package_id, padding_version, is_pre_release, version);
COMMENT ON TABLE package_versions IS 'package version info';
COMMENT ON COLUMN package_versions.id IS 'primary key';
COMMENT ON COLUMN package_versions.gmt_create IS 'create time';
COMMENT ON COLUMN package_versions.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_versions.package_id IS 'package id';
COMMENT ON COLUMN package_versions.package_version_id IS 'package version id';
COMMENT ON COLUMN package_versions.version IS 'package version';
COMMENT ON COLUMN package_versions.abbreviated_dist_id IS 'abbreviated manifest dist id';
COMMENT ON COLUMN package_versions.manifest_dist_id IS 'manifest dist id';
COMMENT ON COLUMN package_versions.tar_dist_id IS 'tar dist id';
COMMENT ON COLUMN package_versions.readme_dist_id IS 'readme dist id';
COMMENT ON COLUMN package_versions.publish_time IS 'publish time';
COMMENT ON COLUMN package_versions.padding_version IS 'token name';
COMMENT ON COLUMN package_versions.is_pre_release IS '是否是先行版本';
CREATE TABLE packages (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
is_private boolean NOT NULL DEFAULT false,
name varchar(214) NOT NULL,
scope varchar(214) NOT NULL,
description varchar(10240) DEFAULT NULL,
abbreviateds_dist_id varchar(24) DEFAULT NULL,
manifests_dist_id varchar(24) DEFAULT NULL,
registry_id varchar(24) DEFAULT NULL
);
CREATE UNIQUE INDEX packages_uk_package_id ON packages (package_id);
CREATE UNIQUE INDEX packages_uk_scope_name ON packages (scope, name);
COMMENT ON TABLE packages IS 'package info';
COMMENT ON COLUMN packages.id IS 'primary key';
COMMENT ON COLUMN packages.gmt_create IS 'create time';
COMMENT ON COLUMN packages.gmt_modified IS 'modified time';
COMMENT ON COLUMN packages.package_id IS 'package id';
COMMENT ON COLUMN packages.is_private IS 'private pkg or not, 1: true, other: false';
COMMENT ON COLUMN packages.name IS 'package name';
COMMENT ON COLUMN packages.scope IS 'package name, empty string meaning no scope';
COMMENT ON COLUMN packages.description IS 'package description';
COMMENT ON COLUMN packages.abbreviateds_dist_id IS 'all abbreviated manifests dist id';
COMMENT ON COLUMN packages.manifests_dist_id IS 'all full manifests dist id';
COMMENT ON COLUMN packages.registry_id IS 'source registry';
CREATE TABLE proxy_caches (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
fullname varchar(214) NOT NULL DEFAULT '',
version varchar(214) DEFAULT NULL,
file_type varchar(30) NOT NULL DEFAULT '',
file_path varchar(512) NOT NULL DEFAULT ''
);
CREATE UNIQUE INDEX proxy_caches_uk_package_version_path_name ON proxy_caches (file_path);
CREATE UNIQUE INDEX proxy_caches_ux_package_version_file_name ON proxy_caches (fullname, file_type, version);
COMMENT ON TABLE proxy_caches IS 'proxy mode cached files index';
COMMENT ON COLUMN proxy_caches.id IS 'primary key';
COMMENT ON COLUMN proxy_caches.gmt_create IS 'create time';
COMMENT ON COLUMN proxy_caches.gmt_modified IS 'modify time';
COMMENT ON COLUMN proxy_caches.fullname IS '@scope/package name';
COMMENT ON COLUMN proxy_caches.version IS 'package version';
COMMENT ON COLUMN proxy_caches.file_type IS 'file type';
COMMENT ON COLUMN proxy_caches.file_path IS 'nfs file path';
CREATE TABLE registries (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
registry_id varchar(24) NOT NULL,
name varchar(256) DEFAULT NULL,
host varchar(4096) DEFAULT NULL,
change_stream varchar(4096) DEFAULT NULL,
type varchar(256) DEFAULT NULL,
user_prefix varchar(256) DEFAULT NULL,
auth_token varchar(256) DEFAULT NULL
);
CREATE UNIQUE INDEX registries_uk_name ON registries (name);
COMMENT ON TABLE registries IS 'registry info';
COMMENT ON COLUMN registries.id IS 'primary key';
COMMENT ON COLUMN registries.gmt_create IS 'create time';
COMMENT ON COLUMN registries.gmt_modified IS 'modified time';
COMMENT ON COLUMN registries.registry_id IS 'registry id';
COMMENT ON COLUMN registries.name IS 'registry name';
COMMENT ON COLUMN registries.host IS 'registry host';
COMMENT ON COLUMN registries.change_stream IS 'change stream url';
COMMENT ON COLUMN registries.type IS 'registry type cnpmjsorg/cnpmcore/npm';
COMMENT ON COLUMN registries.user_prefix IS 'user prefix';
COMMENT ON COLUMN registries.auth_token IS 'registry auth token';
CREATE TABLE scopes (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
scope_id varchar(24) NOT NULL,
name varchar(214) DEFAULT NULL,
registry_id varchar(24) NOT NULL
);
CREATE UNIQUE INDEX scopes_uk_name ON scopes (name);
COMMENT ON TABLE scopes IS 'scope info';
COMMENT ON COLUMN scopes.id IS 'primary key';
COMMENT ON COLUMN scopes.gmt_create IS 'create time';
COMMENT ON COLUMN scopes.gmt_modified IS 'modified time';
COMMENT ON COLUMN scopes.scope_id IS 'scope id';
COMMENT ON COLUMN scopes.name IS 'scope name';
COMMENT ON COLUMN scopes.registry_id IS 'registry id';
CREATE TABLE tasks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
task_id varchar(24) NOT NULL,
type varchar(20) NOT NULL,
state varchar(20) NOT NULL,
target_name varchar(214) NOT NULL,
author_id varchar(24) NOT NULL,
author_ip varchar(100) NOT NULL,
data json DEFAULT NULL,
log_path varchar(512) NOT NULL,
log_store_position varchar(10) NOT NULL,
attempts integer DEFAULT 0,
error text,
biz_id varchar(100) DEFAULT NULL
);
CREATE UNIQUE INDEX tasks_uk_task_id ON tasks (task_id);
CREATE UNIQUE INDEX tasks_uk_biz_id ON tasks (biz_id);
CREATE INDEX tasks_idx_type_state_target_name ON tasks (target_name, type, state);
CREATE INDEX tasks_idx_type_state_gmt_modified ON tasks (type, state, gmt_modified);
CREATE INDEX tasks_idx_gmt_modified ON tasks (gmt_modified);
COMMENT ON TABLE tasks IS 'task info';
COMMENT ON COLUMN tasks.id IS 'primary key';
COMMENT ON COLUMN tasks.gmt_create IS 'create time';
COMMENT ON COLUMN tasks.gmt_modified IS 'modified time';
COMMENT ON COLUMN tasks.task_id IS 'task id';
COMMENT ON COLUMN tasks.type IS 'task type';
COMMENT ON COLUMN tasks.state IS 'task state';
COMMENT ON COLUMN tasks.target_name IS 'target name, like package name / user name';
COMMENT ON COLUMN tasks.author_id IS 'create task user id';
COMMENT ON COLUMN tasks.author_ip IS 'create task user request ip';
COMMENT ON COLUMN tasks.data IS 'task params';
COMMENT ON COLUMN tasks.log_path IS 'access path';
COMMENT ON COLUMN tasks.log_store_position IS 'cloud store disk position';
COMMENT ON COLUMN tasks.attempts IS 'task execute attempts times';
COMMENT ON COLUMN tasks.error IS 'error description';
COMMENT ON COLUMN tasks.biz_id IS 'unique biz id to keep task unique';
CREATE TABLE token_packages (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
token_id varchar(24) NOT NULL,
package_id varchar(24) NOT NULL
);
CREATE UNIQUE INDEX token_packages_uk_token_id_package_id ON token_packages (token_id, package_id);
CREATE INDEX token_packages_idx_token_id ON token_packages (token_id);
CREATE INDEX token_packages_idx_package_id ON token_packages (package_id);
COMMENT ON TABLE token_packages IS 'token allowed packages';
COMMENT ON COLUMN token_packages.id IS 'primary key';
COMMENT ON COLUMN token_packages.gmt_create IS 'create time';
COMMENT ON COLUMN token_packages.gmt_modified IS 'modified time';
COMMENT ON COLUMN token_packages.token_id IS 'token id';
COMMENT ON COLUMN token_packages.package_id IS 'package id';
CREATE TABLE tokens (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
token_id varchar(24) NOT NULL,
token_mark varchar(20) NOT NULL,
token_key varchar(200) NOT NULL,
is_readonly boolean NOT NULL DEFAULT false,
is_automation boolean NOT NULL DEFAULT false,
cidr_whitelist json NOT NULL,
user_id varchar(24) NOT NULL,
name varchar(255) DEFAULT NULL,
type varchar(255) DEFAULT NULL,
description varchar(255) DEFAULT NULL,
allowed_scopes text,
expired_at timestamp(3) DEFAULT NULL,
last_used_at timestamp(3) DEFAULT NULL
);
CREATE UNIQUE INDEX tokens_uk_token_id ON tokens (token_id);
CREATE UNIQUE INDEX tokens_uk_token_key ON tokens (token_key);
CREATE UNIQUE INDEX tokens_uk_user_id_name ON tokens (user_id, name);
CREATE INDEX tokens_idx_user_id ON tokens (user_id);
COMMENT ON TABLE tokens IS 'token info';
COMMENT ON COLUMN tokens.id IS 'primary key';
COMMENT ON COLUMN tokens.gmt_create IS 'create time';
COMMENT ON COLUMN tokens.gmt_modified IS 'modified time';
COMMENT ON COLUMN tokens.token_id IS 'token id';
COMMENT ON COLUMN tokens.token_mark IS 'token mark value';
COMMENT ON COLUMN tokens.token_key IS 'token value sha512 hex';
COMMENT ON COLUMN tokens.is_readonly IS 'readonly token or not, 1: true, other: false';
COMMENT ON COLUMN tokens.is_automation IS 'automation token or not, 1: true, other: false';
COMMENT ON COLUMN tokens.cidr_whitelist IS 'ip list, ["127.0.0.1"]';
COMMENT ON COLUMN tokens.user_id IS 'user id';
COMMENT ON COLUMN tokens.name IS 'token name';
COMMENT ON COLUMN tokens.type IS 'token type, granular or legacy';
COMMENT ON COLUMN tokens.description IS 'token description';
COMMENT ON COLUMN tokens.allowed_scopes IS 'scope allowed list';
COMMENT ON COLUMN tokens.expired_at IS 'token expiration time';
COMMENT ON COLUMN tokens.last_used_at IS 'token last used time';
CREATE TABLE total (
id BIGSERIAL PRIMARY KEY,
total_id varchar(24) NOT NULL,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_count bigint NOT NULL DEFAULT 0,
package_file_size bigint NOT NULL DEFAULT 0,
package_version_count bigint NOT NULL DEFAULT 0,
package_version_delete_count bigint NOT NULL DEFAULT 0,
private_package_count bigint NOT NULL DEFAULT 0,
private_package_file_size bigint NOT NULL DEFAULT 0,
private_package_version_count bigint NOT NULL DEFAULT 0,
private_package_version_delete_count bigint NOT NULL DEFAULT 0,
change_stream_seq varchar(100) DEFAULT NULL
);
CREATE UNIQUE INDEX total_uk_total_id ON total (total_id);
COMMENT ON TABLE total IS 'total info';
COMMENT ON COLUMN total.id IS 'primary key';
COMMENT ON COLUMN total.total_id IS 'total id, should set it to "global"';
COMMENT ON COLUMN total.gmt_create IS 'create time';
COMMENT ON COLUMN total.gmt_modified IS 'modified time';
COMMENT ON COLUMN total.package_count IS 'package count';
COMMENT ON COLUMN total.package_file_size IS 'package all files total size';
COMMENT ON COLUMN total.package_version_count IS 'package version count';
COMMENT ON COLUMN total.package_version_delete_count IS 'package version delete count';
COMMENT ON COLUMN total.private_package_count IS 'private package count';
COMMENT ON COLUMN total.private_package_file_size IS 'private package all files total size';
COMMENT ON COLUMN total.private_package_version_count IS 'private package version count';
COMMENT ON COLUMN total.private_package_version_delete_count IS 'private package version delete count';
COMMENT ON COLUMN total.change_stream_seq IS 'npm change stream sync data seq id';
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
user_id varchar(24) NOT NULL,
name varchar(100) NOT NULL,
email varchar(400) NOT NULL,
password_salt varchar(100) NOT NULL,
password_integrity varchar(512) NOT NULL,
ip varchar(100) NOT NULL,
is_private boolean NOT NULL DEFAULT true,
scopes json DEFAULT NULL
);
CREATE UNIQUE INDEX users_uk_user_id ON users (user_id);
CREATE UNIQUE INDEX users_uk_name ON users (name);
COMMENT ON TABLE users IS 'user info';
COMMENT ON COLUMN users.id IS 'primary key';
COMMENT ON COLUMN users.gmt_create IS 'create time';
COMMENT ON COLUMN users.gmt_modified IS 'modified time';
COMMENT ON COLUMN users.user_id IS 'user id';
COMMENT ON COLUMN users.name IS 'user name';
COMMENT ON COLUMN users.email IS 'user email';
COMMENT ON COLUMN users.password_salt IS 'password salt';
COMMENT ON COLUMN users.password_integrity IS 'password integrity';
COMMENT ON COLUMN users.ip IS 'user login request ip';
COMMENT ON COLUMN users.is_private IS 'private user or not, 1: true, other: false';
COMMENT ON COLUMN users.scopes IS 'white scope list, ["@cnpm", "@foo"]';
CREATE TABLE webauthn_credentials (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
wanc_id varchar(24) NOT NULL,
user_id varchar(24) NOT NULL,
credential_id varchar(200) NOT NULL,
public_key varchar(512) NOT NULL,
browser_type varchar(20) DEFAULT NULL
);
CREATE UNIQUE INDEX webauthn_credentials_uk_wanc_id ON webauthn_credentials (wanc_id);
CREATE INDEX webauthn_credentials_idx_user_id ON webauthn_credentials (user_id);
COMMENT ON TABLE webauthn_credentials IS 'webauthn credential info';
COMMENT ON COLUMN webauthn_credentials.id IS 'primary key';
COMMENT ON COLUMN webauthn_credentials.gmt_create IS 'create time';
COMMENT ON COLUMN webauthn_credentials.gmt_modified IS 'modified time';
COMMENT ON COLUMN webauthn_credentials.wanc_id IS 'webauthn credential id';
COMMENT ON COLUMN webauthn_credentials.user_id IS 'user id';
COMMENT ON COLUMN webauthn_credentials.credential_id IS 'webauthn credential id';
COMMENT ON COLUMN webauthn_credentials.public_key IS 'webauthn credential publick key';
COMMENT ON COLUMN webauthn_credentials.browser_type IS 'user browser name';

704
sql/postgresql/3.68.0.sql Normal file
View File

@@ -0,0 +1,704 @@
CREATE TABLE binaries (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
binary_id varchar(24) NOT NULL,
category varchar(50) NOT NULL,
parent varchar(500) NOT NULL,
name varchar(200) NOT NULL,
is_dir boolean NOT NULL DEFAULT false,
size integer NOT NULL,
date varchar(100) NOT NULL
);
CREATE UNIQUE INDEX binaries_uk_binary_id ON binaries (binary_id);
CREATE UNIQUE INDEX binaries_uk_category_parent_name ON binaries (category, parent, name);
CREATE INDEX binaries_idx_category_parent ON binaries (category, parent);
COMMENT ON TABLE binaries IS 'binary info';
COMMENT ON COLUMN binaries.id IS 'primary key';
COMMENT ON COLUMN binaries.gmt_create IS 'create time';
COMMENT ON COLUMN binaries.gmt_modified IS 'modified time';
COMMENT ON COLUMN binaries.binary_id IS 'binary id';
COMMENT ON COLUMN binaries.category IS 'binary category, e.g.: node, sass';
COMMENT ON COLUMN binaries.parent IS 'binary parent name, e.g.: /, /v1.0.0/, /v1.0.0/docs/';
COMMENT ON COLUMN binaries.name IS 'binary name, dir should ends with /';
COMMENT ON COLUMN binaries.is_dir IS 'is dir or not, 1: true, other: false';
COMMENT ON COLUMN binaries.size IS 'file size';
COMMENT ON COLUMN binaries.date IS 'date display string';
CREATE TABLE changes (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
change_id varchar(24) NOT NULL,
type varchar(50) NOT NULL,
target_name varchar(214) NOT NULL,
data json DEFAULT NULL
);
CREATE UNIQUE INDEX changes_uk_change_id ON changes (change_id);
COMMENT ON TABLE changes IS 'change info';
COMMENT ON COLUMN changes.id IS 'primary key';
COMMENT ON COLUMN changes.gmt_create IS 'create time';
COMMENT ON COLUMN changes.gmt_modified IS 'modified time';
COMMENT ON COLUMN changes.change_id IS 'change id';
COMMENT ON COLUMN changes.type IS 'change type';
COMMENT ON COLUMN changes.target_name IS 'target name, like package name / user name';
COMMENT ON COLUMN changes.data IS 'change params';
CREATE TABLE dists (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
dist_id varchar(24) NOT NULL,
name varchar(428) NOT NULL,
path varchar(767) NOT NULL,
size integer NOT NULL,
shasum varchar(512) NOT NULL,
integrity varchar(512) NOT NULL
);
CREATE UNIQUE INDEX dists_uk_dist_id ON dists (dist_id);
COMMENT ON TABLE dists IS 'dist info';
COMMENT ON COLUMN dists.id IS 'primary key';
COMMENT ON COLUMN dists.gmt_create IS 'create time';
COMMENT ON COLUMN dists.gmt_modified IS 'modified time';
COMMENT ON COLUMN dists.dist_id IS 'dist id';
COMMENT ON COLUMN dists.name IS 'dist name, 2x size of package name';
COMMENT ON COLUMN dists.path IS 'access path';
COMMENT ON COLUMN dists.size IS 'file size';
COMMENT ON COLUMN dists.shasum IS 'dist shasum';
COMMENT ON COLUMN dists.integrity IS 'dist integrity';
CREATE TABLE history_tasks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
task_id varchar(24) NOT NULL,
type varchar(20) NOT NULL,
state varchar(20) NOT NULL,
target_name varchar(214) NOT NULL,
author_id varchar(24) NOT NULL,
author_ip varchar(100) NOT NULL,
data json DEFAULT NULL,
log_path varchar(512) NOT NULL,
log_store_position varchar(10) NOT NULL,
attempts integer DEFAULT 0,
error text
);
CREATE UNIQUE INDEX history_tasks_uk_task_id ON history_tasks (task_id);
COMMENT ON TABLE history_tasks IS 'history task info';
COMMENT ON COLUMN history_tasks.id IS 'primary key';
COMMENT ON COLUMN history_tasks.gmt_create IS 'create time';
COMMENT ON COLUMN history_tasks.gmt_modified IS 'modified time';
COMMENT ON COLUMN history_tasks.task_id IS 'task id';
COMMENT ON COLUMN history_tasks.type IS 'task type';
COMMENT ON COLUMN history_tasks.state IS 'task state';
COMMENT ON COLUMN history_tasks.target_name IS 'target name, like package name / user name';
COMMENT ON COLUMN history_tasks.author_id IS 'create task user id';
COMMENT ON COLUMN history_tasks.author_ip IS 'create task user request ip';
COMMENT ON COLUMN history_tasks.data IS 'task params';
COMMENT ON COLUMN history_tasks.log_path IS 'access path';
COMMENT ON COLUMN history_tasks.log_store_position IS 'cloud store disk position';
COMMENT ON COLUMN history_tasks.attempts IS 'task execute attempts times';
COMMENT ON COLUMN history_tasks.error IS 'error description';
CREATE TABLE hooks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
hook_id varchar(24) NOT NULL,
type varchar(20) NOT NULL,
name varchar(428) NOT NULL,
owner_id varchar(24) NOT NULL,
endpoint varchar(2048) NOT NULL,
secret varchar(200) NOT NULL,
latest_task_id varchar(24) DEFAULT NULL,
enable boolean NOT NULL DEFAULT false
);
CREATE UNIQUE INDEX hooks_uk_type_name_owner_id ON hooks (type, name, owner_id);
CREATE INDEX hooks_idx_type_name_id ON hooks (type, name, id);
COMMENT ON TABLE hooks IS 'task info';
COMMENT ON COLUMN hooks.id IS 'primary key';
COMMENT ON COLUMN hooks.gmt_create IS 'create time';
COMMENT ON COLUMN hooks.gmt_modified IS 'modified time';
COMMENT ON COLUMN hooks.hook_id IS 'hook id';
COMMENT ON COLUMN hooks.type IS 'hook type, scope, name, owner';
COMMENT ON COLUMN hooks.name IS 'hook name';
COMMENT ON COLUMN hooks.owner_id IS 'hook owner id';
COMMENT ON COLUMN hooks.endpoint IS 'hook url';
COMMENT ON COLUMN hooks.secret IS 'sign secret';
COMMENT ON COLUMN hooks.latest_task_id IS 'latest task id';
COMMENT ON COLUMN hooks.enable IS 'hook is enable not, 1: true, other: false';
CREATE TABLE maintainers (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
user_id varchar(24) NOT NULL
);
CREATE UNIQUE INDEX maintainers_uk_package_id_user_id ON maintainers (package_id, user_id);
CREATE INDEX maintainers_idx_package_id ON maintainers (package_id);
CREATE INDEX maintainers_idx_user_id ON maintainers (user_id);
COMMENT ON TABLE maintainers IS 'package maintainers';
COMMENT ON COLUMN maintainers.id IS 'primary key';
COMMENT ON COLUMN maintainers.gmt_create IS 'create time';
COMMENT ON COLUMN maintainers.gmt_modified IS 'modified time';
COMMENT ON COLUMN maintainers.package_id IS 'package id';
COMMENT ON COLUMN maintainers.user_id IS 'user id';
CREATE TABLE package_deps (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_version_id varchar(24) NOT NULL,
package_dep_id varchar(24) NOT NULL,
scope varchar(214) NOT NULL,
name varchar(214) NOT NULL,
spec varchar(100) NOT NULL
);
CREATE UNIQUE INDEX package_deps_uk_package_dep_id ON package_deps (package_dep_id);
CREATE UNIQUE INDEX package_deps_uk_package_version_id_scope_name ON package_deps (package_version_id, scope, name);
COMMENT ON TABLE package_deps IS 'package dependency info';
COMMENT ON COLUMN package_deps.id IS 'primary key';
COMMENT ON COLUMN package_deps.gmt_create IS 'create time';
COMMENT ON COLUMN package_deps.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_deps.package_version_id IS 'package version id';
COMMENT ON COLUMN package_deps.package_dep_id IS 'package dep id';
COMMENT ON COLUMN package_deps.scope IS 'package scope';
COMMENT ON COLUMN package_deps.name IS 'package name';
COMMENT ON COLUMN package_deps.spec IS 'package dep spec';
CREATE TABLE package_tags (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
package_tag_id varchar(24) NOT NULL,
tag varchar(214) NOT NULL,
version varchar(256) NOT NULL
);
CREATE UNIQUE INDEX package_tags_uk_package_tag_id ON package_tags (package_tag_id);
CREATE UNIQUE INDEX package_tags_uk_package_tag ON package_tags (package_id, tag);
COMMENT ON TABLE package_tags IS 'package tag info';
COMMENT ON COLUMN package_tags.id IS 'primary key';
COMMENT ON COLUMN package_tags.gmt_create IS 'create time';
COMMENT ON COLUMN package_tags.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_tags.package_id IS 'package id';
COMMENT ON COLUMN package_tags.package_tag_id IS 'package tag id';
COMMENT ON COLUMN package_tags.tag IS 'package tag';
COMMENT ON COLUMN package_tags.version IS 'package version';
CREATE TABLE package_version_blocks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_version_block_id varchar(24) NOT NULL,
package_id varchar(24) NOT NULL,
version varchar(256) NOT NULL,
reason text NOT NULL
);
CREATE UNIQUE INDEX package_version_blocks_uk_package_version_block_id ON package_version_blocks (package_version_block_id);
CREATE UNIQUE INDEX package_version_blocks_uk_name_version ON package_version_blocks (package_id, version);
COMMENT ON TABLE package_version_blocks IS 'blocklist package versions';
COMMENT ON COLUMN package_version_blocks.id IS 'primary key';
COMMENT ON COLUMN package_version_blocks.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_blocks.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_blocks.package_version_block_id IS 'package version block id';
COMMENT ON COLUMN package_version_blocks.package_id IS 'package id';
COMMENT ON COLUMN package_version_blocks.version IS 'package version, "*" meaning all versions';
COMMENT ON COLUMN package_version_blocks.reason IS 'block reason';
CREATE TABLE package_version_downloads (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
year_month integer NOT NULL,
package_id varchar(214) NOT NULL,
version varchar(256) NOT NULL,
d01 integer NOT NULL DEFAULT 0,
d02 integer NOT NULL DEFAULT 0,
d03 integer NOT NULL DEFAULT 0,
d04 integer NOT NULL DEFAULT 0,
d05 integer NOT NULL DEFAULT 0,
d06 integer NOT NULL DEFAULT 0,
d07 integer NOT NULL DEFAULT 0,
d08 integer NOT NULL DEFAULT 0,
d09 integer NOT NULL DEFAULT 0,
d10 integer NOT NULL DEFAULT 0,
d11 integer NOT NULL DEFAULT 0,
d12 integer NOT NULL DEFAULT 0,
d13 integer NOT NULL DEFAULT 0,
d14 integer NOT NULL DEFAULT 0,
d15 integer NOT NULL DEFAULT 0,
d16 integer NOT NULL DEFAULT 0,
d17 integer NOT NULL DEFAULT 0,
d18 integer NOT NULL DEFAULT 0,
d19 integer NOT NULL DEFAULT 0,
d20 integer NOT NULL DEFAULT 0,
d21 integer NOT NULL DEFAULT 0,
d22 integer NOT NULL DEFAULT 0,
d23 integer NOT NULL DEFAULT 0,
d24 integer NOT NULL DEFAULT 0,
d25 integer NOT NULL DEFAULT 0,
d26 integer NOT NULL DEFAULT 0,
d27 integer NOT NULL DEFAULT 0,
d28 integer NOT NULL DEFAULT 0,
d29 integer NOT NULL DEFAULT 0,
d30 integer NOT NULL DEFAULT 0,
d31 integer NOT NULL DEFAULT 0
);
CREATE UNIQUE INDEX package_version_downloads_uk_year_month_package_id_version ON package_version_downloads (year_month, package_id, version);
CREATE INDEX package_version_downloads_idx_year_month ON package_version_downloads (year_month);
CREATE INDEX package_version_downloads_idx_packageid_yearmonth ON package_version_downloads (package_id, year_month);
COMMENT ON TABLE package_version_downloads IS 'package version download total info';
COMMENT ON COLUMN package_version_downloads.id IS 'primary key';
COMMENT ON COLUMN package_version_downloads.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_downloads.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_downloads.year_month IS 'YYYYMM format';
COMMENT ON COLUMN package_version_downloads.package_id IS 'package id, maybe scope name';
COMMENT ON COLUMN package_version_downloads.version IS 'package version';
COMMENT ON COLUMN package_version_downloads.d01 IS '01 download count';
COMMENT ON COLUMN package_version_downloads.d02 IS '02 download count';
COMMENT ON COLUMN package_version_downloads.d03 IS '03 download count';
COMMENT ON COLUMN package_version_downloads.d04 IS '04 download count';
COMMENT ON COLUMN package_version_downloads.d05 IS '05 download count';
COMMENT ON COLUMN package_version_downloads.d06 IS '06 download count';
COMMENT ON COLUMN package_version_downloads.d07 IS '07 download count';
COMMENT ON COLUMN package_version_downloads.d08 IS '08 download count';
COMMENT ON COLUMN package_version_downloads.d09 IS '09 download count';
COMMENT ON COLUMN package_version_downloads.d10 IS '10 download count';
COMMENT ON COLUMN package_version_downloads.d11 IS '11 download count';
COMMENT ON COLUMN package_version_downloads.d12 IS '12 download count';
COMMENT ON COLUMN package_version_downloads.d13 IS '13 download count';
COMMENT ON COLUMN package_version_downloads.d14 IS '14 download count';
COMMENT ON COLUMN package_version_downloads.d15 IS '15 download count';
COMMENT ON COLUMN package_version_downloads.d16 IS '16 download count';
COMMENT ON COLUMN package_version_downloads.d17 IS '17 download count';
COMMENT ON COLUMN package_version_downloads.d18 IS '18 download count';
COMMENT ON COLUMN package_version_downloads.d19 IS '19 download count';
COMMENT ON COLUMN package_version_downloads.d20 IS '20 download count';
COMMENT ON COLUMN package_version_downloads.d21 IS '21 download count';
COMMENT ON COLUMN package_version_downloads.d22 IS '22 download count';
COMMENT ON COLUMN package_version_downloads.d23 IS '23 download count';
COMMENT ON COLUMN package_version_downloads.d24 IS '24 download count';
COMMENT ON COLUMN package_version_downloads.d25 IS '25 download count';
COMMENT ON COLUMN package_version_downloads.d26 IS '26 download count';
COMMENT ON COLUMN package_version_downloads.d27 IS '27 download count';
COMMENT ON COLUMN package_version_downloads.d28 IS '28 download count';
COMMENT ON COLUMN package_version_downloads.d29 IS '29 download count';
COMMENT ON COLUMN package_version_downloads.d30 IS '30 download count';
COMMENT ON COLUMN package_version_downloads.d31 IS '31 download count';
CREATE TABLE package_version_files (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_version_id varchar(24) NOT NULL,
package_version_file_id varchar(24) NOT NULL,
dist_id varchar(24) NOT NULL,
directory varchar(500) NOT NULL,
name varchar(200) NOT NULL,
content_type varchar(200) NOT NULL,
mtime timestamp(3) NOT NULL
);
CREATE UNIQUE INDEX package_version_files_uk_package_version_file_id ON package_version_files (package_version_file_id);
CREATE UNIQUE INDEX package_version_files_ux_package_version_id_directory_name ON package_version_files (package_version_id, directory, name);
COMMENT ON TABLE package_version_files IS 'package version file';
COMMENT ON COLUMN package_version_files.id IS 'primary key';
COMMENT ON COLUMN package_version_files.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_files.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_files.package_version_id IS 'package version id';
COMMENT ON COLUMN package_version_files.package_version_file_id IS 'package version file id';
COMMENT ON COLUMN package_version_files.dist_id IS 'file dist id';
COMMENT ON COLUMN package_version_files.directory IS 'directory path, e.g.: /bin';
COMMENT ON COLUMN package_version_files.name IS 'file name, e.g.: index.js';
COMMENT ON COLUMN package_version_files.content_type IS 'file content type, e.g.: application/javascript';
COMMENT ON COLUMN package_version_files.mtime IS 'file modified time';
CREATE TABLE package_version_manifests (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
package_version_id varchar(24) NOT NULL,
package_version_manifest_id varchar(24) NOT NULL,
manifest json NOT NULL
);
CREATE UNIQUE INDEX package_version_manifests_uk_package_version_manifest_id ON package_version_manifests (package_version_manifest_id);
CREATE UNIQUE INDEX package_version_manifests_uk_package_version_id ON package_version_manifests (package_version_id);
CREATE INDEX package_version_manifests_idx_package_id ON package_version_manifests (package_id);
COMMENT ON TABLE package_version_manifests IS 'package version manifest';
COMMENT ON COLUMN package_version_manifests.id IS 'primary key';
COMMENT ON COLUMN package_version_manifests.gmt_create IS 'create time';
COMMENT ON COLUMN package_version_manifests.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_version_manifests.package_id IS 'package id';
COMMENT ON COLUMN package_version_manifests.package_version_id IS 'package version id';
COMMENT ON COLUMN package_version_manifests.package_version_manifest_id IS 'package version manifest id';
COMMENT ON COLUMN package_version_manifests.manifest IS 'manifest JSON, including README text';
CREATE TABLE package_versions (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
package_version_id varchar(24) NOT NULL,
version varchar(256) NOT NULL,
abbreviated_dist_id varchar(24) NOT NULL,
manifest_dist_id varchar(24) NOT NULL,
tar_dist_id varchar(24) NOT NULL,
readme_dist_id varchar(24) NOT NULL,
publish_time timestamp(3) NOT NULL,
padding_version varchar(255) DEFAULT NULL,
is_pre_release boolean DEFAULT NULL
);
CREATE UNIQUE INDEX package_versions_uk_package_version_id ON package_versions (package_version_id);
CREATE UNIQUE INDEX package_versions_uk_package_id_version ON package_versions (package_id, version);
CREATE INDEX package_versions_idx_pkg_id_is_pre_release_padding_version ON package_versions (package_id, padding_version, is_pre_release, version);
COMMENT ON TABLE package_versions IS 'package version info';
COMMENT ON COLUMN package_versions.id IS 'primary key';
COMMENT ON COLUMN package_versions.gmt_create IS 'create time';
COMMENT ON COLUMN package_versions.gmt_modified IS 'modified time';
COMMENT ON COLUMN package_versions.package_id IS 'package id';
COMMENT ON COLUMN package_versions.package_version_id IS 'package version id';
COMMENT ON COLUMN package_versions.version IS 'package version';
COMMENT ON COLUMN package_versions.abbreviated_dist_id IS 'abbreviated manifest dist id';
COMMENT ON COLUMN package_versions.manifest_dist_id IS 'manifest dist id';
COMMENT ON COLUMN package_versions.tar_dist_id IS 'tar dist id';
COMMENT ON COLUMN package_versions.readme_dist_id IS 'readme dist id';
COMMENT ON COLUMN package_versions.publish_time IS 'publish time';
COMMENT ON COLUMN package_versions.padding_version IS 'token name';
COMMENT ON COLUMN package_versions.is_pre_release IS '是否是先行版本';
CREATE TABLE packages (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_id varchar(24) NOT NULL,
is_private boolean NOT NULL DEFAULT false,
name varchar(214) NOT NULL,
scope varchar(214) NOT NULL,
description varchar(10240) DEFAULT NULL,
abbreviateds_dist_id varchar(24) DEFAULT NULL,
manifests_dist_id varchar(24) DEFAULT NULL,
registry_id varchar(24) DEFAULT NULL
);
CREATE UNIQUE INDEX packages_uk_package_id ON packages (package_id);
CREATE UNIQUE INDEX packages_uk_scope_name ON packages (scope, name);
COMMENT ON TABLE packages IS 'package info';
COMMENT ON COLUMN packages.id IS 'primary key';
COMMENT ON COLUMN packages.gmt_create IS 'create time';
COMMENT ON COLUMN packages.gmt_modified IS 'modified time';
COMMENT ON COLUMN packages.package_id IS 'package id';
COMMENT ON COLUMN packages.is_private IS 'private pkg or not, 1: true, other: false';
COMMENT ON COLUMN packages.name IS 'package name';
COMMENT ON COLUMN packages.scope IS 'package name, empty string meaning no scope';
COMMENT ON COLUMN packages.description IS 'package description';
COMMENT ON COLUMN packages.abbreviateds_dist_id IS 'all abbreviated manifests dist id';
COMMENT ON COLUMN packages.manifests_dist_id IS 'all full manifests dist id';
COMMENT ON COLUMN packages.registry_id IS 'source registry';
CREATE TABLE proxy_caches (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
fullname varchar(214) NOT NULL DEFAULT '',
version varchar(214) DEFAULT NULL,
file_type varchar(30) NOT NULL DEFAULT '',
file_path varchar(512) NOT NULL DEFAULT ''
);
CREATE UNIQUE INDEX proxy_caches_uk_package_version_path_name ON proxy_caches (file_path);
CREATE UNIQUE INDEX proxy_caches_ux_package_version_file_name ON proxy_caches (fullname, file_type, version);
COMMENT ON TABLE proxy_caches IS 'proxy mode cached files index';
COMMENT ON COLUMN proxy_caches.id IS 'primary key';
COMMENT ON COLUMN proxy_caches.gmt_create IS 'create time';
COMMENT ON COLUMN proxy_caches.gmt_modified IS 'modify time';
COMMENT ON COLUMN proxy_caches.fullname IS '@scope/package name';
COMMENT ON COLUMN proxy_caches.version IS 'package version';
COMMENT ON COLUMN proxy_caches.file_type IS 'file type';
COMMENT ON COLUMN proxy_caches.file_path IS 'nfs file path';
CREATE TABLE registries (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
registry_id varchar(24) NOT NULL,
name varchar(256) DEFAULT NULL,
host varchar(4096) DEFAULT NULL,
change_stream varchar(4096) DEFAULT NULL,
type varchar(256) DEFAULT NULL,
user_prefix varchar(256) DEFAULT NULL,
auth_token varchar(256) DEFAULT NULL
);
CREATE UNIQUE INDEX registries_uk_name ON registries (name);
COMMENT ON TABLE registries IS 'registry info';
COMMENT ON COLUMN registries.id IS 'primary key';
COMMENT ON COLUMN registries.gmt_create IS 'create time';
COMMENT ON COLUMN registries.gmt_modified IS 'modified time';
COMMENT ON COLUMN registries.registry_id IS 'registry id';
COMMENT ON COLUMN registries.name IS 'registry name';
COMMENT ON COLUMN registries.host IS 'registry host';
COMMENT ON COLUMN registries.change_stream IS 'change stream url';
COMMENT ON COLUMN registries.type IS 'registry type cnpmjsorg/cnpmcore/npm';
COMMENT ON COLUMN registries.user_prefix IS 'user prefix';
COMMENT ON COLUMN registries.auth_token IS 'registry auth token';
CREATE TABLE scopes (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
scope_id varchar(24) NOT NULL,
name varchar(214) DEFAULT NULL,
registry_id varchar(24) NOT NULL
);
CREATE UNIQUE INDEX scopes_uk_name ON scopes (name);
COMMENT ON TABLE scopes IS 'scope info';
COMMENT ON COLUMN scopes.id IS 'primary key';
COMMENT ON COLUMN scopes.gmt_create IS 'create time';
COMMENT ON COLUMN scopes.gmt_modified IS 'modified time';
COMMENT ON COLUMN scopes.scope_id IS 'scope id';
COMMENT ON COLUMN scopes.name IS 'scope name';
COMMENT ON COLUMN scopes.registry_id IS 'registry id';
CREATE TABLE tasks (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
task_id varchar(24) NOT NULL,
type varchar(20) NOT NULL,
state varchar(20) NOT NULL,
target_name varchar(214) NOT NULL,
author_id varchar(24) NOT NULL,
author_ip varchar(100) NOT NULL,
data json DEFAULT NULL,
log_path varchar(512) NOT NULL,
log_store_position varchar(10) NOT NULL,
attempts integer DEFAULT 0,
error text,
biz_id varchar(100) DEFAULT NULL
);
CREATE UNIQUE INDEX tasks_uk_task_id ON tasks (task_id);
CREATE UNIQUE INDEX tasks_uk_biz_id ON tasks (biz_id);
CREATE INDEX tasks_idx_type_state_target_name ON tasks (target_name, type, state);
CREATE INDEX tasks_idx_type_state_gmt_modified ON tasks (type, state, gmt_modified);
CREATE INDEX tasks_idx_gmt_modified ON tasks (gmt_modified);
COMMENT ON TABLE tasks IS 'task info';
COMMENT ON COLUMN tasks.id IS 'primary key';
COMMENT ON COLUMN tasks.gmt_create IS 'create time';
COMMENT ON COLUMN tasks.gmt_modified IS 'modified time';
COMMENT ON COLUMN tasks.task_id IS 'task id';
COMMENT ON COLUMN tasks.type IS 'task type';
COMMENT ON COLUMN tasks.state IS 'task state';
COMMENT ON COLUMN tasks.target_name IS 'target name, like package name / user name';
COMMENT ON COLUMN tasks.author_id IS 'create task user id';
COMMENT ON COLUMN tasks.author_ip IS 'create task user request ip';
COMMENT ON COLUMN tasks.data IS 'task params';
COMMENT ON COLUMN tasks.log_path IS 'access path';
COMMENT ON COLUMN tasks.log_store_position IS 'cloud store disk position';
COMMENT ON COLUMN tasks.attempts IS 'task execute attempts times';
COMMENT ON COLUMN tasks.error IS 'error description';
COMMENT ON COLUMN tasks.biz_id IS 'unique biz id to keep task unique';
CREATE TABLE token_packages (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
token_id varchar(24) NOT NULL,
package_id varchar(24) NOT NULL
);
CREATE UNIQUE INDEX token_packages_uk_token_id_package_id ON token_packages (token_id, package_id);
CREATE INDEX token_packages_idx_token_id ON token_packages (token_id);
CREATE INDEX token_packages_idx_package_id ON token_packages (package_id);
COMMENT ON TABLE token_packages IS 'token allowed packages';
COMMENT ON COLUMN token_packages.id IS 'primary key';
COMMENT ON COLUMN token_packages.gmt_create IS 'create time';
COMMENT ON COLUMN token_packages.gmt_modified IS 'modified time';
COMMENT ON COLUMN token_packages.token_id IS 'token id';
COMMENT ON COLUMN token_packages.package_id IS 'package id';
CREATE TABLE tokens (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
token_id varchar(24) NOT NULL,
token_mark varchar(20) NOT NULL,
token_key varchar(200) NOT NULL,
is_readonly boolean NOT NULL DEFAULT false,
is_automation boolean NOT NULL DEFAULT false,
cidr_whitelist json NOT NULL,
user_id varchar(24) NOT NULL,
name varchar(255) DEFAULT NULL,
type varchar(255) DEFAULT NULL,
description varchar(255) DEFAULT NULL,
allowed_scopes text,
expired_at timestamp(3) DEFAULT NULL,
last_used_at timestamp(3) DEFAULT NULL
);
CREATE UNIQUE INDEX tokens_uk_token_id ON tokens (token_id);
CREATE UNIQUE INDEX tokens_uk_token_key ON tokens (token_key);
CREATE UNIQUE INDEX tokens_uk_user_id_name ON tokens (user_id, name);
CREATE INDEX tokens_idx_user_id ON tokens (user_id);
COMMENT ON TABLE tokens IS 'token info';
COMMENT ON COLUMN tokens.id IS 'primary key';
COMMENT ON COLUMN tokens.gmt_create IS 'create time';
COMMENT ON COLUMN tokens.gmt_modified IS 'modified time';
COMMENT ON COLUMN tokens.token_id IS 'token id';
COMMENT ON COLUMN tokens.token_mark IS 'token mark value';
COMMENT ON COLUMN tokens.token_key IS 'token value sha512 hex';
COMMENT ON COLUMN tokens.is_readonly IS 'readonly token or not, 1: true, other: false';
COMMENT ON COLUMN tokens.is_automation IS 'automation token or not, 1: true, other: false';
COMMENT ON COLUMN tokens.cidr_whitelist IS 'ip list, ["127.0.0.1"]';
COMMENT ON COLUMN tokens.user_id IS 'user id';
COMMENT ON COLUMN tokens.name IS 'token name';
COMMENT ON COLUMN tokens.type IS 'token type, granular or legacy';
COMMENT ON COLUMN tokens.description IS 'token description';
COMMENT ON COLUMN tokens.allowed_scopes IS 'scope allowed list';
COMMENT ON COLUMN tokens.expired_at IS 'token expiration time';
COMMENT ON COLUMN tokens.last_used_at IS 'token last used time';
CREATE TABLE total (
id BIGSERIAL PRIMARY KEY,
total_id varchar(24) NOT NULL,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
package_count bigint NOT NULL DEFAULT 0,
package_file_size bigint NOT NULL DEFAULT 0,
package_version_count bigint NOT NULL DEFAULT 0,
package_version_delete_count bigint NOT NULL DEFAULT 0,
private_package_count bigint NOT NULL DEFAULT 0,
private_package_file_size bigint NOT NULL DEFAULT 0,
private_package_version_count bigint NOT NULL DEFAULT 0,
private_package_version_delete_count bigint NOT NULL DEFAULT 0,
change_stream_seq varchar(100) DEFAULT NULL
);
CREATE UNIQUE INDEX total_uk_total_id ON total (total_id);
COMMENT ON TABLE total IS 'total info';
COMMENT ON COLUMN total.id IS 'primary key';
COMMENT ON COLUMN total.total_id IS 'total id, should set it to "global"';
COMMENT ON COLUMN total.gmt_create IS 'create time';
COMMENT ON COLUMN total.gmt_modified IS 'modified time';
COMMENT ON COLUMN total.package_count IS 'package count';
COMMENT ON COLUMN total.package_file_size IS 'package all files total size';
COMMENT ON COLUMN total.package_version_count IS 'package version count';
COMMENT ON COLUMN total.package_version_delete_count IS 'package version delete count';
COMMENT ON COLUMN total.private_package_count IS 'private package count';
COMMENT ON COLUMN total.private_package_file_size IS 'private package all files total size';
COMMENT ON COLUMN total.private_package_version_count IS 'private package version count';
COMMENT ON COLUMN total.private_package_version_delete_count IS 'private package version delete count';
COMMENT ON COLUMN total.change_stream_seq IS 'npm change stream sync data seq id';
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
user_id varchar(24) NOT NULL,
name varchar(100) NOT NULL,
email varchar(400) NOT NULL,
password_salt varchar(100) NOT NULL,
password_integrity varchar(512) NOT NULL,
ip varchar(100) NOT NULL,
is_private boolean NOT NULL DEFAULT true,
scopes json DEFAULT NULL
);
CREATE UNIQUE INDEX users_uk_user_id ON users (user_id);
CREATE UNIQUE INDEX users_uk_name ON users (name);
COMMENT ON TABLE users IS 'user info';
COMMENT ON COLUMN users.id IS 'primary key';
COMMENT ON COLUMN users.gmt_create IS 'create time';
COMMENT ON COLUMN users.gmt_modified IS 'modified time';
COMMENT ON COLUMN users.user_id IS 'user id';
COMMENT ON COLUMN users.name IS 'user name';
COMMENT ON COLUMN users.email IS 'user email';
COMMENT ON COLUMN users.password_salt IS 'password salt';
COMMENT ON COLUMN users.password_integrity IS 'password integrity';
COMMENT ON COLUMN users.ip IS 'user login request ip';
COMMENT ON COLUMN users.is_private IS 'private user or not, 1: true, other: false';
COMMENT ON COLUMN users.scopes IS 'white scope list, ["@cnpm", "@foo"]';
CREATE TABLE webauthn_credentials (
id BIGSERIAL PRIMARY KEY,
gmt_create timestamp(3) NOT NULL,
gmt_modified timestamp(3) NOT NULL,
wanc_id varchar(24) NOT NULL,
user_id varchar(24) NOT NULL,
credential_id varchar(200) NOT NULL,
public_key varchar(512) NOT NULL,
browser_type varchar(20) DEFAULT NULL
);
CREATE UNIQUE INDEX webauthn_credentials_uk_wanc_id ON webauthn_credentials (wanc_id);
CREATE INDEX webauthn_credentials_idx_user_id ON webauthn_credentials (user_id);
COMMENT ON TABLE webauthn_credentials IS 'webauthn credential info';
COMMENT ON COLUMN webauthn_credentials.id IS 'primary key';
COMMENT ON COLUMN webauthn_credentials.gmt_create IS 'create time';
COMMENT ON COLUMN webauthn_credentials.gmt_modified IS 'modified time';
COMMENT ON COLUMN webauthn_credentials.wanc_id IS 'webauthn credential id';
COMMENT ON COLUMN webauthn_credentials.user_id IS 'user id';
COMMENT ON COLUMN webauthn_credentials.credential_id IS 'webauthn credential id';
COMMENT ON COLUMN webauthn_credentials.public_key IS 'webauthn credential publick key';
COMMENT ON COLUMN webauthn_credentials.browser_type IS 'user browser name';

View File

@@ -5,11 +5,13 @@ import { tmpdir } from 'os';
import { mkdtempSync } from 'fs';
import { Readable } from 'stream';
import mysql from 'mysql2';
import pg from 'pg';
import path from 'path';
import crypto from 'crypto';
import { cleanUserPrefix, getScopeAndName } from '../app/common/PackageUtil';
import semver from 'semver';
import { PackageJSONType } from '../app/repository/PackageRepository';
import { database, DATABASE_TYPE } from '../config/database';
type PackageOptions = {
name?: string;
@@ -44,20 +46,14 @@ export class TestUtil {
private static _app;
private static ua = 'npm/7.0.0 cnpmcore-unittest/1.0.0';
static getMySqlConfig() {
static getDatabaseConfig() {
return {
host: process.env.MYSQL_HOST || '127.0.0.1',
port: process.env.MYSQL_PORT || 3306,
user: process.env.MYSQL_USER || 'root',
password: process.env.MYSQL_PASSWORD,
...database,
database: database.name ?? 'cnpmcore_unittest',
multipleStatements: true,
};
}
static getDatabase() {
return process.env.MYSQL_DATABASE || 'cnpmcore_unittest';
}
// 不同的 npm 版本 cli 命令不同
// 通过 coffee 运行时获取对应版本号
static async getNpmVersion() {
@@ -65,26 +61,17 @@ export class TestUtil {
return semver.clean(res.stdout);
}
// 获取当前所有 sql 脚本内容
// 目前统一放置在 ../sql 文件夹中
// 默认根据版本号排序,确保向后兼容
static async getTableSqls(): Promise<string> {
const dirents = await fs.readdir(path.join(__dirname, '../sql'));
let versions = dirents.filter(t => path.extname(t) === '.sql').map(t => path.basename(t, '.sql'));
versions = semver.sort(versions);
const sqls = await Promise.all(versions.map(version => {
return fs.readFile(path.join(__dirname, '../sql', `${version}.sql`), 'utf8');
}));
return sqls.join('\n');
}
static async query(sql: string): Promise<any[]> {
const conn = this.getConnection();
return new Promise((resolve, reject) => {
conn.query(sql, (err: Error, rows: any[]) => {
conn.query(sql, (err: Error, rows: any) => {
if (err) {
return reject(err);
}
if (rows.rows) {
// pg: { rows }
return resolve(rows.rows);
}
return resolve(rows);
});
});
@@ -92,11 +79,15 @@ export class TestUtil {
static getConnection() {
if (!this.connection) {
const config: any = this.getMySqlConfig();
const config = this.getDatabaseConfig();
if (process.env.CI) {
console.log('[TestUtil] connection to mysql: %j', config);
console.log('[TestUtil] connection to database: %j', config);
}
if (config.type === DATABASE_TYPE.MySQL) {
this.connection = mysql.createConnection(config as any);
} else if (config.type === DATABASE_TYPE.PostgreSQL) {
this.connection = new pg.Client(config as any);
}
this.connection = mysql.createConnection(config);
this.connection.connect();
}
return this.connection;
@@ -111,18 +102,26 @@ export class TestUtil {
static async getTableNames() {
if (!this.tables) {
const database = this.getDatabase();
const sql = `SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = '${database}';`;
const rows = await this.query(sql);
this.tables = rows.map(row => row.TABLE_NAME);
const config = this.getDatabaseConfig();
if (config.type === DATABASE_TYPE.MySQL) {
const sql = `
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = '${config.database}';`;
const rows = await this.query(sql);
this.tables = rows.map(row => row.TABLE_NAME);
} else if (config.type === DATABASE_TYPE.PostgreSQL) {
const sql = 'SELECT * FROM pg_catalog.pg_tables where schemaname = \'public\';';
const rows = await this.query(sql);
this.tables = rows.map(row => row.tablename);
}
}
return this.tables;
}
static async truncateDatabase() {
const database = this.getDatabase();
const tables = await this.getTableNames();
await Promise.all(tables.map((table: string) => this.query(`TRUNCATE TABLE ${database}.${table};`)));
await Promise.all(tables.map(async (table: string) => {
await this.query(`TRUNCATE TABLE ${table};`);
}));
}
static get app() {

View File

@@ -4,6 +4,7 @@ import { app, mock } from 'egg-mock/bootstrap';
import { TestUtil } from '../../../../test/TestUtil';
import { PackageVersionFileService } from '../../../../app/core/service/PackageVersionFileService';
import { calculateIntegrity } from '../../../../app/common/PackageUtil';
import { database, DATABASE_TYPE } from '../../../../config/database';
describe('test/port/controller/PackageVersionFileController/listFiles.test.ts', () => {
let publisher;
@@ -151,7 +152,7 @@ describe('test/port/controller/PackageVersionFileController/listFiles.test.ts',
});
});
it('should return the current directory\'s files and directories instead all sub items', async () => {
it('should return the current directory files and directories instead all sub items', async () => {
mock(app.config.cnpmcore, 'allowPublishNonScopePackage', true);
const tarball = await TestUtil.readFixturesFile('unpkg.com/openapi-7.3.3.tgz');
const { integrity } = await calculateIntegrity(tarball);
@@ -184,139 +185,143 @@ describe('test/port/controller/PackageVersionFileController/listFiles.test.ts',
file.lastModified = '2024-05-18T16:00:18.307Z';
}
// console.log(JSON.stringify(res.body, null, 2));
assert.deepEqual(res.body, {
path: '/',
type: 'directory',
files: [
{
path: '/LICENSE',
type: 'file',
contentType: 'text/plain',
integrity: 'sha512-OJCAthMtPqrngGSNaZg5DYzHGQhWG84JV44nxUKqGp8xIuAfZAxbAb7nMATCOqTp8gZv5e4MogcsJCBXiyjXHw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 11357,
},
{
path: '/index.html',
type: 'file',
contentType: 'text/html',
integrity: 'sha512-L4Vxx8DW1PtZfPut4uwP9DSK9+DbFbKDWWGp4KK5TRKGTHSjYoMExqY50WiTKs/bGu1Ecpneiu3dnYlRZ/sDdw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 1437,
},
{
path: '/package.json',
type: 'file',
contentType: 'application/json',
integrity: 'sha512-ke5ybpErJgl+Mul1XCSMvly0uYAt8/5mWa5/yYykxfMCE0OBpzgWoFHC+/RM9AQfNgic3bW/ssHXDUUPZiEKkg==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 2852,
},
{
path: '/CHANGES.md',
type: 'file',
contentType: 'text/markdown',
integrity: 'sha512-xxD+0Mdep4Pprq0JsudGLCKtSfHBeIqJVoGqM0qK1b2B/0sXjSQYinxgAwjK8rKSD0jNSo3R5aK8VbgOXLtbjw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 12346,
},
{
path: '/README.md',
type: 'file',
contentType: 'text/markdown',
integrity: 'sha512-Nnj8b9SsDDobga1LsV7FVE46YrxkdZf5MOMboVHICw56tPHnQ0v1lXvXkWz7k12kTFWbA0z42daaW7WE+AQWfw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 4409,
},
{
path: '/.npmcheckrc.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-EYTJJ5StGM9DUpAbF8XHV4Z02rlmzN9O6k93fu1YXpf1wDBtmFYG64xaTXk2UfB8x0BCotga+Upm1yOgJVIZTQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 105,
},
{
path: '/.redocly.lint-ignore.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-tyPeiIaOGIXb3PNFb2ELAZawxGHSdPZ7IoLdl+tEcDARVFlq6B9yJVAzL5R8L26iCBbvPtlfNGnYkHj4H/5ZMA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 644,
},
{
path: '/index.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-KW7xaZW5F8NOGt72kc9WvLcvkFDmXbm65JdWPM2pYfy9HMX0/6obJD5jhzQSX5ZU8ww0HMlXGXkRviFnDr88ZA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 21379,
},
{
path: '/.eslintrc.yml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-3q0aghG4dBd7pgE4UrbtVn52cfg3BqOPkuNcCSwHZKMSFnKZxWr+sH7/OgnBDaifVsXGK7AN8q7sX0Eds6Ditw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 149,
},
{
path: '/oauth2',
type: 'directory',
files: [],
},
{
path: '/_util',
type: 'directory',
files: [],
},
{
path: '/resource',
type: 'directory',
files: [],
},
{
path: '/parameters',
type: 'directory',
files: [],
},
{
path: '/id',
type: 'directory',
files: [],
},
{
path: '/location',
type: 'directory',
files: [],
},
{
path: '/string',
type: 'directory',
files: [],
},
{
path: '/money',
type: 'directory',
files: [],
},
{
path: '/time',
type: 'directory',
files: [],
},
{
path: '/human',
type: 'directory',
files: [],
},
{
path: '/health',
type: 'directory',
files: [],
},
],
});
if (database.type === DATABASE_TYPE.PostgreSQL) {
assert.equal(res.body.files.length, 20);
} else {
assert.deepEqual(res.body, {
path: '/',
type: 'directory',
files: [
{
path: '/LICENSE',
type: 'file',
contentType: 'text/plain',
integrity: 'sha512-OJCAthMtPqrngGSNaZg5DYzHGQhWG84JV44nxUKqGp8xIuAfZAxbAb7nMATCOqTp8gZv5e4MogcsJCBXiyjXHw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 11357,
},
{
path: '/index.html',
type: 'file',
contentType: 'text/html',
integrity: 'sha512-L4Vxx8DW1PtZfPut4uwP9DSK9+DbFbKDWWGp4KK5TRKGTHSjYoMExqY50WiTKs/bGu1Ecpneiu3dnYlRZ/sDdw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 1437,
},
{
path: '/package.json',
type: 'file',
contentType: 'application/json',
integrity: 'sha512-ke5ybpErJgl+Mul1XCSMvly0uYAt8/5mWa5/yYykxfMCE0OBpzgWoFHC+/RM9AQfNgic3bW/ssHXDUUPZiEKkg==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 2852,
},
{
path: '/CHANGES.md',
type: 'file',
contentType: 'text/markdown',
integrity: 'sha512-xxD+0Mdep4Pprq0JsudGLCKtSfHBeIqJVoGqM0qK1b2B/0sXjSQYinxgAwjK8rKSD0jNSo3R5aK8VbgOXLtbjw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 12346,
},
{
path: '/README.md',
type: 'file',
contentType: 'text/markdown',
integrity: 'sha512-Nnj8b9SsDDobga1LsV7FVE46YrxkdZf5MOMboVHICw56tPHnQ0v1lXvXkWz7k12kTFWbA0z42daaW7WE+AQWfw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 4409,
},
{
path: '/.npmcheckrc.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-EYTJJ5StGM9DUpAbF8XHV4Z02rlmzN9O6k93fu1YXpf1wDBtmFYG64xaTXk2UfB8x0BCotga+Upm1yOgJVIZTQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 105,
},
{
path: '/.redocly.lint-ignore.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-tyPeiIaOGIXb3PNFb2ELAZawxGHSdPZ7IoLdl+tEcDARVFlq6B9yJVAzL5R8L26iCBbvPtlfNGnYkHj4H/5ZMA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 644,
},
{
path: '/index.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-KW7xaZW5F8NOGt72kc9WvLcvkFDmXbm65JdWPM2pYfy9HMX0/6obJD5jhzQSX5ZU8ww0HMlXGXkRviFnDr88ZA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 21379,
},
{
path: '/.eslintrc.yml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-3q0aghG4dBd7pgE4UrbtVn52cfg3BqOPkuNcCSwHZKMSFnKZxWr+sH7/OgnBDaifVsXGK7AN8q7sX0Eds6Ditw==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 149,
},
{
path: '/oauth2',
type: 'directory',
files: [],
},
{
path: '/_util',
type: 'directory',
files: [],
},
{
path: '/resource',
type: 'directory',
files: [],
},
{
path: '/parameters',
type: 'directory',
files: [],
},
{
path: '/id',
type: 'directory',
files: [],
},
{
path: '/location',
type: 'directory',
files: [],
},
{
path: '/string',
type: 'directory',
files: [],
},
{
path: '/money',
type: 'directory',
files: [],
},
{
path: '/time',
type: 'directory',
files: [],
},
{
path: '/human',
type: 'directory',
files: [],
},
{
path: '/health',
type: 'directory',
files: [],
},
],
});
}
res = await app.httpRequest()
.get(`/${pkg.name}/1.0.0/files/id/?meta`);
@@ -326,83 +331,87 @@ describe('test/port/controller/PackageVersionFileController/listFiles.test.ts',
file.lastModified = '2024-05-18T16:00:18.307Z';
}
// console.log(JSON.stringify(res.body, null, 2));
assert.deepEqual(res.body, {
path: '/id',
type: 'directory',
files: [
{
path: '/id/AccountId.d.ts',
type: 'file',
contentType: 'text/plain',
integrity: 'sha512-xj1/RCRAp72pukals97C98DG0b38Gl2xNrUwOi2SRj+EnJKIfQX8WisDpCOSKLFq5j++sGbL0/4wCttrPvi37w==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 787,
},
{
path: '/id/AccountId.js',
type: 'file',
contentType: 'application/javascript',
integrity: 'sha512-kFa+SXSMGbCh2DiuSGmlCS8OCBSE4VRGlq/A2IyY3QxL794soFq4zO3F+UEx4ANUG33urAa4LG4IY2OiUc2Mng==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 1343,
},
{
path: '/id/AccountId.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-R6WB9dXEaNpvqIAH6OdRQ77gSEBlq1GeH2jv2tv1wQEVOmzQtErHlpj+ukvZUwzqf9wTXIPxKjeUhqk6VbfBkA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 571,
},
{
path: '/id/Mode.js',
type: 'file',
contentType: 'application/javascript',
integrity: 'sha512-jfMuIff4LW/ZQ8el9iCww8c9gw+12UK7eZn+6TMDAlStfLhlu8u7jcCSWSEG1zBTty9DIHn4Nbp+dMDjRUnVWQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 3357,
},
{
path: '/id/mode.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-er9S1Da52G8fxwfgxhNbcXPdYz9bzABM7VifDXhgVGX+hwtu8tve9y2aZhPAHcJOy3dClMDQ1eYLAHp7k8TMNQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 1222,
},
{
path: '/id/UUID.js',
type: 'file',
contentType: 'application/javascript',
integrity: 'sha512-bo/JyxOZeRRjbN0OR8vNRz3cTY2GcJfRmRnp3QTGXE5iuKYjrpjYzj+vEXopZV1QYPdZaXUK671EoysPE59pQQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 992,
},
{
path: '/id/UUID.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-Gjr0LNqWQcO5/oaCyMm9oZWpc/D9K6Qe37sGuYv4kbq0I8teZL92xbR81L+2VShkhLSXdg2Qw5WRjwCkSWyfoA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 659,
},
{
path: '/id/legalPerson',
type: 'directory',
files: [],
},
{
path: '/id/naturalPerson',
type: 'directory',
files: [],
},
{
path: '/id/sigedis',
type: 'directory',
files: [],
},
],
});
if (database.type === DATABASE_TYPE.PostgreSQL) {
assert.equal(res.body.files.length, 10);
} else {
assert.deepEqual(res.body, {
path: '/id',
type: 'directory',
files: [
{
path: '/id/AccountId.d.ts',
type: 'file',
contentType: 'text/plain',
integrity: 'sha512-xj1/RCRAp72pukals97C98DG0b38Gl2xNrUwOi2SRj+EnJKIfQX8WisDpCOSKLFq5j++sGbL0/4wCttrPvi37w==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 787,
},
{
path: '/id/AccountId.js',
type: 'file',
contentType: 'application/javascript',
integrity: 'sha512-kFa+SXSMGbCh2DiuSGmlCS8OCBSE4VRGlq/A2IyY3QxL794soFq4zO3F+UEx4ANUG33urAa4LG4IY2OiUc2Mng==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 1343,
},
{
path: '/id/AccountId.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-R6WB9dXEaNpvqIAH6OdRQ77gSEBlq1GeH2jv2tv1wQEVOmzQtErHlpj+ukvZUwzqf9wTXIPxKjeUhqk6VbfBkA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 571,
},
{
path: '/id/Mode.js',
type: 'file',
contentType: 'application/javascript',
integrity: 'sha512-jfMuIff4LW/ZQ8el9iCww8c9gw+12UK7eZn+6TMDAlStfLhlu8u7jcCSWSEG1zBTty9DIHn4Nbp+dMDjRUnVWQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 3357,
},
{
path: '/id/mode.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-er9S1Da52G8fxwfgxhNbcXPdYz9bzABM7VifDXhgVGX+hwtu8tve9y2aZhPAHcJOy3dClMDQ1eYLAHp7k8TMNQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 1222,
},
{
path: '/id/UUID.js',
type: 'file',
contentType: 'application/javascript',
integrity: 'sha512-bo/JyxOZeRRjbN0OR8vNRz3cTY2GcJfRmRnp3QTGXE5iuKYjrpjYzj+vEXopZV1QYPdZaXUK671EoysPE59pQQ==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 992,
},
{
path: '/id/UUID.yaml',
type: 'file',
contentType: 'text/yaml',
integrity: 'sha512-Gjr0LNqWQcO5/oaCyMm9oZWpc/D9K6Qe37sGuYv4kbq0I8teZL92xbR81L+2VShkhLSXdg2Qw5WRjwCkSWyfoA==',
lastModified: '2024-05-18T16:00:18.307Z',
size: 659,
},
{
path: '/id/legalPerson',
type: 'directory',
files: [],
},
{
path: '/id/naturalPerson',
type: 'directory',
files: [],
},
{
path: '/id/sigedis',
type: 'directory',
files: [],
},
],
});
}
res = await app.httpRequest()
.get(`/${pkg.name}/1.0.0/files/id/legalPerson/?meta`);

View File

@@ -232,7 +232,7 @@ describe('test/port/controller/TokenController/createToken.test.ts', () => {
expires: 30,
});
assert.match(res.body.error, /ER_DUP_ENTRY/);
assert.match(res.body.error, /ER_DUP_ENTRY|duplicate key value violates unique constraint/);
});
});

View File

@@ -30,8 +30,10 @@ describe('test/repository/ChangeRepository.test.ts', () => {
it('should list from header', async () => {
const changes = await changeRepository.query(0, 5);
const ids = changes.map(t => t.id);
// pg truncate 后id 不会重置
const firstId = Number(ids[0]) - 1;
assert.deepStrictEqual(ids, [
1, 2, 3, 4, 5,
firstId + 1, firstId + 2, firstId + 3, firstId + 4, firstId + 5,
]);
});
});