<!--- SUMMARY_MARKER --->
## Sweep Summary <sub><a href="https://app.sweep.dev"><img
src="https://raw.githubusercontent.com/sweepai/sweep/main/.assets/sweep-square.png"
width="25" alt="Sweep"></a></sub>
Improves database performance by replacing expensive count queries with
a dedicated totals table that's updated asynchronously via events.
- Created a new `totals` table in `app/repository/model/Total.ts` to
store package and version counts instead of running expensive SQL count
queries.
- Implemented `TotalRepository` in `app/repository/TotalRepository.ts`
with methods to increment and retrieve count values.
- Added event handlers in `app/core/event/TotalHandler.ts` that listen
for package and version additions to update counts asynchronously.
- Modified `PackageRepository.queryTotal()` to fetch counts from the
totals table instead of executing direct SQL count queries.
- Added migration scripts in `sql/mysql/4.3.0.sql` and
`sql/postgresql/4.3.0.sql` to create the totals table and populate it
with existing data.
---
[Ask Sweep AI questions about this PR](https://app.sweep.dev)
<!--- SUMMARY_MARKER --->
> Fix database performance issues caused by doc_count and
doc_version_count queries
1. 💽 Add a corresponding totals table to record statistical information
2. ➕ Add a `PACKAGE_ADDED` event and the original
`PACKAGE_VERSION_ADDED` event to asynchronously update records in the
table
3. ♻️ Add a new existing data migration script to migrate the original
statistical information to the totals table
-----------
> 修复 doc_count 和 doc_version_count 查询导致的数据库性能问题
1. 💽 新增对应 totals 表，用来记录统计信息
2. ➕ 新增 `PACKAGE_ADDED` 事件，和原有 `PACKAGE_VERSION_ADDED` 事件，异步更新表内记录
3. ♻️ 新增存量数据迁移脚本，迁移原有的统计信息到 totals 表
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced persistent tracking of total package and package version
counts, with real-time updates when new packages or versions are added.
- Added new data models and repository methods to manage and retrieve
these total counts.
- Emitted events upon new package creation to update totals
automatically.
- **Database**
- Added a new "totals" table to both MySQL and PostgreSQL databases for
storing aggregate counts initialized from existing data.
- **Bug Fixes**
- Ensured total counts are always returned as numbers in scheduled data
updates.
- **Tests**
- Added and updated tests to verify correct behavior of total count
tracking, incrementing, resetting, and retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Include sql change.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced data processing with additional safeguards to limit
iterations during data retrieval, improving stability and preventing
resource overuse.
- **Chores**
- Refined database indexing by removing an old index and adding a new
one, optimizing query performance for faster data access and a smoother
overall experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/731
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Added support for PostgreSQL alongside MySQL, including a new database
setup script and comprehensive documentation for local development.
- Introduced a new CI job for PostgreSQL testing in the GitHub Actions
workflow.
- Enhanced the `README.md` and `DEVELOPER.md` files to provide clearer
instructions for using both database systems.
- Added new environment variable configurations for PostgreSQL in the
Docker deployment documentation.
- **Bug Fixes**
- Improved error handling in tests for duplicate entries to accommodate
both MySQL and PostgreSQL error messages.
- **Documentation**
- Updated setup instructions for PostgreSQL and clarified MySQL setup in
the documentation.
- Enhanced contributor information in the README.
- Expanded instructions for setting up Elasticsearch and Kibana,
including environment variable configurations.
- **Chores**
- Updated package dependencies to include PostgreSQL client libraries
and modified scripts to support both databases.
- Changed the base image in the Dockerfile to a newer Node.js version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/366
开启代理模式时如果找不到依赖会直接返回上游仓库的manifest信息并缓存于nfs，当请求的tgz文件不存在时从上游仓库获取并返回，同时创建对应版本的同步任务。每小时检查更新已缓存的manifest文件保证上游仓库发布新版本时不会因为缓存落后而404。
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced proxy cache management for package manifests and versions.
- Added new HTTP methods for managing proxy caches.
- Implemented scheduled workers for updating and synchronizing proxy
cache.
- **Updates**
- Expanded `SyncMode` enum to include a new value `proxy`.
- Updated constants with `PROXY_CACHE_DIR_NAME` and
`ABBREVIATED_META_TYPE`.
- **Tests**
- Added comprehensive test cases for `ProxyCacheService`,
`ProxyCacheRepository`, and related controllers.
- Verified functionality of scheduled workers for proxy cache updates
and synchronization.
- Enhanced testing coverage for handling package downloads in proxy
mode.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: fengmk2 <suqian.yf@antgroup.com>
> Add the lastUsedBy field to the Token model
* 🧶 Add `lastUsedBy` field to the token, for platform display.
* 🐞 Fix the issue where the graunlarToken is not expired in read-only
scenarios.
------
> 为 Token 模型添加 lastUsedBy 字段
* 🧶 新增 lastUsedBy 字段，记录 token 最近使用时间，用于平台展示
* 🐞 修复 graunlarToken 过期时，只读场景没有禁用的问题
> ENOENT: no such file or directory, stat
'/root/.cnpmcore/downloads/2023/05/06/unpkg_@iov_wallet-providers@1.0.0_0f152162-9cce-4a80-bacc-41271b7aac3f/package'
> 🚀 Added implementation related to
[granularToken](https://docs.npmjs.com/about-access-tokens#about-granular-access-tokens),
mainly used for web authorization scenarios.
* 📝 Added `1.14.0.sql` to add fields and `token_packages` for
granularToken.
* 🛣️ Added gat related routes, including `create`, `query`, and `delete`
api.
* 🌟 Added `tokenService` to check granularToken access.
* 🔄 Modified Token to perform options and data attribute conversions
internally in the model.
-----------
> 🚀 新增
[granularToken](https://docs.npmjs.com/about-access-tokens#about-granular-access-tokens)
相关实现，主要用于 web 端授权场景
* 📝 新增 `1.14.0.sql` 添加 granularToken 相关字段及 `token_packages` 中间表
* 🛣️ 新增 gat 相关路由，包括`创建`、`查询`、`删除`接口
* 🌟 新增 `tokenService` ，处理 granularToken 鉴权
* 🔄 修改 Token ，在 model 内部进行 options 和 data 属性转换
1. webauth 由 authentication 改造为 session，并增加 web 登录页面，更安全
2. 支持 webauthn 的登录方式，可通过配置控制（默认关闭），更高效
---------------
1. use session instead http authentication on webauth
2. support [webauthn](https://webauthn.guide/), you should set
`enableWebAuthn: true` in the configuration
closes https://github.com/cnpm/cnpmcore/issues/236
---------
Co-authored-by: lanxiu.lwl <lanxiu.lwl@alipay.com>
Co-authored-by: elrrrrrrr <elrrrrrrr@gmail.com>
Supoort add new version, show version and download version tar.
Only support node >= 16.13.0 for (fs|stream/promises) API
closes https://github.com/cnpm/cnpmcore/issues/4
