[skip ci]
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* FFmpeg builds are now available as a supported binary source, offering
static builds for Windows and Linux platforms (x86_64). Both master
branch and latest release versions are accessible.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an npm build script to simplify project compilation.
* **Chores**
* Simplified CI dependency installation to use standard npm install.
* Added a new deployment test job that runs multi-service integration
checks, build, health checks, and graceful shutdown.
* Introduced an environment-variable gate to allow opting into local
filesystem behavior in production.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [actions/checkout](https://redirect.github.com/actions/checkout) |
action | major | `v4` -> `v5` |
---
### Release Notes
<details>
<summary>actions/checkout (actions/checkout)</summary>
### [`v5`](https://redirect.github.com/actions/checkout/compare/v4...v5)
[Compare
Source](https://redirect.github.com/actions/checkout/compare/v4...v5)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/cnpm/cnpmcore).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xNTYuMSIsInVwZGF0ZWRJblZlciI6IjQxLjE1Ni4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbXX0=-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [actions/setup-node](https://redirect.github.com/actions/setup-node) |
action | major | `v4` -> `v6` |
---
### Release Notes
<details>
<summary>actions/setup-node (actions/setup-node)</summary>
###
[`v6`](https://redirect.github.com/actions/setup-node/compare/v5...v6)
[Compare
Source](https://redirect.github.com/actions/setup-node/compare/v5...v6)
###
[`v5`](https://redirect.github.com/actions/setup-node/compare/v4...v5)
[Compare
Source](https://redirect.github.com/actions/setup-node/compare/v4...v5)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/cnpm/cnpmcore).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xNTYuMSIsInVwZGF0ZWRJblZlciI6IjQxLjE1Ni4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbXX0=-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[actions/attest-build-provenance](https://redirect.github.com/actions/attest-build-provenance)
| action | major | `v2` -> `v3` |
---
### Release Notes
<details>
<summary>actions/attest-build-provenance
(actions/attest-build-provenance)</summary>
###
[`v3`](https://redirect.github.com/actions/attest-build-provenance/compare/v2...v3)
[Compare
Source](https://redirect.github.com/actions/attest-build-provenance/compare/v2...v3)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/cnpm/cnpmcore).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xNTYuMSIsInVwZGF0ZWRJblZlciI6IjQxLjE1Ni4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbXX0=-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This commit enhances type safety and fixes type-related issues
throughout the project including:
- Updated type definitions in entities, repositories, and models
- Improved type annotations in services and controllers
- Fixed type issues in adapters and utilities
- Enhanced test file type definitions
- Added typings/index.d.ts for global type declarations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude <noreply@anthropic.com>
- [x] Add `ripgrep-prebuilt` configuration to `config/binaries.ts`
- [x] Create test file for `ripgrep-prebuilt` binary
- [x] Create test fixture data for releases
- [x] Run and validate tests
- [x] Verify linting passes
- [x] Address code review feedback - use assert.equal instead of
assert.ok
<!-- START COPILOT CODING AGENT SUFFIX -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>mirror microsoft/ripgrep-prebuilt</issue_title>
>
<issue_description>https://github.com/microsoft/ripgrep-prebuilt/releases
>
> part of
https://github.com/cnpm/binary-mirror-config/issues/57</issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> </comments>
>
</details>
Fixescnpm/cnpmcore#841
<!-- START COPILOT CODING AGENT TIPS -->
---
💬 Share your feedback on Copilot coding agent for the chance to win a
$200 gift card! Click
[here](https://survey3.medallia.com/?EAHeSx-AP01bZqG0Ld9QLQ) to start
the survey.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: fengmk2 <156269+fengmk2@users.noreply.github.com>
Co-authored-by: MK <fengmk2@gmail.com>
## Overview
This PR significantly enhances the `.github/copilot-instructions.md`
file to provide comprehensive guidance for GitHub Copilot when assisting
with cnpmcore development. The instructions have been expanded from 281
lines to 564 lines, adding critical sections that align with GitHub's
best practices for coding agents.
## What Changed
### New Sections Added
**1. Code Style and Conventions**
- Detailed Oxlint and Prettier configuration rules
- TypeScript conventions (strict typing, avoiding `any`, ES modules)
- Testing conventions with naming patterns and mock usage
- Complete code examples for test structure
**2. Domain-Driven Design (DDD) Architecture**
- Visual layer architecture showing dependency flow
- Detailed responsibilities for each layer:
- Controller: HTTP interface, validation, authentication
- Service: Business logic orchestration
- Repository: Data access and persistence
- Entity: Domain models with business behavior
- Model: ORM definitions
- Repository method naming conventions (`findX`, `saveX`, `removeX`,
`listXs`)
- Request validation trilogy workflow (params → auth → authorization)
- Database model modification guidelines (update all 3 locations)
**3. Infrastructure Adapters**
- Documentation of enterprise customization points
- Adapter types: NFSClientAdapter, QueueAdapter, AuthAdapter,
BinaryAdapter
**4. Semantic Commit Messages**
- Conventional commit format standards
- Real-world examples for feat, fix, docs, chore, test, refactor, perf
### Sections Enhanced
**Adding New Features** (7 lines → 69 lines)
- Step-by-step bottom-up implementation workflow
- Layer-by-layer implementation guidance
- Complete TypeScript controller code example with decorators:
```typescript
@HTTPController()
export class YourController extends AbstractController {
@Inject()
private readonly yourService: YourService;
@HTTPMethod({ path: '/api/path', method: 'GET' })
async yourMethod(@HTTPQuery() params: YourQueryType) {
// 1. Validate params, 2. Authenticate, 3. Authorize, 4. Execute
}
}
```
**Testing** (15 lines → 42 lines)
- Testing philosophy (test all features, test at right layer, mock
dependencies)
- Common test patterns with setup/execute/assert structure
- Examples for both success and error cases
**Performance Notes**
- Added individual test file timing (~12 seconds)
- Added linting speed (<1 second)
- Comprehensive timing reference for CI/CD planning
## Quality Validation
✅ **Markdown Syntax**: All 48 code blocks properly balanced and closed
✅ **Completeness**: All 13 key sections present with practical examples
✅ **Code Examples**: 21 TypeScript code examples throughout
✅ **Best Practices**: Follows GitHub's recommended structure for Copilot
instructions
## Benefits
With these enhancements, GitHub Copilot can now:
- Generate code following cnpmcore's DDD architecture patterns
- Apply correct code style (Prettier + Oxlint rules)
- Create properly structured tests with appropriate mocking
- Follow repository naming conventions
- Implement the 3-step request validation pattern
- Suggest appropriate architectural layers for new features
- Generate semantic commit messages
## Related
Closes #[issue_number]
The instructions now provide comprehensive, actionable guidance that
enables GitHub Copilot to be a more effective development assistant
while maintaining code quality and architectural consistency across the
cnpmcore project.
<!-- START COPILOT CODING AGENT SUFFIX -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>✨ Set up Copilot instructions</issue_title>
> <issue_description>Configure instructions for this repository as
documented in [Best practices for Copilot coding agent in your
repository](https://gh.io/copilot-coding-agent-tips).
>
> <Onboard this repo></issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> </comments>
>
</details>
Fixescnpm/cnpmcore#837
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/cnpm/cnpmcore/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: fengmk2 <156269+fengmk2@users.noreply.github.com>
Co-authored-by: MK (fengmk2) <fengmk2@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved CI parallelization and shard-aware job naming for faster,
clearer test runs.
* **Bug Fixes**
* More robust handling of binary paths and change-stream edge cases to
reduce rare failures.
* Prevented unintended data mutation and tightened minor
version-handling resilience.
* **Chores**
* Streamlined test scripts and updated dependency resolution.
* Added Prettier ignore rules, relaxed selected lint rules, and small
build script formatting cleanup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- [x] Add time field to abbreviated manifests in
`_listPackageAbbreviatedManifests` method
- [x] Fix cache update logic to populate time field when adding versions
to abbreviated manifests
- [x] Fix cache update logic to remove time field when removing versions
from abbreviated manifests
- [x] Add comprehensive test coverage for time field in abbreviated
manifests
- [x] All tests passing (30/30 in ShowPackageController.test.ts)
closes https://github.com/cnpm/cnpmcore/issues/609
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Package metadata now consistently includes time information (created,
modified, and per-version publish timestamps) in both full and
abbreviated manifests.
* Removing a version also removes its corresponding time entry to keep
metadata accurate.
* Added guards to prevent writing invalid time data, improving
stability.
* **Tests**
* Added tests to verify presence and correctness of time fields in
abbreviated manifests, including created/modified and per-version
timestamps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: fengmk2 <156269+fengmk2@users.noreply.github.com>
Co-authored-by: MK (fengmk2) <fengmk2@gmail.com>
This PR adds comprehensive documentation for cnpmcore's internal APIs,
enabling users to make direct HTTP requests without needing the cnpm
client with outdated dependencies.
## Background
Users requested documentation for internal APIs to avoid using the cnpm
client, which has outdated dependencies like `bagpipe` (11 years old)
and `npm-request` (9 years old). The primary use case is package
synchronization functionality that was previously only accessible
through the cnpm CLI.
## Changes
### New Documentation: `docs/internal-api.md`
Created a comprehensive 988-line API reference covering:
**Package Sync API** - The primary requested functionality:
- `PUT /-/package/:fullname/syncs` - Modern sync API
- `PUT /:fullname/sync` - Legacy sync API (cnpmjs.org compatibility)
- `GET /-/package/:fullname/syncs/:taskId` - Check sync status
- `GET /-/package/:fullname/syncs/:taskId/log` - View sync logs
**Additional APIs documented:**
- **Token Management**: Create, list, delete authentication tokens
- **Hook Management**: CRUD operations for webhooks
- **Package Administration**: Block/unblock packages (admin only)
- **Registry Management**: Manage multiple npm registries
- **Binary Sync**: List and browse binary packages
- **User Management**: User registration and authentication
### Documentation Features
- Complete curl examples for all endpoints
- Request/response schemas with validation rules
- Authentication requirements clearly marked
- Error handling with HTTP status codes
- Security considerations and best practices
- Real-world workflow examples
### Updated README.md
Added reference to the new internal API documentation to make it
discoverable.
## Usage Examples
Users can now sync packages directly:
```bash
# Trigger package sync
curl -X PUT \
-H "Authorization: Bearer your-token" \
-H "Content-Type: application/json" \
-d '{"tips": "API sync request"}' \
https://your-registry.com/-/package/lodash/syncs
# Check sync status
curl -H "Authorization: Bearer your-token" \
https://your-registry.com/-/package/lodash/syncs/task-id
# View sync logs
curl -H "Authorization: Bearer your-token" \
https://your-registry.com/-/package/lodash/syncs/task-id/log
```
## Validation
All documented endpoints have been verified against the actual
implementation in the controllers. The API paths, parameters, and
response formats match the codebase exactly.
This enables users to extract sync functionality and other operations
without dealing with outdated cnpm dependencies, directly addressing the
issue's core request.
Fixes#709.
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: fengmk2 <156269+fengmk2@users.noreply.github.com>
Co-authored-by: MK (fengmk2) <fengmk2@gmail.com>
[skip ci]
- [x] Extended the `BinaryType` enum to include `Firefox = 'firefox'`,
following the established pattern for other binary types.
- [x] Implemented a new `FirefoxBinary` class that:
- Extends the `AbstractBinary` base class
- Parses Mozilla's HTML directory listing format using regex patterns
- Handles both version directories (e.g., `131.0.3/`) and binary files
(e.g., `firefox-131.0.3.tar.bz2`)
- Supports all standard platforms: `linux-i686`, `linux-x86_64`, `mac`,
`win32`, `win64`
- **Filters out old Firefox versions < 100.0.0** to reduce noise and
improve performance
- [x] Configured Firefox binary mirroring in `binaries.ts`
- [x] Added intelligent version filtering that:
- Only processes Firefox version directories >= 100.0.0
- Uses the `semver` library for accurate version comparison
- Handles beta/RC versions (e.g., "131.0b3") by extracting the base
version ("131.0")
- Pads two-part versions to semver format (e.g., "131.0" -> "131.0.0")
- Skips directories with versions < 100.0.0 to avoid outdated releases
- [x] **Implemented focused unit tests that work in CI environments**:
- ✅ Binary instantiation and dependency injection tests
- ✅ Version filtering verification (correctly includes >= 100.0.0,
excludes < 100.0.0)
- Tests use realistic test fixtures simulating actual Mozilla archive
structure
- **Test cases with old versions (3.6, 52.0, 78.0, 99.0) to verify
filtering works correctly**
- **Unit tests work without requiring external network access or HTTP
mocking**
- [x] **Updated `131.0.3.html` to match the original Mozilla archive
HTML format with proper line breaks**
- [x] **Enhanced error handling to log version parsing failures instead
of silently ignoring them**:
- Added descriptive error logging with proper tags following project
conventions
- Includes directory name and error details for better troubleshooting
- Uses `this.logger.warn()` for appropriate log level
- [x] **Resolved all linting errors in Firefox binary tests**:
- Fixed unused catch parameter by removing the unused `error` parameter
- Replaced array with Set for better performance when checking existence
- **Removed unused `gte` import from semver library**
- All linting checks now pass successfully
- [x] **Implemented useful preview tests with HTTP mocking**:
- Tests use `app.mockHttpclient()` instead of real network requests
- Provide end-to-end integration testing of Firefox binary fetching
- Include root directory fetching, version filtering, and file parsing
tests
- Safe for CI environments since they don't require external network
access
- Help developers preview and validate the Firefox binary functionality
- [x] **Fixed test failure with improved error messages and debugging
support**:
- Added `filteredResults` variable in test for better error reporting
- Enhanced assertion messages to show actual vs expected results
- Improved test robustness and debugging capabilities
- [x] **Removed redundant HTML parsing logic tests**:
- Cleaned up duplicate test logic that was testing the same
functionality
- Kept the more valuable preview tests that provide end-to-end
integration testing
- Focused test suite on the actual API interface rather than internal
implementation details
This enables Puppeteer users in China to configure Firefox binary
downloads through cnpmcore's mirror infrastructure, improving download
speeds and reliability while focusing on modern Firefox versions.
Fixes#826.
<!-- START COPILOT CODING AGENT TIPS -->
---
💬 Share your feedback on Copilot coding agent for the chance to win a
$200 gift card! Click
[here](https://survey3.medallia.com/?EAHeSx-AP01bZqG0Ld9QLQ) to start
the survey.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added Firefox as a selectable binary source alongside existing
options.
* Browse official Firefox release directories and versions directly from
within the app.
* Automatically filters out legacy versions (<100) and handles
pre-release tags consistently.
* Displays accurate file metadata (date, size) and provides direct
download links for release assets.
* Updated catalog to include Firefox releases, enabling easier setup for
Firefox-based automation workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: fengmk2 <156269+fengmk2@users.noreply.github.com>
Co-authored-by: MK (fengmk2) <fengmk2@gmail.com>
Co-authored-by: elrrrrrrr <5574625+elrrrrrrr@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
https://github.com/node-modules/github-actions/issues/14
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the release workflow configuration for improved automation.
* Upgraded the "oxlint" development dependency to version ^1.11.0.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Updates the oxlint configuration to use the new shared
`@eggjs/oxlint-config` package, significantly simplifying the project's
linting setup while maintaining all existing functionality.
## Changes
- **Added** `@eggjs/oxlint-config@1.0.0` as a dev dependency
- **Updated** `.oxlintrc.json` to extend from the shared configuration
- **Simplified** configuration from 156 lines to 20 lines (87%
reduction)
- **Maintained** all project-specific overrides:
- `max-params: 6` (vs shared config's 5)
- `no-console: "warn"` (vs shared config's "allow")
- `import/no-anonymous-default-export: "error"` (vs shared config's
"allow")
- Added `mocha` environment for test files
- Preserved `index.d.ts` ignore pattern
## Benefits
- **Consistency**: Now uses the same base linting rules as other EggJS
projects
- **Maintainability**: Rule updates are centralized in the shared config
- **Simplicity**: Dramatically reduced configuration complexity
- **Future-proof**: Automatic updates when the shared config is improved
The configuration now only contains project-specific overrides, making
it much easier to understand and maintain. All existing linting behavior
is preserved - the project passes all linting checks with zero warnings
and errors.
<!-- START COPILOT CODING AGENT TIPS -->
---
💬 Share your feedback on Copilot coding agent for the chance to win a
$200 gift card! Click
[here](https://survey.alchemer.com/s3/8343779/Copilot-Coding-agent) to
start the survey.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: fengmk2 <156269+fengmk2@users.noreply.github.com>
close https://github.com/cnpm/cnpmcore/issues/821
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced a GitHub Actions workflow to automate building, tagging,
publishing, and attesting Docker images for multiple platforms.
* **Documentation**
* Updated Docker deployment documentation to reflect the new image
repository and usage instructions.
* **Chores**
* Updated Docker build scripts and Dockerfiles for improved image
building and logging.
* Upgraded the "oxlint" development dependency.
* Removed the "prepare" script from project scripts.
* Adjusted TypeScript configuration to disable declaration file
generation and exclude test files from compilation.
* Updated linter configuration to allow additional code patterns.
* Improved code comments for better linting and error suppression.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated linter configuration to relax certain code style rules.
* Upgraded the oxlint development dependency and corrected project URLs
in metadata.
* **Refactor**
* Simplified arrow function syntax in various controllers, services, and
type declarations for improved code readability.
* **Tests**
* Streamlined mock implementations in test files for more concise
syntax.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> see https://github.com/orgs/community/discussions/152515
-------
* ♻️ 删除 `descending` 参数
------
* ♻️ remove `descending` args in changesStream
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Simplified the process for handling request URLs when fetching data,
removing the explicit addition of certain query parameters. This does
not affect user-facing functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: fengmk2 <fengmk2@gmail.com>
```bash
2025-06-26 15:12:05,133 ERROR 423473 [-/127.0.0.1/ea0a01e1-5a66-4191-92c7-f06aa5bb86ea/19.41ms GET /] nodejs.TypeError: [EventBus] process event PACKAGE_MAINTAINER_REMOVED for handler packageVersionAddedSyncESEvent failed: Cannot read properties of undefined (reading 'undefined')
at PackageSearchService.syncPackage (file:///home/admin/application/app/core/service/PackageSearchService.ts:93:45)
at processTicksAndRejections (node:internal/process/task_queues:105:5)
at PackageVersionAddedSyncESEvent.syncPackage (file:///home/admin/application/app/core/event/SyncESPackage.ts:29:5)
at PackageVersionAddedSyncESEvent.handle (file:///home/admin/application/app/core/event/SyncESPackage.ts:53:5)
at EventHandlerFactory.handle (file:///home/admin/application/node_modules/_@eggjs_tegg-eventbus-runtime@4.0.0-beta.4@@eggjs/tegg-eventbus-runtime/src/EventHandlerFactory.ts:49:7)
at file:///home/admin/application/node_modules/_@eggjs_tegg-eventbus-runtime@4.0.0-beta.4@@eggjs/tegg-eventbus-runtime/src/SingletonEventBus.ts:151:13
at async Promise.all (index 2)
at file:///home/admin/application/node_modules/_@eggjs_tegg-eventbus-runtime@4.0.0-beta.4@@eggjs/tegg-eventbus-runtime/src/SingletonEventBus.ts:149:9
at EggContextHandler.run (file:///home/admin/application/node_modules/_@eggjs_tegg-plugin@4.0.0-beta.4@@eggjs/tegg-plugin/lib/EggContextHandler.ts:19:12)
at Function.ContextHandler.runInContextCallback (file:///home/admin/application/node_modules/_@eggjs_tegg-plugin@4.0.0-beta.4@@eggjs/tegg-plugin/lib/EggContextHandler.ts:27:14)
```
#### PR Dependency Tree
* **PR #814** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved error handling when package version information is missing,
reducing the chance of runtime errors and providing clearer warning
messages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #813** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated linting configuration to explicitly allow import extensions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> https://github.com/orgs/community/discussions/152515
* ⚙️ Added descending parameter to npm changesStream
--------
* ⚙️ 为 npm changesStream 添加 descending 参数
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated internal request behavior to fetch changes in descending
order.
- Locked the version of Mocha to 11.6.0 for improved dependency
management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the development dependency "oxlint" to the latest version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the development dependency "oxlint" to the latest version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Fix the content-type display logic of showFileController, close#693
* 🌐 HTML/XML files now render directly (no forced attachment)
* ⚠️ Note: No changes to actual file storage - only affects presentation
layer
* ♻️ New content automatically converts to standardized formats
-------
> 修复 showFileController 相关接口 content-type 展示逻辑, close#693
* 对于 html,xml 不再默认返回 attachment，改为直接渲染
* 增量部分默认进行格式转换
* 存量部分在 controller 查看时统一返回
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Files with `.xml` and `.html` extensions are now served with a
`text/plain` content type instead of their previous MIME types,
improving content handling and consistency.
- The download behavior for HTML and XML files has been updated; these
files are no longer forced as attachments in the browser.
- **Tests**
- Updated tests to reflect the new content type handling for `.xml` and
`.html` files.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved accuracy in retrieving the latest binary directory by
updating the sorting criterion.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for a new task property to prevent certain tasks from
being merged when waiting, providing more granular control over task
handling.
- **Bug Fixes**
- Improved logic for task merging to consider both task type and a new
optional flag, ensuring correct behavior for historical task
compensation scenarios.
- **Tests**
- Introduced new tests to verify the updated task merging logic and the
effect of the new property.
- **Chores**
- Limited the number of items returned per request when fetching
platform-specific binaries to improve data retrieval efficiency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved synchronization logic to prevent overwriting the last sync
time for Chromium browser snapshots if it is already set.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved handling of the `lastSyncTime` parameter when fetching binary
data to ensure more accurate synchronization.
- **Tests**
- Added a new test case to verify fetching binary data with a provided
`lastSyncTime`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!--- SUMMARY_MARKER --->
## Sweep Summary <sub><a href="https://app.sweep.dev"><img
src="https://raw.githubusercontent.com/sweepai/sweep/main/.assets/sweep-square.png"
width="25" alt="Sweep"></a></sub>
Adds Elasticsearch setup to GitHub Actions workflow to enable running
Elasticsearch-dependent tests in CI.
- Added system configuration steps for Elasticsearch in the GitHub
Actions workflow, including swap and sysctl settings.
- Integrated the official Elastic GitHub Action to run Elasticsearch
8.18.0 during CI tests.
- Implemented a wait mechanism to ensure Elasticsearch is fully ready
before proceeding with tests.
---
[Ask Sweep AI questions about this PR](https://app.sweep.dev)
<!--- SUMMARY_MARKER --->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Enhanced CI workflow to configure system limits and start an
Elasticsearch service before running tests. The workflow now waits for
Elasticsearch to be fully ready before proceeding.
- Improved test diagnostics by adding detailed response data to
assertion messages for better debugging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
should be `https://replicate.npmjs.com/registry` not
`https://replicate.npmjs.com`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the changes stream registry endpoint configuration.
- Standardized assertion methods in tests to use `assert.ok()` for
improved clarity and consistency across the codebase.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: fengmk2 <fengmk2@gmail.com>
Co-authored-by: elrrrrrrr <elrrrrrrr@gmail.com>
<!--- SUMMARY_MARKER --->
## Sweep Summary <sub><a href="https://app.sweep.dev"><img
src="https://raw.githubusercontent.com/sweepai/sweep/main/.assets/sweep-square.png"
width="25" alt="Sweep"></a></sub>
Improves database performance by replacing expensive count queries with
a dedicated totals table that's updated asynchronously via events.
- Created a new `totals` table in `app/repository/model/Total.ts` to
store package and version counts instead of running expensive SQL count
queries.
- Implemented `TotalRepository` in `app/repository/TotalRepository.ts`
with methods to increment and retrieve count values.
- Added event handlers in `app/core/event/TotalHandler.ts` that listen
for package and version additions to update counts asynchronously.
- Modified `PackageRepository.queryTotal()` to fetch counts from the
totals table instead of executing direct SQL count queries.
- Added migration scripts in `sql/mysql/4.3.0.sql` and
`sql/postgresql/4.3.0.sql` to create the totals table and populate it
with existing data.
---
[Ask Sweep AI questions about this PR](https://app.sweep.dev)
<!--- SUMMARY_MARKER --->
> Fix database performance issues caused by doc_count and
doc_version_count queries
1. 💽 Add a corresponding totals table to record statistical information
2. ➕ Add a `PACKAGE_ADDED` event and the original
`PACKAGE_VERSION_ADDED` event to asynchronously update records in the
table
3. ♻️ Add a new existing data migration script to migrate the original
statistical information to the totals table
-----------
> 修复 doc_count 和 doc_version_count 查询导致的数据库性能问题
1. 💽 新增对应 totals 表，用来记录统计信息
2. ➕ 新增 `PACKAGE_ADDED` 事件，和原有 `PACKAGE_VERSION_ADDED` 事件，异步更新表内记录
3. ♻️ 新增存量数据迁移脚本，迁移原有的统计信息到 totals 表
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced persistent tracking of total package and package version
counts, with real-time updates when new packages or versions are added.
- Added new data models and repository methods to manage and retrieve
these total counts.
- Emitted events upon new package creation to update totals
automatically.
- **Database**
- Added a new "totals" table to both MySQL and PostgreSQL databases for
storing aggregate counts initialized from existing data.
- **Bug Fixes**
- Ensured total counts are always returned as numbers in scheduled data
updates.
- **Tests**
- Added and updated tests to verify correct behavior of total count
tracking, incrementing, resetting, and retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> close https://github.com/cnpm/cnpm/issues/459#issue-2998106947
1. Correct the initialization of initRelease to ensure the fetch process
is triggered as expected.
------
> close https://github.com/cnpm/cnpm/issues/459#issue-2998106947
1. 🐛 修复 initRelease 时，被错误初始化为 [] 空数组，导致不会触发 fetch
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Improved handling of uninitialized or absent release data, ensuring
clearer distinction between missing and empty release lists.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Fix potential performance issues caused by using
`information_schema.tables` in distributed databases
1. 📊 `information_schema.tables` has no index, consumes large amounts of
memory, and requires aggregation after being generated per instance.
2. 🚚 Execution plans involve multi-table joins, distributed operations,
GROUP BY, and similar operations.
3. ♻️ Consistently use `SELECT COUNT` with index.
----------
> 修复在分布式 db 中，使用 `information_schema.tables` 可能导致的性能问题
1. 📊 `information_schema.tables` 内部无索引，占用大量内存，需要根据实例数生成后聚合
2. 🚚 执行计划涉及多表连接、分布式操作、GROUP BY 等操作
3. ♻️ 统一使用 `select count`，通过索引计算
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Unified the method for retrieving total row counts across all
databases for improved consistency.
- **Chores**
- Updated linting configuration to allow grouped exports.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/791
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added an HTTP header to indicate replication opt-in.
- Enhanced logging now shows the results count and last sequence
identifier.
- **Refactor**
- Streamlined the change-fetching process by directly iterating over
response data.
- **Tests**
- Updated test cases with refined assertions and a simplified mocking
strategy to align expectations with the new response format.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced the task management system to support binary data
synchronization tasks, enabling more precise handling of binary workflow
processes.
- Improved support for binary task execution and logging to ensure data
accuracy during synchronization.
- **Tests**
- Added new test cases to validate the proper execution and data
handling for binary synchronization tasks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/784
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new option for obtaining redistributable Python builds,
making it easier for users to access pre-packaged releases via GitHub.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Include sql change.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced data processing with additional safeguards to limit
iterations during data retrieval, improving stability and preventing
resource overuse.
- **Chores**
- Refined database indexing by removing an old index and adding a new
one, optimizing query performance for faster data access and a smoother
overall experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced an admin-protected endpoint for binary synchronization with
support for additional data.
- Added enhanced logic to dynamically retrieve and update the latest
binary versions for multiple platforms.
- **Improvements**
- Optimized binary data fetching by migrating to a more dynamic,
cloud-based data source.
- **Dependency Management**
- Updated project dependencies to improve XML parsing capabilities while
removing outdated tools.
- **Tests**
- Revised test workflows to verify the new synchronization parameters
and data structures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
✅Closes: #779
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated configuration validation to require the OSS bucket setting
instead of a CDN URL, with no changes to public interfaces.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: ANMeng <littleliar.im@qq.com>
- unicorn/prefer-number-properties
- typescript/no-import-type-side-effects
- unicorn/no-array-for-each
- typescript/no-dynamic-delete
- no-empty-function
- import/unambiguous
- max-depth 6
- unicorn/prefer-array-some
- unicorn/prefer-string-slice
- no-lonely-if
- max-nested-callbacks 5
- unicorn/explicit-length-check
- unicorn/no-lonely-if
- no-negated-condition
- no-else-return
- unicorn/prefer-date-now
- typescript/prefer-ts-expect-error
- typescript/ban-ts-comment
- no-throw-literal
- typescript/prefer-enum-initializers
- unicorn/no-typeof-undefined
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Upgraded dependency version of `oxlint` to `^0.16.0` for improved code
quality.
- **Refactor**
- Streamlined internal logic for error handling, string manipulation,
and data parsing, enhancing maintainability and performance.
- Simplified conditional logic and iteration methods in various services
and controllers for improved clarity.
- Updated import statements for type clarity and consistency across the
codebase.
- **Tests**
- Improved test assertions and iteration methods to provide clearer
verification of system reliability.
- Enhanced logging functionality in test cases for better visibility of
output and error handling.
- Modified assertions to ensure more precise requirements in test cases.
These behind-the-scenes enhancements contribute to a more robust and
stable application, ensuring a smoother experience for end-users.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated dependency versioning where specific beta numbers were
replaced with a broader beta designation. This change streamlines
dependency management and supports smoother integrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Streamlined asynchronous operation tracking for improved performance
monitoring.
- Enhanced event handling configuration to provide more consistent and
reliable system behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the build command to use revised TypeScript configuration
settings, ensuring a streamlined and consistent build process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
revert https://github.com/cnpm/cnpmcore/pull/763
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Streamlined the package download process by removing outdated package
blocking checks, leading to a smoother and more direct download
experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Enhanced the internal process for retrieving package version details,
resulting in more accurate and reliable version information.
- Streamlined the handling of missing package data to reduce unexpected
issues.
- Optimized overall data processing, contributing to improved stability
and a smoother user experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/766
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Expanded our binary support to include prebuilt binaries for the
node-pty package, enhancing integration possibilities and deployment
options for users relying on these binaries.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> The 'allowPublicRegistration' is enabled by default, which my cause
unexpected users registering arbitrarily
1. ⚙ Modify the default configuration 'allowPublicRegistration' to
'false'`
-------------
> 目前默认开启了 `allowPublicRegistration` 配置，公网部署可能会导致预期外的用户任意注册
1. ⚙ 修改默认配置 `allowPublicRegistration` 为 `false`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Public registration can now be enabled through a new configuration
option, allowing for more flexible user onboarding.
- **Bug Fixes**
- Updated the configuration to disallow public registration by default,
ensuring only administrators can log in unless changed.
- **Documentation**
- Added an informational note in the developer documentation regarding
public registration settings.
- **Tests**
- Introduced a setup method to enable public registration before each
test case runs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> The tgz download interface does not check if the package is blocked,
which may pose additional risks for parsing package-lock.json or other
lock files.
[exp](https://registry.npmmirror.com/joker-su/-/joker-su-1.0.0.tgz)
1. 🛡️ Add validation logic for
DownloadPackageVersionTarController#download to check if the package is
allowed to be downloaded.
2. 🧶 Add PackageVersionService#findBlockInfo to check if the
corresponding package is blocked.
3. ♻️ When a single version is blocked, skip check as per the current
manifest logic.
---------
> tgz 下载接口没有判断包是否被 block，对于 package-lock.json
或者其他依赖锁文件解析可能会有额外风险，[exp](https://registry.npmmirror.com/joker-su/-/joker-su-1.0.0.tgz)
1. 🛡️ `DownloadPackageVersionTarController#download` 接口新增校验逻辑，判断是否允许下载
2. 🧶 新增 PackageVersionService#findBlockInfo 判断对应包是否被全局拦截
3. ♻️ 单版本被 block 时，考虑到误封场景，按目前 manifest 逻辑，不在 tgz 下载时进行拦截操作
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced the package download process with an additional block check.
Now, if a package is flagged, the download will be halted and a clear
error response is returned to inform users of the block.
- Introduced a method to retrieve block information related to package
versions, improving the service's capabilities.
- **Tests**
- Added new test cases to verify the blocking functionality for package
downloads, ensuring the application correctly handles requests for
blocked packages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- Refactor
- Enhanced environment variable handling to trim extra whitespace and
improve default value checks for more robust configuration processing.
- Tests
- Expanded test coverage to validate default behavior, type conversions,
and error handling for various environment variable scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Improve the local development process based on docker-compose
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a centralized configuration utility that validates
environment variable types and provides fallback defaults.
- **Refactor**
- Standardized environment variable handling across configuration files,
improving maintainability and consistency in system setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/750
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Expanded support with additional platform configurations for Ubuntu
24.04 and macOS 15 (including ARM variants).
- Introduced a new Windows configuration for improved compatibility.
- Updated browser configurations to deliver the latest versions across
Chromium, Firefox, Webkit, FFmpeg, and Android.
- **Tests**
- Refined validation checks and error reporting to ensure consistent and
reliable browser setups.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/742
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Improved organization for browser downloads: Secondary browser
components are now integrated under the primary browser category,
resulting in a more intuitive and streamlined binary structure for
users.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
在内部也遇到了，感觉直接加就行，可以试试看看 @fengmk2
close#742
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced enhanced support for the chromium-headless-shell binary
with updated download options across multiple operating systems,
including popular Linux distributions, macOS (with arm64 support), and
Windows. This improvement ensures smoother integration and broader
compatibility for users running different platforms.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: fengmk2 <fengmk2@gmail.com>
proxy时因为一个低级的拼写错误没有正确的携带请求头，导致代理模式时返回的数据不正确。但是现在用户发起的请求中的user-agent和x-forwarded等头部信息也没有正确的携带。虽然影响不大但还是想和跑批时更新的请求做一下区分。
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Enhanced error handling and logging for task execution, improving
traceability.
- **Improvements**
- Updated HTTP header access method for better alignment with context
structure.
- Clarified logic for manifest retrieval based on file type, ensuring
correct API usage.
- Streamlined cache handling and response generation logic in package
management.
- Improved method visibility and organization within the cache service
and controller.
- Simplified task creation logic and cache removal processes in the
controller.
- Updated expected outcomes for cache-related operations in the test
cases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/739
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced error handling for package synchronization, including
specific logging for package not found scenarios.
- Simplified criteria for identifying removed packages.
- **Bug Fixes**
- Corrected documentation for the `syncMode` property.
- **Chores**
- Updated dependency versions in `package.json`.
- **Tests**
- Added new test cases and refined existing assertions to improve
logging and error handling verification.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/730
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced data fetching for Chrome for Testing with new JSON entries
for version management.
- **Bug Fixes**
- Improved data validation in tests to ensure correct structure and
content of fetched data.
- **Tests**
- Added assertions to validate properties of fetched items in the
ChromeForTestingBinary tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/731
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Added support for PostgreSQL alongside MySQL, including a new database
setup script and comprehensive documentation for local development.
- Introduced a new CI job for PostgreSQL testing in the GitHub Actions
workflow.
- Enhanced the `README.md` and `DEVELOPER.md` files to provide clearer
instructions for using both database systems.
- Added new environment variable configurations for PostgreSQL in the
Docker deployment documentation.
- **Bug Fixes**
- Improved error handling in tests for duplicate entries to accommodate
both MySQL and PostgreSQL error messages.
- **Documentation**
- Updated setup instructions for PostgreSQL and clarified MySQL setup in
the documentation.
- Enhanced contributor information in the README.
- Expanded instructions for setting up Elasticsearch and Kibana,
including environment variable configurations.
- **Chores**
- Updated package dependencies to include PostgreSQL client libraries
and modified scripts to support both databases.
- Changed the base image in the Dockerfile to a newer Node.js version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced support for the Deno runtime, providing users with a modern
runtime for JavaScript and TypeScript.
- **Documentation**
- Updated configuration to include Deno, ensuring users can easily
access and utilize this new option.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced error handling across various services by introducing a
centralized timeout error checking function.
- HTTP/2 support enabled in the HTTP client configuration.
- **Bug Fixes**
- Corrected a typographical error in comments for better clarity.
- **Documentation**
- Updated Node.js version requirements in the project configuration.
- **Tests**
- Improved test cases for `NpmChangesStream` and `TaskRepository` to
ensure accurate behavior and performance.
- **Chores**
- Updated Node.js version in CI workflow for more precise testing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Updated the "Supported Versions" section in the SECURITY.md file to
reflect that security updates are now provided for versions 3.0.0 and
above.
- Minor formatting adjustments made in the "Disclosure Policy" section
for consistency.
- **Chores**
- Simplified debugger configurations by removing explicit protocol and
port settings from the `.vscode/launch.json` file.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: elrrrrrrr <elrrrrrrr@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced file fetching capabilities with updated logic to handle a
broader range of file formats and attributes.
- Introduced a new index HTML page for Node.js distribution version
18.15.0, listing downloadable files with metadata.
- **Bug Fixes**
- Improved regex for matching HTML anchor tags to accurately capture
additional file types and structures.
- **Tests**
- Added new test cases for the `fetch()` method to verify functionality
against the Node.js distribution version 18.15.0.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> getaddrinfo ENOTFOUND alinode.aliyun.com
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new property to disable the `alinode` binary, affecting
its availability in the application.
- **Improvements**
- Updated the `ignoreDownloadStatuses` for the `python` binary to
enhance type safety, ensuring proper adherence to expected data types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
兼容Verdaccio下载地址风格，镜像库从Verdaccio切换至cnpmcore后无需大面积调整lock文件
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a comprehensive test suite for the download functionality,
ensuring robust behavior for various package retrieval scenarios.
- Enhanced validation for CORS requests and CDN redirection during
package downloads.
- Added new test cases for downloading version tarballs with scoped
package handling.
- **Bug Fixes**
- Improved error handling for non-existent packages and versions,
providing clearer error messages.
- Ensured proper handling of deprecated download paths and
scoped/non-scoped package names.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> During the sync process, dependency installation errors may occur due
to incorrect timing or incorrect declaration.
1. ⚙️ Added `strictValidatePackageDeps` configuration, disabled by
default.
2. 🔄 In publish process, if the dependencies for the current version do
not exist, it will be interrupted.
3. 📦 In synch process, will automatically enter the queue to wait for
the next synchronization attempt (up to 3 retries).
4. ♻️ Packages that have already been published or synchronized will not
be affected.
-----------
> 在版本同步时，由于同步时机或自身依赖声明错误，导致依赖安装报错。
1. ⚙️ 新增 `strictValidatePackageDeps` 配置，默认关闭
2. 🔄 在包发布时，如果当前版本的 `dependencies` 不存在，则中断发布
3. 📦 在包同步时，如果校验未通过，则中断发布流程，自动进入队列等待下次同步（最多重试3次）
4. ♻️ 已发布、同步的包不受影响
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced strict validation for package dependencies during
publishing and synchronization.
- Added a new configuration option for enforcing dependency validation.
- **Bug Fixes**
- Enhanced error handling for dependency validation failures, allowing
for task retries.
- **Tests**
- Added new test cases to ensure proper validation of package
dependencies under strict settings.
- Created a new test suite for handling invalid dependencies in package
synchronization.
- **Chores**
- Updated logging for package synchronization processes to improve
clarity and error tracking.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for Protocol Buffers with a new binary entry in the
configuration.
- **Tests**
- Introduced a new test case to verify the fetching of release data for
the Protocol Buffers repository from GitHub.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: shixia.ly <shixia.ly@antgroup.com>
closes https://github.com/cnpm/cnpmcore/issues/710
pick from https://github.com/cnpm/cnpmcore/pull/712
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Updated the `binaries` configuration to include new entries and
modified existing ones, enhancing the variety and sources of available
binaries.
- **Bug Fixes**
- Improved the test coverage for the `GithubBinary` class with a new
test case to ensure proper fetching of `skia-canvas` release data from
GitHub.
- **Refactor**
- Removed outdated test cases related to the `skia-canvas` package from
the `NodePreGypBinary` test suite, streamlining the testing process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: ltxhhz <ltxhhz@qq.com>
closes https://github.com/cnpm/cnpmcore/issues/366
开启代理模式时如果找不到依赖会直接返回上游仓库的manifest信息并缓存于nfs，当请求的tgz文件不存在时从上游仓库获取并返回，同时创建对应版本的同步任务。每小时检查更新已缓存的manifest文件保证上游仓库发布新版本时不会因为缓存落后而404。
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced proxy cache management for package manifests and versions.
- Added new HTTP methods for managing proxy caches.
- Implemented scheduled workers for updating and synchronizing proxy
cache.
- **Updates**
- Expanded `SyncMode` enum to include a new value `proxy`.
- Updated constants with `PROXY_CACHE_DIR_NAME` and
`ABBREVIATED_META_TYPE`.
- **Tests**
- Added comprehensive test cases for `ProxyCacheService`,
`ProxyCacheRepository`, and related controllers.
- Verified functionality of scheduled workers for proxy cache updates
and synchronization.
- Enhanced testing coverage for handling package downloads in proxy
mode.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: fengmk2 <suqian.yf@antgroup.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Updated setup instructions to reflect the requirement for MySQL 9,
removing references to MySQL 5.7 and 8.
- Removed outdated troubleshooting information related to MySQL 8
authentication issues.
- **Chores**
- Updated `.gitignore` to include `.egg/` and improved formatting for
`.DS_Store`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> https://github.com/eggjs/tegg/pull/242/files
* 🤖 Updated the duplicate class name, fix the error when dev.
------
> https://github.com/eggjs/tegg/pull/242/files
* 🤖 修改同名 class 名称定义，修复本地启动报错
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced clarity in event handling by renaming various package-related
event classes for better understanding.
- **Bug Fixes**
- Updated test cases to reflect the renamed event classes, ensuring
accurate functionality in the testing framework.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This reverts commit 9a7994090b.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Updated HTTP client configuration to enhance compatibility.
- Adjusted minimum Node.js version requirement for broader support.
- **Bug Fixes**
- Removed HTTP/2 support from the HTTP client configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new configuration option allowing the use of HTTP/2 in
the HTTP client.
- **Updates**
- Updated the minimum required Node.js version to 18.20.0 for improved
performance and compatibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Currently, `triggerHookWorkerMaxConcurrentTasks` is 10 by defualt,
which can lead to some redis queries even hookEnable is not activated.
* ♻️ Follow `CreateTriggerHookWorker`, when hookEnable is not activated,
do not query task queue.
-------
> 目前 triggerHookWorkerMaxConcurrentTasks 默认为 10，在未开启 hookEnable
时会带来一些冗余的 redis 查询
* ♻️ 参照 `CreateTriggerHookWorker` 逻辑，hookEnable 关闭时，不进行存量任务轮训
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Added a check to ensure hooks are enabled before proceeding, improving
reliability and preventing errors when hooks are disabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> remove es index when the target pkg has been blocked or deleted.
1. ⚓ update `SyncESPackage` logic for event register
2. ♻️ do the block check when `syncPackage`
---------
> 包删除或被 block 时，es 索引同步删除
1. ⚓ 修改 `SyncESPackage` 逻辑，更新事件注册
2. ♻️ `syncPackage` 时，重新判断包是否被 block，用于清理
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced event types `PACKAGE_BLOCKED` and `PACKAGE_UNBLOCKED` for
better package management.
- **Bug Fixes**
- Improved logic to handle package blocks before syncing, ensuring
smoother operations.
- **Tests**
- Added test scenarios to cover new package blocking and unblocking
features, enhancing reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> optimize binary sync perf , closes#698
* ♻️ calculate the latestItem only once for the same fetchItems.
--------
> binary 同步性能优化，close #698
* ♻️ binary 最新版本比对时，相同 fetchItems 仅计算一次。
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved logic for identifying the latest item in synchronization
processes, ensuring more accurate date comparisons.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* 📚 add doc for the trends API.
---------
* 📚 添加 trends api 相关文档
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Added sections for `[Token]` and `[Trend]` in the Registry API
documentation.
- Introduced a new endpoint `GET /downloads/range/:start::end/:pkgName`
for fetching package download trends.
- Included sample response data for the `Trend` endpoint.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Update #692, Declare compatibility using + notation
1. 🤖 Ensure version matching by `semver`
---------
> 更新 #692 , 兼容版本声明为 `+` 的场景
1. 🤖 统一通过 `semver` 进行版本匹配判断
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for prerelease versions when checking package version
compatibility.
- **Tests**
- Updated test cases to include operations related to a new object `baz`
with version `*`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/unpkg-white-list/issues/63
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved package version checking to support wildcard (`*`) versions,
ensuring better compatibility and flexibility.
- Fixed issues in handling release candidate (rc) versions in package
version checks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/689
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for `acceptDependencies`, `directories`, and `funding`
metadata in package management.
- **Bug Fixes**
- Improved package metadata synchronization to include new keys.
- **Tests**
- Added new test cases to verify `acceptDependencies` metadata syncing.
- **Chores**
- Updated import statements to use the `node` namespace for better
compatibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
https://github.com/messense/homebrew-macos-cross-toolchains/releases
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for `homebrew-macos-cross-toolchains` in the binaries
list, enhancing cross-toolchain capabilities for macOS users.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/680
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved file and directory handling to ensure accurate listing and
filtering of package version files.
- **Tests**
- Updated test cases to reflect the new logic for file and directory
handling, ensuring more reliable and accurate test results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
see https://github.com/cnpm/unpkg-white-list
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new configuration option `enableSyncUnpkgFilesWhiteList`
to enhance package version file synchronization.
- **Improvements**
- Enhanced logging in package version file operations for better
traceability.
- Simplified file redirection logic for improved performance and
readability.
- **Tests**
- Added test cases for the new `enableSyncUnpkgFilesWhiteList`
configuration to ensure reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
revert https://github.com/cnpm/cnpmcore/pull/650
run test on Node.js 22
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated Node.js version support to 18, 20, and 22.
- Removed unnecessary configuration from the release workflow.
- Cleaned up outdated security-related entries in the package
configuration.
- Updated encryption functions in CryptoUtil.ts for improved security.
- Refactored import statements for ES module syntax in
webauthController.test.ts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
closes https://github.com/cnpm/cnpmcore/issues/674
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Implemented a feature to handle compatibility with unpkg by searching
for and redirecting to possible file entries if the requested file is
not found.
- **Tests**
- Added a new test case to ensure the redirection to possible file
entries functions correctly.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> close#664 Missing a startup document for sync pkgs.
* 📚 Add doc for sync-setup
-------
> close#664 目前缺少一个启动文档，开启包同步服务
* 📚 添加 sync-setup 使用文档
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Added a new guide for setting up package synchronization services
using `cnpmcore`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> follow
[ref](https://github.com/microsoft/playwright/blob/main/packages/playwright-core/src/server/registry/index.ts)
update
* 🔧 Modify Playwright `DOWNLOAD_PATHS` configuration
* 🔧 Add `android` related configuration
* ♻️ Remove `chromium-with-symbols` configuration and `ubuntu18.04`
related versions
---------
> 参照
[ref](https://github.com/microsoft/playwright/blob/main/packages/playwright-core/src/server/registry/index.ts)
变更
* 🔧 修改 playwright `DOWNLOAD_PATHS` 相关配置
* 🔧 添加 `android` 相关配置
* ♻️ 移除 `chromium-with-symbols` 配置，`ubuntu18.04` 相关版本
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for `mac14` and `mac14-arm64` platforms for Chromium and
Chromium tip-of-tree downloads.
- **Bug Fixes**
- Updated download paths for various platforms to ensure compatibility
and correct file retrieval.
- **Tests**
- Adjusted test assertions to reflect changes in download paths and
platform support, ensuring accurate validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added a new configuration option `enableSyncUnpkgFiles` for enhanced
synchronization control.
- **Improvements**
- Improved synchronization logic to check both `enableUnpkg` and
`enableSyncUnpkgFiles` settings before proceeding.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> nodejs.AssertionError: invalid status code: -1
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved error handling to ensure the HTTP status code is valid and
within the correct range.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/594
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced the EdgedriverBinary class to enhance the fetching and
management of Edgedriver binaries.
- Added support for the Edgedriver type in our binary management system.
- Updated the binaries configuration to include the new Edgedriver
category, complete with repository and distribution details.
- **Documentation**
- Updated documentation to reflect the addition of the Edgedriver
category and its functionalities.
- **Tests**
- Implemented new tests for the EdgedriverBinary class to ensure
functionality works as expected.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
https://github.com/cyjake/leoric/pull/419
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated the database management system to enhance performance and
compatibility.
- **Chores**
- Upgraded internal libraries to improve stability and security.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Removed the `mysql2` dependency from the project.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmcore/issues/668
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated the database management package to enhance performance and
security.
- Adjusted supported Node.js versions in the workflow configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes https://github.com/cnpm/cnpmjs.org/issues/1560
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for the `linux-arm64` platform in Cypress binary
handling.
- **Tests**
- Updated tests to include assertions for the newly supported
`linux-arm64` platform.
- **Chores**
- Updated Node.js version to 21 in workflow configuration for improved
job execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Node.JS 安全性修复导致 RSA_PKCS1_PADDING 不可用，会报出以下错误：
> RSA_PKCS1_PADDING is no longer supported for private decryption, this
can be reverted with --security-revert=CVE-2023-46809。
参考链接：https://www.eggjs.org/zh-CN/core/security#revert-cve
> Since the eventBus#cork , version & tag events are triggered at same
time, cause the abnormal triggers of different types of hooks.
* ~~🐞 Fix triggerHook type targetName to be `tagetName:changeId`~~
* 🤖 Only merge sync tasks (binary, package) which in waiting states
--------
> 由于 `eventBus#cork` 机制，版本事件同时触发，导致不同类型 hook 触发异常
* ~~🐞 修改 triggerHook 类型 targetName 为 `包名:changeId`~~
* 🤖 仅合并 waiting 状态下包同步任务
> When accessing the unpkg service, when the packages have not yet been
synchronized, will lead to multiple synchronization attempts
concurrently causing db insert errors.
* 🔒 Added a Redis lock for the `ensurePackageVersionFilesSync` function,
with a default timeout of 60 seconds.
* 🥸 Admin PUT requests and the package version auto sync process are not
restricted by this.
> 当访问 unpkg 服务时，如果访问存量未同步的包，可能导致多次同步并发报错
* 🔒 为 ensurePackageVersionFilesSync 添加 redis 锁，默认超时 60s
* 🥸 管理员手动 PUT 请求和包同步流程不受限制
> closes#574 Fixed the issue where custom tags in publishConfig
prevented the default latest tag.
* 🧶 Modified the `savePackageVersion` API, automatically add latest tag
if no latest tag.
* 🧶 The publish tag parameter has been changed to tags, triggering
corresponding events in batches.
* ♻️ No changes to the package synchronization process.
------
> closes#574 修复 publishConfig 中自定义 tag，导致 latest tag 未设置的问题
* 🧶 修改 savePackageVersion 接口，如果当前包未配置 latest tag，则自动补全
* 🧶 publish tag 参数改为 tags，对应事件分批触发
* ♻️ 包同步流程不做修改
> Auto init the account when auth
* 🧶 Added `findOrCreateUser` method. Initialize account on both login
and authorization, as per the submitted GitHub."
------
> 授权时，默认进行账户初始化
* 🧶 新增 `findOrCreateUser` 方法，登录和授权时均初始化账户
> Attempted to apply the `noImplicitAny`, parameter types should be
specified unless any is manually declared.
* 🐞 Fixed an issue in bugVersionAdvice where AbbreviatedManifest was
being set abnormally.
* 🤖 Added index.d.ts to store declarations for dependencies without
types.
* 🤔 skipLibCheck has no effect on leoric for now, so it cannot be
enabled temporarily.
--------
> 尝试应用 `noImplicitAny` 配置，除非手动声明 any，否则需要指定参数类型
* 🐞 修复 bugVersionAdvice 中，AbbreviatedManifest 设置异常
* 🤖 添加 index.d.ts 存放无类型依赖声明
* 🤔 skipLibCheck 对 leoric 失效，暂时无法开启
> Fixed an issue with the `_npmUser` field setting during the initial
sync.
1. 🧶 Fixed _npmUser field setting issue during initial sync, should use
displayName.
2. 🧶 skip diff abbreviated meta for _npmUser
3. 🤖 Refined some TypeScript definitions, use isEqual to diff metas
-------
> 修复首次同步时，`_npmUser` 字段设置异常
1. 🧶 修复 publisher 匹配，获取逻辑，应当用 displayName 进行匹配
2. 🧶 精简 meta 信息，跳过比较不存在的 _npmUser 字段
2. 🤖 调整部分 ts 定义，使用 isEqual 来进行 diff 比较
> closes#553, fixing the issue introduced by
https://github.com/cnpm/cnpmcore/pull/491, which caused an abnormality
in the _npmUser field in fullManifest.
1. 🧶 Update the `publish` method to pass in the actual operating
publisher information.
2. 🧶 Update the diffMeta function to compare the _npmUser as well.
3. ♻️ Existing data needs to be resynchronized, from 6.2 to 7.20.
-----
> closes#553 , 修复由 https://github.com/cnpm/cnpmcore/pull/491 引入问题，导致
fullManifest 中 _npmUser 字段异常
1. 🧶 更新 .publish 方法，传入实际操作的 publisher 信息
2. 🧶 更新 diffMeta 算法，将 _npmUser 也进行比对
3. ♻️ 存量数据需要重新进行同步，6.2 -> 7.20
> When concurrently executing packet sending, there is a possibility of
version overwrite
* 🧶 Add `usingLock` logic to the publish interface to handle concurrent
execution, which will prevent version overwrite
* 🔨 Modify usingLock to include a return value indicating the success of
lock creation
------
> 并发执行发包时，可能出现版本覆盖问题
1. 🧶 在发布接口中，添加 usingLock 逻辑，包同步场景不涉及
2. 🔨 `usingLock` 添加返回值，标记是否创建锁成功
> Follow https://github.com/cnpm/cnpmcore/pull/488 , add lastUsedAt for
classic tokens.
1. 🧶 Modify the `checkTokenExpired` method to `checkTokenStatus` which
update the token field internally.
2. ♻️ No compensation will be made for existing data, and it should be
updated by the consuming end.
---------
> Follow https://github.com/cnpm/cnpmcore/pull/488, 为 classic token 也添加
lastUsedAt 信息
1. 🧶 修改 `checkTokenExpired` 方法为 `checkTokenStatus`，内部进行 token 字段更新
2. ♻️ 存量数据不做补偿，由消费端控制
> Validate the manifest and tarball info to prevent contamination during
consumption, closes#542.
1. 🔨 Added the "strictValidateTarballPkg" mode to enable validation,
only applicable to the slef registry scenario.
2. 🧶 When the configuration is enabled, validate the relevant fields
during publishing, currently only validating the fields affecting
consumption.
3. ♻️ No corrective actions will be taken for existing scenario data.
-----
> 发布时校验 manifest 和 tarball 字段是否陪陪，防止消费时被污染 closes#542
1. 🔨 新增 strictValidateTarballPkg 配置，仅对在发布当前 registry 场景下生效
2. 🧶 配置开启时，发布时校验相关字段，目前仅校验影响消费相关字段
3. ♻️ 存量场景数据不做订正处理
> the registryInfo in pkg fullManifest should be updated when the
package is migrated to another registry.
1. 🐞 when query from DB, dynamically add registry information.
2. ♻️ when hit cache, updating metadata should already trigger cache
modifications.
--------
> 当包从属 registryId 发生变化时，包 manifest 内的信息也需要同步更新
1. 🐞 从 db 读取元信息时，实时添加 registry 信息
2. ♻️ 从缓存读取元信息时，发布或修改元信息时已触发缓存修改
> When executing npm cli owner add command,
[ref](https://github.com/npm/cli/blob/latest/lib/commands/owner.js#L151),
it causes errors due to duplicate additions，when the selfRegistry
configuration has the userPrefix option enabled.
1. 🧶 Modify the user query api to prioritize returning users from
selfRegistry
2. 🧶 Ensure that the query api uniformly returns displayName
---------
> npm cli 执行 owner add 时
[ref](https://github.com/npm/cli/blob/latest/lib/commands/owner.js#L151)，依赖查询结果做去重，selfRegistry
配置 userPrefix 时会导致重复添加报错
1. 🧶 修改用户查询接口，优先返回 selfRegistry 内的用户
2. 🧶 查询接口统一返回 displayName
> During the syncUpstream process, it will attempt to create sync
repeatedly until it times out, when the pkg has been published in the
self registry.
1. 🐞 When executing the syncTask, filter out scenarios where the target
registry is the self registry.
-------
> 包迁移至当前 registry 时，收到同步请求会产生无效的同步任务，当 `syncUpstream` 时，会尝试重复创建 sync
直到超时。
1. 🐞 syncTask 执行时，先过滤目标 registry 是当前 registry 的场景。
> Add the `_source_registry_manifest` in pkgFullManifest.
* 🧶 Set registryInfo during reading because of the excessive existing
data.
* ♻️ The change takes effect when redis cache expired.
----------
> 在 pkgFullManifest 中添加 _source_registry_manifest 相关字段
1. 🧶 存量数据过多，在读取时统一设置，展示当前对应 registryId
2. ♻️ 在读取 db 时生效，需要等缓存过期
> add token info when invoke `whoami` to notify the caller about the
token's current status.
* 🧶 Add token information to the "whoami" interface.
* 🔨 Modify the query logic for allowedPackages uniformly within the
repository.
-----------
> 当使用 granularToken 调用 whoami 信息时，返回当前 token 信息，告知调用方当前 token 状态
* 🧶 在whoami 接口中添加 token 信息
* 🔨 修改 allowedPackages 查询逻辑，统一在 repository 中集成
> For private packages published in the current registry, add the
"_npmUser" field to align with the npm registry.
* 🧶 Add the "_npmUser" field for new scenarios, without modifying the
abbreviated data. Use the following command: curl -H 'Accept:
application/vnd.npm.install-v1+json'
'https://registry.npmjs.org/cnpmcore'
* ♻️ Existing data cannot be traced and will not be compensated.
-----
> 对于在当前 registry 发布的私有包，添加 _npmUser 字段，和公网 registry 保持一致
* 🧶 新增 _npmUser 字段，abbreviated 场景不做修改, (via `curl -H 'Accept:
application/vnd.npm.install-v1+json'
'https://registry.npmjs.org/cnpmcore'`)
* ♻️ 存量数据无法回溯，不做补偿
> Add the lastUsedBy field to the Token model
* 🧶 Add `lastUsedBy` field to the token, for platform display.
* 🐞 Fix the issue where the graunlarToken is not expired in read-only
scenarios.
------
> 为 Token 模型添加 lastUsedBy 字段
* 🧶 新增 lastUsedBy 字段，记录 token 最近使用时间，用于平台展示
* 🐞 修复 graunlarToken 过期时，只读场景没有禁用的问题
> Make the method for tegg integration mode to be more user-friendly.
* 🤖 Automatically add config.cnpmcore type hints.
* 🧶 Export the default `cnpmcoreConfig` , which needs to be explicitly
declared for app config.
* 📚 Supplement the documentation and field definitions.
------
> 对于 egg 集成模式，提供更加友好的自定义配置方式。
* 🤖 自动添加 config.cnpmcore 类型提示
* 🧶 输出默认的 cnpmcoreConfig 对象，应用集成需显式声明，防止新增配置丢失
* 📚 补充文档及字段定义信息
---------
Co-authored-by: fengmk2 <fengmk2@gmail.com>
> Adjust the logic for unpublishing a package
* 🧶 Determine if a call to unpublish within the removePackageVersion
function
* ♻️ Remove`forceRefresh` in unpublishPackage
-------
> 调整 unpublish package 逻辑
* 🧶 removePackageVersion 内判断是否需要调用 unpublish
* ♻️ unpublishPackage 删除 forceRefresh 逻辑
> Fixed the idempotent issue during unpublish pkg, which caused repeated
triggering of change events and endless sync loops for downstream
registries.
* 🐞 Add idempotent check during unpublish; skip when the package has
already been unpublished.
-------------
> 修复 unpublish 时未做幂等控制，导致删包时，不断触发 change 事件，下游 registry 不断 sync 导致任务循环
* 🐞 统一在 unpublish 进行幂等判断，如果该包已 unpublish，则跳过
avoid oss upload fail
> [SignatureDoesNotMatchError]: The request signature we calculated does
not match the signature you provided. Check your key and signing method.
> ENOENT: no such file or directory, stat
'/root/.cnpmcore/downloads/2023/05/06/unpkg_@iov_wallet-providers@1.0.0_0f152162-9cce-4a80-bacc-41271b7aac3f/package'
> https://github.com/puppeteer/puppeteer/issues/10131 Puppeteer has
updated the default browser to Chrome and added the corresponding
implementation as follows:
🧶 Added a new category `/-/binary/` for Chrome , exp:
`/-/binary/chrome-for-testing/113.0.5672.63/mac-arm64/chrome-mac-arm64.zip`
-----------
> https://github.com/puppeteer/puppeteer/issues/10131 puppeteer
更新了默认浏览器为 chrome，新增对应实现
🧶 `/-/binary/` 新增 chrome binary 分类,
示例链接`/-/binary/chrome-for-testing/113.0.5672.63/mac-arm64/chrome-mac-arm64.zip`
> Add a private field, _source_registry_name in the version manifest.
* 🧶 Add related types for PackageManifestType and adjust relevant unit
tests.
* 🤖 Update the workflow trigger.
* ♻️ No compensation will be made for the _source_registry_name field in
the existing packageVersion.
-------
> 在 version manifest 中新增私有字段，_source_registry_name 用于标记
* 🧶 新增 PackageManifestType 相关类型，并调整相关单测
* 🤖 调整 workflow 触发时机，不限制 target 分支
* ♻️ 存量 packageVersion 内 _source_registry_name 不做补偿
> 🚀 Added implementation related to
[granularToken](https://docs.npmjs.com/about-access-tokens#about-granular-access-tokens),
mainly used for web authorization scenarios.
* 📝 Added `1.14.0.sql` to add fields and `token_packages` for
granularToken.
* 🛣️ Added gat related routes, including `create`, `query`, and `delete`
api.
* 🌟 Added `tokenService` to check granularToken access.
* 🔄 Modified Token to perform options and data attribute conversions
internally in the model.
-----------
> 🚀 新增
[granularToken](https://docs.npmjs.com/about-access-tokens#about-granular-access-tokens)
相关实现，主要用于 web 端授权场景
* 📝 新增 `1.14.0.sql` 添加 granularToken 相关字段及 `token_packages` 中间表
* 🛣️ 新增 gat 相关路由，包括`创建`、`查询`、`删除`接口
* 🌟 新增 `tokenService` ，处理 granularToken 鉴权
* 🔄 修改 Token ，在 model 内部进行 options 和 data 属性转换
> Fixed the issue where the registry was not correctly matched when
synchronizing scoped packages for the first time
* Add scope params in initSpecRegistry
------------
> 修复初次同步 scope 包，未正确匹配 registry 的问题
* 修改 initSpecRegistry 方法，统一传入 scope 参数
> 💥 TaskTimeoutHandler did not have try-catch, the redis lock will cause
all queues to fail when a single task update failed.
* 🛡️ Added try-catch statements in TaskTimeoutHandler.
* 🚧 Restricted updates to the primary key when updating the model in
ModelConvertor.
---------------
> 💥 TaskTimeoutHandler 未添加 try-catch，且有同步锁，导致单个任务更新异常时，所有队列不生效
* 🛡️ TaskTimeoutHandler 统一添加 try-catch
* 🚧 ModelConvertor 更新模型时，统一限制不允许更新主键
1. webauth 由 authentication 改造为 session，并增加 web 登录页面，更安全
2. 支持 webauthn 的登录方式，可通过配置控制（默认关闭），更高效
---------------
1. use session instead http authentication on webauth
2. support [webauthn](https://webauthn.guide/), you should set
`enableWebAuthn: true` in the configuration
closes https://github.com/cnpm/cnpmcore/issues/236
---------
Co-authored-by: lanxiu.lwl <lanxiu.lwl@alipay.com>
Co-authored-by: elrrrrrrr <elrrrrrrr@gmail.com>
> 在禁止自动创建同步任务时也可以关闭 redirectNotFound，实现在私有化部署时用户仅能使用当前仓库内已有的依赖
--------------
> Allow to turn off redirectNotFound when disabling the automatic
creation of sync tasks, enabling users to use only existing dependencies
in the current repository when deploying privately.
---------
Co-authored-by: fengmk2 <fengmk2@gmail.com>
> Support for migrating packages into current registry
1. 🆕 Add `ensureSelfRegistry` method to initialize the current
configuration to the DB
2. 🧹 Add displayName to hide userPrefix info
3. 🧶 Uniformly determine publish access with `checkPublishAccess` and
`ensurePublishAccess`
--------------
> 支持将包迁移至当前 registry，避免不再进行包同步
1. 🆕 `ensureSelfRegistry` 方法，将当前配置初始化至 DB
2. 🧹 添加 displayName，外部不再展示 userPrefix 信息
3. 🧶 通过 checkPublishAccess 及 ensurePublishAccess 统一判断发布权限
> 当前请求 changesStream 失败时，需等待 15 分钟超时调度。
* 原 suspendTaskWhenExit 重构为 suspendSync ，支持传入 exit 参数，控制是否继续等待
* 请求 changesStream 失败时，主动挂起任务
------
> Wait 15 minutes for timeout scheduling if the current request
changesStream fails
* `suspendTaskWhenExit` is refactored to `suspendSync`, add exit
parameter to control whether to exiting the queue
* Suspend task when request changesStream fails
> restrict binaryName type , the single source is the `config/binary.js`
file.
* export `BinaryName` & `CategoryName` type
* use `BinaryNameRule` typebox validator in controller
* `binaryName: string` => `binaryName: BinaryName`
Fix partial `config/binaries` file path in binary.
Prevent js parsing issues when cnpmcore is required as an npm module.
This is the part that was missed in the previous pr .
https://github.com/cnpm/cnpmcore/pull/384
cnpmcore is a TypeScript-based private NPM registry implementation built with Egg.js framework. It provides enterprise-grade package management with support for MySQL/PostgreSQL databases, Redis caching, and optional Elasticsearch.
**ALWAYS reference these instructions first** and fallback to search or bash commands only when you encounter unexpected information that does not match the information here.
## Code Style and Conventions
### Linting and Formatting
- **Linter**: Oxlint (fast Rust-based linter)
- **Formatter**: Prettier with specific configuration
- **Pre-commit hooks**: Husky + lint-staged automatically format and lint on commit
**Code Style Rules:**
```javascript
// From .prettierrc
{
"singleQuote":true,// Use single quotes
"trailingComma":"es5",// ES5 trailing commas
"tabWidth":2,// 2-space indentation
"printWidth":120,// 120 character line width
"arrowParens":"avoid"// Avoid parens when possible
}
// From .oxlintrc.json
{
"max-params":6,// Maximum 6 function parameters
"no-console":"warn",// Warn on console usage
"import/no-anonymous-default-export":"error"
}
```
**Linting Commands:**
```bash
npm run lint # Check for linting errors
npm run lint:fix # Auto-fix linting issues
npm run typecheck # TypeScript type checking without build
```
### TypeScript Conventions
- Use strict TypeScript with comprehensive type definitions
- Avoid `any` types - use proper typing or `unknown`
- Export types and interfaces for reusability
- Use ES modules (`import/export`) syntax throughout
### Testing Conventions
- Test files use `.test.ts` suffix
- Use `@eggjs/mock` for mocking and testing
- Tests organized to mirror source structure in `test/` directory
- Use `assert` from `node:assert/strict` for assertions
- Mock external dependencies using `mock()` from `@eggjs/mock`
- **UserController**: User authentication and profile management
### Infrastructure Adapters (`app/infra/`)
Enterprise customization layer for PaaS integration. cnpmcore provides default implementations, but enterprises should implement their own based on their infrastructure:
# Defines two custom environment variables for the workflow. These are used for the Container registry domain, and a name for the Docker image that this workflow builds.
env:
REGISTRY:ghcr.io
IMAGE_NAME:${{ github.repository }}
# There is a single job in this workflow. It's configured to run on the latest available version of Ubuntu.
# Sets the permissions granted to the `GITHUB_TOKEN` for the actions in this job.
permissions:
contents:read
packages:write
attestations:write
id-token:write
steps:
- name:Checkout repository
uses:actions/checkout@v5
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
- name:Log in to the Container registry
if:github.event_name != 'pull_request'
uses:docker/login-action@v3
with:
registry:${{ env.REGISTRY }}
username:${{ github.actor }}
password:${{ secrets.GITHUB_TOKEN }}
- name:Set up QEMU
uses:docker/setup-qemu-action@v3
- name:Set up Docker Buildx
uses:docker/setup-buildx-action@v3
# This step uses [docker/metadata-action](https://github.com/docker/metadata-action#about) to extract tags and labels that will be applied to the specified image. The `id` "meta" allows the output of this step to be referenced in a subsequent step. The `images` value provides the base name for the tags and labels.
- name:Extract metadata (tags, labels) for Docker
id:meta
uses:docker/metadata-action@v5
with:
images:${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# This step uses the `docker/build-push-action` action to build the image, based on your repository's `Dockerfile`. If the build succeeds, it pushes the image to GitHub Packages.
# It uses the `context` parameter to define the build's context as the set of files located in the specified path. For more information, see [Usage](https://github.com/docker/build-push-action#usage) in the README of the `docker/build-push-action` repository.
# It uses the `tags` and `labels` parameters to tag and label the image with the output from the "meta" step.
- name:Build and push Docker image
id:push
uses:docker/build-push-action@v6
with:
context:.
file:.docker/debian/Dockerfile
platforms:linux/amd64,linux/arm64
push:${{ github.event_name != 'pull_request' }}
tags:${{ steps.meta.outputs.tags }}
labels:${{ steps.meta.outputs.labels }}
# This step generates an artifact attestation for the image, which is a tamper-proof statement about where and how it was built. It increases supply chain security for people who consume the image. For more information, see [Using artifact attestations to establish provenance for builds](/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds).
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
cnpmcore is a TypeScript-based private NPM registry implementation for enterprise use. It's built on the Egg.js framework using Domain-Driven Design (DDD) architecture principles and supports both MySQL and PostgreSQL databases.
## Essential Commands
### Development
```bash
# Start development server (MySQL)
npm run dev
# Start development server (PostgreSQL)
npm run dev:postgresql
# Lint code
npm run lint
# Fix linting issues
npm run lint:fix
# TypeScript type checking
npm run typecheck
```
### Testing
```bash
# Run all tests with MySQL (takes 4+ minutes)
npm run test
# Run all tests with PostgreSQL (takes 4+ minutes)
npm run test:postgresql
# Run single test file (faster iteration, ~12 seconds)
* [[`4e8700c`](http://github.com/cnpm/cnpmcore/commit/4e8700c4f7c6fb5c4f4d4a2b9a9546096c5d10e2)] - fix: only create createHookTask if hook enable (#299) (killa <<killa123@126.com>>)
**others**
* [[`e06c841`](http://github.com/cnpm/cnpmcore/commit/e06c841537113fdb0c00beb22b0a55378c61ce80)] - 🐛 FIX: Should sync public package when registryName not exists (#303) (fengmk2 <<fengmk2@gmail.com>>)
* [[`709d65b`](http://github.com/cnpm/cnpmcore/commit/709d65bd0473856c9bfc4416ea2ca375136e354f)] - 🤖 TEST: Use diff bucket on OSS test (#301) (fengmk2 <<fengmk2@gmail.com>>)
* [[`7106807`](http://github.com/cnpm/cnpmcore/commit/710680742a078b2faf4cb18c3a39c0397308712e)] - 🐛 FIX: Should show queue size on logging (#280) (fengmk2 <<fengmk2@gmail.com>>)
* [[`c2b7d5a`](http://github.com/cnpm/cnpmcore/commit/c2b7d5aa98b5ba8649ec246c616574a22e9a74b8)] - feat: use sort set to impl queue (#277) (killa <<killa123@126.com>>)
1.9.1 / 2022-07-29
==================
**fixes**
* [[`c54aa21`](http://github.com/cnpm/cnpmcore/commit/c54aa2165c3938dcbb5a2b3b54e66a0d961cc813)] - fix: check executingCount after task is done (#276) (killa <<killa123@126.com>>)
**others**
* [[`3268d03`](http://github.com/cnpm/cnpmcore/commit/3268d030b620825c8c2e6331e1745c1788066c61)] - 🤖 TEST: show package not use cache if isSync (#273) (fengmk2 <<fengmk2@gmail.com>>)
* [[`b49a38c`](http://github.com/cnpm/cnpmcore/commit/b49a38c77e044c978e6de32a9d3e257cc90ea7c1)] - feat: use Model with inject (#269) (killa <<killa123@126.com>>)
1.7.1 / 2022-07-20
==================
**fixes**
* [[`52fca55`](http://github.com/cnpm/cnpmcore/commit/52fca55aa883865f0ae70bfc1ff274c313b8f76a)] - fix: show package not use cache if isSync (#268) (killa <<killa123@126.com>>)
* [[`4ea0ef6`](http://github.com/cnpm/cnpmcore/commit/4ea0ef63b7af9fd4dcc247c2c2ac8e4d579f941a)] - fix: query changes with order by id asc (#251) (killa <<killa123@126.com>>)
Reimplementation based on [cnpmjs.org](https://github.com/cnpm/cnpmjs.org) with TypeScript.
Reimplement based on [cnpmjs.org](https://github.com/cnpm/cnpmjs.org) with TypeScript.
## Registry HTTP API
See https://github.com/cnpm/cnpmjs.org/blob/master/docs/registry-api.md#npm-registry-api
See [registry-api.md](docs/registry-api.md)
## Internal API for Direct HTTP Requests
See [internal-api.md](docs/internal-api.md) for comprehensive documentation of cnpmcore's internal APIs that allow direct HTTP requests for package synchronization, administration, and other advanced operations.
// {"message":"API rate limit exceeded for 47.57.239.54. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
