Skip to content

Commit 832e3ab

Browse files
authored
docs(release): add README for Dragonfly v2.4.0 (#136)
Signed-off-by: Gaius <gaius.qi@gmail.com>
1 parent 39b55b0 commit 832e3ab

File tree

9 files changed

+264
-0
lines changed

9 files changed

+264
-0
lines changed

.markdownlint.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
MD010: false
22
MD013:
33
line_length: 1024
4+
MD024: false
45
MD046:
56
style: "fenced"

releases/2.4.0/README-zh.md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Dragonfly v2.4.0 发布了
2+
3+
Dragonfly v2.4.0 发布了!🎉🎉🎉 感谢为此版本做出贡献的[贡献者们](https://github.com/dragonflyoss/dragonfly/graphs/contributors),欢迎访问 [d7y.io](https://d7y.io) 网站。
4+
5+
![dragonfly](images/dragonfly-linear.png)
6+
7+
## 新增特性
8+
9+
### 负载感知调度算法
10+
11+
调度算法升级为一种两阶段调度算法,结合中央调度与节点级二次调度,基于实时负载感知来优化 P2P 下载性能。
12+
13+
![p1](images/p1.svg)
14+
15+
更多信息,请参阅 [Scheduling](https://d7y.io/docs/next/operations/deployment/applications/scheduler/#scheduling)
16+
17+
### Vortex 协议支持 P2P 文件传输
18+
19+
Dragonfly 提供了基于 TLV 的新型 Vortex 传输协议,以提升 P2P 下载性能。使用 TLV(Tag-Length-Value)格式作为轻量级协议,替代 gRPC 进行节点间的数据传输。
20+
与 gRPC 相比,基于 TCP 的 Vortex 可将大文件下载时间减少 50%,基于 QUIC 的 Vortex 可减少 40%,两者都能有效降低峰值内存使用。
21+
22+
更多信息,请参阅 [TCP Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/tcp-protocol/tcp-protocol.md)[QUIC Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/quic-protocol/quic-protocol.md)
23+
24+
### Request SDK
25+
26+
新增 SDK 用于将用户请求路由到 Seed Peer,使用一致性哈希算法,取代之前的 Kubernetes Service 负载均衡方式。
27+
28+
![p2](images/p2.svg)
29+
30+
更多信息,请参阅 [Request SDK](https://d7y.io/docs/next/advanced-guides/request-sdk/)
31+
32+
### 指定集群 ID 实现多集群 Kubernetes 简化部署
33+
34+
Dragonfly 支持一种简化功能,用于部署和管理多个 Kubernetes 集群,方法是为每个集群显式分配一个 `schedulerClusterID`
35+
这种方式允许用户直接控制集群亲和性,而无需依赖基于位置的调度元数据,例如 IDC、主机名或 IP 地址。
36+
37+
使用此功能,每个 Peer、Seed Peer 和 Scheduler 都通过明确定义的 Scheduler Cluster ID 来确定其目标调度器集群。
38+
这确保了集群之间的精确隔离以及可预测的跨集群行为。
39+
40+
![p3](images/p3.png)
41+
42+
更多信息,请参阅 [Create Dragonfly Cluster Simple](https://d7y.io/docs/next/getting-started/quick-start/multi-cluster-kubernetes/#create-dragonfly-cluster-simple)
43+
44+
### Manager 和 Scheduler 组件的性能和资源优化
45+
46+
增强了 Manager 和 Scheduler 组件的性能和资源利用率,同时显著降低了 CPU 和内存开销,提升了系统效率和资源管理能力。
47+
48+
### 增强预热功能
49+
50+
- 支持预热到指定节点:采用基于优先级的选择逻辑,其中 IP 指定具有最高优先级,其次是基于数量的选择,最后是基于百分比的选择。
51+
52+
- 支持在单个预热任务中预热多个 URL。
53+
54+
- 支持通过 Scheduler gRPC 接口预热文件和镜像。
55+
56+
![p4](images/p4.png)
57+
58+
### 基于镜像 Blob SHA256 计算 ID 以避免重复下载
59+
60+
客户端现已支持直接根据镜像 Blob 的 SHA256 哈希值计算任务 ID,而不再使用下载 URL。此增强功能可以防止当同一 Blob 从不同镜像仓库
61+
域名访问时产生的重复下载和数据冗余问题,新版本默认启用此功能。
62+
63+
### 缓存 HTTP 307 重定向 URL
64+
65+
新增对 HTTP 307(临时重定向)响应 URL 的缓存支持,以优化 Dragonfly 的多分片下载性能。当下载 URL 被拆分为多个分片时,
66+
重定向目标地址现在会被缓存,从而消除冗余的重定向请求并降低延迟。
67+
68+
### Go 客户端已弃用,并由 Rust 客户端取代
69+
70+
Go 客户端已被弃用,并由 [Rust 客户端](https://github.com/dragonflyoss/client) 取代。未来所有的开发和维护工作将专注于 Rust 客户端,它提供了更好的性能、稳定性和可靠性。
71+
72+
更多信息,请参阅 [dragoflyoss/client](https://github.com/dragonflyoss/client)
73+
74+
### 附加功能增强
75+
76+
- 在 Rust 客户端中启用 ARM64 架构的 64K 页面大小支持。
77+
- 修复 dfget 版本输出中缺失的 git commit 元数据问题。
78+
- 支持 containerd V3 配置中 `io.containerd.cri.v1.images` 插件的 `config_path` 配置。
79+
-`reqwest` 中使用 `hickory-dns` 替换 `glibc DNS` 解析器,以实现 DNS 缓存并防止分片下载过程中过多的 DNS 查询。
80+
- 支持 `--include-files` 配置参数,可选择性地从目录中下载文件。
81+
- 添加 `--no-progress` 配置参数,可禁用下载进度条输出。
82+
- 支持 `backend` 操作中的自定义请求头,为 HTTP 请求提供灵活的请求头配置。
83+
- 重构日志输出,减少冗余日志记录并提高日志可读性。
84+
85+
## 重要修复
86+
87+
- 将数据库字段类型从 `text` 修改为 `longtext`,以支持存储预热任务的信息。
88+
- 修复了 Scheduler 关闭期间重复停止 Seed Peer 服务时导致的 panic 问题。
89+
- 修复了在指定 Redis 密码但未设置用户名时,Broker 认证失败的问题。
90+
91+
## Nydus
92+
93+
### 新增特性
94+
95+
- Nydusd:为 RAFS V5 和 V6 格式添加运行时 CRC32 校验支持,增强数据完整性验证。
96+
- Nydusd:支持在 Nydusd 恢复过程中重新发送 FUSE 请求,提高进程恢复的可靠性。
97+
- Nydusd:增强 VFS 状态保存机制,用于进程热升级和故障转移。
98+
- Nydusify:支持 Nydus 到 OCI 的反向转换能力(实验性功能),支持无缝迁移回 OCI 格式。
99+
- Nydusify:实现镜像拷贝时零磁盘传输,显著减少复制操作期间对本地磁盘的使用。
100+
- Snapshotter:将 `blob.meta` 打包到 Bootstrap 中,提高 RAFS V6 镜像的 Blob 下载时的可靠性
101+
102+
### 重要修复
103+
104+
- Nydusd:修复镜像仓库认证中 `access_token` 字段的认证令牌获取问题。
105+
- Nydusd:为 umount API 添加了递归的 inode/dentry 失效处理。
106+
- Nydus Image:修复 optimize 子命令中的多个问题,并添加后端配置支持。
107+
- Snapshotter:为代理模式实现延迟父快照恢复,修复父快照缺失问题。
108+
109+
## 其他
110+
111+
您可以在 [CHANGELOG](https://github.com/dragonflyoss/dragonfly/blob/main/CHANGELOG.md) 中查看更多详细信息。
112+
113+
## 链接
114+
115+
- Dragonfly 网站:<https://d7y.io/>
116+
- Dragonfly 仓库:<https://github.com/dragonflyoss/dragonfly>
117+
- Dragonfly 客户端仓库:<https://github.com/dragonflyoss/client>
118+
- Dragonfly 控制台仓库:<https://github.com/dragonflyoss/console>
119+
- Dragonfly Charts 仓库:<https://github.com/dragonflyoss/helm-charts>
120+
- Dragonfly 监控仓库:<https://github.com/dragonflyoss/monitoring>
121+
122+
## Dragonfly Github
123+
124+
![qrcode](images/qrcode.webp)

releases/2.4.0/README.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Dragonfly v2.4.0 is released
2+
3+
Dragonfly v2.4.0 is released!🎉🎉🎉 Thanks the [contributors](https://github.com/dragonflyoss/dragonfly/graphs/contributors) who made this release happend and welcome you to visit [d7y.io](https://d7y.io) website.
4+
5+
![dragonfly](images/dragonfly-linear.png)
6+
7+
## New features and enhancements
8+
9+
### load-aware scheduling algorithm
10+
11+
A two-stage scheduling algorithm combining central scheduling with node-level secondary scheduling to
12+
optimize P2P download performance based on real-time load awareness.
13+
14+
![p1](images/p1.svg)
15+
16+
For more information, please refer to the [Scheduling](https://d7y.io/docs/next/operations/deployment/applications/scheduler/#scheduling).
17+
18+
### Vortex Protocol Support for P2P File Transfer
19+
20+
Dragonfly provide the new Vortex transfer protocol based on TLV to improve the download performance in internal network.
21+
Use the TLV (Tag-Length-Value) format as a lightweight protocol to replace gRPC for data transfer between peers.
22+
TCP-based Vortex reduces large file download time by `50%` and QUIC-based Vortex by `40%` compared to gRPC, both effectively reducing peak memory usage.
23+
24+
For more information, please refer to the [TCP Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/tcp-protocol/tcp-protocol.md) and [QUIC Protocol Support for P2P File Transfer](https://github.com/dragonflyoss/design/blob/main/systems-analysis/quic-protocol/quic-protocol.md).
25+
26+
### Request SDK
27+
28+
A SDK for routing User requests to Seed Peers using consistent hashing, replacing the previous Kubernetes Service load balancing approach.
29+
30+
![p2](images/p2.svg)
31+
32+
For more details, please refer to [Request SDK](https://d7y.io/docs/next/advanced-guides/request-sdk/).
33+
34+
### Simple Multi‑Cluster Kubernetes Deployment with Scheduler Cluster ID
35+
36+
Dragonfly supports a simplified feature for deploying and managing multiple Kubernetes clusters by explicitly assigning a `schedulerClusterID` to each cluster.
37+
This approach allows users to directly control cluster affinity without relying on location‑based scheduling metadata such as IDC, hostname, or IP.
38+
39+
Using this feature, each Peer, Seed Peer, and Scheduler determines its target scheduler cluster through a clearly defined scheduler cluster ID.
40+
This ensures precise separation between clusters and predictable cross‑cluster behavior.
41+
42+
![p3](images/p3.png)
43+
44+
For more information, please refer to the [Create Dragonfly Cluster Simple](https://d7y.io/docs/next/getting-started/quick-start/multi-cluster-kubernetes/#create-dragonfly-cluster-simple).
45+
46+
### Performance and Resource Optimization for Manager and Scheduler Components
47+
48+
Enhanced service performance and resource utilization across Manager and Scheduler components while significantly reducing
49+
CPU and memory overhead, delivering improved system efficiency and better resource management.
50+
51+
### Enhanced Preheating
52+
53+
- Support for IP-based peer selection in preheating jobs with priority-based selection logic where IP specification
54+
takes highest priority, followed by count-based and percentage-based selection
55+
56+
- Support for preheating multiple URLs in a single request.
57+
58+
- Support for preheating file and image via Scheduler gRPC interface.
59+
60+
![p4](images/p4.png)
61+
62+
### Calculate task ID based on image blob SHA256 to avoid redundant downloads
63+
64+
The Client now supports calculating task IDs directly from the SHA256 hash of image blobs instead of using the download URL.
65+
This enhancement prevents redundant downloads and data duplication when the same blob is accessed from
66+
different registry domains.
67+
68+
### Cache HTTP 307 redirects for split downloads
69+
70+
Support for caching HTTP 307 (Temporary Redirect) responses to optimize Dragonfly's multi-piece download performance.
71+
When a download URL is split into multiple pieces, the redirect target is now cached,
72+
eliminating redundant redirect requests and reducing latency.
73+
74+
### Go Client Deprecated and Replaced by Rust Client
75+
76+
The Go client has been deprecated and replaced by the [Rust Client](https://github.com/dragonflyoss/client). All future development and maintenance will focus
77+
exclusively on the Rust client, which offers improved performance, stability, and reliability.
78+
79+
For more information, please refer to the [dragoflyoss/client](https://github.com/dragonflyoss/client).
80+
81+
### Additional Enhancements
82+
83+
- Enable 64K page size support for ARM64 in the Dragonfly Rust client.
84+
- Fix missing git commit metadata in dfget version output.
85+
- Support for `config_path` of `io.containerd.cri.v1.images` plugin for containerd V3 configuration.
86+
- Replaces glibc DNS resolver with hickory-dns in reqwest to implement DNS caching and prevent excessive DNS lookups during piece downloads.
87+
- Support for the --include-files flag to selectively download files from a directory.
88+
- Add the `--no-progress flag` to disable the download progress bar output.
89+
- Support for custom request headers in backend operations, enabling flexible header configuration for HTTP requests.
90+
- Refactored log output to reduce redundant logging and improve overall logging efficiency.
91+
92+
## Significant bug fixes
93+
94+
- Modified the database field type from `text` to `longtext` to support storing the information of preheating job.
95+
- Fixed panic on repeated seed peer service stops during Scheduler shutdown.
96+
- Fixed broker authentication failure when specifying the Redis password without setting a username.
97+
98+
## Nydus
99+
100+
### New features and enhancements
101+
102+
- Nydusd: Add CRC32 validation support for both RAFS V5 and V6 formats, enhancing data integrity verification.
103+
- Nydusd: Support resending FUSE requests during nydusd restoration, improving daemon recovery reliability.
104+
- Nydusd: Enhance VFS state saving mechanism for daemon hot upgrade and failover.
105+
- Nydusify: Introduce Nydus-to-OCI reverse conversion capability, enabling seamless migration back to OCI format.
106+
- Nydusify: Implement zero-disk transfer for image copy, significantly reducing local disk usage during copy operations.
107+
- Snapshotter: Builtin blob.meta in bootstrap for blob fetch reliability for RAFS v6 image.
108+
109+
### Significant bug fixes
110+
111+
- Nydusd: Fix auth token fetching for `access_token` field in registry authentication.
112+
- Nydusd: Add recursive inode/dentry invalidation for umount API.
113+
- Nydus Image: Fix multiple issues in optimize subcommand and add backend configuration support.
114+
- Snapshotter: Implement lazy parent recovery for proxy mode to handle missing parent snapshots.
115+
116+
## Others
117+
118+
You can see [CHANGELOG](https://github.com/dragonflyoss/dragonfly/blob/main/CHANGELOG.md) for more details.
119+
120+
## Links
121+
122+
- Dragonfly Website: <https://d7y.io/>
123+
- Dragonfly Repository: <https://github.com/dragonflyoss/dragonfly>
124+
- Dragonfly Client Repository: <https://github.com/dragonflyoss/client>
125+
- Dragonfly Console Repository: <https://github.com/dragonflyoss/console>
126+
- Dragonfly Charts Repository: <https://github.com/dragonflyoss/helm-charts>
127+
- Dragonfly Monitor Repository: <https://github.com/dragonflyoss/monitoring>
128+
129+
## Dragonfly Github
130+
131+
![qrcode](images/qrcode.webp)
36.7 KB
Loading

releases/2.4.0/images/p1.svg

Lines changed: 4 additions & 0 deletions
Loading

releases/2.4.0/images/p2.svg

Lines changed: 4 additions & 0 deletions
Loading

releases/2.4.0/images/p3.png

125 KB
Loading

releases/2.4.0/images/p4.png

408 KB
Loading

releases/2.4.0/images/qrcode.webp

4.29 KB
Loading

0 commit comments

Comments
 (0)