38 Commits

Author SHA1 Message Date
starry
61f09192bb Update README.md 2025-06-27 09:06:44 +08:00
starry
d876809086 完善一些小细节 2025-06-27 08:50:04 +08:00
user123456
fe9156f878 Merge commit 'refs/pull/origin/28' 2025-06-21 00:30:51 +08:00
starry
35651e214f proxy字段修复 2025-06-21 00:15:27 +08:00
user123456
d373e0104d 获取更多镜像tag 2025-06-20 23:44:13 +08:00
starry
207a03a511 Merge pull request #25 from beck-8/me/op_proxy
优化代理配置
2025-06-19 23:00:44 +08:00
beck-8
5bd32cd6c1 go fmt . 2025-06-19 22:53:20 +08:00
beck-8
8c127a795b op http client proxy 2025-06-19 22:52:51 +08:00
user123456
2567652a7d 更新配置说明 2025-06-18 22:26:19 +08:00
user123456
c023e6a9c4 清理冗余written字段 2025-06-18 22:05:28 +08:00
user123456
44c6e4cd7b 修复双重写入 2025-06-18 21:29:56 +08:00
user123456
c22bd0637a 更新默认配置 2025-06-18 20:49:45 +08:00
user123456
a94b476726 移除冗余的限流智能判断逻辑 2025-06-18 20:44:26 +08:00
user123456
4c6751b862 限流改为全局应用 2025-06-18 19:44:32 +08:00
user123456
acc63d7b68 删除热重载 2025-06-18 19:14:13 +08:00
starry
d0b1ea8582 LF 2025-06-18 17:08:14 +08:00
starry
c607061dae LF 2025-06-18 17:07:43 +08:00
starry
143de7b254 Normalize all line endings to LF 2025-06-18 17:03:29 +08:00
user123456
51ace73b78 优化离线镜像的防抖以及日志 2025-06-18 16:04:53 +08:00
user123456
fa9e9210ab 默认为原始压缩层 2025-06-18 15:14:33 +08:00
user123456
f308410920 修复函数调用点传递 2025-06-18 15:00:41 +08:00
user123456
252dc319c6 优化离线包体积 2025-06-18 14:55:35 +08:00
user123456
29ceeef45b IPv6日志适配 2025-06-17 18:49:34 +08:00
user123456
182dced403 修复ipv6标准化的潜在BUG 2025-06-17 18:38:48 +08:00
user123456
aea36939a3 增加支持走代理 2025-06-17 18:18:17 +08:00
starry
4240c1452a Update README.md 2025-06-16 00:51:06 +08:00
starry
212c8e529d Update README.md 2025-06-15 16:18:54 +08:00
starry
3fd630159b Update config.toml 2025-06-14 14:11:14 +08:00
starry
17d827f50b Update README.md 2025-06-14 14:10:31 +08:00
starry
7dcbc839c6 Update README.md 2025-06-14 14:10:07 +08:00
starry
45ffebc820 Update README.md 2025-06-14 14:08:08 +08:00
starry
3027b1f218 Update README.md 2025-06-13 18:31:13 +08:00
starry
3d2c419ebe Update README.md 2025-06-13 18:30:14 +08:00
starry
b529fbfdd2 Update README.md 2025-06-13 18:29:38 +08:00
user123456
737c1dbf46 io.Copy 2025-06-13 17:58:13 +08:00
user123456
a67ef6c52c 离线镜像下载去掉缓存,避免缓存不完整导致空指针 2025-06-13 17:00:47 +08:00
starry
0adf11099e add 2025-06-13 16:25:27 +08:00
starry
dbb9432eb0 Create LICENSE 2025-06-13 14:11:56 +08:00
22 changed files with 3017 additions and 2744 deletions

1
.gitattributes vendored Normal file
View File

@@ -0,0 +1 @@
* text=auto eol=lf

4
.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
.idea
.vscode
.DS_Store
hubproxy*

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 sky22333
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

136
README.md
View File

@@ -2,7 +2,7 @@
🚀 **Docker 和 GitHub 加速代理服务器**
一个轻量级、高性能的多功能代理服务,提供 Docker 镜像加速、GitHub 文件加速等功能。
一个轻量级、高性能的多功能代理服务,提供 Docker 镜像加速、GitHub 文件加速、下载离线镜像、在线搜索 Docker 镜像等功能。
## ✨ 特性
@@ -14,7 +14,8 @@
- 🚫 **仓库审计** - 强大的自定义黑名单白名单同时审计镜像仓库和GitHub仓库
- 🔍 **镜像搜索** - 在线搜索 Docker 镜像
-**轻量高效** - 基于 Go 语言,单二进制文件运行,资源占用低,优雅的内存清理机制。
- 🔧 **配置热重载** - 统一配置管理,部分配置项支持热重载,无需重启服务
- 🔧 **统一配置** - 统一配置管理
## 🚀 快速开始
@@ -29,12 +30,14 @@ docker run -d \
### 一键安装
### 一键脚本安装
```bash
curl -fsSL https://raw.githubusercontent.com/sky22333/hubproxy/main/install-service.sh | sudo bash
```
也可以直接下载二进制文件执行`./hubproxy`使用无需配置文件即可启动内置默认配置支持所有功能。初始内存占用约18M二进制文件大小约12M
这个命令会:
- 🔍 自动检测系统架构AMD64/ARM64
- 📥 从 GitHub Releases 下载最新版本
@@ -55,7 +58,9 @@ docker pull nginx
docker pull yourdomain.com/nginx
# ghcr加速
docker pull yourdomain.com/ghcr.io/user/images
docker pull yourdomain.com/ghcr.io/sky22333/hubproxy
# 符合Docker Registry API v2标准的仓库都支持
```
### GitHub 文件加速
@@ -66,16 +71,134 @@ https://github.com/user/repo/releases/download/v1.0.0/file.tar.gz
# 加速链接
https://yourdomain.com/https://github.com/user/repo/releases/download/v1.0.0/file.tar.gz
# 加速下载仓库
git clone https://yourdomain.com/https://github.com/sky22333/hubproxy.git
```
## ⚙️ 配置
<details>
<summary>config.toml配置说明</summary>
## ⚙️ 提示
此配置是默认配置
主配置文件位于 `/opt/hubproxy/config.toml`
```
[server]
host = "0.0.0.0"
# 监听端口
port = 5000
# Github文件大小限制字节默认2GB
fileSize = 2147483648
[rateLimit]
# 每个IP每小时允许的请求数(注意Docker镜像会有多个层会消耗多个次数)
requestLimit = 500
# 限流周期(小时)
periodHours = 1.0
[security]
# IP白名单支持单个IP或IP段
# 白名单中的IP不受限流限制
whiteList = [
"127.0.0.1",
"172.17.0.0/16",
"192.168.1.0/24"
]
# IP黑名单支持单个IP或IP段
# 黑名单中的IP将被直接拒绝访问
blackList = [
"192.168.100.1",
"192.168.100.0/24"
]
[proxy]
# 代理服务白名单支持GitHub仓库和Docker镜像支持通配符
# 只允许访问白名单中的仓库/镜像,为空时不限制
whiteList = []
# 代理服务黑名单支持GitHub仓库和Docker镜像支持通配符
# 禁止访问黑名单中的仓库/镜像
blackList = [
"baduser/malicious-repo",
"*/malicious-repo",
"baduser/*"
]
# 代理配置,支持有用户名/密码认证和无认证模式
# 无认证: socks5://127.0.0.1:1080
# 有认证: socks5://username:password@127.0.0.1:1080
# HTTP 代理示例
# http://username:password@127.0.0.1:7890
# SOCKS5 代理示例
# socks5://username:password@127.0.0.1:1080
# SOCKS5H 代理示例
# socks5h://username:password@127.0.0.1:1080
# 留空不使用代理
proxy = ""
[download]
# 批量下载离线镜像数量限制
maxImages = 10
# Registry映射配置支持多种镜像仓库上游
[registries]
# GitHub Container Registry
[registries."ghcr.io"]
upstream = "ghcr.io"
authHost = "ghcr.io/token"
authType = "github"
enabled = true
# Google Container Registry
[registries."gcr.io"]
upstream = "gcr.io"
authHost = "gcr.io/v2/token"
authType = "google"
enabled = true
# Quay.io Container Registry
[registries."quay.io"]
upstream = "quay.io"
authHost = "quay.io/v2/auth"
authType = "quay"
enabled = true
# Kubernetes Container Registry
[registries."registry.k8s.io"]
upstream = "registry.k8s.io"
authHost = "registry.k8s.io"
authType = "anonymous"
enabled = true
[tokenCache]
# 是否启用缓存(同时控制Token和Manifest缓存)显著提升性能
enabled = true
# 默认缓存时间(分钟)
defaultTTL = "20m"
```
</details>
容器内的配置文件位于 `/root/config.toml`
脚本部署配置文件位于 `/opt/hubproxy/config.toml`
为了IP限流能够正常运行反向代理需要传递IP头用来获取访客真实IP以caddy为例
```
example.com {
reverse_proxy {
to 127.0.0.1:5000
header_up X-Real-IP {remote}
header_up X-Forwarded-For {remote}
header_up X-Forwarded-Proto {scheme}
}
}
```
cloudflare CDN
```
example.com {
reverse_proxy 127.0.0.1:5000 {
header_up X-Forwarded-For {http.request.header.CF-Connecting-IP}
@@ -87,6 +210,7 @@ example.com {
```
## ⚠️ 免责声明
- 本程序仅供学习交流使用,请勿用于非法用途

View File

@@ -1,5 +1,5 @@
services:
ghproxy:
hubproxy:
build: .
restart: always
ports:

View File

@@ -1,220 +1,212 @@
package main
import (
"strings"
"sync"
)
// ResourceType 资源类型
type ResourceType string
const (
ResourceTypeGitHub ResourceType = "github"
ResourceTypeDocker ResourceType = "docker"
)
// AccessController 统一访问控制器
type AccessController struct {
mu sync.RWMutex
}
// DockerImageInfo Docker镜像信息
type DockerImageInfo struct {
Namespace string
Repository string
Tag string
FullName string
}
// 全局访问控制器实例
var GlobalAccessController = &AccessController{}
// ParseDockerImage 解析Docker镜像名称
func (ac *AccessController) ParseDockerImage(image string) DockerImageInfo {
image = strings.TrimPrefix(image, "docker://")
var tag string
if idx := strings.LastIndex(image, ":"); idx != -1 {
part := image[idx+1:]
if !strings.Contains(part, "/") {
tag = part
image = image[:idx]
}
}
if tag == "" {
tag = "latest"
}
var namespace, repository string
if strings.Contains(image, "/") {
parts := strings.Split(image, "/")
if len(parts) >= 2 {
if strings.Contains(parts[0], ".") {
if len(parts) >= 3 {
namespace = parts[1]
repository = parts[2]
} else {
namespace = "library"
repository = parts[1]
}
} else {
namespace = parts[0]
repository = parts[1]
}
}
} else {
namespace = "library"
repository = image
}
fullName := namespace + "/" + repository
return DockerImageInfo{
Namespace: namespace,
Repository: repository,
Tag: tag,
FullName: fullName,
}
}
// CheckDockerAccess 检查Docker镜像访问权限
func (ac *AccessController) CheckDockerAccess(image string) (allowed bool, reason string) {
cfg := GetConfig()
// 解析镜像名称
imageInfo := ac.ParseDockerImage(image)
// 检查白名单(如果配置了白名单,则只允许白名单中的镜像)
if len(cfg.Proxy.WhiteList) > 0 {
if !ac.matchImageInList(imageInfo, cfg.Proxy.WhiteList) {
return false, "不在Docker镜像白名单内"
}
}
// 检查黑名单
if len(cfg.Proxy.BlackList) > 0 {
if ac.matchImageInList(imageInfo, cfg.Proxy.BlackList) {
return false, "Docker镜像在黑名单内"
}
}
return true, ""
}
// CheckGitHubAccess 检查GitHub仓库访问权限
func (ac *AccessController) CheckGitHubAccess(matches []string) (allowed bool, reason string) {
if len(matches) < 2 {
return false, "无效的GitHub仓库格式"
}
cfg := GetConfig()
// 检查白名单
if len(cfg.Proxy.WhiteList) > 0 && !ac.checkList(matches, cfg.Proxy.WhiteList) {
return false, "不在GitHub仓库白名单内"
}
// 检查黑名单
if len(cfg.Proxy.BlackList) > 0 && ac.checkList(matches, cfg.Proxy.BlackList) {
return false, "GitHub仓库在黑名单内"
}
return true, ""
}
// matchImageInList 检查Docker镜像是否在指定列表中
func (ac *AccessController) matchImageInList(imageInfo DockerImageInfo, list []string) bool {
fullName := strings.ToLower(imageInfo.FullName)
namespace := strings.ToLower(imageInfo.Namespace)
for _, item := range list {
item = strings.ToLower(strings.TrimSpace(item))
if item == "" {
continue
}
if fullName == item {
return true
}
if item == namespace || item == namespace+"/*" {
return true
}
if strings.HasSuffix(item, "*") {
prefix := strings.TrimSuffix(item, "*")
if strings.HasPrefix(fullName, prefix) {
return true
}
}
if strings.HasPrefix(item, "*/") {
repoPattern := strings.TrimPrefix(item, "*/")
if strings.HasSuffix(repoPattern, "*") {
repoPrefix := strings.TrimSuffix(repoPattern, "*")
if strings.HasPrefix(imageInfo.Repository, repoPrefix) {
return true
}
} else {
if strings.ToLower(imageInfo.Repository) == repoPattern {
return true
}
}
}
if strings.HasPrefix(fullName, item+"/") {
return true
}
}
return false
}
// checkList GitHub仓库检查逻辑
func (ac *AccessController) checkList(matches, list []string) bool {
if len(matches) < 2 {
return false
}
username := strings.ToLower(strings.TrimSpace(matches[0]))
repoName := strings.ToLower(strings.TrimSpace(strings.TrimSuffix(matches[1], ".git")))
fullRepo := username + "/" + repoName
for _, item := range list {
item = strings.ToLower(strings.TrimSpace(item))
if item == "" {
continue
}
// 支持多种匹配模式
if fullRepo == item {
return true
}
// 用户级匹配
if item == username || item == username+"/*" {
return true
}
// 前缀匹配(支持通配符)
if strings.HasSuffix(item, "*") {
prefix := strings.TrimSuffix(item, "*")
if strings.HasPrefix(fullRepo, prefix) {
return true
}
}
// 子仓库匹配(防止 user/repo 匹配到 user/repo-fork
if strings.HasPrefix(fullRepo, item+"/") {
return true
}
}
return false
}
// Reload 热重载访问控制规则
func (ac *AccessController) Reload() {
ac.mu.Lock()
defer ac.mu.Unlock()
// 访问控制器本身不缓存配置
}
package main
import (
"strings"
"sync"
)
// ResourceType 资源类型
type ResourceType string
const (
ResourceTypeGitHub ResourceType = "github"
ResourceTypeDocker ResourceType = "docker"
)
// AccessController 统一访问控制器
type AccessController struct {
mu sync.RWMutex
}
// DockerImageInfo Docker镜像信息
type DockerImageInfo struct {
Namespace string
Repository string
Tag string
FullName string
}
// 全局访问控制器实例
var GlobalAccessController = &AccessController{}
// ParseDockerImage 解析Docker镜像名称
func (ac *AccessController) ParseDockerImage(image string) DockerImageInfo {
image = strings.TrimPrefix(image, "docker://")
var tag string
if idx := strings.LastIndex(image, ":"); idx != -1 {
part := image[idx+1:]
if !strings.Contains(part, "/") {
tag = part
image = image[:idx]
}
}
if tag == "" {
tag = "latest"
}
var namespace, repository string
if strings.Contains(image, "/") {
parts := strings.Split(image, "/")
if len(parts) >= 2 {
if strings.Contains(parts[0], ".") {
if len(parts) >= 3 {
namespace = parts[1]
repository = parts[2]
} else {
namespace = "library"
repository = parts[1]
}
} else {
namespace = parts[0]
repository = parts[1]
}
}
} else {
namespace = "library"
repository = image
}
fullName := namespace + "/" + repository
return DockerImageInfo{
Namespace: namespace,
Repository: repository,
Tag: tag,
FullName: fullName,
}
}
// CheckDockerAccess 检查Docker镜像访问权限
func (ac *AccessController) CheckDockerAccess(image string) (allowed bool, reason string) {
cfg := GetConfig()
// 解析镜像名称
imageInfo := ac.ParseDockerImage(image)
// 检查白名单(如果配置了白名单,则只允许白名单中的镜像)
if len(cfg.Access.WhiteList) > 0 {
if !ac.matchImageInList(imageInfo, cfg.Access.WhiteList) {
return false, "不在Docker镜像白名单内"
}
}
// 检查黑名单
if len(cfg.Access.BlackList) > 0 {
if ac.matchImageInList(imageInfo, cfg.Access.BlackList) {
return false, "Docker镜像在黑名单内"
}
}
return true, ""
}
// CheckGitHubAccess 检查GitHub仓库访问权限
func (ac *AccessController) CheckGitHubAccess(matches []string) (allowed bool, reason string) {
if len(matches) < 2 {
return false, "无效的GitHub仓库格式"
}
cfg := GetConfig()
// 检查白名单
if len(cfg.Access.WhiteList) > 0 && !ac.checkList(matches, cfg.Access.WhiteList) {
return false, "不在GitHub仓库白名单内"
}
// 检查黑名单
if len(cfg.Access.BlackList) > 0 && ac.checkList(matches, cfg.Access.BlackList) {
return false, "GitHub仓库在黑名单内"
}
return true, ""
}
// matchImageInList 检查Docker镜像是否在指定列表中
func (ac *AccessController) matchImageInList(imageInfo DockerImageInfo, list []string) bool {
fullName := strings.ToLower(imageInfo.FullName)
namespace := strings.ToLower(imageInfo.Namespace)
for _, item := range list {
item = strings.ToLower(strings.TrimSpace(item))
if item == "" {
continue
}
if fullName == item {
return true
}
if item == namespace || item == namespace+"/*" {
return true
}
if strings.HasSuffix(item, "*") {
prefix := strings.TrimSuffix(item, "*")
if strings.HasPrefix(fullName, prefix) {
return true
}
}
if strings.HasPrefix(item, "*/") {
repoPattern := strings.TrimPrefix(item, "*/")
if strings.HasSuffix(repoPattern, "*") {
repoPrefix := strings.TrimSuffix(repoPattern, "*")
if strings.HasPrefix(imageInfo.Repository, repoPrefix) {
return true
}
} else {
if strings.ToLower(imageInfo.Repository) == repoPattern {
return true
}
}
}
if strings.HasPrefix(fullName, item+"/") {
return true
}
}
return false
}
// checkList GitHub仓库检查逻辑
func (ac *AccessController) checkList(matches, list []string) bool {
if len(matches) < 2 {
return false
}
username := strings.ToLower(strings.TrimSpace(matches[0]))
repoName := strings.ToLower(strings.TrimSpace(strings.TrimSuffix(matches[1], ".git")))
fullRepo := username + "/" + repoName
for _, item := range list {
item = strings.ToLower(strings.TrimSpace(item))
if item == "" {
continue
}
// 支持多种匹配模式
if fullRepo == item {
return true
}
// 用户级匹配
if item == username || item == username+"/*" {
return true
}
// 前缀匹配(支持通配符)
if strings.HasSuffix(item, "*") {
prefix := strings.TrimSuffix(item, "*")
if strings.HasPrefix(fullRepo, prefix) {
return true
}
}
// 子仓库匹配(防止 user/repo 匹配到 user/repo-fork
if strings.HasPrefix(fullRepo, item+"/") {
return true
}
}
return false
}

View File

@@ -1,367 +1,276 @@
package main
import (
"fmt"
"os"
"strconv"
"strings"
"sync"
"time"
"github.com/pelletier/go-toml/v2"
"github.com/spf13/viper"
"github.com/fsnotify/fsnotify"
)
// RegistryMapping Registry映射配置
type RegistryMapping struct {
Upstream string `toml:"upstream"` // 上游Registry地址
AuthHost string `toml:"authHost"` // 认证服务器地址
AuthType string `toml:"authType"` // 认证类型: docker/github/google/basic
Enabled bool `toml:"enabled"` // 是否启用
}
// AppConfig 应用配置结构体
type AppConfig struct {
Server struct {
Host string `toml:"host"` // 监听地址
Port int `toml:"port"` // 监听端口
FileSize int64 `toml:"fileSize"` // 文件大小限制(字节)
} `toml:"server"`
RateLimit struct {
RequestLimit int `toml:"requestLimit"` // 每小时请求限制
PeriodHours float64 `toml:"periodHours"` // 限制周期(小时)
} `toml:"rateLimit"`
Security struct {
WhiteList []string `toml:"whiteList"` // 白名单IP/CIDR列表
BlackList []string `toml:"blackList"` // 黑名单IP/CIDR列表
} `toml:"security"`
Proxy struct {
WhiteList []string `toml:"whiteList"` // 代理白名单(仓库级别)
BlackList []string `toml:"blackList"` // 代理黑名单(仓库级别)
} `toml:"proxy"`
Download struct {
MaxImages int `toml:"maxImages"` // 单次下载最大镜像数量限制
} `toml:"download"`
Registries map[string]RegistryMapping `toml:"registries"`
TokenCache struct {
Enabled bool `toml:"enabled"` // 是否启用token缓存
DefaultTTL string `toml:"defaultTTL"` // 默认缓存时间
} `toml:"tokenCache"`
}
var (
appConfig *AppConfig
appConfigLock sync.RWMutex
isViperEnabled bool
viperInstance *viper.Viper
cachedConfig *AppConfig
configCacheTime time.Time
configCacheTTL = 5 * time.Second
configCacheMutex sync.RWMutex
)
// DefaultConfig 返回默认配置
func DefaultConfig() *AppConfig {
return &AppConfig{
Server: struct {
Host string `toml:"host"`
Port int `toml:"port"`
FileSize int64 `toml:"fileSize"`
}{
Host: "0.0.0.0",
Port: 5000,
FileSize: 2 * 1024 * 1024 * 1024, // 2GB
},
RateLimit: struct {
RequestLimit int `toml:"requestLimit"`
PeriodHours float64 `toml:"periodHours"`
}{
RequestLimit: 20,
PeriodHours: 1.0,
},
Security: struct {
WhiteList []string `toml:"whiteList"`
BlackList []string `toml:"blackList"`
}{
WhiteList: []string{},
BlackList: []string{},
},
Proxy: struct {
WhiteList []string `toml:"whiteList"`
BlackList []string `toml:"blackList"`
}{
WhiteList: []string{},
BlackList: []string{},
},
Download: struct {
MaxImages int `toml:"maxImages"`
}{
MaxImages: 10, // 默认值最多同时下载10个镜像
},
Registries: map[string]RegistryMapping{
"ghcr.io": {
Upstream: "ghcr.io",
AuthHost: "ghcr.io/token",
AuthType: "github",
Enabled: true,
},
"gcr.io": {
Upstream: "gcr.io",
AuthHost: "gcr.io/v2/token",
AuthType: "google",
Enabled: true,
},
"quay.io": {
Upstream: "quay.io",
AuthHost: "quay.io/v2/auth",
AuthType: "quay",
Enabled: true,
},
"registry.k8s.io": {
Upstream: "registry.k8s.io",
AuthHost: "registry.k8s.io",
AuthType: "anonymous",
Enabled: true,
},
},
TokenCache: struct {
Enabled bool `toml:"enabled"`
DefaultTTL string `toml:"defaultTTL"`
}{
Enabled: true, // docker认证的匿名Token缓存配置用于提升性能
DefaultTTL: "20m",
},
}
}
// GetConfig 安全地获取配置副本
func GetConfig() *AppConfig {
configCacheMutex.RLock()
if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL {
config := cachedConfig
configCacheMutex.RUnlock()
return config
}
configCacheMutex.RUnlock()
// 缓存过期,重新生成配置
configCacheMutex.Lock()
defer configCacheMutex.Unlock()
// 双重检查,防止重复生成
if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL {
return cachedConfig
}
appConfigLock.RLock()
if appConfig == nil {
appConfigLock.RUnlock()
defaultCfg := DefaultConfig()
cachedConfig = defaultCfg
configCacheTime = time.Now()
return defaultCfg
}
// 生成新的配置深拷贝
configCopy := *appConfig
configCopy.Security.WhiteList = append([]string(nil), appConfig.Security.WhiteList...)
configCopy.Security.BlackList = append([]string(nil), appConfig.Security.BlackList...)
configCopy.Proxy.WhiteList = append([]string(nil), appConfig.Proxy.WhiteList...)
configCopy.Proxy.BlackList = append([]string(nil), appConfig.Proxy.BlackList...)
appConfigLock.RUnlock()
cachedConfig = &configCopy
configCacheTime = time.Now()
return cachedConfig
}
// setConfig 安全地设置配置
func setConfig(cfg *AppConfig) {
appConfigLock.Lock()
defer appConfigLock.Unlock()
appConfig = cfg
configCacheMutex.Lock()
cachedConfig = nil
configCacheMutex.Unlock()
}
// LoadConfig 加载配置文件
func LoadConfig() error {
// 首先使用默认配置
cfg := DefaultConfig()
// 尝试加载TOML配置文件
if data, err := os.ReadFile("config.toml"); err == nil {
if err := toml.Unmarshal(data, cfg); err != nil {
return fmt.Errorf("解析配置文件失败: %v", err)
}
} else {
fmt.Println("未找到config.toml使用默认配置")
}
// 从环境变量覆盖配置
overrideFromEnv(cfg)
// 设置配置
setConfig(cfg)
if !isViperEnabled {
go enableViperHotReload()
}
return nil
}
func enableViperHotReload() {
if isViperEnabled {
return
}
// 创建Viper实例
viperInstance = viper.New()
// 配置Viper
viperInstance.SetConfigName("config")
viperInstance.SetConfigType("toml")
viperInstance.AddConfigPath(".")
// 读取配置文件
if err := viperInstance.ReadInConfig(); err != nil {
fmt.Printf("读取配置失败,继续使用当前配置: %v\n", err)
return
}
isViperEnabled = true
viperInstance.WatchConfig()
viperInstance.OnConfigChange(func(e fsnotify.Event) {
fmt.Printf("检测到配置文件变化: %s\n", e.Name)
hotReloadWithViper()
})
}
func hotReloadWithViper() {
start := time.Now()
fmt.Println("🔄 自动热重载...")
// 创建新配置
cfg := DefaultConfig()
// 使用Viper解析配置到结构体
if err := viperInstance.Unmarshal(cfg); err != nil {
fmt.Printf("❌ 配置解析失败: %v\n", err)
return
}
overrideFromEnv(cfg)
setConfig(cfg)
// 异步更新受影响的组件
go func() {
updateAffectedComponents()
fmt.Printf("✅ Viper配置热重载完成耗时: %v\n", time.Since(start))
}()
}
func updateAffectedComponents() {
// 重新初始化限流器
if globalLimiter != nil {
fmt.Println("📡 重新初始化限流器...")
initLimiter()
}
// 重新加载访问控制
fmt.Println("🔒 重新加载访问控制规则...")
if GlobalAccessController != nil {
GlobalAccessController.Reload()
}
fmt.Println("🌐 更新Registry配置映射...")
reloadRegistryConfig()
// 其他需要重新初始化的组件可以在这里添加
fmt.Println("🔧 组件更新完成")
}
func reloadRegistryConfig() {
cfg := GetConfig()
enabledCount := 0
// 统计启用的Registry数量
for _, mapping := range cfg.Registries {
if mapping.Enabled {
enabledCount++
}
}
fmt.Printf("🌐 Registry配置已更新: %d个启用\n", enabledCount)
}
// overrideFromEnv 从环境变量覆盖配置
func overrideFromEnv(cfg *AppConfig) {
// 服务器配置
if val := os.Getenv("SERVER_HOST"); val != "" {
cfg.Server.Host = val
}
if val := os.Getenv("SERVER_PORT"); val != "" {
if port, err := strconv.Atoi(val); err == nil && port > 0 {
cfg.Server.Port = port
}
}
if val := os.Getenv("MAX_FILE_SIZE"); val != "" {
if size, err := strconv.ParseInt(val, 10, 64); err == nil && size > 0 {
cfg.Server.FileSize = size
}
}
// 限流配置
if val := os.Getenv("RATE_LIMIT"); val != "" {
if limit, err := strconv.Atoi(val); err == nil && limit > 0 {
cfg.RateLimit.RequestLimit = limit
}
}
if val := os.Getenv("RATE_PERIOD_HOURS"); val != "" {
if period, err := strconv.ParseFloat(val, 64); err == nil && period > 0 {
cfg.RateLimit.PeriodHours = period
}
}
// IP限制配置
if val := os.Getenv("IP_WHITELIST"); val != "" {
cfg.Security.WhiteList = append(cfg.Security.WhiteList, strings.Split(val, ",")...)
}
if val := os.Getenv("IP_BLACKLIST"); val != "" {
cfg.Security.BlackList = append(cfg.Security.BlackList, strings.Split(val, ",")...)
}
// 下载限制配置
if val := os.Getenv("MAX_IMAGES"); val != "" {
if maxImages, err := strconv.Atoi(val); err == nil && maxImages > 0 {
cfg.Download.MaxImages = maxImages
}
}
}
// CreateDefaultConfigFile 创建默认配置文件
func CreateDefaultConfigFile() error {
cfg := DefaultConfig()
data, err := toml.Marshal(cfg)
if err != nil {
return fmt.Errorf("序列化默认配置失败: %v", err)
}
return os.WriteFile("config.toml", data, 0644)
}
package main
import (
"fmt"
"os"
"strconv"
"strings"
"sync"
"time"
"github.com/pelletier/go-toml/v2"
)
// RegistryMapping Registry映射配置
type RegistryMapping struct {
Upstream string `toml:"upstream"` // 上游Registry地址
AuthHost string `toml:"authHost"` // 认证服务器地址
AuthType string `toml:"authType"` // 认证类型: docker/github/google/basic
Enabled bool `toml:"enabled"` // 是否启用
}
// AppConfig 应用配置结构体
type AppConfig struct {
Server struct {
Host string `toml:"host"` // 监听地址
Port int `toml:"port"` // 监听端口
FileSize int64 `toml:"fileSize"` // 文件大小限制(字节)
} `toml:"server"`
RateLimit struct {
RequestLimit int `toml:"requestLimit"` // 每小时请求限制
PeriodHours float64 `toml:"periodHours"` // 限制周期(小时)
} `toml:"rateLimit"`
Security struct {
WhiteList []string `toml:"whiteList"` // 白名单IP/CIDR列表
BlackList []string `toml:"blackList"` // 黑名单IP/CIDR列表
} `toml:"security"`
Access struct {
WhiteList []string `toml:"whiteList"` // 代理白名单(仓库级别)
BlackList []string `toml:"blackList"` // 代理黑名单(仓库级别)
Proxy string `toml:"proxy"` // 代理地址: 支持 http/https/socks5/socks5h
} `toml:"access"`
Download struct {
MaxImages int `toml:"maxImages"` // 单次下载最大镜像数量限制
} `toml:"download"`
Registries map[string]RegistryMapping `toml:"registries"`
TokenCache struct {
Enabled bool `toml:"enabled"` // 是否启用token缓存
DefaultTTL string `toml:"defaultTTL"` // 默认缓存时间
} `toml:"tokenCache"`
}
var (
appConfig *AppConfig
appConfigLock sync.RWMutex
cachedConfig *AppConfig
configCacheTime time.Time
configCacheTTL = 5 * time.Second
configCacheMutex sync.RWMutex
)
// todo:Refactoring is needed
// DefaultConfig 返回默认配置
func DefaultConfig() *AppConfig {
return &AppConfig{
Server: struct {
Host string `toml:"host"`
Port int `toml:"port"`
FileSize int64 `toml:"fileSize"`
}{
Host: "0.0.0.0",
Port: 5000,
FileSize: 2 * 1024 * 1024 * 1024, // 2GB
},
RateLimit: struct {
RequestLimit int `toml:"requestLimit"`
PeriodHours float64 `toml:"periodHours"`
}{
RequestLimit: 20,
PeriodHours: 1.0,
},
Security: struct {
WhiteList []string `toml:"whiteList"`
BlackList []string `toml:"blackList"`
}{
WhiteList: []string{},
BlackList: []string{},
},
Access: struct {
WhiteList []string `toml:"whiteList"`
BlackList []string `toml:"blackList"`
Proxy string `toml:"proxy"`
}{
WhiteList: []string{},
BlackList: []string{},
Proxy: "", // 默认不使用代理
},
Download: struct {
MaxImages int `toml:"maxImages"`
}{
MaxImages: 10, // 默认值最多同时下载10个镜像
},
Registries: map[string]RegistryMapping{
"ghcr.io": {
Upstream: "ghcr.io",
AuthHost: "ghcr.io/token",
AuthType: "github",
Enabled: true,
},
"gcr.io": {
Upstream: "gcr.io",
AuthHost: "gcr.io/v2/token",
AuthType: "google",
Enabled: true,
},
"quay.io": {
Upstream: "quay.io",
AuthHost: "quay.io/v2/auth",
AuthType: "quay",
Enabled: true,
},
"registry.k8s.io": {
Upstream: "registry.k8s.io",
AuthHost: "registry.k8s.io",
AuthType: "anonymous",
Enabled: true,
},
},
TokenCache: struct {
Enabled bool `toml:"enabled"`
DefaultTTL string `toml:"defaultTTL"`
}{
Enabled: true, // docker认证的匿名Token缓存配置用于提升性能
DefaultTTL: "20m",
},
}
}
// GetConfig 安全地获取配置副本
func GetConfig() *AppConfig {
configCacheMutex.RLock()
if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL {
config := cachedConfig
configCacheMutex.RUnlock()
return config
}
configCacheMutex.RUnlock()
// 缓存过期,重新生成配置
configCacheMutex.Lock()
defer configCacheMutex.Unlock()
// 双重检查,防止重复生成
if cachedConfig != nil && time.Since(configCacheTime) < configCacheTTL {
return cachedConfig
}
appConfigLock.RLock()
if appConfig == nil {
appConfigLock.RUnlock()
defaultCfg := DefaultConfig()
cachedConfig = defaultCfg
configCacheTime = time.Now()
return defaultCfg
}
// 生成新的配置深拷贝
configCopy := *appConfig
configCopy.Security.WhiteList = append([]string(nil), appConfig.Security.WhiteList...)
configCopy.Security.BlackList = append([]string(nil), appConfig.Security.BlackList...)
configCopy.Access.WhiteList = append([]string(nil), appConfig.Access.WhiteList...)
configCopy.Access.BlackList = append([]string(nil), appConfig.Access.BlackList...)
appConfigLock.RUnlock()
cachedConfig = &configCopy
configCacheTime = time.Now()
return cachedConfig
}
// setConfig 安全地设置配置
func setConfig(cfg *AppConfig) {
appConfigLock.Lock()
defer appConfigLock.Unlock()
appConfig = cfg
configCacheMutex.Lock()
cachedConfig = nil
configCacheMutex.Unlock()
}
// LoadConfig 加载配置文件
func LoadConfig() error {
// 首先使用默认配置
cfg := DefaultConfig()
// 尝试加载TOML配置文件
if data, err := os.ReadFile("config.toml"); err == nil {
if err := toml.Unmarshal(data, cfg); err != nil {
return fmt.Errorf("解析配置文件失败: %v", err)
}
} else {
fmt.Println("未找到config.toml使用默认配置")
}
// 从环境变量覆盖配置
overrideFromEnv(cfg)
// 设置配置
setConfig(cfg)
return nil
}
// overrideFromEnv 从环境变量覆盖配置
func overrideFromEnv(cfg *AppConfig) {
// 服务器配置
if val := os.Getenv("SERVER_HOST"); val != "" {
cfg.Server.Host = val
}
if val := os.Getenv("SERVER_PORT"); val != "" {
if port, err := strconv.Atoi(val); err == nil && port > 0 {
cfg.Server.Port = port
}
}
if val := os.Getenv("MAX_FILE_SIZE"); val != "" {
if size, err := strconv.ParseInt(val, 10, 64); err == nil && size > 0 {
cfg.Server.FileSize = size
}
}
// 限流配置
if val := os.Getenv("RATE_LIMIT"); val != "" {
if limit, err := strconv.Atoi(val); err == nil && limit > 0 {
cfg.RateLimit.RequestLimit = limit
}
}
if val := os.Getenv("RATE_PERIOD_HOURS"); val != "" {
if period, err := strconv.ParseFloat(val, 64); err == nil && period > 0 {
cfg.RateLimit.PeriodHours = period
}
}
// IP限制配置
if val := os.Getenv("IP_WHITELIST"); val != "" {
cfg.Security.WhiteList = append(cfg.Security.WhiteList, strings.Split(val, ",")...)
}
if val := os.Getenv("IP_BLACKLIST"); val != "" {
cfg.Security.BlackList = append(cfg.Security.BlackList, strings.Split(val, ",")...)
}
// 下载限制配置
if val := os.Getenv("MAX_IMAGES"); val != "" {
if maxImages, err := strconv.Atoi(val); err == nil && maxImages > 0 {
cfg.Download.MaxImages = maxImages
}
}
}
// CreateDefaultConfigFile 创建默认配置文件
func CreateDefaultConfigFile() error {
cfg := DefaultConfig()
data, err := toml.Marshal(cfg)
if err != nil {
return fmt.Errorf("序列化默认配置失败: %v", err)
}
return os.WriteFile("config.toml", data, 0644)
}

View File

@@ -1,89 +1,94 @@
[server]
# 监听地址,默认监听所有接口
host = "0.0.0.0"
# 监听端口
port = 5000
# 文件大小限制字节默认2GB
fileSize = 2147483648
[rateLimit]
# 每个IP每小时允许的请求数(Docker镜像每个层为一个请求)
requestLimit = 200
# 限流周期(小时)
periodHours = 1.0
[security]
# IP白名单支持单个IP或CIDR格式
# 白名单中的IP不受限流限制
whiteList = [
"127.0.0.1",
"192.168.1.0/24"
]
# IP黑名单支持单个IP或CIDR格式
# 黑名单中的IP将被直接拒绝访问
blackList = [
"192.168.100.1"
]
[proxy]
# 代理服务白名单支持GitHub仓库和Docker镜像支持通配符
# 只允许访问白名单中的仓库/镜像,为空时不限制
whiteList = []
# 代理服务黑名单支持GitHub仓库和Docker镜像支持通配符
# 禁止访问黑名单中的仓库/镜像
blackList = [
"baduser/malicious-repo",
"*/malicious-repo",
"baduser/*"
]
[download]
# 单次并发下载离线镜像数量限制
maxImages = 10
# Registry映射配置支持多种Container Registry
[registries]
# GitHub Container Registry
[registries."ghcr.io"]
upstream = "ghcr.io"
authHost = "ghcr.io/token"
authType = "github"
enabled = true
# Google Container Registry
[registries."gcr.io"]
upstream = "gcr.io"
authHost = "gcr.io/v2/token"
authType = "google"
enabled = true
# Quay.io Container Registry
[registries."quay.io"]
upstream = "quay.io"
authHost = "quay.io/v2/auth"
authType = "quay"
enabled = true
# Kubernetes Container Registry
[registries."registry.k8s.io"]
upstream = "registry.k8s.io"
authHost = "registry.k8s.io"
authType = "anonymous"
enabled = true
# 私有Registry示例默认禁用
# [registries."harbor.company.com"]
# upstream = "harbor.company.com"
# authHost = "harbor.company.com/service/token"
# authType = "basic"
# enabled = false
# 缓存配置Docker临时Token和Manifest统一管理显著提升性能
[tokenCache]
# 是否启用缓存(同时控制Token和Manifest缓存)
enabled = true
# 默认缓存时间
defaultTTL = "20m"
[server]
host = "0.0.0.0"
# 监听端口
port = 5000
# Github文件大小限制字节默认2GB
fileSize = 2147483648
[rateLimit]
# 每个IP每小时允许的请求数(注意Docker镜像会有多个层会消耗多个次数)
requestLimit = 500
# 限流周期(小时)
periodHours = 1.0
[security]
# IP白名单支持单个IP或IP段
# 白名单中的IP不受限流限制
whiteList = [
"127.0.0.1",
"172.17.0.0/16",
"192.168.1.0/24"
]
# IP黑名单支持单个IP或IP段
# 黑名单中的IP将被直接拒绝访问
blackList = [
"192.168.100.1",
"192.168.100.0/24"
]
[access]
# 代理服务白名单支持GitHub仓库和Docker镜像支持通配符
# 只允许访问白名单中的仓库/镜像,为空时不限制
whiteList = []
# 代理服务黑名单支持GitHub仓库和Docker镜像支持通配符
# 禁止访问黑名单中的仓库/镜像
blackList = [
"baduser/malicious-repo",
"*/malicious-repo",
"baduser/*"
]
# 代理配置,支持有用户名/密码认证和无认证模式
# 无认证: socks5://127.0.0.1:1080
# 有认证: socks5://username:password@127.0.0.1:1080
# HTTP 代理示例
# http://username:password@127.0.0.1:7890
# SOCKS5 代理示例
# socks5://username:password@127.0.0.1:1080
# SOCKS5H 代理示例
# socks5h://username:password@127.0.0.1:1080
# 留空不使用代理
proxy = ""
[download]
# 批量下载离线镜像数量限制
maxImages = 10
# Registry映射配置支持多种镜像仓库上游
[registries]
# GitHub Container Registry
[registries."ghcr.io"]
upstream = "ghcr.io"
authHost = "ghcr.io/token"
authType = "github"
enabled = true
# Google Container Registry
[registries."gcr.io"]
upstream = "gcr.io"
authHost = "gcr.io/v2/token"
authType = "google"
enabled = true
# Quay.io Container Registry
[registries."quay.io"]
upstream = "quay.io"
authHost = "quay.io/v2/auth"
authType = "quay"
enabled = true
# Kubernetes Container Registry
[registries."registry.k8s.io"]
upstream = "registry.k8s.io"
authHost = "registry.k8s.io"
authType = "anonymous"
enabled = true
[tokenCache]
# 是否启用缓存(同时控制Token和Manifest缓存)显著提升性能
enabled = true
# 默认缓存时间(分钟)
defaultTTL = "20m"

File diff suppressed because it is too large Load Diff

View File

@@ -3,11 +3,9 @@ module hubproxy
go 1.24.0
require (
github.com/fsnotify/fsnotify v1.8.0
github.com/gin-gonic/gin v1.10.0
github.com/google/go-containerregistry v0.20.5
github.com/pelletier/go-toml/v2 v2.2.3
github.com/spf13/viper v1.20.1
golang.org/x/time v0.11.0
)
@@ -25,11 +23,11 @@ require (
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.20.0 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
@@ -38,18 +36,11 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/sagikazarmark/locafero v0.7.0 // indirect
github.com/rogpeppe/go-internal v1.9.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.12.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect
github.com/vbatts/tar-split v0.12.1 // indirect
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
golang.org/x/arch v0.8.0 // indirect
golang.org/x/crypto v0.32.0 // indirect
golang.org/x/net v0.33.0 // indirect
@@ -57,5 +48,6 @@ require (
golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.21.0 // indirect
google.golang.org/protobuf v1.36.3 // indirect
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

View File

@@ -8,6 +8,7 @@ github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8=
github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -17,10 +18,6 @@ github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBi
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8=
github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M=
github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
@@ -35,8 +32,6 @@ github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJn
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
github.com/go-viper/mapstructure/v2 v2.2.1 h1:ZAaOCxANMuZx5RCeg0mBdEZk7DZasvvZIxtHqx8aGss=
github.com/go-viper/mapstructure/v2 v2.2.1/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -52,8 +47,11 @@ github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa02
github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
@@ -77,22 +75,11 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/sagikazarmark/locafero v0.7.0 h1:5MqpDsTGNDhY8sGp0Aowyf0qKsPrhewaLSsFaodPcyo=
github.com/sagikazarmark/locafero v0.7.0/go.mod h1:2za3Cg5rMaTMoG/2Ulr9AwtFaIppKXTRYnozin4aB5k=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
github.com/spf13/afero v1.12.0 h1:UcOPyRBYczmFn6yvphxkn9ZEOY65cpwGKb5mL36mrqs=
github.com/spf13/afero v1.12.0/go.mod h1:ZTlWwG4/ahT8W7T0WQ5uYmjI9duaLQGy3Q2OAl4sk/4=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.20.1 h1:ZMi+z/lvLyPSCoNtFCpqjy0S4kPbirhpTMwl8BkW9X4=
github.com/spf13/viper v1.20.1/go.mod h1:P9Mdzt1zoHIG8m2eZQinpiBjo6kCmZSKBClNNqjJvu4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
@@ -101,20 +88,14 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
github.com/vbatts/tar-split v0.12.1 h1:CqKoORW7BUWBe7UL/iqTVvkTBOF8UvOMKOIZykxnnbo=
github.com/vbatts/tar-split v0.12.1/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
@@ -136,8 +117,10 @@ golang.org/x/time v0.11.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=
google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -1,59 +1,68 @@
package main
import (
"net"
"net/http"
"time"
)
var (
// 全局HTTP客户端 - 用于代理请求(长超时)
globalHTTPClient *http.Client
// 搜索HTTP客户端 - 用于API请求短超时
searchHTTPClient *http.Client
)
// initHTTPClients 初始化HTTP客户端
func initHTTPClients() {
// 代理客户端配置 - 适用于大文件传输
globalHTTPClient = &http.Client{
Transport: &http.Transport{
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
MaxIdleConns: 1000,
MaxIdleConnsPerHost: 1000,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
ResponseHeaderTimeout: 300 * time.Second,
},
}
// 搜索客户端配置 - 适用于API调用
searchHTTPClient = &http.Client{
Timeout: 10 * time.Second,
Transport: &http.Transport{
DialContext: (&net.Dialer{
Timeout: 5 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
MaxIdleConns: 100,
MaxIdleConnsPerHost: 10,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 5 * time.Second,
DisableCompression: false,
},
}
}
// GetGlobalHTTPClient 获取全局HTTP客户端用于代理
func GetGlobalHTTPClient() *http.Client {
return globalHTTPClient
}
// GetSearchHTTPClient 获取搜索HTTP客户端用于API调用
func GetSearchHTTPClient() *http.Client {
return searchHTTPClient
}
package main
import (
"net"
"net/http"
"os"
"time"
)
var (
// 全局HTTP客户端 - 用于代理请求(长超时)
globalHTTPClient *http.Client
// 搜索HTTP客户端 - 用于API请求短超时
searchHTTPClient *http.Client
)
// initHTTPClients 初始化HTTP客户端
func initHTTPClients() {
cfg := GetConfig()
if p := cfg.Access.Proxy; p != "" {
os.Setenv("HTTP_PROXY", p)
os.Setenv("HTTPS_PROXY", p)
}
// 代理客户端配置 - 适用于大文件传输
globalHTTPClient = &http.Client{
Transport: &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
MaxIdleConns: 1000,
MaxIdleConnsPerHost: 1000,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
ResponseHeaderTimeout: 300 * time.Second,
},
}
// 搜索客户端配置 - 适用于API调用
searchHTTPClient = &http.Client{
Timeout: 10 * time.Second,
Transport: &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 5 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
MaxIdleConns: 100,
MaxIdleConnsPerHost: 10,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 5 * time.Second,
DisableCompression: false,
},
}
}
// GetGlobalHTTPClient 获取全局HTTP客户端用于代理
func GetGlobalHTTPClient() *http.Client {
return globalHTTPClient
}
// GetSearchHTTPClient 获取搜索HTTP客户端用于API调用
func GetSearchHTTPClient() *http.Client {
return searchHTTPClient
}

View File

@@ -33,16 +33,18 @@ type DebounceEntry struct {
// DownloadDebouncer 下载防抖器
type DownloadDebouncer struct {
mu sync.RWMutex
entries map[string]*DebounceEntry
window time.Duration
mu sync.RWMutex
entries map[string]*DebounceEntry
window time.Duration
lastCleanup time.Time
}
// NewDownloadDebouncer 创建下载防抖器
func NewDownloadDebouncer(window time.Duration) *DownloadDebouncer {
return &DownloadDebouncer{
entries: make(map[string]*DebounceEntry),
window: window,
entries: make(map[string]*DebounceEntry),
window: window,
lastCleanup: time.Now(),
}
}
@@ -50,27 +52,28 @@ func NewDownloadDebouncer(window time.Duration) *DownloadDebouncer {
func (d *DownloadDebouncer) ShouldAllow(userID, contentKey string) bool {
d.mu.Lock()
defer d.mu.Unlock()
key := userID + ":" + contentKey
now := time.Now()
if entry, exists := d.entries[key]; exists {
if now.Sub(entry.LastRequest) < d.window {
return false // 在防抖窗口内,拒绝请求
}
}
// 更新或创建条目
d.entries[key] = &DebounceEntry{
LastRequest: now,
UserID: userID,
}
// 清理过期条目(简单策略每100次请求清理一次)
if len(d.entries)%100 == 0 {
// 清理过期条目(每5分钟清理一次)
if time.Since(d.lastCleanup) > 5*time.Minute {
d.cleanup(now)
d.lastCleanup = now
}
return true
}
@@ -89,10 +92,10 @@ func generateContentFingerprint(images []string, platform string) string {
sortedImages := make([]string, len(images))
copy(sortedImages, images)
sort.Strings(sortedImages)
// 组合内容:镜像列表 + 平台信息
content := strings.Join(sortedImages, "|") + ":" + platform
// 生成MD5哈希
hash := md5.Sum([]byte(content))
return hex.EncodeToString(hash[:])
@@ -104,14 +107,14 @@ func getUserID(c *gin.Context) string {
if sessionID, err := c.Cookie("session_id"); err == nil && sessionID != "" {
return "session:" + sessionID
}
// 备用方案IP + User-Agent组合
ip := c.ClientIP()
userAgent := c.GetHeader("User-Agent")
if userAgent == "" {
userAgent = "unknown"
}
// 生成简短标识
combined := ip + ":" + userAgent
hash := md5.Sum([]byte(combined))
@@ -128,8 +131,8 @@ var (
func initDebouncer() {
// 单个镜像5秒防抖窗口
singleImageDebouncer = NewDownloadDebouncer(5 * time.Second)
// 批量镜像:30秒防抖窗口(影响更大,需要更长保护)
batchImageDebouncer = NewDownloadDebouncer(30 * time.Second)
// 批量镜像:60秒防抖窗口
batchImageDebouncer = NewDownloadDebouncer(60 * time.Second)
}
// ImageStreamer 镜像流式下载器
@@ -171,14 +174,15 @@ func NewImageStreamer(config *ImageStreamerConfig) *ImageStreamer {
// StreamOptions 下载选项
type StreamOptions struct {
Platform string
Compression bool
Platform string
Compression bool
UseCompressedLayers bool // 是否保存原始压缩层,默认开启
}
// StreamImageToWriter 流式下载镜像到Writer
func (is *ImageStreamer) StreamImageToWriter(ctx context.Context, imageRef string, writer io.Writer, options *StreamOptions) error {
if options == nil {
options = &StreamOptions{}
options = &StreamOptions{UseCompressedLayers: true}
}
ref, err := name.ParseReference(imageRef)
@@ -211,63 +215,20 @@ func (is *ImageStreamer) getImageDescriptor(ref name.Reference, options []remote
// getImageDescriptorWithPlatform 获取指定平台的镜像描述符
func (is *ImageStreamer) getImageDescriptorWithPlatform(ref name.Reference, options []remote.Option, platform string) (*remote.Descriptor, error) {
if isCacheEnabled() {
var reference string
if tagged, ok := ref.(name.Tag); ok {
reference = tagged.TagStr()
} else if digested, ok := ref.(name.Digest); ok {
reference = digested.DigestStr()
}
if reference != "" {
cacheKey := buildManifestCacheKeyWithPlatform(ref.Context().String(), reference, platform)
if cachedItem := globalCache.Get(cacheKey); cachedItem != nil {
desc := &remote.Descriptor{
Manifest: cachedItem.Data,
}
log.Printf("使用缓存的manifest: %s (平台: %s)", ref.String(), platform)
return desc, nil
}
}
}
desc, err := remote.Get(ref, options...)
if err != nil {
return nil, err
}
if isCacheEnabled() {
var reference string
if tagged, ok := ref.(name.Tag); ok {
reference = tagged.TagStr()
} else if digested, ok := ref.(name.Digest); ok {
reference = digested.DigestStr()
}
if reference != "" {
cacheKey := buildManifestCacheKeyWithPlatform(ref.Context().String(), reference, platform)
ttl := getManifestTTL(reference)
headers := map[string]string{
"Docker-Content-Digest": desc.Digest.String(),
}
globalCache.Set(cacheKey, desc.Manifest, string(desc.MediaType), headers, ttl)
log.Printf("缓存manifest: %s (平台: %s, TTL: %v)", ref.String(), platform, ttl)
}
}
return desc, nil
// 直接从网络获取完整的descriptor确保对象完整性
return remote.Get(ref, options...)
}
// StreamImageToGin 流式响应到Gin
func (is *ImageStreamer) StreamImageToGin(ctx context.Context, imageRef string, c *gin.Context, options *StreamOptions) error {
if options == nil {
options = &StreamOptions{}
options = &StreamOptions{UseCompressedLayers: true}
}
filename := strings.ReplaceAll(imageRef, "/", "_") + ".tar"
c.Header("Content-Type", "application/octet-stream")
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", filename))
if options.Compression {
c.Header("Content-Encoding", "gzip")
}
@@ -320,32 +281,32 @@ func (is *ImageStreamer) streamImageLayers(ctx context.Context, img v1.Image, wr
log.Printf("镜像包含 %d 层", len(layers))
return is.streamDockerFormat(ctx, tarWriter, img, layers, configFile, imageRef)
return is.streamDockerFormat(ctx, tarWriter, img, layers, configFile, imageRef, options)
}
// streamDockerFormat 生成Docker格式
func (is *ImageStreamer) streamDockerFormat(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string) error {
return is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, nil, nil)
func (is *ImageStreamer) streamDockerFormat(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, options *StreamOptions) error {
return is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, nil, nil, options)
}
// streamDockerFormatWithReturn 生成Docker格式并返回manifest和repositories信息
func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, manifestOut *map[string]interface{}, repositoriesOut *map[string]map[string]string) error {
func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWriter *tar.Writer, img v1.Image, layers []v1.Layer, configFile *v1.ConfigFile, imageRef string, manifestOut *map[string]interface{}, repositoriesOut *map[string]map[string]string, options *StreamOptions) error {
configDigest, err := img.ConfigName()
if err != nil {
return err
}
configData, err := json.Marshal(configFile)
if err != nil {
return err
}
configHeader := &tar.Header{
Name: configDigest.String() + ".json",
Size: int64(len(configData)),
Mode: 0644,
}
if err := tarWriter.WriteHeader(configHeader); err != nil {
return err
}
@@ -374,17 +335,29 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
Typeflag: tar.TypeDir,
Mode: 0755,
}
if err := tarWriter.WriteHeader(layerHeader); err != nil {
return err
}
uncompressedSize, err := partial.UncompressedSize(layer)
if err != nil {
return err
var layerSize int64
var layerReader io.ReadCloser
// 根据配置选择使用压缩层或未压缩层
if options != nil && options.UseCompressedLayers {
layerSize, err = layer.Size()
if err != nil {
return err
}
layerReader, err = layer.Compressed()
} else {
layerSize, err = partial.UncompressedSize(layer)
if err != nil {
return err
}
layerReader, err = layer.Uncompressed()
}
layerReader, err := layer.Uncompressed()
if err != nil {
return err
}
@@ -392,10 +365,10 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
layerTarHeader := &tar.Header{
Name: layerDir + "/layer.tar",
Size: uncompressedSize,
Size: layerSize,
Mode: 0644,
}
if err := tarWriter.WriteHeader(layerTarHeader); err != nil {
return err
}
@@ -412,12 +385,11 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
log.Printf("已处理层 %d/%d", i+1, len(layers))
}
// 构建单个镜像的manifest信息
singleManifest := map[string]interface{}{
"Config": configDigest.String() + ".json",
"RepoTags": []string{imageRef},
"Layers": func() []string {
"Layers": func() []string {
var layers []string
for _, digest := range layerDigests {
layers = append(layers, digest+"/layer.tar")
@@ -444,22 +416,22 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
// 单镜像下载直接写入manifest.json
manifest := []map[string]interface{}{singleManifest}
manifestData, err := json.Marshal(manifest)
if err != nil {
return err
}
manifestHeader := &tar.Header{
Name: "manifest.json",
Size: int64(len(manifestData)),
Mode: 0644,
}
if err := tarWriter.WriteHeader(manifestHeader); err != nil {
return err
}
if _, err := tarWriter.Write(manifestData); err != nil {
return err
}
@@ -469,22 +441,46 @@ func (is *ImageStreamer) streamDockerFormatWithReturn(ctx context.Context, tarWr
if err != nil {
return err
}
repositoriesHeader := &tar.Header{
Name: "repositories",
Size: int64(len(repositoriesData)),
Mode: 0644,
}
if err := tarWriter.WriteHeader(repositoriesHeader); err != nil {
return err
}
_, err = tarWriter.Write(repositoriesData)
return err
}
// streamSingleImageForBatch 为批量下载流式处理单个镜像
// processImageForBatch 处理镜像的公共逻辑,用于批量下载
func (is *ImageStreamer) processImageForBatch(ctx context.Context, img v1.Image, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) {
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
var manifest map[string]interface{}
var repositories map[string]map[string]string
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories, options)
if err != nil {
return nil, nil, err
}
return manifest, repositories, nil
}
func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWriter *tar.Writer, imageRef string, options *StreamOptions) (map[string]interface{}, map[string]map[string]string, error) {
ref, err := name.ParseReference(imageRef)
if err != nil {
@@ -498,80 +494,28 @@ func (is *ImageStreamer) streamSingleImageForBatch(ctx context.Context, tarWrite
return nil, nil, fmt.Errorf("获取镜像描述失败: %w", err)
}
var manifest map[string]interface{}
var repositories map[string]map[string]string
var img v1.Image
switch desc.MediaType {
case types.OCIImageIndex, types.DockerManifestList:
// 处理多架构镜像,复用单个下载的逻辑
img, err := is.selectPlatformImage(desc, options)
// 处理多架构镜像
img, err = is.selectPlatformImage(desc, options)
if err != nil {
return nil, nil, fmt.Errorf("选择平台镜像失败: %w", err)
}
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
if err != nil {
return nil, nil, err
}
case types.OCIManifestSchema1, types.DockerManifestSchema2:
img, err := desc.Image()
img, err = desc.Image()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
}
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
if err != nil {
return nil, nil, err
}
default:
img, err := desc.Image()
img, err = desc.Image()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像失败: %w", err)
}
layers, err := img.Layers()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像层失败: %w", err)
}
configFile, err := img.ConfigFile()
if err != nil {
return nil, nil, fmt.Errorf("获取镜像配置失败: %w", err)
}
log.Printf("镜像包含 %d 层", len(layers))
err = is.streamDockerFormatWithReturn(ctx, tarWriter, img, layers, configFile, imageRef, &manifest, &repositories)
if err != nil {
return nil, nil, err
}
}
return manifest, repositories, nil
return is.processImageForBatch(ctx, img, tarWriter, imageRef, options)
}
// selectPlatformImage 从多架构镜像中选择合适的平台镜像
@@ -592,7 +536,7 @@ func (is *ImageStreamer) selectPlatformImage(desc *remote.Descriptor, options *S
if m.Platform == nil {
continue
}
if options.Platform != "" {
platformParts := strings.Split(options.Platform, "/")
if len(platformParts) >= 2 {
@@ -602,10 +546,10 @@ func (is *ImageStreamer) selectPlatformImage(desc *remote.Descriptor, options *S
if len(platformParts) >= 3 {
targetVariant = platformParts[2]
}
if m.Platform.OS == targetOS &&
m.Platform.Architecture == targetArch &&
m.Platform.Variant == targetVariant {
if m.Platform.OS == targetOS &&
m.Platform.Architecture == targetArch &&
m.Platform.Variant == targetVariant {
selectedDesc = &m
break
}
@@ -637,7 +581,6 @@ var globalImageStreamer *ImageStreamer
// initImageStreamer 初始化镜像下载器
func initImageStreamer() {
globalImageStreamer = NewImageStreamer(nil)
// 镜像下载器初始化完成
}
// formatPlatformText 格式化平台文本
@@ -652,9 +595,9 @@ func formatPlatformText(platform string) string {
func initImageTarRoutes(router *gin.Engine) {
imageAPI := router.Group("/api/image")
{
imageAPI.GET("/download/:image", RateLimitMiddleware(globalLimiter), handleDirectImageDownload)
imageAPI.GET("/info/:image", RateLimitMiddleware(globalLimiter), handleImageInfo)
imageAPI.POST("/batch", RateLimitMiddleware(globalLimiter), handleSimpleBatchDownload)
imageAPI.GET("/download/:image", handleDirectImageDownload)
imageAPI.GET("/info/:image", handleImageInfo)
imageAPI.POST("/batch", handleSimpleBatchDownload)
}
}
@@ -669,6 +612,7 @@ func handleDirectImageDownload(c *gin.Context) {
imageRef := strings.ReplaceAll(imageParam, "_", "/")
platform := c.Query("platform")
tag := c.DefaultQuery("tag", "")
useCompressed := c.DefaultQuery("compressed", "true") == "true"
if tag != "" && !strings.Contains(imageRef, ":") && !strings.Contains(imageRef, "@") {
imageRef = imageRef + ":" + tag
@@ -684,18 +628,19 @@ func handleDirectImageDownload(c *gin.Context) {
// 防抖检查
userID := getUserID(c)
contentKey := generateContentFingerprint([]string{imageRef}, platform)
if !singleImageDebouncer.ShouldAllow(userID, contentKey) {
c.JSON(http.StatusTooManyRequests, gin.H{
"error": "请求过于频繁,请稍后再试",
"error": "请求过于频繁,请稍后再试",
"retry_after": 5,
})
return
}
options := &StreamOptions{
Platform: platform,
Compression: false,
Platform: platform,
Compression: false,
UseCompressedLayers: useCompressed,
}
ctx := c.Request.Context()
@@ -711,8 +656,9 @@ func handleDirectImageDownload(c *gin.Context) {
// handleSimpleBatchDownload 处理批量下载
func handleSimpleBatchDownload(c *gin.Context) {
var req struct {
Images []string `json:"images" binding:"required"`
Platform string `json:"platform"`
Images []string `json:"images" binding:"required"`
Platform string `json:"platform"`
UseCompressedLayers *bool `json:"useCompressedLayers"`
}
if err := c.ShouldBindJSON(&req); err != nil {
@@ -742,25 +688,31 @@ func handleSimpleBatchDownload(c *gin.Context) {
// 批量下载防抖检查
userID := getUserID(c)
contentKey := generateContentFingerprint(req.Images, req.Platform)
if !batchImageDebouncer.ShouldAllow(userID, contentKey) {
c.JSON(http.StatusTooManyRequests, gin.H{
"error": "批量下载请求过于频繁,请稍后再试",
"retry_after": 30,
"error": "批量下载请求过于频繁,请稍后再试",
"retry_after": 60,
})
return
}
useCompressed := true // 默认启用原始压缩层
if req.UseCompressedLayers != nil {
useCompressed = *req.UseCompressedLayers
}
options := &StreamOptions{
Platform: req.Platform,
Compression: false,
Platform: req.Platform,
Compression: false,
UseCompressedLayers: useCompressed,
}
ctx := c.Request.Context()
log.Printf("批量下载 %d 个镜像 (平台: %s)", len(req.Images), formatPlatformText(req.Platform))
filename := fmt.Sprintf("batch_%d_images.tar", len(req.Images))
c.Header("Content-Type", "application/octet-stream")
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", filename))
@@ -833,7 +785,7 @@ func handleImageInfo(c *gin.Context) {
// StreamMultipleImages 批量下载多个镜像
func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []string, writer io.Writer, options *StreamOptions) error {
if options == nil {
options = &StreamOptions{}
options = &StreamOptions{UseCompressedLayers: true}
}
var finalWriter io.Writer = writer
@@ -858,12 +810,12 @@ func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []s
}
log.Printf("处理镜像 %d/%d: %s", i+1, len(imageRefs), imageRef)
// 防止单个镜像处理时间过长
timeoutCtx, cancel := context.WithTimeout(ctx, 15*time.Minute)
manifest, repositories, err := is.streamSingleImageForBatch(timeoutCtx, tarWriter, imageRef, options)
cancel()
if err != nil {
log.Printf("下载镜像 %s 失败: %v", imageRef, err)
return fmt.Errorf("下载镜像 %s 失败: %w", imageRef, err)
@@ -892,17 +844,17 @@ func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []s
if err != nil {
return fmt.Errorf("序列化manifest失败: %w", err)
}
manifestHeader := &tar.Header{
Name: "manifest.json",
Size: int64(len(manifestData)),
Mode: 0644,
}
if err := tarWriter.WriteHeader(manifestHeader); err != nil {
return fmt.Errorf("写入manifest header失败: %w", err)
}
if _, err := tarWriter.Write(manifestData); err != nil {
return fmt.Errorf("写入manifest数据失败: %w", err)
}
@@ -912,21 +864,21 @@ func (is *ImageStreamer) StreamMultipleImages(ctx context.Context, imageRefs []s
if err != nil {
return fmt.Errorf("序列化repositories失败: %w", err)
}
repositoriesHeader := &tar.Header{
Name: "repositories",
Size: int64(len(repositoriesData)),
Mode: 0644,
}
if err := tarWriter.WriteHeader(repositoriesHeader); err != nil {
return fmt.Errorf("写入repositories header失败: %w", err)
}
if _, err := tarWriter.Write(repositoriesData); err != nil {
return fmt.Errorf("写入repositories数据失败: %w", err)
}
log.Printf("批量下载完成,共处理 %d 个镜像", len(imageRefs))
return nil
}
}

View File

@@ -1,381 +1,379 @@
package main
import (
"embed"
"fmt"
"io"
"log"
"net/http"
"regexp"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
)
//go:embed public/*
var staticFiles embed.FS
// 服务嵌入的静态文件
func serveEmbedFile(c *gin.Context, filename string) {
data, err := staticFiles.ReadFile(filename)
if err != nil {
c.Status(404)
return
}
contentType := "text/html; charset=utf-8"
if strings.HasSuffix(filename, ".ico") {
contentType = "image/x-icon"
}
c.Data(200, contentType, data)
}
var (
exps = []*regexp.Regexp{
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:releases|archive)/.*$`),
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:blob|raw)/.*$`),
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:info|git-).*$`),
regexp.MustCompile(`^(?:https?://)?raw\.github(?:usercontent|)\.com/([^/]+)/([^/]+)/.+?/.+$`),
regexp.MustCompile(`^(?:https?://)?gist\.github(?:usercontent|)\.com/([^/]+)/.+?/.+`),
regexp.MustCompile(`^(?:https?://)?api\.github\.com/repos/([^/]+)/([^/]+)/.*`),
regexp.MustCompile(`^(?:https?://)?huggingface\.co(?:/spaces)?/([^/]+)/(.+)$`),
regexp.MustCompile(`^(?:https?://)?cdn-lfs\.hf\.co(?:/spaces)?/([^/]+)/([^/]+)(?:/(.*))?$`),
regexp.MustCompile(`^(?:https?://)?download\.docker\.com/([^/]+)/.*\.(tgz|zip)$`),
regexp.MustCompile(`^(?:https?://)?(github|opengraph)\.githubassets\.com/([^/]+)/.+?$`),
}
globalLimiter *IPRateLimiter
// 服务启动时间
serviceStartTime = time.Now()
)
func main() {
// 加载配置
if err := LoadConfig(); err != nil {
fmt.Printf("配置加载失败: %v\n", err)
return
}
// 初始化HTTP客户端
initHTTPClients()
// 初始化限流器
initLimiter()
// 初始化Docker流式代理
initDockerProxy()
// 初始化镜像流式下载器
initImageStreamer()
// 初始化防抖器
initDebouncer()
gin.SetMode(gin.ReleaseMode)
router := gin.Default()
// 全局Panic恢复保护
router.Use(gin.CustomRecovery(func(c *gin.Context, recovered interface{}) {
log.Printf("🚨 Panic recovered: %v", recovered)
c.JSON(http.StatusInternalServerError, gin.H{
"error": "Internal server error",
"code": "INTERNAL_ERROR",
})
}))
// 初始化监控端点
initHealthRoutes(router)
// 初始化镜像tar下载路由
initImageTarRoutes(router)
// 静态文件路由
router.GET("/", func(c *gin.Context) {
serveEmbedFile(c, "public/index.html")
})
router.GET("/public/*filepath", func(c *gin.Context) {
filepath := strings.TrimPrefix(c.Param("filepath"), "/")
serveEmbedFile(c, "public/"+filepath)
})
router.GET("/images.html", func(c *gin.Context) {
serveEmbedFile(c, "public/images.html")
})
router.GET("/search.html", func(c *gin.Context) {
serveEmbedFile(c, "public/search.html")
})
router.GET("/favicon.ico", func(c *gin.Context) {
serveEmbedFile(c, "public/favicon.ico")
})
// 注册dockerhub搜索路由
RegisterSearchRoute(router)
// 注册Docker认证路由(/token*
router.Any("/token", RateLimitMiddleware(globalLimiter), ProxyDockerAuthGin)
router.Any("/token/*path", RateLimitMiddleware(globalLimiter), ProxyDockerAuthGin)
// 注册Docker Registry代理路由
router.Any("/v2/*path", RateLimitMiddleware(globalLimiter), ProxyDockerRegistryGin)
// 注册NoRoute处理器
router.NoRoute(RateLimitMiddleware(globalLimiter), handler)
cfg := GetConfig()
fmt.Printf("🚀 HubProxy 启动成功\n")
fmt.Printf("📡 监听地址: %s:%d\n", cfg.Server.Host, cfg.Server.Port)
fmt.Printf("⚡ 限流配置: %d请求/%g小时\n", cfg.RateLimit.RequestLimit, cfg.RateLimit.PeriodHours)
fmt.Printf("🔗 项目地址: https://github.com/sky22333/hubproxy\n")
err := router.Run(fmt.Sprintf("%s:%d", cfg.Server.Host, cfg.Server.Port))
if err != nil {
fmt.Printf("启动服务失败: %v\n", err)
}
}
func handler(c *gin.Context) {
rawPath := strings.TrimPrefix(c.Request.URL.RequestURI(), "/")
for strings.HasPrefix(rawPath, "/") {
rawPath = strings.TrimPrefix(rawPath, "/")
}
if !strings.HasPrefix(rawPath, "http") {
c.String(http.StatusForbidden, "无效输入")
return
}
matches := checkURL(rawPath)
if matches != nil {
// GitHub仓库访问控制检查
if allowed, reason := GlobalAccessController.CheckGitHubAccess(matches); !allowed {
// 构建仓库名用于日志
var repoPath string
if len(matches) >= 2 {
username := matches[0]
repoName := strings.TrimSuffix(matches[1], ".git")
repoPath = username + "/" + repoName
}
fmt.Printf("GitHub仓库 %s 访问被拒绝: %s\n", repoPath, reason)
c.String(http.StatusForbidden, reason)
return
}
} else {
c.String(http.StatusForbidden, "无效输入")
return
}
if exps[1].MatchString(rawPath) {
rawPath = strings.Replace(rawPath, "/blob/", "/raw/", 1)
}
proxy(c, rawPath)
}
func proxy(c *gin.Context, u string) {
proxyWithRedirect(c, u, 0)
}
func proxyWithRedirect(c *gin.Context, u string, redirectCount int) {
// 限制最大重定向次数,防止无限递归
const maxRedirects = 20
if redirectCount > maxRedirects {
c.String(http.StatusLoopDetected, "重定向次数过多,可能存在循环重定向")
return
}
req, err := http.NewRequest(c.Request.Method, u, c.Request.Body)
if err != nil {
c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err))
return
}
for key, values := range c.Request.Header {
for _, value := range values {
req.Header.Add(key, value)
}
}
req.Header.Del("Host")
resp, err := GetGlobalHTTPClient().Do(req)
if err != nil {
c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err))
return
}
defer func() {
if err := resp.Body.Close(); err != nil {
fmt.Printf("关闭响应体失败: %v\n", err)
}
}()
// 检查文件大小限制
cfg := GetConfig()
if contentLength := resp.Header.Get("Content-Length"); contentLength != "" {
if size, err := strconv.ParseInt(contentLength, 10, 64); err == nil && size > cfg.Server.FileSize {
c.String(http.StatusRequestEntityTooLarge,
fmt.Sprintf("文件过大,限制大小: %d MB", cfg.Server.FileSize/(1024*1024)))
return
}
}
// 清理安全相关的头
resp.Header.Del("Content-Security-Policy")
resp.Header.Del("Referrer-Policy")
resp.Header.Del("Strict-Transport-Security")
// 获取真实域名
realHost := c.Request.Header.Get("X-Forwarded-Host")
if realHost == "" {
realHost = c.Request.Host
}
// 如果域名中没有协议前缀添加https://
if !strings.HasPrefix(realHost, "http://") && !strings.HasPrefix(realHost, "https://") {
realHost = "https://" + realHost
}
if strings.HasSuffix(strings.ToLower(u), ".sh") {
isGzipCompressed := resp.Header.Get("Content-Encoding") == "gzip"
processedBody, processedSize, err := ProcessSmart(resp.Body, isGzipCompressed, realHost)
if err != nil {
fmt.Printf("智能处理失败,回退到直接代理: %v\n", err)
processedBody = resp.Body
processedSize = 0
}
// 智能设置响应头
if processedSize > 0 {
resp.Header.Del("Content-Length")
resp.Header.Del("Content-Encoding")
resp.Header.Set("Transfer-Encoding", "chunked")
}
// 复制其他响应头
for key, values := range resp.Header {
for _, value := range values {
c.Header(key, value)
}
}
if location := resp.Header.Get("Location"); location != "" {
if checkURL(location) != nil {
c.Header("Location", "/"+location)
} else {
proxyWithRedirect(c, location, redirectCount+1)
return
}
}
c.Status(resp.StatusCode)
// 输出处理后的内容
if _, err := io.Copy(c.Writer, processedBody); err != nil {
return
}
} else {
for key, values := range resp.Header {
for _, value := range values {
c.Header(key, value)
}
}
// 处理重定向
if location := resp.Header.Get("Location"); location != "" {
if checkURL(location) != nil {
c.Header("Location", "/"+location)
} else {
proxyWithRedirect(c, location, redirectCount+1)
return
}
}
c.Status(resp.StatusCode)
// 直接流式转发
if _, err := io.Copy(c.Writer, resp.Body); err != nil {
fmt.Printf("直接代理失败: %v\n", err)
}
}
}
func checkURL(u string) []string {
for _, exp := range exps {
if matches := exp.FindStringSubmatch(u); matches != nil {
return matches[1:]
}
}
return nil
}
// 初始化健康监控路由
func initHealthRoutes(router *gin.Engine) {
// 健康检查端点
router.GET("/health", func(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{
"status": "healthy",
"timestamp": time.Now().Unix(),
"uptime": time.Since(serviceStartTime).Seconds(),
"service": "hubproxy",
})
})
// 就绪检查端点
router.GET("/ready", func(c *gin.Context) {
checks := make(map[string]string)
allReady := true
if GetConfig() != nil {
checks["config"] = "ok"
} else {
checks["config"] = "failed"
allReady = false
}
// 检查全局缓存状态
if globalCache != nil {
checks["cache"] = "ok"
} else {
checks["cache"] = "failed"
allReady = false
}
// 检查限流器状态
if globalLimiter != nil {
checks["ratelimiter"] = "ok"
} else {
checks["ratelimiter"] = "failed"
allReady = false
}
// 检查镜像下载器状态
if globalImageStreamer != nil {
checks["imagestreamer"] = "ok"
} else {
checks["imagestreamer"] = "failed"
allReady = false
}
// 检查HTTP客户端状态
if GetGlobalHTTPClient() != nil {
checks["httpclient"] = "ok"
} else {
checks["httpclient"] = "failed"
allReady = false
}
status := http.StatusOK
if !allReady {
status = http.StatusServiceUnavailable
}
c.JSON(status, gin.H{
"ready": allReady,
"checks": checks,
"timestamp": time.Now().Unix(),
"uptime": time.Since(serviceStartTime).Seconds(),
})
})
}
package main
import (
"embed"
"fmt"
"io"
"log"
"net/http"
"regexp"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
)
//go:embed public/*
var staticFiles embed.FS
// 服务嵌入的静态文件
func serveEmbedFile(c *gin.Context, filename string) {
data, err := staticFiles.ReadFile(filename)
if err != nil {
c.Status(404)
return
}
contentType := "text/html; charset=utf-8"
if strings.HasSuffix(filename, ".ico") {
contentType = "image/x-icon"
}
c.Data(200, contentType, data)
}
var (
exps = []*regexp.Regexp{
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:releases|archive)/.*$`),
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:blob|raw)/.*$`),
regexp.MustCompile(`^(?:https?://)?github\.com/([^/]+)/([^/]+)/(?:info|git-).*$`),
regexp.MustCompile(`^(?:https?://)?raw\.github(?:usercontent|)\.com/([^/]+)/([^/]+)/.+?/.+$`),
regexp.MustCompile(`^(?:https?://)?gist\.github(?:usercontent|)\.com/([^/]+)/.+?/.+`),
regexp.MustCompile(`^(?:https?://)?api\.github\.com/repos/([^/]+)/([^/]+)/.*`),
regexp.MustCompile(`^(?:https?://)?huggingface\.co(?:/spaces)?/([^/]+)/(.+)$`),
regexp.MustCompile(`^(?:https?://)?cdn-lfs\.hf\.co(?:/spaces)?/([^/]+)/([^/]+)(?:/(.*))?$`),
regexp.MustCompile(`^(?:https?://)?download\.docker\.com/([^/]+)/.*\.(tgz|zip)$`),
regexp.MustCompile(`^(?:https?://)?(github|opengraph)\.githubassets\.com/([^/]+)/.+?$`),
}
globalLimiter *IPRateLimiter
// 服务启动时间
serviceStartTime = time.Now()
)
func main() {
// 加载配置
if err := LoadConfig(); err != nil {
fmt.Printf("配置加载失败: %v\n", err)
return
}
// 初始化HTTP客户端
initHTTPClients()
// 初始化限流器
initLimiter()
// 初始化Docker流式代理
initDockerProxy()
// 初始化镜像流式下载器
initImageStreamer()
// 初始化防抖器
initDebouncer()
gin.SetMode(gin.ReleaseMode)
router := gin.Default()
// 全局Panic恢复保护
router.Use(gin.CustomRecovery(func(c *gin.Context, recovered interface{}) {
log.Printf("🚨 Panic recovered: %v", recovered)
c.JSON(http.StatusInternalServerError, gin.H{
"error": "Internal server error",
"code": "INTERNAL_ERROR",
})
}))
// 全局限流中间件 - 应用到所有路由
router.Use(RateLimitMiddleware(globalLimiter))
// 初始化监控端点
initHealthRoutes(router)
// 初始化镜像tar下载路由
initImageTarRoutes(router)
// 静态文件路由
router.GET("/", func(c *gin.Context) {
serveEmbedFile(c, "public/index.html")
})
router.GET("/public/*filepath", func(c *gin.Context) {
filepath := strings.TrimPrefix(c.Param("filepath"), "/")
serveEmbedFile(c, "public/"+filepath)
})
router.GET("/images.html", func(c *gin.Context) {
serveEmbedFile(c, "public/images.html")
})
router.GET("/search.html", func(c *gin.Context) {
serveEmbedFile(c, "public/search.html")
})
router.GET("/favicon.ico", func(c *gin.Context) {
serveEmbedFile(c, "public/favicon.ico")
})
// 注册dockerhub搜索路由
RegisterSearchRoute(router)
// 注册Docker认证路由/token*
router.Any("/token", ProxyDockerAuthGin)
router.Any("/token/*path", ProxyDockerAuthGin)
// 注册Docker Registry代理路由
router.Any("/v2/*path", ProxyDockerRegistryGin)
// 注册NoRoute处理器
router.NoRoute(handler)
cfg := GetConfig()
fmt.Printf("🚀 HubProxy 启动成功\n")
fmt.Printf("📡 监听地址: %s:%d\n", cfg.Server.Host, cfg.Server.Port)
fmt.Printf("⚡ 限流配置: %d请求/%g小时\n", cfg.RateLimit.RequestLimit, cfg.RateLimit.PeriodHours)
fmt.Printf("🔗 项目地址: https://github.com/sky22333/hubproxy\n")
err := router.Run(fmt.Sprintf("%s:%d", cfg.Server.Host, cfg.Server.Port))
if err != nil {
fmt.Printf("启动服务失败: %v\n", err)
}
}
func handler(c *gin.Context) {
rawPath := strings.TrimPrefix(c.Request.URL.RequestURI(), "/")
for strings.HasPrefix(rawPath, "/") {
rawPath = strings.TrimPrefix(rawPath, "/")
}
if !strings.HasPrefix(rawPath, "http") {
c.String(http.StatusForbidden, "无效输入")
return
}
matches := checkURL(rawPath)
if matches != nil {
// GitHub仓库访问控制检查
if allowed, reason := GlobalAccessController.CheckGitHubAccess(matches); !allowed {
// 构建仓库名用于日志
var repoPath string
if len(matches) >= 2 {
username := matches[0]
repoName := strings.TrimSuffix(matches[1], ".git")
repoPath = username + "/" + repoName
}
fmt.Printf("GitHub仓库 %s 访问被拒绝: %s\n", repoPath, reason)
c.String(http.StatusForbidden, reason)
return
}
} else {
c.String(http.StatusForbidden, "无效输入")
return
}
if exps[1].MatchString(rawPath) {
rawPath = strings.Replace(rawPath, "/blob/", "/raw/", 1)
}
proxyRequest(c, rawPath)
}
func proxyRequest(c *gin.Context, u string) {
proxyWithRedirect(c, u, 0)
}
func proxyWithRedirect(c *gin.Context, u string, redirectCount int) {
// 限制最大重定向次数,防止无限递归
const maxRedirects = 20
if redirectCount > maxRedirects {
c.String(http.StatusLoopDetected, "重定向次数过多,可能存在循环重定向")
return
}
req, err := http.NewRequest(c.Request.Method, u, c.Request.Body)
if err != nil {
c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err))
return
}
for key, values := range c.Request.Header {
for _, value := range values {
req.Header.Add(key, value)
}
}
req.Header.Del("Host")
resp, err := GetGlobalHTTPClient().Do(req)
if err != nil {
c.String(http.StatusInternalServerError, fmt.Sprintf("server error %v", err))
return
}
defer func() {
if err := resp.Body.Close(); err != nil {
fmt.Printf("关闭响应体失败: %v\n", err)
}
}()
// 检查文件大小限制
cfg := GetConfig()
if contentLength := resp.Header.Get("Content-Length"); contentLength != "" {
if size, err := strconv.ParseInt(contentLength, 10, 64); err == nil && size > cfg.Server.FileSize {
c.String(http.StatusRequestEntityTooLarge,
fmt.Sprintf("文件过大,限制大小: %d MB", cfg.Server.FileSize/(1024*1024)))
return
}
}
// 清理安全相关的头
resp.Header.Del("Content-Security-Policy")
resp.Header.Del("Referrer-Policy")
resp.Header.Del("Strict-Transport-Security")
// 获取真实域名
realHost := c.Request.Header.Get("X-Forwarded-Host")
if realHost == "" {
realHost = c.Request.Host
}
// 如果域名中没有协议前缀添加https://
if !strings.HasPrefix(realHost, "http://") && !strings.HasPrefix(realHost, "https://") {
realHost = "https://" + realHost
}
if strings.HasSuffix(strings.ToLower(u), ".sh") {
isGzipCompressed := resp.Header.Get("Content-Encoding") == "gzip"
processedBody, processedSize, err := ProcessSmart(resp.Body, isGzipCompressed, realHost)
if err != nil {
fmt.Printf("智能处理失败,回退到直接代理: %v\n", err)
processedBody = resp.Body
processedSize = 0
}
// 智能设置响应头
if processedSize > 0 {
resp.Header.Del("Content-Length")
resp.Header.Del("Content-Encoding")
resp.Header.Set("Transfer-Encoding", "chunked")
}
// 复制其他响应头
for key, values := range resp.Header {
for _, value := range values {
c.Header(key, value)
}
}
if location := resp.Header.Get("Location"); location != "" {
if checkURL(location) != nil {
c.Header("Location", "/"+location)
} else {
proxyWithRedirect(c, location, redirectCount+1)
return
}
}
c.Status(resp.StatusCode)
// 输出处理后的内容
if _, err := io.Copy(c.Writer, processedBody); err != nil {
return
}
} else {
for key, values := range resp.Header {
for _, value := range values {
c.Header(key, value)
}
}
// 处理重定向
if location := resp.Header.Get("Location"); location != "" {
if checkURL(location) != nil {
c.Header("Location", "/"+location)
} else {
proxyWithRedirect(c, location, redirectCount+1)
return
}
}
c.Status(resp.StatusCode)
// 直接流式转发
io.Copy(c.Writer, resp.Body)
}
}
func checkURL(u string) []string {
for _, exp := range exps {
if matches := exp.FindStringSubmatch(u); matches != nil {
return matches[1:]
}
}
return nil
}
// 初始化健康监控路由
func initHealthRoutes(router *gin.Engine) {
// 健康检查端点
router.GET("/health", func(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{
"status": "healthy",
"timestamp": time.Now().Unix(),
"uptime": time.Since(serviceStartTime).Seconds(),
"service": "hubproxy",
})
})
// 就绪检查端点
router.GET("/ready", func(c *gin.Context) {
checks := make(map[string]string)
allReady := true
if GetConfig() != nil {
checks["config"] = "ok"
} else {
checks["config"] = "failed"
allReady = false
}
// 检查全局缓存状态
if globalCache != nil {
checks["cache"] = "ok"
} else {
checks["cache"] = "failed"
allReady = false
}
// 检查限流器状态
if globalLimiter != nil {
checks["ratelimiter"] = "ok"
} else {
checks["ratelimiter"] = "failed"
allReady = false
}
// 检查镜像下载器状态
if globalImageStreamer != nil {
checks["imagestreamer"] = "ok"
} else {
checks["imagestreamer"] = "failed"
allReady = false
}
// 检查HTTP客户端状态
if GetGlobalHTTPClient() != nil {
checks["httpclient"] = "ok"
} else {
checks["httpclient"] = "failed"
allReady = false
}
status := http.StatusOK
if !allReady {
status = http.StatusServiceUnavailable
}
c.JSON(status, gin.H{
"ready": allReady,
"checks": checks,
"timestamp": time.Now().Unix(),
"uptime": time.Since(serviceStartTime).Seconds(),
})
})
}

View File

@@ -1,95 +1,95 @@
package main
import (
"bytes"
"compress/gzip"
"fmt"
"io"
"regexp"
"strings"
)
// GitHub URL正则表达式
var githubRegex = regexp.MustCompile(`https?://(?:github\.com|raw\.githubusercontent\.com|raw\.github\.com|gist\.githubusercontent\.com|gist\.github\.com|api\.github\.com)[^\s'"]+`)
// ProcessSmart Shell脚本智能处理函数
func ProcessSmart(input io.ReadCloser, isCompressed bool, host string) (io.Reader, int64, error) {
defer input.Close()
content, err := readShellContent(input, isCompressed)
if err != nil {
return nil, 0, fmt.Errorf("内容读取失败: %v", err)
}
if len(content) == 0 {
return strings.NewReader(""), 0, nil
}
if len(content) > 10*1024*1024 {
return strings.NewReader(content), int64(len(content)), nil
}
if !strings.Contains(content, "github.com") && !strings.Contains(content, "githubusercontent.com") {
return strings.NewReader(content), int64(len(content)), nil
}
processed := processGitHubURLs(content, host)
return strings.NewReader(processed), int64(len(processed)), nil
}
func readShellContent(input io.ReadCloser, isCompressed bool) (string, error) {
var reader io.Reader = input
// 处理gzip压缩
if isCompressed {
peek := make([]byte, 2)
n, err := input.Read(peek)
if err != nil && err != io.EOF {
return "", fmt.Errorf("读取数据失败: %v", err)
}
if n >= 2 && peek[0] == 0x1f && peek[1] == 0x8b {
combinedReader := io.MultiReader(bytes.NewReader(peek[:n]), input)
gzReader, err := gzip.NewReader(combinedReader)
if err != nil {
return "", fmt.Errorf("gzip解压失败: %v", err)
}
defer gzReader.Close()
reader = gzReader
} else {
reader = io.MultiReader(bytes.NewReader(peek[:n]), input)
}
}
data, err := io.ReadAll(reader)
if err != nil {
return "", fmt.Errorf("读取内容失败: %v", err)
}
return string(data), nil
}
func processGitHubURLs(content, host string) string {
return githubRegex.ReplaceAllStringFunc(content, func(url string) string {
return transformURL(url, host)
})
}
// transformURL URL转换函数
func transformURL(url, host string) string {
if strings.Contains(url, host) {
return url
}
if strings.HasPrefix(url, "http://") {
url = "https" + url[4:]
} else if !strings.HasPrefix(url, "https://") && !strings.HasPrefix(url, "//") {
url = "https://" + url
}
cleanHost := strings.TrimPrefix(host, "https://")
cleanHost = strings.TrimPrefix(cleanHost, "http://")
cleanHost = strings.TrimSuffix(cleanHost, "/")
return cleanHost + "/" + url
}
package main
import (
"bytes"
"compress/gzip"
"fmt"
"io"
"regexp"
"strings"
)
// GitHub URL正则表达式
var githubRegex = regexp.MustCompile(`https?://(?:github\.com|raw\.githubusercontent\.com|raw\.github\.com|gist\.githubusercontent\.com|gist\.github\.com|api\.github\.com)[^\s'"]+`)
// ProcessSmart Shell脚本智能处理函数
func ProcessSmart(input io.ReadCloser, isCompressed bool, host string) (io.Reader, int64, error) {
defer input.Close()
content, err := readShellContent(input, isCompressed)
if err != nil {
return nil, 0, fmt.Errorf("内容读取失败: %v", err)
}
if len(content) == 0 {
return strings.NewReader(""), 0, nil
}
if len(content) > 10*1024*1024 {
return strings.NewReader(content), int64(len(content)), nil
}
if !strings.Contains(content, "github.com") && !strings.Contains(content, "githubusercontent.com") {
return strings.NewReader(content), int64(len(content)), nil
}
processed := processGitHubURLs(content, host)
return strings.NewReader(processed), int64(len(processed)), nil
}
func readShellContent(input io.ReadCloser, isCompressed bool) (string, error) {
var reader io.Reader = input
// 处理gzip压缩
if isCompressed {
peek := make([]byte, 2)
n, err := input.Read(peek)
if err != nil && err != io.EOF {
return "", fmt.Errorf("读取数据失败: %v", err)
}
if n >= 2 && peek[0] == 0x1f && peek[1] == 0x8b {
combinedReader := io.MultiReader(bytes.NewReader(peek[:n]), input)
gzReader, err := gzip.NewReader(combinedReader)
if err != nil {
return "", fmt.Errorf("gzip解压失败: %v", err)
}
defer gzReader.Close()
reader = gzReader
} else {
reader = io.MultiReader(bytes.NewReader(peek[:n]), input)
}
}
data, err := io.ReadAll(reader)
if err != nil {
return "", fmt.Errorf("读取内容失败: %v", err)
}
return string(data), nil
}
func processGitHubURLs(content, host string) string {
return githubRegex.ReplaceAllStringFunc(content, func(url string) string {
return transformURL(url, host)
})
}
// transformURL URL转换函数
func transformURL(url, host string) string {
if strings.Contains(url, host) {
return url
}
if strings.HasPrefix(url, "http://") {
url = "https" + url[4:]
} else if !strings.HasPrefix(url, "https://") && !strings.HasPrefix(url, "//") {
url = "https://" + url
}
cleanHost := strings.TrimPrefix(host, "https://")
cleanHost = strings.TrimPrefix(cleanHost, "http://")
cleanHost = strings.TrimSuffix(cleanHost, "/")
return cleanHost + "/" + url
}

View File

@@ -399,6 +399,67 @@
100% { transform: rotate(360deg); }
}
/* 切换开关样式 */
.switch-container {
display: flex;
align-items: center;
gap: 0.75rem;
margin-bottom: 1.5rem;
}
.switch {
position: relative;
display: inline-block;
width: 50px;
height: 24px;
}
.switch input {
opacity: 0;
width: 0;
height: 0;
}
.slider {
position: absolute;
cursor: pointer;
top: 0;
left: 0;
right: 0;
bottom: 0;
background-color: var(--muted);
transition: 0.2s;
border-radius: 24px;
border: 1px solid var(--border);
}
.slider:before {
position: absolute;
content: "";
height: 18px;
width: 18px;
left: 2px;
bottom: 2px;
background-color: white;
transition: 0.2s;
border-radius: 50%;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
}
input:checked + .slider {
background-color: var(--primary);
}
input:checked + .slider:before {
transform: translateX(26px);
}
.switch-label {
font-weight: 500;
color: var(--foreground);
cursor: pointer;
}
.hidden {
display: none;
}
@@ -520,7 +581,7 @@
</div>
<div class="feature">
<span class="feature-icon">💾</span>
<span>无需打包</span>
<span>无需等待</span>
</div>
<div class="feature">
<span class="feature-icon">🏗️</span>
@@ -559,6 +620,14 @@
</div>
</div>
<div class="switch-container">
<label class="switch">
<input type="checkbox" id="compressedToggle" checked>
<span class="slider"></span>
</label>
<label for="compressedToggle" class="switch-label">使用压缩层(减小包体积)</label>
</div>
<button type="submit" class="btn btn-primary btn-full" id="downloadBtn">
<span id="downloadText">立即下载</span>
<span id="downloadLoading" class="loading hidden"></span>
@@ -573,7 +642,7 @@
<form id="batchForm">
<div class="form-group">
<label class="form-label" for="imagesTextarea">镜像列表,每行一个,会将多个镜像自动合并,符合官方标准,完全兼容docker load</label>
<label class="form-label" for="imagesTextarea">镜像列表每行一个会将多个镜像自动合并符合官方标准兼容docker load</label>
<textarea
id="imagesTextarea"
class="textarea"
@@ -595,6 +664,14 @@
</div>
</div>
<div class="switch-container">
<label class="switch">
<input type="checkbox" id="batchCompressedToggle" checked>
<span class="slider"></span>
</label>
<label for="batchCompressedToggle" class="switch-label">使用压缩层(减小包体积)</label>
</div>
<button type="submit" class="btn btn-primary btn-full" id="batchDownloadBtn">
<span id="batchDownloadText">开始下载</span>
<span id="batchDownloadLoading" class="loading hidden"></span>
@@ -651,12 +728,18 @@
}
}
function buildDownloadUrl(imageName, platform = '') {
function buildDownloadUrl(imageName, platform = '', useCompressed = true) {
const encodedImage = imageName.replace(/\//g, '_');
let url = `/api/image/download/${encodedImage}`;
const params = new URLSearchParams();
if (platform && platform.trim()) {
url += `?platform=${encodeURIComponent(platform.trim())}`;
params.append('platform', platform.trim());
}
params.append('compressed', useCompressed.toString());
if (params.toString()) {
url += '?' + params.toString();
}
return url;
@@ -672,11 +755,12 @@
}
const platform = document.getElementById('platformInput').value.trim();
const useCompressed = document.getElementById('compressedToggle').checked;
hideStatus('singleStatus');
setButtonLoading('downloadBtn', 'downloadText', 'downloadLoading', true);
const downloadUrl = buildDownloadUrl(imageName, platform);
const downloadUrl = buildDownloadUrl(imageName, platform, useCompressed);
const link = document.createElement('a');
link.href = downloadUrl;
@@ -711,9 +795,11 @@
}
const platform = document.getElementById('batchPlatformInput').value.trim();
const useCompressed = document.getElementById('batchCompressedToggle').checked;
const options = {
images: images
images: images,
useCompressedLayers: useCompressed
};
if (platform) {

View File

@@ -609,10 +609,10 @@
<div class="card">
<div class="card-header">
<h2 class="card-title">
⚡ 快速生成加速链接
⚡ 快速转换加速链接
</h2>
<p class="card-description">
输入GitHub文件或仓库链接自动转换加速链接可以直接在Github域名前面加上本站域名使用。
输入GitHub文件链接自动转换加速链接可以直接在Github文件链接前加上本站域名使用。
</p>
</div>
@@ -622,7 +622,7 @@
type="text"
class="input"
id="githubLinkInput"
placeholder="请输入GitHub链接例如https://github.com/user/repo/releases/download/..."
placeholder="请输入GitHub文件链接例如https://github.com/user/repo/releases/download/..."
>
<button class="button button-primary" id="formatButton">
获取加速链接
@@ -653,12 +653,12 @@
🐳 Docker 镜像加速
</h3>
<p class="card-description">
支持多种Registry,在镜像名前添加本站域名即可加速下载。
支持多种镜像仓库,在镜像名前添加本站域名即可加速下载。
</p>
</div>
<button class="docker-button" id="dockerButton">
查看 Docker 镜像加速配置
查看 Docker 镜像加速使用说明
</button>
</div>
</div>
@@ -669,23 +669,23 @@
<button class="close-button" id="closeModal">&times;</button>
<div class="modal-header">
<h2 class="modal-title">Docker 镜像加速</h2>
<p>支持多种Registry,在镜像名前添加本站域名即可加速下载。</p>
<p>支持多种镜像仓库,在镜像名前添加本站域名即可加速下载。</p>
</div>
<div class="domain-examples">
<strong>Docker Hub 官方镜像:</strong>
<strong>Docker 官方镜像:</strong>
docker pull <span class="domain-base"></span>/nginx
<strong>Docker Hub 第三方镜像:</strong>
<strong>Docker 镜像:</strong>
docker pull <span class="domain-base"></span>/user/image
<strong>GitHub Container Registry</strong>
<strong>ghcr.io 镜像</strong>
docker pull <span class="domain-base"></span>/ghcr.io/user/image
<strong>Quay.io Registry</strong>
<strong>Quay.io 镜像</strong>
docker pull <span class="domain-base"></span>/quay.io/org/image
<strong>Kubernetes Registry</strong>
<strong>K8s 镜像</strong>
docker pull <span class="domain-base"></span>/registry.k8s.io/pause:3.8
</div>
</div>

View File

@@ -778,7 +778,12 @@
</div>
</div>
<div class="tag-list" id="tagList"></div>
<div class="tag-list" id="tagList">
<div class="pagination" id="tagPagination" style="display: none;">
<button id="tagPrevPage" disabled>上一页</button>
<button id="tagNextPage" disabled>下一页</button>
</div>
</div>
</div>
<div id="toast"></div>
@@ -853,6 +858,10 @@
let totalPages = 1;
let currentQuery = '';
let currentRepo = null;
// 标签分页相关变量
let currentTagPage = 1;
let totalTagPages = 1;
document.getElementById('searchButton').addEventListener('click', () => {
currentPage = 1;
@@ -884,6 +893,21 @@
showSearchResults();
});
// 使用事件委托处理分页按钮点击避免DOM重建导致事件丢失
document.addEventListener('click', (e) => {
if (e.target.id === 'tagPrevPage') {
if (currentTagPage > 1) {
currentTagPage--;
loadTagPage();
}
} else if (e.target.id === 'tagNextPage') {
if (currentTagPage < totalTagPages) {
currentTagPage++;
loadTagPage();
}
}
});
function showLoading() {
document.querySelector('.loading').style.display = 'block';
}
@@ -901,71 +925,135 @@
}, 3000);
}
function updatePagination() {
const prevButton = document.getElementById('prevPage');
const nextButton = document.getElementById('nextPage');
// 统一分页更新函数(支持搜索和标签分页)
function updatePagination(config = {}) {
const {
currentPage: page = currentPage,
totalPages: total = totalPages,
prefix = ''
} = config;
const prevButtonId = prefix ? `${prefix}PrevPage` : 'prevPage';
const nextButtonId = prefix ? `${prefix}NextPage` : 'nextPage';
const paginationId = prefix ? `${prefix}Pagination` : '.pagination';
const prevButton = document.getElementById(prevButtonId);
const nextButton = document.getElementById(nextButtonId);
const paginationDiv = prefix ? document.getElementById(paginationId) : document.querySelector(paginationId);
prevButton.disabled = currentPage <= 1;
nextButton.disabled = currentPage >= totalPages;
if (!prevButton || !nextButton || !paginationDiv) {
return; // 静默处理,避免控制台警告
}
// 更新按钮状态
prevButton.disabled = page <= 1;
nextButton.disabled = page >= total;
// 更新或创建页面信息
const pageInfoId = prefix ? `${prefix}PageInfo` : 'pageInfo';
let pageInfo = document.getElementById(pageInfoId);
const paginationDiv = document.querySelector('.pagination');
let pageInfo = document.getElementById('pageInfo');
if (!pageInfo) {
const container = document.createElement('div');
container.id = 'pageInfo';
container.style.margin = '0 10px';
container.style.display = 'flex';
container.style.alignItems = 'center';
container.style.gap = '10px';
const pageText = document.createElement('span');
pageText.id = 'pageText';
const jumpInput = document.createElement('input');
jumpInput.type = 'number';
jumpInput.min = '1';
jumpInput.id = 'jumpPage';
jumpInput.style.width = '60px';
jumpInput.style.padding = '4px';
jumpInput.style.borderRadius = '4px';
jumpInput.style.border = '1px solid var(--border)';
jumpInput.style.backgroundColor = 'var(--input)';
jumpInput.style.color = 'var(--foreground)';
const jumpButton = document.createElement('button');
jumpButton.textContent = '跳转';
jumpButton.className = 'btn search-button';
jumpButton.style.padding = '4px 8px';
jumpButton.onclick = () => {
const page = parseInt(jumpInput.value);
if (page && page >= 1 && page <= totalPages) {
currentPage = page;
performSearch();
} else {
showToast('请输入有效的页码');
}
};
container.appendChild(pageText);
container.appendChild(jumpInput);
container.appendChild(jumpButton);
paginationDiv.insertBefore(container, nextButton);
pageInfo = container;
pageInfo = createPageInfo(pageInfoId, prefix, total);
paginationDiv.insertBefore(pageInfo, nextButton);
}
const pageText = document.getElementById('pageText');
pageText.textContent = `${currentPage} / ${totalPages || 1} 页 共 ${totalPages || 1}`;
const jumpInput = document.getElementById('jumpPage');
if (jumpInput) {
jumpInput.max = totalPages;
jumpInput.value = currentPage;
}
paginationDiv.style.display = totalPages > 1 ? 'flex' : 'none';
updatePageInfo(pageInfo, page, total, prefix);
paginationDiv.style.display = total > 1 ? 'flex' : 'none';
}
// 创建页面信息元素
function createPageInfo(pageInfoId, prefix, total) {
const container = document.createElement('div');
container.id = pageInfoId;
container.style.cssText = 'margin: 0 10px; display: flex; align-items: center; gap: 10px;';
const pageText = document.createElement('span');
pageText.id = prefix ? `${prefix}PageText` : 'pageText';
const jumpInput = document.createElement('input');
jumpInput.type = 'number';
jumpInput.min = '1';
jumpInput.max = prefix === 'tag' ? total : Math.min(total, 100); // 搜索页面限制100页
jumpInput.id = prefix ? `${prefix}JumpPage` : 'jumpPage';
jumpInput.style.cssText = 'width: 60px; padding: 4px; border-radius: 4px; border: 1px solid var(--border); background-color: var(--input); color: var(--foreground);';
const jumpButton = document.createElement('button');
jumpButton.textContent = '跳转';
jumpButton.className = 'btn search-button';
jumpButton.style.padding = '4px 8px';
jumpButton.onclick = () => handlePageJump(jumpInput, prefix, total);
container.append(pageText, jumpInput, jumpButton);
return container;
}
// 更新页面信息显示
function updatePageInfo(pageInfo, page, total, prefix) {
const pageText = pageInfo.querySelector('span');
const jumpInput = pageInfo.querySelector('input');
// 标签分页显示策略:根据是否确定总页数显示不同格式
const isTagPagination = prefix === 'tag';
const maxDisplayPages = isTagPagination ? total : Math.min(total, 100);
const pageTextContent = isTagPagination
? `${page}` + (total > page ? ` (至少 ${total} 页)` : ` (共 ${total} 页)`)
: `${page} / ${maxDisplayPages} 页 共 ${maxDisplayPages}` + (total > 100 ? ' (最多100页)' : '');
pageText.textContent = pageTextContent;
jumpInput.max = maxDisplayPages;
jumpInput.value = page;
}
// 处理页面跳转
function handlePageJump(jumpInput, prefix, total) {
const inputPage = parseInt(jumpInput.value);
const maxPage = prefix === 'tag' ? total : Math.min(total, 100);
if (!inputPage || inputPage < 1 || inputPage > maxPage) {
const limitText = prefix === 'tag' ? '页码' : '页码 (最多100页)';
showToast(`请输入有效的${limitText}`);
return;
}
if (prefix === 'tag') {
currentTagPage = inputPage;
loadTagPage();
} else {
currentPage = inputPage;
performSearch();
}
}
// 统一仓库信息处理
function parseRepositoryInfo(repo) {
const namespace = repo.namespace || (repo.is_official ? 'library' : '');
let name = repo.name || repo.repo_name || '';
// 清理名称,确保不包含命名空间前缀
if (name.includes('/')) {
const parts = name.split('/');
name = parts[parts.length - 1];
}
const cleanName = name.replace(/^library\//, '');
const fullRepoName = repo.is_official ? cleanName : `${namespace}/${cleanName}`;
return {
namespace,
name,
cleanName,
fullRepoName
};
}
// 分页更新函数
const updateSearchPagination = () => updatePagination();
const updateTagPagination = () => updatePagination({
currentPage: currentTagPage,
totalPages: totalTagPages,
prefix: 'tag'
});
function showSearchResults() {
document.querySelector('.search-results').style.display = 'block';
document.querySelector('.tag-list').style.display = 'none';
@@ -1006,7 +1094,7 @@
throw new Error(data.error || '搜索请求失败');
}
totalPages = Math.ceil(data.count / 25);
totalPages = Math.min(Math.ceil(data.count / 25), 100);
updatePagination();
displayResults(data.results, targetRepo);
@@ -1108,23 +1196,58 @@
});
}
// 内存管理
async function loadTags(namespace, name) {
currentTagPage = 1;
await loadTagPage(namespace, name);
}
async function loadTagPage(namespace = null, name = null) {
showLoading();
try {
if (!namespace || !name) {
// 如果传入了新的namespace和name更新currentRepo
if (namespace && name) {
// 清理旧数据,防止内存泄露
cleanupOldTagData();
}
// 获取当前仓库信息
const repoInfo = parseRepositoryInfo(currentRepo);
const currentNamespace = namespace || repoInfo.namespace;
const currentName = name || repoInfo.name;
// 调试日志
console.log(`loadTagPage: namespace=${currentNamespace}, name=${currentName}, page=${currentTagPage}`);
if (!currentNamespace || !currentName) {
showToast('命名空间和镜像名称不能为空');
return;
}
const response = await fetch(`/tags/${encodeURIComponent(namespace)}/${encodeURIComponent(name)}`);
const response = await fetch(`/tags/${encodeURIComponent(currentNamespace)}/${encodeURIComponent(currentName)}?page=${currentTagPage}&page_size=100`);
if (!response.ok) {
const errorText = await response.text();
throw new Error(errorText || '获取标签信息失败');
}
const data = await response.json();
displayTags(data);
showTagList();
// 改进的总页数计算:使用更准确的分页策略
if (data.has_more) {
// 如果还有更多页面,至少有当前页+1页但可能更多
totalTagPages = Math.max(currentTagPage + 1, totalTagPages);
} else {
// 如果没有更多页面,当前页就是最后一页
totalTagPages = currentTagPage;
}
displayTags(data.tags, data.has_more);
updateTagPagination();
if (namespace && name) {
showTagList();
}
} catch (error) {
console.error('加载标签错误:', error);
showToast(error.message || '获取标签信息失败,请稍后重试');
@@ -1133,12 +1256,24 @@
}
}
function displayTags(tags) {
function cleanupOldTagData() {
// 清理全局变量,释放内存
if (window.currentPageTags) {
window.currentPageTags.length = 0;
window.currentPageTags = null;
}
// 清理DOM缓存
const tagsContainer = document.getElementById('tagsContainer');
if (tagsContainer) {
tagsContainer.innerHTML = '';
}
}
function displayTags(tags, hasMore = false) {
const tagList = document.getElementById('tagList');
const namespace = currentRepo.namespace || (currentRepo.is_official ? 'library' : '');
const name = currentRepo.name || currentRepo.repo_name || '';
const cleanName = name.replace(/^library\//, '');
const fullRepoName = currentRepo.is_official ? cleanName : `${namespace}/${cleanName}`;
const repoInfo = parseRepositoryInfo(currentRepo);
const { fullRepoName } = repoInfo;
let header = `
<div class="tag-header">
@@ -1165,22 +1300,60 @@
<button class="tag-search-clear" onclick="clearTagSearch()">×</button>
</div>
<div id="tagsContainer"></div>
<div class="pagination" id="tagPagination" style="display: none;">
<button id="tagPrevPage" disabled>上一页</button>
<button id="tagNextPage" disabled>下一页</button>
</div>
`;
tagList.innerHTML = header;
window.allTags = tags;
// 存储当前页标签数据
window.currentPageTags = tags;
renderFilteredTags(tags);
}
function renderFilteredTags(filteredTags) {
const tagsContainer = document.getElementById('tagsContainer');
const namespace = currentRepo.namespace || (currentRepo.is_official ? 'library' : '');
const name = currentRepo.name || currentRepo.repo_name || '';
const cleanName = name.replace(/^library\//, '');
const fullRepoName = currentRepo.is_official ? cleanName : `${namespace}/${cleanName}`;
const repoInfo = parseRepositoryInfo(currentRepo);
const { fullRepoName } = repoInfo;
let tagsHtml = filteredTags.map(tag => {
if (filteredTags.length === 0) {
tagsContainer.innerHTML = '<div class="text-center" style="padding: 20px;">未找到匹配的标签</div>';
return;
}
// 渐进式渲染:分批处理大数据集
const BATCH_SIZE = 50;
if (filteredTags.length <= BATCH_SIZE) {
// 小数据集:直接渲染
renderTagsBatch(filteredTags, fullRepoName, tagsContainer, true);
} else {
// 大数据集:分批渲染
tagsContainer.innerHTML = ''; // 清空容器
let currentBatch = 0;
function renderNextBatch() {
const start = currentBatch * BATCH_SIZE;
const end = Math.min(start + BATCH_SIZE, filteredTags.length);
const batch = filteredTags.slice(start, end);
renderTagsBatch(batch, fullRepoName, tagsContainer, false);
currentBatch++;
if (end < filteredTags.length) {
// 使用requestAnimationFrame确保UI响应性
requestAnimationFrame(renderNextBatch);
}
}
renderNextBatch();
}
}
function renderTagsBatch(tags, fullRepoName, container, replaceContent = false) {
const tagsHtml = tags.map(tag => {
const vulnIndicators = Object.entries(tag.vulnerabilities || {})
.map(([level, count]) => count > 0 ? `<span class="vulnerability-dot vulnerability-${level.toLowerCase()}" title="${level}: ${count}"></span>` : '')
.join('');
@@ -1212,23 +1385,23 @@
`;
}).join('');
if (filteredTags.length === 0) {
tagsHtml = '<div class="text-center" style="padding: 20px;">未找到匹配的标签</div>';
if (replaceContent) {
container.innerHTML = tagsHtml;
} else {
container.insertAdjacentHTML('beforeend', tagsHtml);
}
tagsContainer.innerHTML = tagsHtml;
}
function filterTags(searchText) {
if (!window.allTags) return;
if (!window.currentPageTags) return;
const searchLower = searchText.toLowerCase();
let filteredTags;
if (!searchText) {
filteredTags = window.allTags;
filteredTags = window.currentPageTags;
} else {
const scoredTags = window.allTags.map(tag => {
const scoredTags = window.currentPageTags.map(tag => {
const name = tag.name.toLowerCase();
let score = 0;
@@ -1263,6 +1436,8 @@
}
}
function copyToClipboard(text) {
navigator.clipboard.writeText(text).then(() => {
showToast('已复制到剪贴板');

View File

@@ -1,299 +1,301 @@
package main
import (
"fmt"
"net"
"strings"
"sync"
"time"
"github.com/gin-gonic/gin"
"golang.org/x/time/rate"
)
const (
// 清理间隔
CleanupInterval = 10 * time.Minute
MaxIPCacheSize = 10000
)
// IPRateLimiter IP限流器结构体
type IPRateLimiter struct {
ips map[string]*rateLimiterEntry // IP到限流器的映射
mu *sync.RWMutex // 读写锁,保证并发安全
r rate.Limit // 速率限制(每秒允许的请求数)
b int // 令牌桶容量(突发请求数)
whitelist []*net.IPNet // 白名单IP段
blacklist []*net.IPNet // 黑名单IP段
}
// rateLimiterEntry 限流器条目
type rateLimiterEntry struct {
limiter *rate.Limiter
lastAccess time.Time
}
// initGlobalLimiter 初始化全局限流器
func initGlobalLimiter() *IPRateLimiter {
cfg := GetConfig()
whitelist := make([]*net.IPNet, 0, len(cfg.Security.WhiteList))
for _, item := range cfg.Security.WhiteList {
if item = strings.TrimSpace(item); item != "" {
if !strings.Contains(item, "/") {
item = item + "/32" // 单个IP转为CIDR格式
}
_, ipnet, err := net.ParseCIDR(item)
if err == nil {
whitelist = append(whitelist, ipnet)
} else {
fmt.Printf("警告: 无效的白名单IP格式: %s\n", item)
}
}
}
// 解析黑名单IP段
blacklist := make([]*net.IPNet, 0, len(cfg.Security.BlackList))
for _, item := range cfg.Security.BlackList {
if item = strings.TrimSpace(item); item != "" {
if !strings.Contains(item, "/") {
item = item + "/32" // 单个IP转为CIDR格式
}
_, ipnet, err := net.ParseCIDR(item)
if err == nil {
blacklist = append(blacklist, ipnet)
} else {
fmt.Printf("警告: 无效的黑名单IP格式: %s\n", item)
}
}
}
// 计算速率:将 "每N小时X个请求" 转换为 "每秒Y个请求"
ratePerSecond := rate.Limit(float64(cfg.RateLimit.RequestLimit) / (cfg.RateLimit.PeriodHours * 3600))
burstSize := cfg.RateLimit.RequestLimit
if burstSize < 1 {
burstSize = 1
}
limiter := &IPRateLimiter{
ips: make(map[string]*rateLimiterEntry),
mu: &sync.RWMutex{},
r: ratePerSecond,
b: burstSize,
whitelist: whitelist,
blacklist: blacklist,
}
// 启动定期清理goroutine
go limiter.cleanupRoutine()
return limiter
}
// initLimiter 初始化限流器
func initLimiter() {
globalLimiter = initGlobalLimiter()
}
// cleanupRoutine 定期清理过期的限流器
func (i *IPRateLimiter) cleanupRoutine() {
ticker := time.NewTicker(CleanupInterval)
defer ticker.Stop()
for range ticker.C {
now := time.Now()
expired := make([]string, 0)
// 查找过期的条目
i.mu.RLock()
for ip, entry := range i.ips {
// 如果最后访问时间超过1小时认为过期
if now.Sub(entry.lastAccess) > 1*time.Hour {
expired = append(expired, ip)
}
}
i.mu.RUnlock()
// 如果有过期条目或者缓存过大,进行清理
if len(expired) > 0 || len(i.ips) > MaxIPCacheSize {
i.mu.Lock()
// 删除过期条目
for _, ip := range expired {
delete(i.ips, ip)
}
// 如果缓存仍然过大,全部清理
if len(i.ips) > MaxIPCacheSize {
i.ips = make(map[string]*rateLimiterEntry)
}
i.mu.Unlock()
}
}
}
// extractIPFromAddress 从地址中提取纯IP,去除端口号
func extractIPFromAddress(address string) string {
// 处理IPv6地址 [::1]:8080 格式
if strings.HasPrefix(address, "[") {
if endIndex := strings.Index(address, "]"); endIndex != -1 {
return address[1:endIndex]
}
}
// 处理IPv4地址 192.168.1.1:8080 格式
if lastColon := strings.LastIndex(address, ":"); lastColon != -1 {
return address[:lastColon]
}
return address
}
// isIPInCIDRList 检查IP是否在CIDR列表中
func isIPInCIDRList(ip string, cidrList []*net.IPNet) bool {
// 先提取纯IP地址
cleanIP := extractIPFromAddress(ip)
parsedIP := net.ParseIP(cleanIP)
if parsedIP == nil {
return false
}
for _, cidr := range cidrList {
if cidr.Contains(parsedIP) {
return true
}
}
return false
}
// GetLimiter 获取指定IP的限流器同时返回是否允许访问
func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
// 提取纯IP地址
cleanIP := extractIPFromAddress(ip)
// 检查是否在黑名单中
if isIPInCIDRList(cleanIP, i.blacklist) {
return nil, false
}
// 检查是否在白名单中
if isIPInCIDRList(cleanIP, i.whitelist) {
return rate.NewLimiter(rate.Inf, i.b), true
}
now := time.Now()
i.mu.RLock()
entry, exists := i.ips[cleanIP]
i.mu.RUnlock()
if exists {
i.mu.Lock()
if entry, stillExists := i.ips[cleanIP]; stillExists {
entry.lastAccess = now
i.mu.Unlock()
return entry.limiter, true
}
i.mu.Unlock()
}
i.mu.Lock()
if entry, exists := i.ips[cleanIP]; exists {
entry.lastAccess = now
i.mu.Unlock()
return entry.limiter, true
}
entry = &rateLimiterEntry{
limiter: rate.NewLimiter(i.r, i.b),
lastAccess: now,
}
i.ips[cleanIP] = entry
i.mu.Unlock()
return entry.limiter, true
}
// RateLimitMiddleware 速率限制中间件
func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
return func(c *gin.Context) {
// 获取客户端真实IP
var ip string
// 优先尝试从请求头获取真实IP
if forwarded := c.GetHeader("X-Forwarded-For"); forwarded != "" {
// X-Forwarded-For可能包含多个IP取第一个
ips := strings.Split(forwarded, ",")
ip = strings.TrimSpace(ips[0])
} else if realIP := c.GetHeader("X-Real-IP"); realIP != "" {
// 如果有X-Real-IP头
ip = realIP
} else if remoteIP := c.GetHeader("X-Original-Forwarded-For"); remoteIP != "" {
// 某些代理可能使用此头
ips := strings.Split(remoteIP, ",")
ip = strings.TrimSpace(ips[0])
} else {
// 回退到ClientIP方法
ip = c.ClientIP()
}
// 提取纯IP地址去除端口号
cleanIP := extractIPFromAddress(ip)
// 日志记录请求IP和头信息
fmt.Printf("请求IP: %s (去除端口后: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
ip,
cleanIP,
c.GetHeader("X-Forwarded-For"),
c.GetHeader("X-Real-IP"))
// 获取限流器并检查是否允许访问
ipLimiter, allowed := limiter.GetLimiter(cleanIP)
// 如果IP在黑名单中
if !allowed {
c.JSON(403, gin.H{
"error": "您已被限制访问",
})
c.Abort()
return
}
// 智能限流判断:检查是否应该跳过限流计数
shouldSkip := smartLimiter.ShouldSkipRateLimit(cleanIP, c.Request.URL.Path)
// 只有在不跳过的情况下才检查限流
if !shouldSkip && !ipLimiter.Allow() {
c.JSON(429, gin.H{
"error": "请求频率过快,暂时限制访问",
})
c.Abort()
return
}
c.Next()
}
}
// ApplyRateLimit 应用限流到特定路由
func ApplyRateLimit(router *gin.Engine, path string, method string, handler gin.HandlerFunc) {
// 使用全局限流器
limiter := globalLimiter
if limiter == nil {
limiter = initGlobalLimiter()
}
// 根据HTTP方法应用限流
switch method {
case "GET":
router.GET(path, RateLimitMiddleware(limiter), handler)
case "POST":
router.POST(path, RateLimitMiddleware(limiter), handler)
case "PUT":
router.PUT(path, RateLimitMiddleware(limiter), handler)
case "DELETE":
router.DELETE(path, RateLimitMiddleware(limiter), handler)
default:
router.Any(path, RateLimitMiddleware(limiter), handler)
}
}
package main
import (
"fmt"
"net"
"strings"
"sync"
"time"
"github.com/gin-gonic/gin"
"golang.org/x/time/rate"
)
const (
// 清理间隔
CleanupInterval = 10 * time.Minute
MaxIPCacheSize = 10000
)
// IPRateLimiter IP限流器结构体
type IPRateLimiter struct {
ips map[string]*rateLimiterEntry // IP到限流器的映射
mu *sync.RWMutex // 读写锁,保证并发安全
r rate.Limit // 速率限制(每秒允许的请求数)
b int // 令牌桶容量(突发请求数)
whitelist []*net.IPNet // 白名单IP段
blacklist []*net.IPNet // 黑名单IP段
}
// rateLimiterEntry 限流器条目
type rateLimiterEntry struct {
limiter *rate.Limiter
lastAccess time.Time
}
// initGlobalLimiter 初始化全局限流器
func initGlobalLimiter() *IPRateLimiter {
cfg := GetConfig()
whitelist := make([]*net.IPNet, 0, len(cfg.Security.WhiteList))
for _, item := range cfg.Security.WhiteList {
if item = strings.TrimSpace(item); item != "" {
if !strings.Contains(item, "/") {
item = item + "/32" // 单个IP转为CIDR格式
}
_, ipnet, err := net.ParseCIDR(item)
if err == nil {
whitelist = append(whitelist, ipnet)
} else {
fmt.Printf("警告: 无效的白名单IP格式: %s\n", item)
}
}
}
// 解析黑名单IP段
blacklist := make([]*net.IPNet, 0, len(cfg.Security.BlackList))
for _, item := range cfg.Security.BlackList {
if item = strings.TrimSpace(item); item != "" {
if !strings.Contains(item, "/") {
item = item + "/32" // 单个IP转为CIDR格式
}
_, ipnet, err := net.ParseCIDR(item)
if err == nil {
blacklist = append(blacklist, ipnet)
} else {
fmt.Printf("警告: 无效的黑名单IP格式: %s\n", item)
}
}
}
// 计算速率:将 "每N小时X个请求" 转换为 "每秒Y个请求"
ratePerSecond := rate.Limit(float64(cfg.RateLimit.RequestLimit) / (cfg.RateLimit.PeriodHours * 3600))
burstSize := cfg.RateLimit.RequestLimit
if burstSize < 1 {
burstSize = 1
}
limiter := &IPRateLimiter{
ips: make(map[string]*rateLimiterEntry),
mu: &sync.RWMutex{},
r: ratePerSecond,
b: burstSize,
whitelist: whitelist,
blacklist: blacklist,
}
// 启动定期清理goroutine
go limiter.cleanupRoutine()
return limiter
}
// initLimiter 初始化限流器
func initLimiter() {
globalLimiter = initGlobalLimiter()
}
// cleanupRoutine 定期清理过期的限流器
func (i *IPRateLimiter) cleanupRoutine() {
ticker := time.NewTicker(CleanupInterval)
defer ticker.Stop()
for range ticker.C {
now := time.Now()
expired := make([]string, 0)
// 查找过期的条目
i.mu.RLock()
for ip, entry := range i.ips {
// 如果最后访问时间超过1小时认为过期
if now.Sub(entry.lastAccess) > 1*time.Hour {
expired = append(expired, ip)
}
}
i.mu.RUnlock()
// 如果有过期条目或者缓存过大,进行清理
if len(expired) > 0 || len(i.ips) > MaxIPCacheSize {
i.mu.Lock()
// 删除过期条目
for _, ip := range expired {
delete(i.ips, ip)
}
// 如果缓存仍然过大,全部清理
if len(i.ips) > MaxIPCacheSize {
i.ips = make(map[string]*rateLimiterEntry)
}
i.mu.Unlock()
}
}
}
// extractIPFromAddress 从地址中提取纯IP
func extractIPFromAddress(address string) string {
if host, _, err := net.SplitHostPort(address); err == nil {
return host
}
return address
}
// normalizeIPForRateLimit 标准化IP地址用于限流IPv4保持不变IPv6标准化为/64网段
func normalizeIPForRateLimit(ipStr string) string {
ip := net.ParseIP(ipStr)
if ip == nil {
return ipStr // 解析失败,返回原值
}
if ip.To4() != nil {
return ipStr // IPv4保持不变
}
// IPv6标准化为 /64 网段
ipv6 := ip.To16()
for i := 8; i < 16; i++ {
ipv6[i] = 0 // 清零后64位
}
return ipv6.String() + "/64"
}
// isIPInCIDRList 检查IP是否在CIDR列表中
func isIPInCIDRList(ip string, cidrList []*net.IPNet) bool {
// 先提取纯IP地址
cleanIP := extractIPFromAddress(ip)
parsedIP := net.ParseIP(cleanIP)
if parsedIP == nil {
return false
}
for _, cidr := range cidrList {
if cidr.Contains(parsedIP) {
return true
}
}
return false
}
// GetLimiter 获取指定IP的限流器同时返回是否允许访问
func (i *IPRateLimiter) GetLimiter(ip string) (*rate.Limiter, bool) {
// 提取纯IP地址
cleanIP := extractIPFromAddress(ip)
// 检查是否在黑名单中
if isIPInCIDRList(cleanIP, i.blacklist) {
return nil, false
}
// 检查是否在白名单中
if isIPInCIDRList(cleanIP, i.whitelist) {
return rate.NewLimiter(rate.Inf, i.b), true
}
// 标准化IP用于限流IPv4保持不变IPv6标准化为/64网段
normalizedIP := normalizeIPForRateLimit(cleanIP)
now := time.Now()
i.mu.RLock()
entry, exists := i.ips[normalizedIP]
i.mu.RUnlock()
if exists {
i.mu.Lock()
if entry, stillExists := i.ips[normalizedIP]; stillExists {
entry.lastAccess = now
i.mu.Unlock()
return entry.limiter, true
}
i.mu.Unlock()
}
i.mu.Lock()
if entry, exists := i.ips[normalizedIP]; exists {
entry.lastAccess = now
i.mu.Unlock()
return entry.limiter, true
}
entry = &rateLimiterEntry{
limiter: rate.NewLimiter(i.r, i.b),
lastAccess: now,
}
i.ips[normalizedIP] = entry
i.mu.Unlock()
return entry.limiter, true
}
// RateLimitMiddleware 速率限制中间件
func RateLimitMiddleware(limiter *IPRateLimiter) gin.HandlerFunc {
return func(c *gin.Context) {
// 静态文件豁免:跳过限流检查
path := c.Request.URL.Path
if path == "/" || path == "/favicon.ico" || path == "/images.html" || path == "/search.html" ||
strings.HasPrefix(path, "/public/") {
c.Next()
return
}
// 获取客户端真实IP
var ip string
// 优先尝试从请求头获取真实IP
if forwarded := c.GetHeader("X-Forwarded-For"); forwarded != "" {
// X-Forwarded-For可能包含多个IP取第一个
ips := strings.Split(forwarded, ",")
ip = strings.TrimSpace(ips[0])
} else if realIP := c.GetHeader("X-Real-IP"); realIP != "" {
// 如果有X-Real-IP头
ip = realIP
} else if remoteIP := c.GetHeader("X-Original-Forwarded-For"); remoteIP != "" {
// 某些代理可能使用此头
ips := strings.Split(remoteIP, ",")
ip = strings.TrimSpace(ips[0])
} else {
// 回退到ClientIP方法
ip = c.ClientIP()
}
// 提取纯IP地址去除可能存在的端口
cleanIP := extractIPFromAddress(ip)
// 日志记录请求IP和头信息
normalizedIP := normalizeIPForRateLimit(cleanIP)
if cleanIP != normalizedIP {
fmt.Printf("请求IP: %s (提纯后: %s, 限流段: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
ip, cleanIP, normalizedIP,
c.GetHeader("X-Forwarded-For"),
c.GetHeader("X-Real-IP"))
} else {
fmt.Printf("请求IP: %s (提纯后: %s), X-Forwarded-For: %s, X-Real-IP: %s\n",
ip, cleanIP,
c.GetHeader("X-Forwarded-For"),
c.GetHeader("X-Real-IP"))
}
// 获取限流器并检查是否允许访问
ipLimiter, allowed := limiter.GetLimiter(cleanIP)
// 如果IP在黑名单中
if !allowed {
c.JSON(403, gin.H{
"error": "您已被限制访问",
})
c.Abort()
return
}
// 检查限流
if !ipLimiter.Allow() {
c.JSON(429, gin.H{
"error": "请求频率过快,暂时限制访问",
})
c.Abort()
return
}
c.Next()
}
}

View File

@@ -66,14 +66,21 @@ type Image struct {
Size int64 `json:"size"`
}
// TagPageResult 分页标签结果
type TagPageResult struct {
Tags []TagInfo `json:"tags"`
HasMore bool `json:"has_more"`
}
type cacheEntry struct {
data interface{}
timestamp time.Time
expiresAt time.Time // 存储过期时间
}
const (
maxCacheSize = 1000 // 最大缓存条目数
cacheTTL = 30 * time.Minute
maxCacheSize = 1000 // 最大缓存条目数
maxPaginationCache = 200 // 分页缓存最大条目数
cacheTTL = 30 * time.Minute
)
type Cache struct {
@@ -98,7 +105,8 @@ func (c *Cache) Get(key string) (interface{}, bool) {
return nil, false
}
if time.Since(entry.timestamp) > cacheTTL {
// 比较过期时间
if time.Now().After(entry.expiresAt) {
c.mu.Lock()
delete(c.data, key)
c.mu.Unlock()
@@ -109,40 +117,36 @@ func (c *Cache) Get(key string) (interface{}, bool) {
}
func (c *Cache) Set(key string, data interface{}) {
c.SetWithTTL(key, data, cacheTTL)
}
func (c *Cache) SetWithTTL(key string, data interface{}, ttl time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
now := time.Now()
for k, v := range c.data {
if now.Sub(v.timestamp) > cacheTTL {
delete(c.data, k)
}
}
// 惰性清理:仅在容量超限时清理过期项
if len(c.data) >= c.maxSize {
toDelete := len(c.data) / 4
for k := range c.data {
if toDelete <= 0 {
break
}
delete(c.data, k)
toDelete--
}
c.cleanupExpiredLocked()
}
// 计算过期时间
c.data[key] = cacheEntry{
data: data,
timestamp: now,
expiresAt: time.Now().Add(ttl),
}
}
func (c *Cache) Cleanup() {
c.mu.Lock()
defer c.mu.Unlock()
c.cleanupExpiredLocked()
}
// cleanupExpiredLocked 清理过期缓存(需要已持有锁)
func (c *Cache) cleanupExpiredLocked() {
now := time.Now()
for key, entry := range c.data {
if now.Sub(entry.timestamp) > cacheTTL {
if now.After(entry.expiresAt) {
delete(c.data, key)
}
}
@@ -152,6 +156,8 @@ func (c *Cache) Cleanup() {
func init() {
go func() {
ticker := time.NewTicker(5 * time.Minute)
defer ticker.Stop() // 确保ticker资源释放
for range ticker.C {
searchCache.Cleanup()
}
@@ -214,8 +220,43 @@ func filterSearchResults(results []Repository, query string) []Repository {
return filtered
}
// normalizeRepository 统一规范化仓库信息(消除重复逻辑)
func normalizeRepository(repo *Repository) {
if repo.IsOfficial {
repo.Namespace = "library"
if !strings.Contains(repo.Name, "/") {
repo.Name = "library/" + repo.Name
}
} else {
// 处理用户仓库设置命名空间但保持Name为纯仓库名
if repo.Namespace == "" && repo.RepoOwner != "" {
repo.Namespace = repo.RepoOwner
}
// 如果Name包含斜杠提取纯仓库名
if strings.Contains(repo.Name, "/") {
parts := strings.Split(repo.Name, "/")
if len(parts) > 1 {
if repo.Namespace == "" {
repo.Namespace = parts[0]
}
repo.Name = parts[len(parts)-1] // 取最后部分作为仓库名
}
}
}
}
// searchDockerHub 搜索镜像
func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*SearchResult, error) {
return searchDockerHubWithDepth(ctx, query, page, pageSize, 0)
}
// searchDockerHubWithDepth 搜索镜像(带递归深度控制)
func searchDockerHubWithDepth(ctx context.Context, query string, page, pageSize int, depth int) (*SearchResult, error) {
// 防止无限递归最多允许1次递归调用
if depth > 1 {
return nil, fmt.Errorf("搜索请求过于复杂,请尝试更具体的关键词")
}
cacheKey := fmt.Sprintf("search:%s:%d:%d", query, page, pageSize)
// 尝试从缓存获取
@@ -264,11 +305,7 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
if err != nil {
return nil, fmt.Errorf("请求Docker Hub API失败: %v", err)
}
defer func() {
if err := resp.Body.Close(); err != nil {
fmt.Printf("关闭搜索响应体失败: %v\n", err)
}
}()
defer safeCloseResponseBody(resp.Body, "搜索响应体")
body, err := io.ReadAll(resp.Body)
if err != nil {
@@ -281,8 +318,8 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
return nil, fmt.Errorf("请求过于频繁,请稍后重试")
case http.StatusNotFound:
if isUserRepo && namespace != "" {
// 如果用户仓库搜索失败,尝试普通搜索
return searchDockerHub(ctx, repoName, page, pageSize)
// 如果用户仓库搜索失败,尝试普通搜索(递归调用)
return searchDockerHubWithDepth(ctx, repoName, page, pageSize, depth+1)
}
return nil, fmt.Errorf("未找到相关镜像")
case http.StatusBadGateway, http.StatusServiceUnavailable:
@@ -318,18 +355,16 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
for _, repo := range userRepos.Results {
// 如果指定了仓库名,只保留匹配的结果
if repoName == "" || strings.Contains(strings.ToLower(repo.Name), strings.ToLower(repoName)) {
// 确保设置正确的命名空间和名称
// 设置命名空间并使用统一的规范化函数
repo.Namespace = namespace
if !strings.Contains(repo.Name, "/") {
repo.Name = fmt.Sprintf("%s/%s", namespace, repo.Name)
}
normalizeRepository(&repo)
result.Results = append(result.Results, repo)
}
}
// 如果没有找到结果,尝试普通搜索
// 如果没有找到结果,尝试普通搜索(递归调用)
if len(result.Results) == 0 {
return searchDockerHub(ctx, repoName, page, pageSize)
return searchDockerHubWithDepth(ctx, repoName, page, pageSize, depth+1)
}
result.Count = len(result.Results)
@@ -340,23 +375,9 @@ func searchDockerHub(ctx context.Context, query string, page, pageSize int) (*Se
return nil, fmt.Errorf("解析响应失败: %v", err)
}
// 处理搜索结果
// 处理搜索结果:使用统一的规范化函数
for i := range result.Results {
if result.Results[i].IsOfficial {
if !strings.Contains(result.Results[i].Name, "/") {
result.Results[i].Name = "library/" + result.Results[i].Name
}
result.Results[i].Namespace = "library"
} else {
parts := strings.Split(result.Results[i].Name, "/")
if len(parts) > 1 {
result.Results[i].Namespace = parts[0]
result.Results[i].Name = parts[1]
} else if result.Results[i].RepoOwner != "" {
result.Results[i].Namespace = result.Results[i].RepoOwner
result.Results[i].Name = fmt.Sprintf("%s/%s", result.Results[i].RepoOwner, result.Results[i].Name)
}
}
normalizeRepository(&result.Results[i])
}
// 如果是用户/仓库搜索,过滤结果
@@ -394,61 +415,150 @@ func isRetryableError(err error) bool {
return false
}
// getRepositoryTags 获取仓库标签信息
func getRepositoryTags(ctx context.Context, namespace, name string) ([]TagInfo, error) {
// getRepositoryTags 获取仓库标签信息(支持分页)
func getRepositoryTags(ctx context.Context, namespace, name string, page, pageSize int) ([]TagInfo, bool, error) {
if namespace == "" || name == "" {
return nil, fmt.Errorf("无效输入:命名空间和名称不能为空")
return nil, false, fmt.Errorf("无效输入:命名空间和名称不能为空")
}
cacheKey := fmt.Sprintf("tags:%s:%s", namespace, name)
// 默认参数
if page <= 0 {
page = 1
}
if pageSize <= 0 || pageSize > 100 {
pageSize = 100
}
// 分页缓存key
cacheKey := fmt.Sprintf("tags:%s:%s:page_%d", namespace, name, page)
if cached, ok := searchCache.Get(cacheKey); ok {
return cached.([]TagInfo), nil
result := cached.(TagPageResult)
return result.Tags, result.HasMore, nil
}
// 构建API URL
baseURL := fmt.Sprintf("https://registry.hub.docker.com/v2/repositories/%s/%s/tags", namespace, name)
params := url.Values{}
params.Set("page_size", "100")
params.Set("page", fmt.Sprintf("%d", page))
params.Set("page_size", fmt.Sprintf("%d", pageSize))
params.Set("ordering", "last_updated")
fullURL := baseURL + "?" + params.Encode()
// 使用统一的搜索HTTP客户端
resp, err := GetSearchHTTPClient().Get(fullURL)
// 获取当前页数据
pageResult, err := fetchTagPage(ctx, fullURL, 3)
if err != nil {
return nil, fmt.Errorf("发送请求失败: %v", err)
return nil, false, fmt.Errorf("获取标签失败: %v", err)
}
defer func() {
if err := resp.Body.Close(); err != nil {
fmt.Printf("关闭搜索响应体失败: %v\n", err)
hasMore := pageResult.Next != ""
// 缓存结果(分页缓存时间较短)
result := TagPageResult{Tags: pageResult.Results, HasMore: hasMore}
searchCache.SetWithTTL(cacheKey, result, 30*time.Minute)
return pageResult.Results, hasMore, nil
}
// fetchTagPage 获取单页标签数据,带重试机制
func fetchTagPage(ctx context.Context, url string, maxRetries int) (*struct {
Count int `json:"count"`
Next string `json:"next"`
Previous string `json:"previous"`
Results []TagInfo `json:"results"`
}, error) {
var lastErr error
for retry := 0; retry < maxRetries; retry++ {
if retry > 0 {
// 重试前等待一段时间
time.Sleep(time.Duration(retry) * 500 * time.Millisecond)
}
}()
// 读取响应体
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("读取响应失败: %v", err)
}
resp, err := GetSearchHTTPClient().Get(url)
if err != nil {
lastErr = err
if isRetryableError(err) && retry < maxRetries-1 {
continue
}
return nil, fmt.Errorf("发送请求失败: %v", err)
}
// 检查响应状态码
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("请求失败: 状态码=%d, 响应=%s", resp.StatusCode, string(body))
}
// 读取响应体立即关闭避免defer在循环中累积
body, err := func() ([]byte, error) {
defer safeCloseResponseBody(resp.Body, "标签响应体")
return io.ReadAll(resp.Body)
}()
if err != nil {
lastErr = err
if retry < maxRetries-1 {
continue
}
return nil, fmt.Errorf("读取响应失败: %v", err)
}
// 解析响应
var result struct {
Count int `json:"count"`
Next string `json:"next"`
Previous string `json:"previous"`
Results []TagInfo `json:"results"`
}
if err := json.Unmarshal(body, &result); err != nil {
return nil, fmt.Errorf("解析响应失败: %v", err)
}
// 检查响应状态码
if resp.StatusCode != http.StatusOK {
lastErr = fmt.Errorf("状态码=%d, 响应=%s", resp.StatusCode, string(body))
// 4xx错误通常不需要重试
if resp.StatusCode >= 400 && resp.StatusCode < 500 && resp.StatusCode != 429 {
return nil, fmt.Errorf("请求失败: %v", lastErr)
}
if retry < maxRetries-1 {
continue
}
return nil, fmt.Errorf("请求失败: %v", lastErr)
}
// 缓存结果
searchCache.Set(cacheKey, result.Results)
return result.Results, nil
// 解析响应
var result struct {
Count int `json:"count"`
Next string `json:"next"`
Previous string `json:"previous"`
Results []TagInfo `json:"results"`
}
if err := json.Unmarshal(body, &result); err != nil {
lastErr = err
if retry < maxRetries-1 {
continue
}
return nil, fmt.Errorf("解析响应失败: %v", err)
}
return &result, nil
}
return nil, lastErr
}
// parsePaginationParams 解析分页参数
func parsePaginationParams(c *gin.Context, defaultPageSize int) (page, pageSize int) {
page = 1
pageSize = defaultPageSize
if p := c.Query("page"); p != "" {
fmt.Sscanf(p, "%d", &page)
}
if ps := c.Query("page_size"); ps != "" {
fmt.Sscanf(ps, "%d", &pageSize)
}
return page, pageSize
}
// safeCloseResponseBody 安全关闭HTTP响应体统一资源管理
func safeCloseResponseBody(body io.ReadCloser, context string) {
if body != nil {
if err := body.Close(); err != nil {
fmt.Printf("关闭%s失败: %v\n", context, err)
}
}
}
// sendErrorResponse 统一错误响应处理
func sendErrorResponse(c *gin.Context, message string) {
c.JSON(http.StatusBadRequest, gin.H{"error": message})
}
// RegisterSearchRoute 注册搜索相关路由
@@ -457,22 +567,15 @@ func RegisterSearchRoute(r *gin.Engine) {
r.GET("/search", func(c *gin.Context) {
query := c.Query("q")
if query == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "搜索关键词不能为空"})
sendErrorResponse(c, "搜索关键词不能为空")
return
}
page := 1
pageSize := 25
if p := c.Query("page"); p != "" {
fmt.Sscanf(p, "%d", &page)
}
if ps := c.Query("page_size"); ps != "" {
fmt.Sscanf(ps, "%d", &pageSize)
}
page, pageSize := parsePaginationParams(c, 25)
result, err := searchDockerHub(c.Request.Context(), query, page, pageSize)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
sendErrorResponse(c, err.Error())
return
}
@@ -485,16 +588,27 @@ func RegisterSearchRoute(r *gin.Engine) {
name := c.Param("name")
if namespace == "" || name == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "命名空间和名称不能为空"})
sendErrorResponse(c, "命名空间和名称不能为空")
return
}
tags, err := getRepositoryTags(c.Request.Context(), namespace, name)
page, pageSize := parsePaginationParams(c, 100)
tags, hasMore, err := getRepositoryTags(c.Request.Context(), namespace, name, page, pageSize)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
sendErrorResponse(c, err.Error())
return
}
c.JSON(http.StatusOK, tags)
if c.Query("page") != "" || c.Query("page_size") != "" {
c.JSON(http.StatusOK, gin.H{
"tags": tags,
"has_more": hasMore,
"page": page,
"page_size": pageSize,
})
} else {
c.JSON(http.StatusOK, tags)
}
})
}

View File

@@ -1,108 +0,0 @@
package main
import (
"strings"
"sync"
"time"
)
// SmartRateLimit 智能限流会话管理
type SmartRateLimit struct {
sessions sync.Map
}
// PullSession Docker拉取会话
type PullSession struct {
LastManifestTime time.Time
RequestCount int
}
// 全局智能限流实例
var smartLimiter = &SmartRateLimit{}
const (
// manifest请求后的活跃窗口时间
activeWindowDuration = 3 * time.Minute
// 活跃窗口内最大免费blob请求数(防止滥用)
maxFreeBlobRequests = 100
sessionCleanupInterval = 10 * time.Minute
sessionExpireTime = 30 * time.Minute
)
func init() {
go smartLimiter.cleanupSessions()
}
// ShouldSkipRateLimit 判断是否应该跳过限流计数
func (s *SmartRateLimit) ShouldSkipRateLimit(ip, path string) bool {
requestType, _ := parseRequestInfo(path)
if requestType != "manifests" && requestType != "blobs" {
return false
}
sessionKey := ip
sessionInterface, _ := s.sessions.LoadOrStore(sessionKey, &PullSession{})
session := sessionInterface.(*PullSession)
now := time.Now()
if requestType == "manifests" {
session.LastManifestTime = now
session.RequestCount = 0
return false
}
if requestType == "blobs" {
if !session.LastManifestTime.IsZero() &&
now.Sub(session.LastManifestTime) <= activeWindowDuration {
session.RequestCount++
if session.RequestCount <= maxFreeBlobRequests {
return true
}
}
}
return false
}
func parseRequestInfo(path string) (requestType, imageRef string) {
path = strings.TrimPrefix(path, "/v2/")
if idx := strings.Index(path, "/manifests/"); idx != -1 {
return "manifests", path[:idx]
}
if idx := strings.Index(path, "/blobs/"); idx != -1 {
return "blobs", path[:idx]
}
if idx := strings.Index(path, "/tags/"); idx != -1 {
return "tags", path[:idx]
}
return "unknown", ""
}
// cleanupSessions 定期清理过期会话,防止内存泄露
func (s *SmartRateLimit) cleanupSessions() {
ticker := time.NewTicker(sessionCleanupInterval)
defer ticker.Stop()
for range ticker.C {
now := time.Now()
expiredKeys := make([]string, 0)
s.sessions.Range(func(key, value interface{}) bool {
session := value.(*PullSession)
if !session.LastManifestTime.IsZero() &&
now.Sub(session.LastManifestTime) > sessionExpireTime {
expiredKeys = append(expiredKeys, key.(string))
}
return true
})
for _, key := range expiredKeys {
s.sessions.Delete(key)
}
}
}

View File

@@ -13,10 +13,10 @@ import (
// CachedItem 通用缓存项支持Token和Manifest
type CachedItem struct {
Data []byte // 缓存数据(token字符串或manifest字节)
ContentType string // 内容类型
Data []byte // 缓存数据(token字符串或manifest字节)
ContentType string // 内容类型
Headers map[string]string // 额外的响应头
ExpiresAt time.Time // 过期时间
ExpiresAt time.Time // 过期时间
}
// UniversalCache 通用缓存支持Token和Manifest
@@ -71,14 +71,6 @@ func buildManifestCacheKey(imageRef, reference string) string {
return buildCacheKey("manifest", key)
}
func buildManifestCacheKeyWithPlatform(imageRef, reference, platform string) string {
if platform == "" {
platform = "default"
}
key := fmt.Sprintf("%s:%s@%s", imageRef, reference, platform)
return buildCacheKey("manifest", key)
}
func getManifestTTL(reference string) time.Duration {
cfg := GetConfig()
defaultTTL := 30 * time.Minute
@@ -87,19 +79,18 @@ func getManifestTTL(reference string) time.Duration {
defaultTTL = parsed
}
}
if strings.HasPrefix(reference, "sha256:") {
return 24 * time.Hour
}
// mutable tag的智能判断
if reference == "latest" || reference == "main" || reference == "master" ||
reference == "dev" || reference == "develop" {
if reference == "latest" || reference == "main" || reference == "master" ||
reference == "dev" || reference == "develop" {
// 热门可变标签: 短期缓存
return 10 * time.Minute
}
// 普通tag: 中等缓存时间
return defaultTTL
}
@@ -108,17 +99,17 @@ func extractTTLFromResponse(responseBody []byte) time.Duration {
var tokenResp struct {
ExpiresIn int `json:"expires_in"`
}
// 默认30分钟TTL确保稳定性
defaultTTL := 30 * time.Minute
if json.Unmarshal(responseBody, &tokenResp) == nil && tokenResp.ExpiresIn > 0 {
safeTTL := time.Duration(tokenResp.ExpiresIn-300) * time.Second
if safeTTL > 5*time.Minute {
return safeTTL
}
}
return defaultTTL
}
@@ -131,12 +122,12 @@ func writeCachedResponse(c *gin.Context, item *CachedItem) {
if item.ContentType != "" {
c.Header("Content-Type", item.ContentType)
}
// 设置额外的响应头
for key, value := range item.Headers {
c.Header(key, value)
}
// 返回数据
c.Data(200, item.ContentType, item.Data)
}
@@ -150,4 +141,28 @@ func isCacheEnabled() bool {
// isTokenCacheEnabled 检查token缓存是否启用(向后兼容)
func isTokenCacheEnabled() bool {
return isCacheEnabled()
}
}
// 定期清理过期缓存,防止内存泄漏
func init() {
go func() {
ticker := time.NewTicker(20 * time.Minute)
defer ticker.Stop()
for range ticker.C {
now := time.Now()
expiredKeys := make([]string, 0)
globalCache.cache.Range(func(key, value interface{}) bool {
if cached := value.(*CachedItem); now.After(cached.ExpiresAt) {
expiredKeys = append(expiredKeys, key.(string))
}
return true
})
for _, key := range expiredKeys {
globalCache.cache.Delete(key)
}
}
}()
}