Architecture
Design Architecture
设计架构
OpenList Worker is a Serverless-first rewrite of the OpenList Go backend in TypeScript. The system is divided into three layers: edge runtime, data access, and frontend static assets.
OpenList Worker 是将 OpenList Go 后端以 TypeScript 重写的 Serverless 优先架构。系统分为边缘运行时、数据访问和前端静态资源三层。
Tech Stack
技术栈
Backend
后端
| Component | Technology | Description |
|---|---|---|
| HTTP framework | Hono.js | Lightweight, edge-native web framework |
| Runtime | Cloudflare Workers / EdgeOne Functions / ESA | Edge compute platforms |
| Language | TypeScript | Fully typed, compiled via esbuild / Vite |
| ORM | Drizzle ORM | Type-safe SQL query builder for D1 / MySQL |
| Build tool | esbuild / Vite | Single-file Worker bundle |
| 组件 | 技术 | 说明 |
|---|---|---|
| HTTP 框架 | Hono.js | 轻量级、边缘原生 Web 框架 |
| 运行时 | Cloudflare Workers / EdgeOne 云函数 / ESA | 边缘计算平台 |
| 语言 | TypeScript | 全类型，使用 esbuild / Vite 编译 |
| ORM | Drizzle ORM | 为 D1 / MySQL 提供类型安全的 SQL 查询构建器 |
| 构建工具 | esbuild / Vite | 单文件 Worker 产物 |
Frontend
前端
| Component | Technology | Description |
|---|---|---|
| Framework | SolidJS + TypeScript | Reactive SPA frontend |
| UI library | Hope UI (@hope-ui/solid) | Component library |
| Build tool | Vite | Fast frontend build |
| Bundled with | Workers Static Assets | Served from the same origin as the API |
| 组件 | 技术 | 说明 |
|---|---|---|
| 框架 | SolidJS + TypeScript | 响应式 SPA 前端 |
| UI 库 | Hope UI（@hope-ui/solid） | 组件库 |
| 构建工具 | Vite | 快速前端构建 |
| 与 Worker 同源 | Workers Static Assets | API 与前端同源部署，无跨域问题 |
Data Storage
数据存储
Storage Format (DB_FORMAT)
存储格式（DB_FORMAT）
The DB_FORMAT variable controls how data is serialized:
| Value | Description | Best for |
|---|---|---|
map (default) | Whole object serialized as a single JSON value | KV / Blob storage |
key | Per-key storage, one record per entity | KV with high read frequency |
sql | Relational tables, identical schema to the Go backend | D1 / MySQL — enables Go ↔ TS migration |
DB_FORMAT 控制数据序列化方式：
| 值 | 说明 | 最适用场景 |
|---|---|---|
map（默认） | 整对象序列化为单个 JSON 值 | KV / Blob 存储 |
key | 分 key 存储，每实体一条记录 | 高频读写 KV |
sql | 关系表，与 Go 后端 schema 完全一致 | D1 / MySQL——支持 Go ↔ TS 迁移 |
Storage Driver (DB_DRIVER)
存储驱动（DB_DRIVER）
The DB_DRIVER variable selects the physical storage backend:
| Value | Platform | Description |
|---|---|---|
auto (default) | Universal | Auto-detect: blob → cfkv → kv → d1 → memory |
blob | EdgeOne / ESA | EdgeOne Blob or Alibaba ESA Blob |
cfkv | Universal | Cloudflare KV via REST API (cross-platform) |
kv | Cloudflare Workers | Cloudflare KV binding |
d1 | Cloudflare Workers | Cloudflare D1 (SQLite) |
do | Cloudflare Workers | Durable Objects (strong consistency) |
mysql | Node.js container | External MySQL / MariaDB |
DB_DRIVER 选择物理存储后端：
| 值 | 平台 | 说明 |
|---|---|---|
auto（默认） | 通用 | 自动检测：blob → cfkv → kv → d1 → memory |
blob | EdgeOne / ESA | EdgeOne Blob 或阿里云 ESA Blob |
cfkv | 通用 | Cloudflare KV REST API（跨平台远程调用） |
kv | Cloudflare Workers | Cloudflare KV 绑定 |
d1 | Cloudflare Workers | Cloudflare D1（SQLite） |
do | Cloudflare Workers | Durable Objects（强一致性） |
mysql | Node.js 容器 | 外部 MySQL / MariaDB |
SQL Table Alignment with Go Backend
与 Go 后端的 SQL 表对齐
When DB_FORMAT = "sql", the TS Worker uses the same table names and schema as the Go backend (GORM, default prefix x_), so the two backends can share the same physical database:
| Go struct | Table name |
|---|---|
SettingItem | x_setting_items |
SharingDB | x_sharing_dbs |
Storage | x_storages |
User | x_users |
Meta | x_metas |
| (TS only) | x_plugins |
| (TS runtime) | x_singleflight |
The prefix is fixed to x_, matching the Go backend default.
x_singleflight is a runtime table used for cross-isolate request deduplication (see below). It is created automatically by the SQL driver on initialization, holds no configuration, and is intentionally excluded from the configuration tables — the Go backend has no such table, so sharing a physical database is unaffected.
当 DB_FORMAT = "sql" 时，TS Worker 使用与 Go 后端（GORM，默认前缀 x_）相同的表名与 schema，两个后端可共享同一物理数据库：
| Go 结构体 | 表名 |
|---|---|
SettingItem | x_setting_items |
SharingDB | x_sharing_dbs |
Storage | x_storages |
User | x_users |
Meta | x_metas |
| （仅 TS） | x_plugins |
| （TS 运行时） | x_singleflight |
表名前缀固定为 x_，与 Go 后端默认值一致。
x_singleflight 是用于跨 isolate 请求去重的运行时表（见下文）。它由 SQL 驱动在初始化时自动创建，不承载任何配置，并被刻意排除在配置表之外——Go 后端没有这张表，因此不影响共享同一物理数据库。
Concurrent Deduplication (singleflight)
并发去重（singleflight）
A single deployment runs across multiple isolates, so an in-process map alone cannot merge duplicate calls across instances. Every concurrent request for the same directory listing or metadata would then hit the upstream cloud-drive API separately — a major source of rate limiting, account bans and CPU timeouts. OpenList Worker deduplicates these calls at two levels:
| Level | Mechanism | Effective when |
|---|---|---|
| L1 — in-process | Map<key, Promise> merge | All modes |
| L2 — cross-instance | Grab a lock in the x_singleflight table and share the result | db mode only |
The mode is controlled by SINGLEFLIGHT (auto / db / memory / off, default auto): when a SQL backend (d1 / mysql / do) is available the table is used for coordination, otherwise it falls back to in-process merging.
Design notes:
- Locking without transactions:
INSERT OR IGNORE(MySQL:INSERT IGNORE) plus a read-back ofownerto confirm ownership. - Heartbeat renewal: the executor renews the lock every
lockTtl / 3; if it crashes, the lock expires and another instance can take it over. - Waiting side: polls for the result; on timeout it executes the call itself, so requests are never left hanging.
- No result caching: only calls that observe a running executor on arrival reuse the result; a newly arrived call reclaims the old record and re-executes, which rules out stale reads such as "a deleted file still shows up after refresh".
- Fault isolation: exceptions thrown by the business function are rethrown as-is without retry; coordination-layer failures (missing table, DB timeout, non-serializable result, executor crash) all degrade to "each caller executes once", so the business call never fails.
Integration scope: listItems / getItem in the storage operation layer, with the dedup key fs.list|<storage.id>:<storage.modified>:<virtualPath>. One integration therefore covers /fs/list, /fs/get, /fs/dir, search, WebDAV and MCP. Runtime statistics are exposed in the singleflight field of /api/debug/info.
一个部署会同时运行多个 isolate，仅靠进程内的 Map 无法跨实例合并重复调用，导致同一目录列表 / 元信息的并发请求各自打一次上游网盘 API——这正是限流、封号、CPU 超时的主要来源。OpenList Worker 对这类调用做两级去重：
| 层级 | 机制 | 生效条件 |
|---|---|---|
| L1 进程内 | Map<key, Promise> 合并 | 所有模式 |
| L2 跨实例 | 在 x_singleflight 表中抢锁并共享结果 | 仅 db 模式 |
模式由 SINGLEFLIGHT 控制（auto / db / memory / off，默认 auto）：有 SQL 库（d1 / mysql / do）时用表协调，否则退回进程内合并。
设计要点：
- 不依赖事务抢锁：
INSERT OR IGNORE（MySQL 为INSERT IGNORE）+ 回读owner确认归属。 - 心跳续租：执行者按
lockTtl / 3周期续租；崩溃后锁到期，可被其他实例接管。 - 等待方：轮询读取结果，超时后自行执行，不会把请求挂死。
- 不做结果缓存：只有「到达时观察到执行者正在运行」的调用才复用结果；新到达的调用会回收旧记录并重新执行，因此不会出现「删除文件后刷新仍看到被删文件」这类脏读。
- 故障隔离：业务函数抛出的异常原样抛出且不重试；协调层异常（表缺失 / 数据库超时 / 结果不可序列化 / 执行者崩溃）一律降级为「各自执行一次」，业务不会失败。
接入范围：存储操作层的 listItems / getItem，去重键为 fs.list|<storage.id>:<storage.modified>:<virtualPath>。因此一次接入即覆盖 /fs/list、/fs/get、/fs/dir、搜索、WebDAV 与 MCP。运行统计见 /api/debug/info 的 singleflight 字段。
Project Structure
项目结构
OpenList-Worker/
├── src/
│ ├── backend/ # Hono.js Worker entry & backend logic
│ │ ├── worker.ts # Cloudflare Workers entry
│ │ ├── drivers/ # Storage driver implementations (kv / d1 / blob / mysql …)
│ │ ├── server/ # Route registrations & middleware
│ │ ├── pkg/ # Shared utilities & helpers
│ │ └── internal/ # Core business logic (auth, storage, meta, …)
│ └── frontend/ # Built-in frontend (SolidJS + Vite)
├── dist/ # Build output (Worker bundle + frontend assets)
├── esa-entry.ts # Alibaba Cloud ESA entry
├── wrangler.toml # Cloudflare Workers configuration
├── esa.jsonc # Alibaba Cloud ESA configuration
├── edgeone.json # EdgeOne schedules configuration
└── package.jsonOpenList-Worker/
├── src/
│ ├── backend/ # Hono.js Worker 入口与后端逻辑
│ │ ├── worker.ts # Cloudflare Workers 入口
│ │ ├── drivers/ # 存储驱动实现（kv / d1 / blob / mysql …）
│ │ ├── server/ # 路由注册与中间件
│ │ ├── pkg/ # 公共工具与辅助函数
│ │ └── internal/ # 核心业务逻辑（认证、存储、元数据 …）
│ └── frontend/ # 内置前端（SolidJS + Vite）
├── dist/ # 构建产物（Worker bundle + 前端资源）
├── esa-entry.ts # 阿里云 ESA 入口
├── wrangler.toml # Cloudflare Workers 配置
├── esa.jsonc # 阿里云 ESA 配置
├── edgeone.json # EdgeOne 定时任务配置
└── package.jsonSupported Platforms
支持平台
| Platform | Entry | Persistence | Notes |
|---|---|---|---|
| Cloudflare Workers | worker.ts | D1 / KV / DO | One-click deploy supported |
| Tencent Cloud EdgeOne | worker.ts | Blob / KV | One-click deploy supported |
| Alibaba Cloud ESA | esa-entry.ts | EdgeKV | Manual build & deploy |
| Node.js container | worker.ts (with adapter) | MySQL | Self-hosted |
| 平台 | 入口 | 持久化 | 备注 |
|---|---|---|---|
| Cloudflare Workers | worker.ts | D1 / KV / DO | 支持一键部署 |
| 腾讯云 EdgeOne | worker.ts | Blob / KV | 支持一键部署 |
| 阿里云 ESA | esa-entry.ts | EdgeKV | 手动构建部署 |
| Node.js 容器 | worker.ts（带适配层） | MySQL | 自托管 |