← All articles

Don't Let the 'System Perspective' Take You Hostage: An Independent Developer's Blood-and-Tears Rebuild of a 47GB Physical Asset Library

Why does engineer thinking always produce a parallel world where 'image host backup' and 'image assets' coexist? A field report on rebuilding assets, from a USB drive as the single source of truth to a win for both machines and humans.

Don't Let the 'System Perspective' Take You Hostage: An Independent Developer's Blood-and-Tears Rebuild of a 47GB Physical Asset Library
On this page 4

Today I opened the 2TB high-speed portable SSD plugged into my Mac (mount point /Volumes/AI素材资源) to pick a few Chinese-style character shots for the next video edit.

The moment I saw the root directory, I froze: two folders were sitting side by side at the top level — one called 图片素材/ (2.0 GB), the other 图床备份/ (11.0 GB).

I opened 图片素材: inside were agent头像, 山鬼映画, 好意美, logo, picx; then I opened 图床备份: inside were avatars, brand/shangui, hym, gallery, brand/studio.

To find a Shangui Studio brand logo I had to hop back and forth between two big directories; for an agent avatar, both sides had a batch and I couldn’t tell which one was current.

I paid real money for this drive and did the asset consolidation precisely so that content would be classified by business, in one place. So why did I end up with two parallel freaks instead?


Post-mortem: How Engineers Turned the Assets into a Junk Pile, Step by Step

Digging through the Git history and the automation script logs was what finally reconstructed the crime scene:

  1. Phase one (natural human classification): When the drive was initialized, it was classified by the media types a human creator finds most obvious:

    • 🎵 音乐作品/ (original music masters)
    • 🔊 音频素材/ (BGM and sound-effects library)
    • 🎬 视频素材/ (finished-video footage)
    • 🔤 字体素材/ (local font assets)
    • 🧠 开源模型/ (LLM weights)
    • 🖼 图片素材/ (all sorts of images for local use)

    Nothing about that structure felt wrong, and every image naturally lived under 图片素材/.

  2. Phase two (the system’s lazy intervention): Later, to protect against losing the cloud copy, I wrote an automated sync command, cs resource sync-usb, that pulls data from Cloudflare R2 object storage back down to local. To save himself trouble, the engineer writing the script mapped directories straight off the cloud S3 bucket names:

    • The R2 bucket was named voxflow-backup → create a local voxflow备份/;
    • The R2 bucket was named fonts → pull it into 字体素材/;
    • The R2 bucket was named images → the engineer had a brain fart and pulled a brand-new directory straight into the USB root: 图床备份/!
  3. The disaster: From then on, the same class of assets was hacked into two halves on the physical drive:

    • Images that came from the cloud all lived in 图床备份/ (English keys: avatars/, brand/, characters/);
    • Images organized locally all piled up in 图片素材/ (Chinese names: agent头像/, 山鬼映画/, 好意美/).

This is the textbook case of classifying lazily by physical provenance instead of by business content.


The Core Argument: Business Mental Model > Physical Provenance

Asset management is like database design: you have to answer one fundamental question first — who does this drive actually serve?

On this drive I am the creator and the AI is my automated editing assistant. The mental path we both take when looking for assets is:

  • “I’m making a new Shangui Studio video — find its brand assets and Chinese-style character art”;
  • “I’m publishing a blog post — find the header image for it”;
  • “I’m setting up an agent — find its dedicated avatar”.

Nobody looking for an asset ever thinks: “do I want the image that came from the Cloudflare R2 bucket, or the image that came from the local disk?”

Provenance (cloud vs. local, online delivery vs. offline archive) is only a physical channel; content (which IP it belongs to, which character, what it is used for) is the business source of truth. Slicing directories along the physical channel makes every user pay for the underlying technical implementation.


The Rebuild: Killing the Parallel Worlds for Good

Riding the anger of the moment, I went straight to surgery:

1. Consolidate the Root Directory

Delete the 图床备份/ shell at the root entirely! The top level may only ever hold genuine media types:

/Volumes/AI素材资源/
├── 📁 图片素材/      # 13.08 GB / 6,875 个文件(所有图片资产的唯一真源)
├── 📁 视频素材/      #  6.62 GB /   247 个文件
├── 📁 开源模型/      # 23.29 GB /    90 个文件
├── 📁 音频素材/      #  0.90 GB /    48 个文件
├── 📁 音乐作品/      #  0.34 GB /    28 个文件
├── 📁 字体素材/      #  0.16 GB /    24 个文件
└── 📁 tools/         # 极速检索工具链

2. Deep Content Merge and Deduplication

Merge the old 图床备份 and 图片素材 into one tree along business lines:

  • Avatars consolidated: the old agent头像/ and avatars/ merged into 图片素材/avatars/, deduplicated automatically by SHA-256;
  • Brands consolidated: 山鬼映画 moved to brand/shangui/, 好意美 to brand/hym/, webkubor to brand/studio/, logo to brand/logo/;
  • Gallery wallpapers: shangui-xhs-gallery/ and gallery/ merged;
  • Character art: 光影-白T/ and 表情包系列/ moved under characters/;
  • Clear out the junk: permanently delete the pointless 桌面截图/, and disable the rogue cron job that silently shuttled screenshots around in the background.

3. Rewrite the Underlying Sync Loop (resource_sync_usb.go)

Rewrite the sync engine:

// 修复前:机械按桶名在根目录建文件夹
{"images", "图床备份", "图像资产全量"},

// 修复后:直接将全量图片对齐到图片唯一真源
{"images", "图片素材", "图像资产全量(合入图片素材唯一真源目录)"},

Then run cs resource sync-usb once more:

── images → 图片素材
   图像资产全量(合入图片素材唯一真源目录)
   ✅ 5841 个文件一致
✅ R2 上的资产 U 盘全都有。

New images added in the cloud now merge incrementally into the matching business directory, and the root will never grow a second deformed folder.


Summary: The Independent Developer’s Laws of Asset Management

  1. The USB drive is master, the cloud is servant: physical hardware is the source of truth; the cloud is only a distribution copy. Never let the cloud’s naming structure leak back and contaminate your physical archive.
  2. Refuse to create directories by provenance: don’t create trash-can directories like “Baidu Netdisk downloads/”, “WeChat received/”, “image host sync/”. From day one in the asset library, everything files under its business line.
  3. A win for both machines and humans: the AI editing pipeline needs stable directory paths to consume assets, and the human creator needs intuitive business categories. One local command, rebuild.sh, generates a single-file offline HTML page (INDEX.html) with millisecond search — that is what genuinely sustainable personal digital asset infrastructure looks like.
🚀相关开源工具与实践
读完不过瘾?把文中思考变成可落地的生产力
🧭

path-guard

开发工具

查出你敲的命令实际跑的是哪个副本、被谁遮蔽、更新到底生效没有。

🖥️

dsh-env-inspector

DSH 工具

DSH 电脑环境自检:端口释放、CLI工具链、局域网与秘钥安全探测。

🔐

kyvault

安全底座

本地加密密钥管理 CLI —— AES-256-GCM 存储,AI 只能拿到别名注入,永不泄露明文。

webkubor

关于作者 · 栖洲 (webkubor)

把审美做成工程 —— 经得起生产环境检验的 AI 系统、独立开发工作流与极客主题。