Backing Up 39 Products And CSVs On A 2 GB Server
Learn how to back up 39 digital products and CSV files on a 1 vCPU and 2 GB RAM server after losing a full evening of work in March.
Stock photo for illustration only, not from the actual event
- Assuming cloud host backups were active led to losing an entire evening of work last March.
- Splitting 39 digital products totaling 856 MB into three distinct tiers established a reliable backup schedule.
- CSV files and upload tracking scripts are just as vital as final assets because they encode critical editorial decisions.
- Running archive jobs on a 1 vCPU and 2 GB RAM box requires careful sequencing to avoid out-of-memory crashes.
I lost an entire evening last March because I mistakenly assumed my host's automated backups were covering my data. The nightly snapshot on my 1 vCPU and 2 GB RAM box only retained records for seven days, but I desperately needed a CSV file from eleven days prior. That specific file was the Adobe 5-column upload sheet for a batch of 40 assets I had already submitted. Rebuilding it meant re-running make_metadata.py against a directory of source files that I had since renamed. Recovering it was only possible because I kept a messy, uncleaned system folder. That single incident transformed backup from an afterthought into a core part of my pipeline design.
My catalogue consists of 39 finished products, translating to 856 MB of packed output data. That specific number dictates the entire strategy. At 856 MB, the output is small enough to version control or sync efficiently, but the underlying raw material is not. The parent project contains approximately 19,000 lines of pipeline code and over 300 pages of research notes. If you only back up the resulting zip files, you entirely lose the capability to regenerate or repackage any of the assets.
Stock photo for illustration only, not from the actual event
Consequently, I split the product catalogue into three distinct tiers:
- Tier One: Research notes and raw source inputs
- Tier Two: Pipeline code and automation scripts
- Tier Three: Ready-to-submit output files and final product zip packs
My previous mistake was treating every component identically and ultimately backing up none of them correctly. Creators frequently preserve their visual artwork while completely neglecting metadata, which is entirely backwards. Artwork exists as standalone files on disk, whereas metadata encodes critical decisions: specific titles, keywords, and upload sorting orders. Stock platforms reject incorrect CSV headers instantly. Adobe demands a 5-column structure, Vecteezy requires 4 columns, and Dreamstime asks for 15 columns. The metadata generation script writes the exact header format each platform requires.
Operating a lightweight server environment with tight hardware constraints like 1 vCPU and 2 GB of RAM highlights the importance of minimalist software architecture. Solo developers and digital creators benefit greatly from writing custom, lightweight scripts with minimal external dependencies. Understanding kernel constraints prevents fatal OOM (Out of Memory) kills when heavy archiving tasks coincide with automated rendering processes.
The exact hardware constraints governing rendering also dictate backup protocols. Having only one vCPU means executing processes strictly one at a time, including archive compression jobs. Running a tar or rsync utility while a render is actively in flight risks triggering an out-of-memory crash and corrupting the archive. To manage this safely, I operate three separate routines: a daily commit of trackers, CSVs, and notes; a weekly sync of product packs; and a monthly verified restore drill.
Stock photo for illustration only, not from the actual event
What I deliberately choose not to maintain is a hot disaster recovery site capable of running the entire pipeline instantly. If the virtual server fails completely, I can restore every product and CSV to resume selling, but provisioning a fresh 1 vCPU runtime environment still takes an afternoon of setup. For a solo operation managing 39 products, that trade-off is honest, pragmatic, and acceptable. All four core Python scripts, along with the folder architecture, are packaged into my compliance metadata kit.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment