Rethinking Digital Storage for Community-Based Archives

ASF@5 Convening in New Orleans, Nov 2023

Over the last few decades, the standard procedure for storing and backing up data has been ‘put it in the cloud.’ Many of us use cloud storage every day whether or not we think about it. Our social media data is kept on servers that are run and managed by a big corporation. Whenever we upload photos to facebook or instagram, or share documents through Google Drive, we are increasing our reliance on cloud storage. For digital archives, this not only raises operational costs but also poses risk to collections — exposing potentially sensitive material to government surveillance, corporate censorship, data monetization, hacks and AI training.

Over the past three years, a diverse team led by the SHIFT Collective and funded by the Filecoin Foundation has been working to develop an alternative: Community-Cloud-Storage: A private, self-hosted distributed storage model built for the unique needs of community-based archives.

What is Community-Cloud-Storage?

Community-Cloud-Storage is a way for organizations to keep their digital records on their own hardware — machines you own that sit on a shelf — rather than renting space from a tech giant. The intention of this system is to return digital chain of custody to communities. Instead of your files sitting in a distant data center, data is encrypted and shared across a private network of “nodes” — small servers owned and operated by trusted partner organizations.

Because it is built on a private network, it is invisible to the public web and shielded from the prying eyes of data miners and automated AI scanners. It is Virtual Private Network (VPN) infrastructure built on mutual trust rather than a corporate contract, providing a safe, accessible home for sensitive stories.

Why Now:

We are at a tipping point where “digital convenience” has become a trap for community memory. For archives and ongoing movements, the corporate cloud is no longer a neutral storage locker — it is a liability.

When we rely on corporate giants, we are subject to their terms: data can be subpoenaed without our knowledge, access can be throttled by shifting paywalls, and our most precious and personal histories are now being harvested to “train” models without consent. For many under-represented communities, this isn’t a technicality; it is a continuation of extraction.

Here at the Warrior Women Project, an archive accountable to living narrators and ongoing movements, we have real concerns about where our materials live, who controls them, and under what conditions they can be accessed and cited. Our current workflow relies on a patchwork of platforms: Iconik and Google for remote team workflow, Vimeo and Wordpress for public video, and Airtable for organizing our master database. While these tools enable our work, they also tether us to expensive corporate systems beyond our control.

Community Cloud Storage started as a multi-year inquiry into what distributed storage models could offer for community-based archives. We compared ethical and legal risks of corporate cloud storage and distributed systems from the perspective of our unique collections and the diverse communities they come from, and began to build a system to mitigate these risks as best we could. Our intention: Give communities a way to keep their digital materials safe, accessible, and governed by the people they are entrusted to.

Year 1:

A significant portion of Year 1 was spent unpacking what “community-based archives” actually means. Rather than treating it as a single, fixed model, the group examined the wide range of organizations and collections that fall under this loosely defined umbrella. These include Indigenous cultural collections, LGBTQ+ history projects, oral history initiatives, legal and human rights documentation, grassroots media and historical activist records. These collections vary dramatically in scale, risk exposure, governance structures, and expectations of access. Any storage model that assumed uniform needs or universal public access was quickly understood to be inadequate.

Throughout year 1, we centered the reality that many of these archives hold materials that are sensitive by design: testimony of violence, culturally restricted knowledge, evolving personal narratives, and evidence gathered under conditions of personal risk. For these organizations, concepts often celebrated in decentralized systems, such as immutability, permanence, transparency — potential risks were quickly shown to outweigh the benefits. Permanence (like in a blockchain environment) removes the option for edit or removal, and transparency of ownership and custody can become a problem in a shifting political and legal landscape.

These conversations were shaped by a diverse research faculty, including:

• Paulette Brown-Hinds, PhD (Black Voice News / Voice Media Ventures)
• Beth Castle, PhD (Warrior Women Project)
• Chris Sherertz (Warrior Women Project)
• Samip Mallick (SAADA)
• Yusef Omowale (Southern California Library)
• Teressa Raiford (Don’t Shoot Portland)
• Holly A. Smith (Spelman College Archives)
• Gabriel Solís (After Violence Project)
• Maigen Sullivan (Invisible Histories)
• Ed Summers (Stanford University)

In November 2023, we gathered in New Orleans to close out our first year of research and meet face-to-face with the organizations, community leaders, and funders who comprise this network. Meeting in person reinforced a core truth: trusted networks are built by people, not just technology.

Against the backdrop of the city’s brass bands and drum lines, we mapped the uneven terrain of community-based archives — discussing what is at risk, what has already been lost, and what it actually means to steward memory outside of corporate institutions. These conversations were about the social responsibility of preservation. While we didn’t yet have an answer, we left New Orleans with a shared sense of accountability, ensuring that our technical roadmap would always be led by the needs of the people behind the records.

Year 2:

Rather than designing in the abstract, Year 2 centered six community-based archive partners, each contributing public digital collections through Historypin. These partners represented a wide spectrum of community memory work, risk profiles, and organizational structures:

• **Asian & Pacific Islander Americans in Historic Preservation (APIAHiP)**, documenting sites and histories significant to APIA communities.
• **Black in Appalachia**, preserving African American histories in the Mountain South.
• **Invisible Histories**, stewarding LGBTQ history in the Deep South.
• **Manilatown Heritage Foundation**, preserving Filipino American history and struggles for justice.
• **Mescalero Community Library**, serving Tribal cultural, governmental, and community knowledge on Mescalero/Apache lands.
• **Southside US Colored Troop Coalition**, a descendant-led effort to protect Reconstruction-era Black neighborhoods in Indianapolis.

Working with these partners made clear that “community archives” are not only diverse in content, but in workflow, labor capacity, technical comfort, and tolerance for risk.

Year 2 focused on evaluating how Historypin — a platform historically oriented toward public access and activation — could responsibly support long-term preservation without undermining consent or safety. Crucially, Year 2 tested the entire user journey: signing up, adding content, editing or deleting materials, and accessing or downloading preserved data. In November 2024, project partners gathered in person on Mescalero/Apache lands in New Mexico to review progress, surface concerns, and assess what was working — and what wasn’t. Workshops and in-person convenings surfaced concrete concerns around labor, security, consent changes, and deletion — concerns that directly informed Year 3 development priorities. The convening reinforced a core lesson of Year 2: distributed storage is as much a social system as a technical one.

Year 3

Year 3 was the transition from research and testing to implementation. The project moved into building and operating a real, functioning distributed storage system — not as a proof of concept, but as shared infrastructure used by community archives with very different needs, risks, and capacities. We built a five-node private storage cluster, using QNAP hardware with IPFS Cluster and Tailscale run as docker containers. The system was intentionally hardware-independent and designed so that no single organization controlled the network. Each node was operated by a trusted partner. Within this infrastructure, we housed three distinct networks of content, testing the cluster’s ability to meet the varying needs of the organizations. These networks included:

APIAHiP Network, including collections from:

• Filipino American National Historical Society (FANHS National Archives Committee), Vietnamese American Roundtable (San Jose & Bay Area), Hawaiian National Archive (HNA), Kansha History — Nikkei Farm Project, AjA Project (City Heights, San Diego), 1882 Foundation (Chinese Americans in DC & Sacramento), Locke Foundation (Chinese Americans in the California Delta), Hmong Cultural Center and Museum of Minnesota, Khmer/Cambodian archives, including “Hau Proleung: Calling the Soul” and “Sok Sabbay Tham Phaluv”, APIA of Middle Tennessee, Red Thread Archive (Chinese Americans of Baltimore)

Manitos Network, including:

• Chimayó Cultural Preservation Association, El Valle Community Center — Community History, Community Story Project, El Valle Community Center — Valley School Community History Media Arts Project, Following the Manito Trail, Menaul Historical Library of the Southwest, Peñasco Valley Historical Preservation Society, Pueblo de Abiquiú Library & Cultural Center Archive, Taos County Historical Society, Voices of the Northern Rio Grande

SHIFT Collective CBA Network, including:

• Black Bottom Archives, Black Lesbian Archives, Blacksky Algorithms, Civil Rights Institute of Inland Southern California, Eternal Seeds, Heaux History Project, La Historia, Lawndale Pop-Up Spot, Los Angeles Poverty Department / Skid Row History Museum & Archive, Mapping Black California, Mexican American Civil Rights Institute, Rangzen, Shorefront Legacy Center, Warrior Women Project, ZIKORA Media & Arts African Cultural Heritage Organization

Trust Networks: Infrastructure Built in Real Life

This project affirmed that distributed storage infrastructure relies on human relationship. Community-cloud-storage takes shape through shared responsibility, dialogue, and accountability among the people who steward it. The gatherings in New Orleans and on Mescalero/Apache lands provided space for sustained conversation, collective decision making, and alignment around care, risk, and responsibility.

Questions of access, consent, and stewardship were approached as lived practices. Archive representatives worked together to define hosting roles, access boundaries, and deletion authority through direct conversation before translating an ethical framework into technical form. The resulting infrastructure carries the imprint of those relationships.

What’s Next — and Why We’re Proceeding Carefully

With the core system established, the work ahead focuses on continuity and care. We are strengthening the community-cloud-storage toolkit while building durable archival and media workflows that link long-term preservation files, access copies, transcripts, and descriptive metadata within a secure, unified architecture. Automated ingest and processing pipelines support the ongoing addition of new materials, while in-browser access, full-text search, and integrated media playback enable researchers and community members to engage interviews, transcripts, and contextual materials together.

Next phases emphasize deeper relational access and accountable stewardship: interactive transcript-linked video, expanded cultural and contextual metadata, collaborative review and annotation workflows, and discovery tools that surface thematic connections across interviews beyond keyword search. These systems are being developed alongside role-based access controls, consent-aligned permissions, and secure sharing tools that support community and institutional partners. Deliberate pacing ensures that infrastructure does not outrun governance, anchoring long-term stewardship in collective responsibility and community authority.

Learn more and build your own Community-Cloud-Storage system here: github.com/Historypin/community-cloud-storage/

Read the reports from SHIFT Collective here: shiftcollective.us/ffdw