Fula Storage API
S3-Compatible Decentralized Storage Engine powered by IPFS
Overview
Fula Storage provides an Amazon S3-compatible API backed by a decentralized network of IPFS nodes. It enables developers to build applications using familiar S3 tools and SDKs while benefiting from true data ownership and decentralization.
Unlike traditional cloud storage where your data lives on servers controlled by a single company, Fula distributes your data across a network of individually owned nodes. You maintain full control over your data with client-side encryption.
Decentralized Storage
Data stored across a network of community-owned IPFS nodes. No single point of failure or control.
End-to-End Encryption
HPKE encryption happens on your device. Storage nodes never see your plaintext data.
Verified Streaming
BLAKE3/Bao ensures data integrity. Detect tampering even from untrusted nodes.
Conflict-Free Sync
CRDT-based metadata enables distributed updates without coordination.
Why Fula?
| Feature | Traditional Cloud (S3, GCS) | Fula Storage |
|---|---|---|
| Data Location | Corporate data centers | Distributed across community nodes |
| Data Control | Provider can access your data | Only you can decrypt your data |
| Encryption | Server-side (provider holds keys) | Client-side (you hold keys) |
| Censorship | Provider can delete/restrict | Content-addressed, censorship-resistant |
| Vendor Lock-in | Proprietary APIs | S3-compatible, use any SDK |
| Data Integrity | Trust the provider | Cryptographically verifiable |
| Single Point of Failure | Yes | No - distributed network |
Architecture
Decentralized network of community-owned storage nodes
How It Works
Client-Side Encryption (Optional)
If encryption is enabled, data is encrypted locally using HPKE before leaving your device. Only you hold the keys.
encrypted_data = encrypt(plaintext, your_public_key)
Upload to Gateway
Your app sends a standard S3 PUT request to the Fula gateway. The gateway receives encrypted (or plain) bytes.
PUT /my-bucket/photo.jpg → Gateway
Content Chunking
Large files are split into content-defined chunks for efficient deduplication and transfer.
FastCDC chunking → [chunk1, chunk2, ...]
Content Addressing
Each chunk gets a unique CID (Content Identifier) based on its BLAKE3 hash.
CID = hash(chunk_data)
Distributed Storage
Chunks are stored across multiple IPFS nodes with configurable replication.
IPFS Cluster → replicate(chunk, n_copies)
Metadata Indexing
Object metadata is stored in Prolly Trees for efficient O(log n) lookups and CRDT sync.
prolly_tree.insert(key, metadata)
Encryption & Security
🔐 HPKE (Hybrid Public Key Encryption)
RFC 9180 compliant encryption combining:
- X25519 - Elliptic curve Diffie-Hellman key exchange
- HKDF-SHA256 - Key derivation
- ChaCha20-Poly1305 - Authenticated encryption
// Encrypt for recipient sealed = hpke.seal(recipient_public_key, plaintext) // Only recipient can decrypt plaintext = hpke.open(recipient_secret_key, sealed)
⚡ BLAKE3 Hashing
Modern cryptographic hash function:
- Fast - Faster than MD5, SHA-1, SHA-256
- Secure - 256-bit security level
- Parallelizable - Scales with CPU cores
- Verified streaming - Incremental verification
📊 Bao Streaming Verification
Verified streaming for large files:
- Verify data before fully downloading
- Detect corruption in any chunk
- Works with untrusted sources
- Based on Merkle trees
🔑 Key Management
You control your encryption keys:
- KEK - Key Encryption Key (your master key)
- DEK - Data Encryption Key (per-object)
- Key wrapping - DEKs encrypted with KEK
- Sharing - Encrypt DEK for recipients
Data Structures
🌳 Prolly Trees
Probabilistic B-trees for verifiable indexing:
- O(log n) insert, lookup, delete
- Deterministic structure from content
- Efficient diffs between versions
- Merkle proofs for any key-value pair
[root hash]
/ \
[node A] [node B]
/ \ / \
[leaf] [leaf] [leaf] [leaf]
🔄 CRDTs (Conflict-Free Replicated Data Types)
Enable distributed updates without coordination:
- Eventually consistent across nodes
- Automatic conflict resolution
- Offline-first - sync when connected
- LWW-Register for metadata
Node A: set(key, "v1", t=1)
Node B: set(key, "v2", t=2)
↓ merge
Result: key = "v2" (latest timestamp wins)
📦 Content-Defined Chunking
FastCDC algorithm for efficient deduplication:
- Rolling hash finds chunk boundaries
- Shift-resistant - small changes = few new chunks
- Configurable size - 256KB default
- Deduplication across all files
File: [=====|====|======|===]
↓ content-defined boundaries
Chunks: [CID1] [CID2] [CID3] [CID4]
Getting Started
1. Setup Your Environment
🐳 Docker Compose
Quickest way to run locally:
git clone https://github.com/functionland/fula-api cd fula-api docker-compose up -d
🔧 From Source
Build and run with Cargo:
cargo build --release ./target/release/fula-cli serve
⚙️ Configuration
Environment variables:
FULA_HOST=0.0.0.0 FULA_PORT=9000 IPFS_API_URL=http://localhost:5001 JWT_SECRET=your-secret-key
2. Upload & Manage Photos (JavaScript)
// 1. Install dependencies // npm install @aws-sdk/client-s3 crypto-js import { S3Client, PutObjectCommand, ListObjectsV2Command } from '@aws-sdk/client-s3'; // 2. Configure the Fula client const fula = new S3Client({ endpoint: 'http://localhost:9000', region: 'us-east-1', credentials: { accessKeyId: 'YOUR_ACCESS_KEY', secretAccessKey: 'YOUR_SECRET_KEY', }, forcePathStyle: true, }); const BUCKET = 'my-photos'; // 3. Helper: Encrypt data before upload (client-side encryption) async function encryptData(data: ArrayBuffer, password: string): Promise<ArrayBuffer> { const encoder = new TextEncoder(); const keyMaterial = await crypto.subtle.importKey( 'raw', encoder.encode(password), 'PBKDF2', false, ['deriveKey'] ); const key = await crypto.subtle.deriveKey( { name: 'PBKDF2', salt: encoder.encode('fula-salt'), iterations: 100000, hash: 'SHA-256' }, keyMaterial, { name: 'AES-GCM', length: 256 }, false, ['encrypt'] ); const iv = crypto.getRandomValues(new Uint8Array(12)); const encrypted = await crypto.subtle.encrypt({ name: 'AES-GCM', iv }, key, data); // Prepend IV to encrypted data const result = new Uint8Array(iv.length + encrypted.byteLength); result.set(iv); result.set(new Uint8Array(encrypted), iv.length); return result.buffer; } // 4. Upload an encrypted photo async function uploadPhoto(filename: string, photoData: ArrayBuffer, password: string) { console.log(`📤 Encrypting and uploading: ${filename}`); // Encrypt locally BEFORE sending to gateway const encryptedData = await encryptData(photoData, password); await fula.send(new PutObjectCommand({ Bucket: BUCKET, Key: `photos/${filename}`, Body: new Uint8Array(encryptedData), ContentType: 'application/octet-stream', // Encrypted = opaque bytes Metadata: { 'x-amz-meta-encrypted': 'true', 'x-amz-meta-original-type': 'image/jpeg', }, })); console.log(`✅ Uploaded: photos/${filename}`); } // 5. List all photos in the folder async function listPhotos() { console.log('📂 Listing photos folder...'); const response = await fula.send(new ListObjectsV2Command({ Bucket: BUCKET, Prefix: 'photos/', })); const files = response.Contents || []; console.log(`Found ${files.length} photo(s):`); files.forEach(file => { console.log(` 📷 ${file.Key} (${file.Size} bytes)`); }); return files; } // 6. Run the complete workflow async function main() { const password = 'my-secret-encryption-key'; // Simulate photo data (in real app, read from file/camera) const photo1 = new TextEncoder().encode('fake-jpeg-data-for-sunset.jpg'); const photo2 = new TextEncoder().encode('fake-jpeg-data-for-beach.jpg'); // Upload first photo (encrypted) await uploadPhoto('sunset.jpg', photo1.buffer, password); // List folder - should show 1 photo await listPhotos(); // Output: Found 1 photo(s): 📷 photos/sunset.jpg // Upload second photo (encrypted) await uploadPhoto('beach.jpg', photo2.buffer, password); // List folder again - should show 2 photos await listPhotos(); // Output: Found 2 photo(s): 📷 photos/sunset.jpg, 📷 photos/beach.jpg } main().catch(console.error);
📤 Encrypting and uploading: sunset.jpg ✅ Uploaded: photos/sunset.jpg 📂 Listing photos folder... Found 1 photo(s): 📷 photos/sunset.jpg (156 bytes) 📤 Encrypting and uploading: beach.jpg ✅ Uploaded: photos/beach.jpg 📂 Listing photos folder... Found 2 photo(s): 📷 photos/sunset.jpg (156 bytes) 📷 photos/beach.jpg (152 bytes)