Fula API

Fula Storage API

S3-Compatible Decentralized Storage Engine powered by IPFS

🌐 Decentralized 🔒 End-to-End Encrypted ✅ Verifiable 🔄 Conflict-Free

Overview

Fula Storage provides an Amazon S3-compatible API backed by a decentralized network of IPFS nodes. It enables developers to build applications using familiar S3 tools and SDKs while benefiting from true data ownership and decentralization.

Unlike traditional cloud storage where your data lives on servers controlled by a single company, Fula distributes your data across a network of individually owned nodes. You maintain full control over your data with client-side encryption.

🌐

Decentralized Storage

Data stored across a network of community-owned IPFS nodes. No single point of failure or control.

🔒

End-to-End Encryption

HPKE encryption happens on your device. Storage nodes never see your plaintext data.

Verified Streaming

BLAKE3/Bao ensures data integrity. Detect tampering even from untrusted nodes.

🔄

Conflict-Free Sync

CRDT-based metadata enables distributed updates without coordination.

Why Fula?

Feature Traditional Cloud (S3, GCS) Fula Storage
Data Location Corporate data centers Distributed across community nodes
Data Control Provider can access your data Only you can decrypt your data
Encryption Server-side (provider holds keys) Client-side (you hold keys)
Censorship Provider can delete/restrict Content-addressed, censorship-resistant
Vendor Lock-in Proprietary APIs S3-compatible, use any SDK
Data Integrity Trust the provider Cryptographically verifiable
Single Point of Failure Yes No - distributed network

Architecture

Application Layer
boto3 AWS SDK MinIO SDK cURL Any S3 Client
Fula Gateway
Auth JWT / API Keys
Rate Limiter Request throttling
S3 Handlers API compatibility
fula-core
Prolly Trees O(log n) indexing
Buckets Object management
CRDTs Conflict-free sync
fula-blockstore
IPFS Client Block storage
IPFS Cluster Replication
Chunking Content-defined
fula-crypto
HPKE Hybrid encryption
BLAKE3 Fast hashing
Bao Streaming verification
IPFS Network
📦 Node 📦 Node 📦 Node 📦 Node 📦 Node 📦 ...

Decentralized network of community-owned storage nodes

How It Works

1

Client-Side Encryption (Optional)

If encryption is enabled, data is encrypted locally using HPKE before leaving your device. Only you hold the keys.

encrypted_data = encrypt(plaintext, your_public_key)
2

Upload to Gateway

Your app sends a standard S3 PUT request to the Fula gateway. The gateway receives encrypted (or plain) bytes.

PUT /my-bucket/photo.jpg → Gateway
3

Content Chunking

Large files are split into content-defined chunks for efficient deduplication and transfer.

FastCDC chunking → [chunk1, chunk2, ...]
4

Content Addressing

Each chunk gets a unique CID (Content Identifier) based on its BLAKE3 hash.

CID = hash(chunk_data)
5

Distributed Storage

Chunks are stored across multiple IPFS nodes with configurable replication.

IPFS Cluster → replicate(chunk, n_copies)
6

Metadata Indexing

Object metadata is stored in Prolly Trees for efficient O(log n) lookups and CRDT sync.

prolly_tree.insert(key, metadata)

Encryption & Security

🔐 HPKE (Hybrid Public Key Encryption)

RFC 9180 compliant encryption combining:

  • X25519 - Elliptic curve Diffie-Hellman key exchange
  • HKDF-SHA256 - Key derivation
  • ChaCha20-Poly1305 - Authenticated encryption
// Encrypt for recipient
sealed = hpke.seal(recipient_public_key, plaintext)

// Only recipient can decrypt
plaintext = hpke.open(recipient_secret_key, sealed)

⚡ BLAKE3 Hashing

Modern cryptographic hash function:

  • Fast - Faster than MD5, SHA-1, SHA-256
  • Secure - 256-bit security level
  • Parallelizable - Scales with CPU cores
  • Verified streaming - Incremental verification

📊 Bao Streaming Verification

Verified streaming for large files:

  • Verify data before fully downloading
  • Detect corruption in any chunk
  • Works with untrusted sources
  • Based on Merkle trees

🔑 Key Management

You control your encryption keys:

  • KEK - Key Encryption Key (your master key)
  • DEK - Data Encryption Key (per-object)
  • Key wrapping - DEKs encrypted with KEK
  • Sharing - Encrypt DEK for recipients

Data Structures

🌳 Prolly Trees

Probabilistic B-trees for verifiable indexing:

  • O(log n) insert, lookup, delete
  • Deterministic structure from content
  • Efficient diffs between versions
  • Merkle proofs for any key-value pair
        [root hash]
        /         \
   [node A]    [node B]
   /    \      /    \
[leaf] [leaf] [leaf] [leaf]

🔄 CRDTs (Conflict-Free Replicated Data Types)

Enable distributed updates without coordination:

  • Eventually consistent across nodes
  • Automatic conflict resolution
  • Offline-first - sync when connected
  • LWW-Register for metadata
Node A: set(key, "v1", t=1)
Node B: set(key, "v2", t=2)
        ↓ merge
Result: key = "v2" (latest timestamp wins)

📦 Content-Defined Chunking

FastCDC algorithm for efficient deduplication:

  • Rolling hash finds chunk boundaries
  • Shift-resistant - small changes = few new chunks
  • Configurable size - 256KB default
  • Deduplication across all files
File: [=====|====|======|===]
           ↓ content-defined boundaries
Chunks: [CID1] [CID2] [CID3] [CID4]

Getting Started

1. Setup Your Environment

🐳 Docker Compose

Quickest way to run locally:

git clone https://github.com/functionland/fula-api
cd fula-api
docker-compose up -d

🔧 From Source

Build and run with Cargo:

cargo build --release
./target/release/fula-cli serve

⚙️ Configuration

Environment variables:

FULA_HOST=0.0.0.0
FULA_PORT=9000
IPFS_API_URL=http://localhost:5001
JWT_SECRET=your-secret-key

2. Upload & Manage Photos (JavaScript)

📸 Complete Example: Encrypt, Upload, and List Photos JavaScript / TypeScript
// 1. Install dependencies
// npm install @aws-sdk/client-s3 crypto-js

import { S3Client, PutObjectCommand, ListObjectsV2Command } from '@aws-sdk/client-s3';

// 2. Configure the Fula client
const fula = new S3Client({
  endpoint: 'http://localhost:9000',
  region: 'us-east-1',
  credentials: {
    accessKeyId: 'YOUR_ACCESS_KEY',
    secretAccessKey: 'YOUR_SECRET_KEY',
  },
  forcePathStyle: true,
});

const BUCKET = 'my-photos';

// 3. Helper: Encrypt data before upload (client-side encryption)
async function encryptData(data: ArrayBuffer, password: string): Promise<ArrayBuffer> {
  const encoder = new TextEncoder();
  const keyMaterial = await crypto.subtle.importKey(
    'raw', encoder.encode(password), 'PBKDF2', false, ['deriveKey']
  );
  const key = await crypto.subtle.deriveKey(
    { name: 'PBKDF2', salt: encoder.encode('fula-salt'), iterations: 100000, hash: 'SHA-256' },
    keyMaterial, { name: 'AES-GCM', length: 256 }, false, ['encrypt']
  );
  const iv = crypto.getRandomValues(new Uint8Array(12));
  const encrypted = await crypto.subtle.encrypt({ name: 'AES-GCM', iv }, key, data);
  // Prepend IV to encrypted data
  const result = new Uint8Array(iv.length + encrypted.byteLength);
  result.set(iv);
  result.set(new Uint8Array(encrypted), iv.length);
  return result.buffer;
}

// 4. Upload an encrypted photo
async function uploadPhoto(filename: string, photoData: ArrayBuffer, password: string) {
  console.log(`📤 Encrypting and uploading: ${filename}`);
  
  // Encrypt locally BEFORE sending to gateway
  const encryptedData = await encryptData(photoData, password);
  
  await fula.send(new PutObjectCommand({
    Bucket: BUCKET,
    Key: `photos/${filename}`,
    Body: new Uint8Array(encryptedData),
    ContentType: 'application/octet-stream', // Encrypted = opaque bytes
    Metadata: {
      'x-amz-meta-encrypted': 'true',
      'x-amz-meta-original-type': 'image/jpeg',
    },
  }));
  
  console.log(`✅ Uploaded: photos/${filename}`);
}

// 5. List all photos in the folder
async function listPhotos() {
  console.log('📂 Listing photos folder...');
  
  const response = await fula.send(new ListObjectsV2Command({
    Bucket: BUCKET,
    Prefix: 'photos/',
  }));
  
  const files = response.Contents || [];
  console.log(`Found ${files.length} photo(s):`);
  files.forEach(file => {
    console.log(`  📷 ${file.Key} (${file.Size} bytes)`);
  });
  
  return files;
}

// 6. Run the complete workflow
async function main() {
  const password = 'my-secret-encryption-key';
  
  // Simulate photo data (in real app, read from file/camera)
  const photo1 = new TextEncoder().encode('fake-jpeg-data-for-sunset.jpg');
  const photo2 = new TextEncoder().encode('fake-jpeg-data-for-beach.jpg');
  
  // Upload first photo (encrypted)
  await uploadPhoto('sunset.jpg', photo1.buffer, password);
  
  // List folder - should show 1 photo
  await listPhotos();
  // Output: Found 1 photo(s): 📷 photos/sunset.jpg
  
  // Upload second photo (encrypted)
  await uploadPhoto('beach.jpg', photo2.buffer, password);
  
  // List folder again - should show 2 photos
  await listPhotos();
  // Output: Found 2 photo(s): 📷 photos/sunset.jpg, 📷 photos/beach.jpg
}

main().catch(console.error);
Expected Output
📤 Encrypting and uploading: sunset.jpg
✅ Uploaded: photos/sunset.jpg
📂 Listing photos folder...
Found 1 photo(s):
  📷 photos/sunset.jpg (156 bytes)

📤 Encrypting and uploading: beach.jpg
✅ Uploaded: photos/beach.jpg
📂 Listing photos folder...
Found 2 photo(s):
  📷 photos/sunset.jpg (156 bytes)
  📷 photos/beach.jpg (152 bytes)