Skip to content

Garage: S3 Object Storage

Introduction

Garage Overview

Garage is an open-source, self-hosted object storage system that implements the Amazon S3 API, specifically designed for small-to-medium deployments on consumer-grade hardware and internet connections. Unlike traditional distributed storage solutions that rely on consensus algorithms like Raft, Garage operates as a geo-distributed cluster where nodes can be added or removed without a central leader, allowing all requests to be processed independently and avoiding single-point bottlenecks.

This architecture emphasizes resilience and low overhead, making it particularly suitable for home or edge environments where nodes can span multiple physical locations. The platform supports standard S3 features including multipart uploads, bucket management, access-key authentication, and static website hosting, while providing an admin API for cluster control and Prometheus-format metrics for performance monitoring. These characteristics make Garage an attractive option for organizations seeking a cost-effective, decentralized storage solution that maintains S3 compatibility without the infrastructure requirements of commercial cloud providers.

Purpose of this Guide

This guide outlines how to join a collaborative Garage storage cluster, a model where participants contribute their own storage capacity to create a resilient, geo-distributed network. By pooling resources, each member gains the security benefit of having their critical data replicated off-site at a trusted peer's home, ensuring redundancy and protection against local hardware failures without relying on commercial cloud providers. This approach transforms individual storage into a shared, secure infrastructure where everyone contributes to and benefits from a decentralized, self-hosted backup ecosystem.

Things to Know Before Setup

Shared Administration

Because this is a collaborative storage cluster, everyone is an administrator. All members can add, remove, view, and modify keys, buckets, the cluster, nodes, etc. Before storing any meaningful data, please consider encrypting it.

Deleting objects in Garage can permanently remove keys or buckets, potentially causing data loss. The Garage WebUI warns you when you try to delete a non-empty bucket, but stay vigilant.

Tip

Never modify keys, buckets, or nodes that belong to someone else without first contacting the owner. Good communication keeps the cluster healthy.

Bring Your Own Storage: Storage-Usage Guidelines

Plan to provide at least three times the total size of your backup data, since each piece of data is replicated three ways across the cluster. This ensures that there is enough space to keep the overall storage cluster healthy.

Example

You have two backup directories that you plan to backup to the Garage storage cluster. Those directories are 600 GB and 400 GB in size for a total of 1.0 TB. You should contribute at least 3 TB of capacity to the cluster layout.

Shared Key for Joining the Collaborative Storage Cluster

This guide will instruct you on how to create your own Garage storage cluster. Once you are familiar with the setup and have some hands-on experience then you can join the collaborative storage cluster. You will need the shared rpc_secret in order to join that cluster. This can be obtained by contacting the administrator.

This section contains the recommended configuration files and directory layout for running a single Garage node and Garage WebUI. Additional nodes can be added once intial setup is complete.

garage/
├─ data/
│  ├─ meta/          # Metadata directory for garage (recommended on SSD)
│  ├─ garage/        # Data / blob storage (can be on an SSD or HDD)
│  └─ garage.toml    # Configuration file used by garage and garage-webui
└─ docker-compose.yaml

Docker-Compose File

docker-compose.yaml
services:

  garage:
    image: dxflrs/garage:v2.2.0
    container_name: garage
    ports:
      - 3900:3900   # S3 API endpoint
      - 3901:3901   # Garage RPC endpoint
      - 3903:3903   # Garage admin API endpoint
    volumes:
      - ./data/garage.toml:/etc/garage.toml  # Config file
      - ./data/meta:/var/lib/garage/meta     # Metadata storage (recommended on SSD)
      - ./data/garage:/var/lib/garage/data   # Data storage (can be on SSD or HDD)
    restart: unless-stopped

  webui:
    image: khairul169/garage-webui:latest
    container_name: garage-webui
    volumes:
      - ./data/garage.toml:/etc/garage.toml:ro
    ports:
      - 3909:3909   # WebUI admin panel
    environment:
      API_BASE_URL: "http://garage:3903"
      S3_ENDPOINT_URL: "http://garage:3900"
    restart: unless-stopped

garage.toml (Core Settings)

garage.toml
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"
metadata_auto_snapshot_interval = "6h"
metrics_require_token = true
use_local_tz = false

block_size = "1MiB"
block_ram_buffer_max = "512MiB"               # Adjust to available RAM
block_max_concurrent_reads = 16
block_max_concurrent_writes_per_request = 15  # 10‑30 for NVMe, 3‑10 for HDD

replication_factor = 3          # DO NOT CHANGE: Must be same on all nodes
consistency_mode = "consistent" # DO NOT CHANGE: this affects failure read/write behavior
compression_level = 2           # fast, decent compression

rpc_bind_addr = "[::]:3901"
rpc_public_addr = "mydomain.co:3901"   # <- Replace with your public node address
rpc_secret = "my_secret"               # <- Generate with: openssl rand -hex 32 (same on all nodes)

# Peer IDs have the format: "<node public key>@<node public IP or hostname>:<port>"
# TODO: Update bootstrap_peers with the address for each other node this one should connect with.
bootstrap_peers = [
    "<node1-public-key>@<node1-public-IP-or-hostname>:<port>",
    "<node2-public-key>@<node2-public-IP-or-hostname>:<port>"
]

[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = ".s3.mydomain.co"     # <- Enables virtual-hosted-style access (e.g., mybucket.s3.mydomain.co)

[admin]
api_bind_addr = "[::]:3903"
admin_token = "my_token"            # Generate with: openssl rand -base64 32
metrics_token = "my_metrics_token"  # Generate with: openssl rand -base64 32

Key Configuration Items to Edit

  1. rpc_public_addr - Public address other nodes will use to reach this node.
  2. rpc_secret - A 32-byte hex string; generate with openssl rand -hex 32. Must be identical on every node.
  3. bootstrap_peers - List of known peers (optional but speeds up discovery).
  4. admin_token - Generate with openssl rand -base64 32. Keep it secret.

Advanced Configuration

The following are links to external documentation for the Garage and Garage WebUI projects. These are useful for further customizing your installation.

Deploying & Configuring Garage

Running Garage with Docker

  1. Install Docker and Docker-Compose on your host.
  2. Place the directory structure shown above.
  3. Edit garage.toml with your values.
  4. Run docker compose up -d. The service will start and expose the ports defined earlier.

Adding a New Node

  1. Open Garage WebUI (http://localhost:3909) -> Cluster.
  2. Click Connect, paste the new node's ID (<public-key>@<hostname>:<port>).
  3. Save - the node appears in the list.
  4. Assign the node:
    • Click the three-dot menu -> Assign.
    • Choose a Zone (one per home lab).
    • Set the Capacity you're contributing.
    • (Recommended) Add Tags for easy identification.
  5. After confirming, click Apply to push the new cluster layout. The cluster will rebalance data as needed.

Creating a New Key

  1. Garage WebUI -> Keys -> Create Key.
  2. Give the key a unique name.
  3. Record the displayed Key ID and Secret Key (store them securely).

Creating a Bucket & Setting Permissions

  1. Garage WebUI -> Buckets -> Create Bucket -> give it a unique name.
  2. Click Manage -> Permissions on the new bucket.
  3. Press Allow Key, select the desired keys, and grant the needed rights (Read, Write, Owner are typical for write-heavy workloads).
  4. Submit - the permissions table updates accordingly.

Setting Up a Reverse Proxy

Goal: Expose the S3 API and RPC endpoints via friendly domain names.

  1. Point a DNS record (e.g., s3.mydomain.co) to the IP of a garage node and configure your reverse proxy (Caddy, Nginx, Traefik, ...) to forward port 3900 to that node.
  2. Create separate DNS records for each node (e.g., node1.mydomain.co, node2.mydomain.co) and proxy port 3901 to the corresponding node's RPC endpoint.
  3. Ensure all nodes can resolve each other's domain names; this is required for inter-node communication.

Configuring a Backup Application (Kopia Example)

When adding a new repository in Kopia UI:

Field Value
Bucket Name of the Garage bucket
Server Endpoint http://<node-address>:3900 (or the DNS name you created)
Override Region garage (or leave blank)
Access Key ID The Key ID you generated
Secret Access Key The matching Secret Key

Kopia will encrypt data locally before sending it to Garage.

Running Additional Garage Nodes

Additional nodes only need the garage service in their docker-compose.yaml. The Garage WebUI container is optional and typically runs on a single "management" node.

Each additional node's garage.toml must use the same rpc_secret, replication_factor, and consistency_mode as existing nodes. Set rpc_public_addr to the new node's address and include at least one existing node in bootstrap_peers.

Joining the Collaborative Storage Cluster

When you are ready to join the collaborative storage cluster you will need the shared rpc_secret. This can be obtained by contacting the administrator.

Final Thoughts

  • Collaboration is key - keep communication open when modifying shared resources.
  • Security matters - never expose rpc_secret or admin_token publicly.
  • Capacity planning - aim for three-times replication storage to avoid saturation.