Compare commits

...

4 Commits

Author SHA1 Message Date
Ludwig Lehnert
61b698890e better folder/share names (1) 2026-03-17 10:15:37 +01:00
Ludwig Lehnert
f34e90b303 better folder/share names 2026-03-17 10:12:10 +01:00
Ludwig Lehnert
972c1a649f better redeploy: reduced downtime 2026-03-17 10:08:26 +01:00
Ludwig Lehnert
6da6db6955 better backups 2026-03-17 10:04:50 +01:00
8 changed files with 618 additions and 139 deletions

View File

@@ -16,3 +16,8 @@ FSLOGIX_GROUP_SID=S-1-5-21-1111111111-2222222222-3333333333-513
# BACKUP_DESTINATION=smb://DOMAIN%5Cuser:pass@backup.example.com/Backups/samba # BACKUP_DESTINATION=smb://DOMAIN%5Cuser:pass@backup.example.com/Backups/samba
# BACKUP_DESTINATION=davfs://user:pass@webdav.example.com/remote.php/dav/files/backup # BACKUP_DESTINATION=davfs://user:pass@webdav.example.com/remote.php/dav/files/backup
# BACKUP_DESTINATION=sftp://user:pass@sftp.example.com/exports/samba # BACKUP_DESTINATION=sftp://user:pass@sftp.example.com/exports/samba
# BACKUP_START_HOUR=2
# BACKUP_RETENTION_DAILY=3
# BACKUP_RETENTION_WEEKLY=2
# BACKUP_RETENTION_MONTHLY=2
# BACKUP_RETENTION_YEARLY=1

View File

@@ -1,12 +1,12 @@
# AD-Integrated Containerized Samba File Server # AD-Integrated Containerized Samba File Server
This repository provides a production-oriented Samba file server container that joins an existing Active Directory domain and exposes three SMB shares: `Privat`, `Data`, and `FSLogix`. This repository provides a production-oriented Samba file server container that joins an existing Active Directory domain and exposes three SMB shares: `Private`, `Data`, and `FSLogix`.
## Architecture ## Architecture
- Samba runs in ADS mode with `winbind` identity mapping. - Samba runs in ADS mode with `winbind` identity mapping.
- Static shares: - Static shares:
- `\\server\Privat` -> `/data/private` - `\\server\Private` -> `/data/private`
- `\\server\Data` -> `/data/groups/data` - `\\server\Data` -> `/data/groups/data`
- `\\server\FSLogix` -> `/data/fslogix` - `\\server\FSLogix` -> `/data/fslogix`
- FS_* groups are projected as folders inside the Data share (`/data/groups/data/<groupName>`). - FS_* groups are projected as folders inside the Data share (`/data/groups/data/<groupName>`).
@@ -27,8 +27,7 @@ This repository provides a production-oriented Samba file server container that
- once on startup - once on startup
- every 5 minutes via cron - every 5 minutes via cron
- Backup is executed: - Backup is executed:
- once on startup (when enabled) - daily at `BACKUP_START_HOUR` (default: `2`, i.e. 02:00)
- every 30 minutes via cron
## Data Folder Lifecycle ## Data Folder Lifecycle
@@ -63,8 +62,9 @@ CREATE TABLE shares (
- Initial admin credentials with rights to create/reset `FileShare_ServiceAccount` during `./setup`. - Initial admin credentials with rights to create/reset `FileShare_ServiceAccount` during `./setup`.
- `FileShare_ServiceAccount` must be allowed to join computers to the domain (`net ads join`) in your AD policy. - `FileShare_ServiceAccount` must be allowed to join computers to the domain (`net ads join`) in your AD policy.
- Dynamic group discovery primarily uses machine-account LDAP (`net ads search -P`); join credentials are only used as a fallback LDAP bind path. - Dynamic group discovery primarily uses machine-account LDAP (`net ads search -P`); join credentials are only used as a fallback LDAP bind path.
- Group naming convention for Data folders: - Group naming convention for Data folder eligibility:
- `FS_<FolderName>` - `FS_<Anything>`
- Folder names use AD group display names (`displayName`, then `name`/`cn` fallback), not pre-2000 (`sAMAccountName`) names.
## DNS Requirements ## DNS Requirements
@@ -109,6 +109,11 @@ Kerberos requires close time alignment.
- `DOMAIN_ADMINS_SID` - `DOMAIN_ADMINS_SID`
- optional `FSLOGIX_GROUP_SID` (defaults to `DOMAIN_USERS_SID`) - optional `FSLOGIX_GROUP_SID` (defaults to `DOMAIN_USERS_SID`)
- optional `BACKUP_DESTINATION` (empty disables backup) - optional `BACKUP_DESTINATION` (empty disables backup)
- optional `BACKUP_START_HOUR` (0-23, default `2`)
- optional `BACKUP_RETENTION_DAILY` (default `3`)
- optional `BACKUP_RETENTION_WEEKLY` (default `2`)
- optional `BACKUP_RETENTION_MONTHLY` (default `2`)
- optional `BACKUP_RETENTION_YEARLY` (default `1`)
Optional: Optional:
- `SAMBA_HOSTNAME` (defaults to `adsambafsrv`) - `SAMBA_HOSTNAME` (defaults to `adsambafsrv`)
@@ -133,14 +138,14 @@ Kerberos requires close time alignment.
## SMB Shares ## SMB Shares
### Privat ### Private
- Share: `\\server\Privat` - Share: `\\server\Private`
- Root path: `/data/private` - Root path: `/data/private`
- Per-user path: `/data/private/<samAccountName>` - Per-user path: `/data/private/<samAccountName>`
- Script ensures user directories exist and assigns ownership through winbind identity resolution. - Script ensures user directories exist and assigns ownership through winbind identity resolution.
- Root `/data/private` is enforced read/execute-only (`0555`) to prevent folder creation directly under `\\server\Privat`. - Root `/data/private` is enforced read/execute-only (`0555`) to prevent folder creation directly under `\\server\Private`.
- SMB-side ACL changes on `\\server\Privat` are blocked (`nt acl support = no`). - SMB-side ACL changes on `\\server\Private` are blocked (`nt acl support = no`).
- Auto-creation skips well-known/service/non-login accounts (disabled, locked, or expired). - Auto-creation skips well-known/service/non-login accounts (disabled, locked, or expired).
- Each private user tree is reconciled recursively to homogeneous permissions (dirs `0700`, files `0600`, user/admin ACLs). - Each private user tree is reconciled recursively to homogeneous permissions (dirs `0700`, files `0600`, user/admin ACLs).
- Permissions: - Permissions:
@@ -168,12 +173,25 @@ Kerberos requires close time alignment.
## Backups ## Backups
- Backups are enabled only if `BACKUP_DESTINATION` is non-empty. - Backups are enabled only if `BACKUP_DESTINATION` is non-empty.
- Each run creates a timestamped snapshot under `snapshots/YYYYMMDDTHHMMSSZ` at the destination.
- Backup job is scheduled daily at `BACKUP_START_HOUR` in container local time.
- Sources synced to destination on each run: - Sources synced to destination on each run:
- `/data/private` -> `data/private` - `/data/private` -> `data/private`
- `/data/groups` -> `data/groups` - `/data/groups` -> `data/groups`
- `/data/fslogix` -> `data/fslogix` - `/data/fslogix` -> `data/fslogix`
- `/state` -> `state` - `/state` -> `state`
- `/var/lib/samba/private` -> `samba/private` - `/var/lib/samba/private` -> `samba/private`
- Retention policy env vars (defaults):
- `BACKUP_RETENTION_YEARLY=1`
- `BACKUP_RETENTION_MONTHLY=2`
- `BACKUP_RETENTION_WEEKLY=2`
- `BACKUP_RETENTION_DAILY=3`
- Retention logic:
- daily: newest N snapshots
- weekly: newest N snapshots created on week start (Monday)
- monthly: newest N snapshots created on day 1
- yearly: newest N snapshots created on Jan 1
- snapshots selected by any tier are retained; all others are pruned
- Supported destination schemes: - Supported destination schemes:
- `rsync://user:pass@host/module/path` - `rsync://user:pass@host/module/path`
- `smb://user:pass@host/share/path` (domain user example: `smb://DOMAIN%5Cuser:pass@host/share/path`) - `smb://user:pass@host/share/path` (domain user example: `smb://DOMAIN%5Cuser:pass@host/share/path`)
@@ -185,6 +203,11 @@ Kerberos requires close time alignment.
```env ```env
BACKUP_DESTINATION=sftp://backupuser:StrongPassword@sftp.example.com/exports/samba BACKUP_DESTINATION=sftp://backupuser:StrongPassword@sftp.example.com/exports/samba
BACKUP_START_HOUR=2
BACKUP_RETENTION_DAILY=3
BACKUP_RETENTION_WEEKLY=2
BACKUP_RETENTION_MONTHLY=2
BACKUP_RETENTION_YEARLY=1
``` ```
## Useful Commands ## Useful Commands
@@ -257,7 +280,7 @@ docker compose exec samba sh -lc 'tail -n 200 /var/log/backup.log'
docker compose exec samba sh -lc 'mods="$(smbd -b | sed -n "s/^ *MODULESDIR: //p" | head -n1)/vfs"; ls -1 "$mods"/acl_xattr.so "$mods"/full_audit.so' docker compose exec samba sh -lc 'mods="$(smbd -b | sed -n "s/^ *MODULESDIR: //p" | head -n1)/vfs"; ls -1 "$mods"/acl_xattr.so "$mods"/full_audit.so'
``` ```
### Permissions in Privat share are incorrect ### Permissions in Private share are incorrect
- Re-run reconciliation to rebuild private directories and ACLs: - Re-run reconciliation to rebuild private directories and ACLs:

View File

@@ -1,16 +1,20 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import datetime as dt
import fcntl import fcntl
import os import os
import re
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
from dataclasses import dataclass from dataclasses import dataclass
from typing import Dict, List, Optional, Tuple from typing import Dict, List, Optional, Set, Tuple
from urllib.parse import SplitResult, unquote, urlsplit from urllib.parse import SplitResult, unquote, urlsplit
LOCK_PATH = "/state/backup.lock" LOCK_PATH = "/state/backup.lock"
SNAPSHOT_NAME_RE = re.compile(r"^\d{8}T\d{6}Z$")
BACKUP_SOURCES: List[Tuple[str, str]] = [ BACKUP_SOURCES: List[Tuple[str, str]] = [
("/data/private", "data/private"), ("/data/private", "data/private"),
("/data/groups", "data/groups"), ("/data/groups", "data/groups"),
@@ -30,6 +34,12 @@ RCLONE_SCHEME_MAP = {
"https": "webdav", "https": "webdav",
} }
DEFAULT_BACKUP_START_HOUR = 2
DEFAULT_RETENTION_DAILY = 3
DEFAULT_RETENTION_WEEKLY = 2
DEFAULT_RETENTION_MONTHLY = 2
DEFAULT_RETENTION_YEARLY = 1
@dataclass @dataclass
class Destination: class Destination:
@@ -43,6 +53,20 @@ class Destination:
path: str path: str
@dataclass
class RetentionPolicy:
daily: int
weekly: int
monthly: int
yearly: int
@dataclass(frozen=True)
class Snapshot:
name: str
timestamp: dt.datetime
def log(message: str) -> None: def log(message: str) -> None:
print(f"[backup] {message}", flush=True) print(f"[backup] {message}", flush=True)
@@ -51,9 +75,16 @@ def run_command(
command: List[str], command: List[str],
*, *,
env: Optional[Dict[str, str]] = None, env: Optional[Dict[str, str]] = None,
input_text: Optional[str] = None,
check: bool = True, check: bool = True,
) -> subprocess.CompletedProcess: ) -> subprocess.CompletedProcess:
result = subprocess.run(command, capture_output=True, text=True, env=env) result = subprocess.run(
command,
capture_output=True,
text=True,
env=env,
input=input_text,
)
if check and result.returncode != 0: if check and result.returncode != 0:
output = result.stderr.strip() or result.stdout.strip() output = result.stderr.strip() or result.stdout.strip()
raise RuntimeError(f"Command failed ({command[0]}): {output}") raise RuntimeError(f"Command failed ({command[0]}): {output}")
@@ -127,14 +158,57 @@ def join_path(prefix: str, suffix: str) -> str:
return left or right return left or right
def obscure_secret(secret: str) -> str: def parse_int_env(
result = run_command(["rclone", "obscure", secret]) name: str, default: int, *, minimum: int, maximum: Optional[int]
value = result.stdout.strip() ) -> int:
if not value: raw_value = os.getenv(name, "").strip()
raise RuntimeError("rclone obscure returned an empty value") if not raw_value:
return default
try:
value = int(raw_value)
except ValueError:
log(f"Invalid {name}='{raw_value}', using default {default}")
return default
if value < minimum or (maximum is not None and value > maximum):
if maximum is None:
log(f"Invalid {name}='{raw_value}', using default {default}")
else:
log(
f"Invalid {name}='{raw_value}' (expected {minimum}-{maximum}), using default {default}"
)
return default
return value return value
def parse_retention_policy() -> RetentionPolicy:
return RetentionPolicy(
daily=parse_int_env(
"BACKUP_RETENTION_DAILY", DEFAULT_RETENTION_DAILY, minimum=0, maximum=None
),
weekly=parse_int_env(
"BACKUP_RETENTION_WEEKLY",
DEFAULT_RETENTION_WEEKLY,
minimum=0,
maximum=None,
),
monthly=parse_int_env(
"BACKUP_RETENTION_MONTHLY",
DEFAULT_RETENTION_MONTHLY,
minimum=0,
maximum=None,
),
yearly=parse_int_env(
"BACKUP_RETENTION_YEARLY",
DEFAULT_RETENTION_YEARLY,
minimum=0,
maximum=None,
),
)
def parse_smb_identity(username: str) -> Tuple[str, str]: def parse_smb_identity(username: str) -> Tuple[str, str]:
if not username: if not username:
return "", "" return "", ""
@@ -147,34 +221,102 @@ def parse_smb_identity(username: str) -> Tuple[str, str]:
return "", username return "", username
def sync_with_rsync(destination: Destination, sources: List[Tuple[str, str]]) -> None: def obscure_secret(secret: str) -> str:
module_path = destination.path.lstrip("/") result = run_command(["rclone", "obscure", secret])
if not module_path: value = result.stdout.strip()
raise RuntimeError( if not value:
"rsync destinations must include a module path (example: rsync://user:pass@host/module/path)" raise RuntimeError("rclone obscure returned an empty value")
return value
def parse_snapshot_name(name: str) -> Optional[dt.datetime]:
if not SNAPSHOT_NAME_RE.match(name):
return None
try:
parsed = dt.datetime.strptime(name, "%Y%m%dT%H%M%SZ")
except ValueError:
return None
return parsed.replace(tzinfo=dt.timezone.utc)
def choose_snapshot_name(existing_names: Set[str]) -> str:
base = dt.datetime.now(dt.timezone.utc)
for offset in range(0, 120):
candidate = (base + dt.timedelta(seconds=offset)).strftime("%Y%m%dT%H%M%SZ")
if candidate not in existing_names:
return candidate
raise RuntimeError("Unable to generate a unique snapshot name")
def is_week_start(timestamp: dt.datetime) -> bool:
return timestamp.weekday() == 0
def is_month_start(timestamp: dt.datetime) -> bool:
return timestamp.day == 1
def is_year_start(timestamp: dt.datetime) -> bool:
return timestamp.month == 1 and timestamp.day == 1
def select_newest(snapshot_pool: List[Snapshot], limit: int) -> Set[str]:
if limit <= 0:
return set()
sorted_pool = sorted(snapshot_pool, key=lambda entry: entry.timestamp, reverse=True)
return {entry.name for entry in sorted_pool[:limit]}
def compute_retained_snapshots(
snapshots: List[Snapshot], policy: RetentionPolicy
) -> Set[str]:
retained: Set[str] = set()
retained.update(select_newest(snapshots, policy.daily))
retained.update(
select_newest(
[entry for entry in snapshots if is_week_start(entry.timestamp)],
policy.weekly,
)
)
retained.update(
select_newest(
[entry for entry in snapshots if is_month_start(entry.timestamp)],
policy.monthly,
)
)
retained.update(
select_newest(
[entry for entry in snapshots if is_year_start(entry.timestamp)],
policy.yearly,
)
)
return retained
class RcloneBackend:
def __init__(self, destination: Destination):
self.base_prefix = ""
options, self.base_prefix = self._build_remote(destination)
self.config_path = self._write_config(options)
def close(self) -> None:
if os.path.exists(self.config_path):
os.remove(self.config_path)
def _run(
self,
args: List[str],
*,
check: bool = True,
input_text: Optional[str] = None,
) -> subprocess.CompletedProcess:
return run_command(
["rclone", *args, "--config", self.config_path],
check=check,
input_text=input_text,
) )
host = format_host(destination.hostname) def _build_remote(self, destination: Destination) -> Tuple[Dict[str, str], str]:
if destination.port is not None:
host = f"{host}:{destination.port}"
user_prefix = f"{destination.username}@" if destination.username else ""
remote_base = f"rsync://{user_prefix}{host}/{module_path.rstrip('/')}"
command_env = os.environ.copy()
if destination.password:
command_env["RSYNC_PASSWORD"] = destination.password
for source_path, destination_path in sources:
remote_path = f"{remote_base}/{destination_path.strip('/')}/"
log(f"Syncing {source_path} to rsync destination")
run_command(
["rsync", "-a", "--delete", f"{source_path}/", remote_path],
env=command_env,
)
def build_rclone_remote(destination: Destination) -> Tuple[Dict[str, str], str]:
backend = RCLONE_SCHEME_MAP.get(destination.scheme) backend = RCLONE_SCHEME_MAP.get(destination.scheme)
if backend is None: if backend is None:
supported = ", ".join(["rsync", *sorted(RCLONE_SCHEME_MAP.keys())]) supported = ", ".join(["rsync", *sorted(RCLONE_SCHEME_MAP.keys())])
@@ -197,7 +339,9 @@ def build_rclone_remote(destination: Destination) -> Tuple[Dict[str, str], str]:
return options, remote_prefix return options, remote_prefix
if backend == "smb": if backend == "smb":
path_segments = [segment for segment in destination.path.split("/") if segment] path_segments = [
segment for segment in destination.path.split("/") if segment
]
if not path_segments: if not path_segments:
raise RuntimeError( raise RuntimeError(
"smb destinations must include a share name in the path (example: smb://user:pass@host/share/path)" "smb destinations must include a share name in the path (example: smb://user:pass@host/share/path)"
@@ -238,8 +382,7 @@ def build_rclone_remote(destination: Destination) -> Tuple[Dict[str, str], str]:
return options, remote_prefix return options, remote_prefix
def _write_config(self, options: Dict[str, str]) -> str:
def write_rclone_config(options: Dict[str, str]) -> str:
with tempfile.NamedTemporaryFile("w", encoding="utf-8", delete=False) as handle: with tempfile.NamedTemporaryFile("w", encoding="utf-8", delete=False) as handle:
handle.write("[backup]\n") handle.write("[backup]\n")
for key, value in options.items(): for key, value in options.items():
@@ -249,29 +392,221 @@ def write_rclone_config(options: Dict[str, str]) -> str:
os.chmod(config_path, 0o600) os.chmod(config_path, 0o600)
return config_path return config_path
def _snapshots_root(self) -> str:
return join_path(self.base_prefix, "snapshots")
def sync_with_rclone(destination: Destination, sources: List[Tuple[str, str]]) -> None: def _snapshot_root(self, snapshot_name: str) -> str:
options, remote_prefix = build_rclone_remote(destination) return join_path(self._snapshots_root(), snapshot_name)
config_path = write_rclone_config(options)
try: def sync_source(
for source_path, destination_path in sources: self, snapshot_name: str, source_path: str, destination_path: str
remote_path = join_path(remote_prefix, destination_path) ) -> None:
log(f"Syncing {source_path} to {destination.scheme} destination") remote_path = join_path(self._snapshot_root(snapshot_name), destination_path)
run_command( self._run(
[ [
"rclone",
"sync", "sync",
f"{source_path}/", f"{source_path}/",
f"backup:{remote_path}", f"backup:{remote_path}",
"--config",
config_path,
"--create-empty-src-dirs", "--create-empty-src-dirs",
] ]
) )
def write_marker(self, snapshot_name: str) -> None:
marker_path = join_path(self._snapshot_root(snapshot_name), ".backup_complete")
self._run(
["rcat", f"backup:{marker_path}"],
input_text=f"{dt.datetime.now(dt.timezone.utc).isoformat()}\n",
)
def _snapshot_has_marker(self, snapshot_name: str) -> bool:
result = self._run(
[
"lsf",
f"backup:{self._snapshot_root(snapshot_name)}",
"--files-only",
"--include",
".backup_complete",
],
check=False,
)
if result.returncode != 0:
return False
return any(
line.strip() == ".backup_complete" for line in result.stdout.splitlines()
)
def list_snapshots(self) -> List[str]:
result = self._run(
["lsf", f"backup:{self._snapshots_root()}", "--dirs-only", "--format", "p"],
check=False,
)
if result.returncode != 0:
output = (result.stderr.strip() or result.stdout.strip()).lower()
if "not found" in output or "doesn't exist" in output:
return []
raise RuntimeError(result.stderr.strip() or result.stdout.strip())
names: List[str] = []
for line in result.stdout.splitlines():
candidate = line.strip().rstrip("/")
if not SNAPSHOT_NAME_RE.match(candidate):
continue
if self._snapshot_has_marker(candidate):
names.append(candidate)
return sorted(set(names))
def delete_snapshot(self, snapshot_name: str) -> None:
result = self._run(
["purge", f"backup:{self._snapshot_root(snapshot_name)}"],
check=False,
)
if result.returncode != 0:
log(
f"Failed to delete snapshot {snapshot_name}: {result.stderr.strip() or result.stdout.strip()}"
)
class RsyncBackend:
def __init__(self, destination: Destination):
module_path = destination.path.lstrip("/")
if not module_path:
raise RuntimeError(
"rsync destinations must include a module path (example: rsync://user:pass@host/module/path)"
)
host = format_host(destination.hostname)
if destination.port is not None:
host = f"{host}:{destination.port}"
user_prefix = f"{destination.username}@" if destination.username else ""
self.remote_base = f"rsync://{user_prefix}{host}/{module_path.rstrip('/')}"
self.command_env = os.environ.copy()
if destination.password:
self.command_env["RSYNC_PASSWORD"] = destination.password
def close(self) -> None:
return
def _remote_path(self, path: str) -> str:
trimmed = path.strip("/")
if not trimmed:
return self.remote_base
return f"{self.remote_base}/{trimmed}"
def sync_source(
self, snapshot_name: str, source_path: str, destination_path: str
) -> None:
target = self._remote_path(
join_path(f"snapshots/{snapshot_name}", destination_path)
)
run_command(
["rsync", "-a", "--delete", f"{source_path}/", f"{target}/"],
env=self.command_env,
)
def write_marker(self, snapshot_name: str) -> None:
marker_remote = self._remote_path(f"snapshots/{snapshot_name}/.backup_complete")
marker_file = None
try:
with tempfile.NamedTemporaryFile(
"w", encoding="utf-8", delete=False
) as handle:
marker_file = handle.name
handle.write(f"{dt.datetime.now(dt.timezone.utc).isoformat()}\n")
run_command(
["rsync", "-a", marker_file, marker_remote], env=self.command_env
)
finally: finally:
if os.path.exists(config_path): if marker_file and os.path.exists(marker_file):
os.remove(config_path) os.remove(marker_file)
def _snapshot_has_marker(self, snapshot_name: str) -> bool:
marker_remote = self._remote_path(f"snapshots/{snapshot_name}/.backup_complete")
result = run_command(
["rsync", "--list-only", marker_remote],
env=self.command_env,
check=False,
)
return result.returncode == 0
def list_snapshots(self) -> List[str]:
root = self._remote_path("snapshots")
result = run_command(
["rsync", "--list-only", f"{root}/"],
env=self.command_env,
check=False,
)
if result.returncode != 0:
output = result.stderr.strip() or result.stdout.strip()
lower = output.lower()
if (
"no such file" in lower
or "not found" in lower
or "chdir failed" in lower
):
return []
raise RuntimeError(output)
names: List[str] = []
for line in result.stdout.splitlines():
line = line.strip()
if not line or line.startswith("receiving"):
continue
parts = line.split()
if not parts:
continue
candidate = parts[-1].rstrip("/")
if not SNAPSHOT_NAME_RE.match(candidate):
continue
if self._snapshot_has_marker(candidate):
names.append(candidate)
return sorted(set(names))
def delete_snapshot(self, snapshot_name: str) -> None:
snapshot_remote = self._remote_path(f"snapshots/{snapshot_name}")
empty_dir = tempfile.mkdtemp(prefix="backup-empty-")
try:
run_command(
["rsync", "-a", "--delete", f"{empty_dir}/", f"{snapshot_remote}/"],
env=self.command_env,
check=False,
)
run_command(
[
"rsync",
"-a",
"--delete",
"--prune-empty-dirs",
"--include",
f"/{snapshot_name}/***",
"--exclude",
"*",
f"{empty_dir}/",
f"{self._remote_path('snapshots')}/",
],
env=self.command_env,
check=False,
)
finally:
os.rmdir(empty_dir)
def build_backend(destination: Destination):
if destination.scheme == "rsync":
return RsyncBackend(destination)
return RcloneBackend(destination)
def parse_snapshot_inventory(snapshot_names: List[str]) -> List[Snapshot]:
snapshots: List[Snapshot] = []
for name in snapshot_names:
timestamp = parse_snapshot_name(name)
if timestamp is None:
continue
snapshots.append(Snapshot(name=name, timestamp=timestamp))
snapshots.sort(key=lambda entry: entry.timestamp, reverse=True)
return snapshots
def run_backup() -> int: def run_backup() -> int:
@@ -280,21 +615,45 @@ def run_backup() -> int:
log("BACKUP_DESTINATION is unset, skipping backup") log("BACKUP_DESTINATION is unset, skipping backup")
return 0 return 0
policy = parse_retention_policy()
sources = available_sources() sources = available_sources()
if not sources: if not sources:
log("No backup sources are available, skipping backup") log("No backup sources are available, skipping backup")
return 0 return 0
destination = parse_destination(destination_url) destination = parse_destination(destination_url)
backend = build_backend(destination)
try:
log(f"Starting backup to {redact_destination(destination.raw_url)}") log(f"Starting backup to {redact_destination(destination.raw_url)}")
if destination.scheme == "rsync": existing = set(backend.list_snapshots())
sync_with_rsync(destination, sources) snapshot_name = choose_snapshot_name(existing)
else: log(f"Creating snapshot {snapshot_name}")
sync_with_rclone(destination, sources)
log("Backup completed") for source_path, destination_path in sources:
log(f"Syncing {source_path}")
backend.sync_source(snapshot_name, source_path, destination_path)
backend.write_marker(snapshot_name)
all_snapshot_names = backend.list_snapshots()
snapshots = parse_snapshot_inventory(all_snapshot_names)
retained = compute_retained_snapshots(snapshots, policy)
deleted_count = 0
for snapshot in snapshots:
if snapshot.name in retained:
continue
log(f"Pruning snapshot {snapshot.name}")
backend.delete_snapshot(snapshot.name)
deleted_count += 1
log(
f"Backup completed (snapshots total={len(snapshots)}, retained={len(retained)}, pruned={deleted_count})"
)
return 0 return 0
finally:
backend.close()
def with_lock() -> int: def with_lock() -> int:
@@ -311,6 +670,15 @@ def with_lock() -> int:
def main() -> int: def main() -> int:
backup_hour = parse_int_env(
"BACKUP_START_HOUR",
DEFAULT_BACKUP_START_HOUR,
minimum=0,
maximum=23,
)
if backup_hour != DEFAULT_BACKUP_START_HOUR:
log(f"Configured backup start hour is {backup_hour}:00")
try: try:
return with_lock() return with_lock()
except Exception as exc: # pylint: disable=broad-except except Exception as exc: # pylint: disable=broad-except

View File

@@ -62,6 +62,17 @@ derive_netbios_name() {
export NETBIOS_NAME="${cleaned_name:0:15}" export NETBIOS_NAME="${cleaned_name:0:15}"
} }
derive_backup_start_hour() {
local raw_hour="${BACKUP_START_HOUR:-2}"
if [[ "$raw_hour" =~ ^[0-9]+$ ]] && (( raw_hour >= 0 && raw_hour <= 23 )); then
printf '%d\n' "$raw_hour"
return
fi
log "Invalid BACKUP_START_HOUR '${raw_hour}', defaulting to 2."
printf '2\n'
}
resolve_sid_to_group() { resolve_sid_to_group() {
local sid="$1" local sid="$1"
local resolved_name="" local resolved_name=""
@@ -159,9 +170,22 @@ write_runtime_env_file() {
printf 'export DOMAIN_USERS_GROUP=%q\n' "$DOMAIN_USERS_GROUP" printf 'export DOMAIN_USERS_GROUP=%q\n' "$DOMAIN_USERS_GROUP"
printf 'export DOMAIN_ADMINS_GROUP=%q\n' "$DOMAIN_ADMINS_GROUP" printf 'export DOMAIN_ADMINS_GROUP=%q\n' "$DOMAIN_ADMINS_GROUP"
printf 'export FSLOGIX_GROUP=%q\n' "$FSLOGIX_GROUP" printf 'export FSLOGIX_GROUP=%q\n' "$FSLOGIX_GROUP"
printf 'export BACKUP_START_HOUR=%q\n' "$BACKUP_START_HOUR"
if [[ -n "${BACKUP_DESTINATION:-}" ]]; then if [[ -n "${BACKUP_DESTINATION:-}" ]]; then
printf 'export BACKUP_DESTINATION=%q\n' "$BACKUP_DESTINATION" printf 'export BACKUP_DESTINATION=%q\n' "$BACKUP_DESTINATION"
fi fi
if [[ -n "${BACKUP_RETENTION_DAILY:-}" ]]; then
printf 'export BACKUP_RETENTION_DAILY=%q\n' "$BACKUP_RETENTION_DAILY"
fi
if [[ -n "${BACKUP_RETENTION_WEEKLY:-}" ]]; then
printf 'export BACKUP_RETENTION_WEEKLY=%q\n' "$BACKUP_RETENTION_WEEKLY"
fi
if [[ -n "${BACKUP_RETENTION_MONTHLY:-}" ]]; then
printf 'export BACKUP_RETENTION_MONTHLY=%q\n' "$BACKUP_RETENTION_MONTHLY"
fi
if [[ -n "${BACKUP_RETENTION_YEARLY:-}" ]]; then
printf 'export BACKUP_RETENTION_YEARLY=%q\n' "$BACKUP_RETENTION_YEARLY"
fi
if [[ -n "${JOIN_USER:-}" ]]; then if [[ -n "${JOIN_USER:-}" ]]; then
printf 'export JOIN_USER=%q\n' "$JOIN_USER" printf 'export JOIN_USER=%q\n' "$JOIN_USER"
fi fi
@@ -216,8 +240,8 @@ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
EOF EOF
if [[ -n "${BACKUP_DESTINATION:-}" ]]; then if [[ -n "${BACKUP_DESTINATION:-}" ]]; then
cat >> /etc/cron.d/reconcile-shares <<'EOF' cat >> /etc/cron.d/reconcile-shares <<EOF
*/30 * * * * root source /app/runtime.env && /usr/bin/python3 /app/backup_to_destination.py >> /var/log/backup.log 2>&1 0 ${BACKUP_START_HOUR} * * * root source /app/runtime.env && /usr/bin/python3 /app/backup_to_destination.py >> /var/log/backup.log 2>&1
EOF EOF
fi fi
@@ -231,6 +255,7 @@ require_env DOMAIN_USERS_SID
require_env DOMAIN_ADMINS_SID require_env DOMAIN_ADMINS_SID
export REALM WORKGROUP DOMAIN export REALM WORKGROUP DOMAIN
export BACKUP_START_HOUR="$(derive_backup_start_hour)"
export FSLOGIX_GROUP_SID="${FSLOGIX_GROUP_SID:-${DOMAIN_USERS_SID}}" export FSLOGIX_GROUP_SID="${FSLOGIX_GROUP_SID:-${DOMAIN_USERS_SID}}"
export DOMAIN_USERS_GROUP="${DOMAIN_USERS_SID}" export DOMAIN_USERS_GROUP="${DOMAIN_USERS_SID}"
export DOMAIN_ADMINS_GROUP="${DOMAIN_ADMINS_SID}" export DOMAIN_ADMINS_GROUP="${DOMAIN_ADMINS_SID}"
@@ -266,12 +291,9 @@ log 'Running startup reconciliation'
python3 /app/reconcile_shares.py python3 /app/reconcile_shares.py
if [[ -n "${BACKUP_DESTINATION:-}" ]]; then if [[ -n "${BACKUP_DESTINATION:-}" ]]; then
log 'Running startup backup' log "Backups enabled: daily at ${BACKUP_START_HOUR}:00 (container local time)."
if ! python3 /app/backup_to_destination.py; then
log 'Startup backup failed; continuing service startup.'
fi
else else
log 'BACKUP_DESTINATION is unset; startup backup skipped' log 'BACKUP_DESTINATION is unset; scheduled backup disabled'
fi fi
install_cron_job install_cron_job

View File

@@ -24,11 +24,12 @@ FSLOGIX_ROOT = "/data/fslogix"
LDAP_FILTER = "(&(objectClass=group)(sAMAccountName=FS_*))" LDAP_FILTER = "(&(objectClass=group)(sAMAccountName=FS_*))"
GROUP_PREFIXES = ("FS_",) GROUP_PREFIXES = ("FS_",)
GROUP_TITLE_ATTRS = ("displayname", "name", "cn")
USER_STATUS_FILTER = "(&(objectClass=user)(!(objectClass=computer))(sAMAccountName=*))" USER_STATUS_FILTER = "(&(objectClass=user)(!(objectClass=computer))(sAMAccountName=*))"
REQUIRED_ENV = ["REALM", "WORKGROUP", "DOMAIN"] REQUIRED_ENV = ["REALM", "WORKGROUP", "DOMAIN"]
ATTR_RE = re.compile(r"^([^:]+)(::?)\s*(.*)$") ATTR_RE = re.compile(r"^([^:]+)(::?)\s*(.*)$")
GROUP_FOLDER_INVALID_RE = re.compile(r"[\\/:*?\"<>|;\[\],+=]") GROUP_FOLDER_INVALID_RE = re.compile(r"[\\/:*?\"<>|]")
PRIVATE_SKIP_EXACT = { PRIVATE_SKIP_EXACT = {
"krbtgt", "krbtgt",
"administrator", "administrator",
@@ -122,6 +123,15 @@ def derive_share_name(sam_account_name: str) -> Optional[str]:
return None return None
def derive_group_title(entry: Dict[str, Tuple[str, bool]]) -> Optional[str]:
for attr in GROUP_TITLE_ATTRS:
if attr in entry:
value = entry[attr][0].strip()
if value:
return value
return None
def parse_groups_from_ldap_output(output: str) -> List[Dict[str, str]]: def parse_groups_from_ldap_output(output: str) -> List[Dict[str, str]]:
entries = parse_ldap_entries(output) entries = parse_ldap_entries(output)
@@ -132,7 +142,7 @@ def parse_groups_from_ldap_output(output: str) -> List[Dict[str, str]]:
sam_value, _ = entry["samaccountname"] sam_value, _ = entry["samaccountname"]
sam = sam_value.strip() sam = sam_value.strip()
share_name = derive_share_name(sam) share_name = derive_group_title(entry) or derive_share_name(sam)
if not share_name: if not share_name:
continue continue
@@ -223,7 +233,18 @@ def next_available_path(path: str) -> str:
def fetch_groups_via_net_ads() -> List[Dict[str, str]]: def fetch_groups_via_net_ads() -> List[Dict[str, str]]:
result = run_command( result = run_command(
["net", "ads", "search", "-P", LDAP_FILTER, "objectGUID", "sAMAccountName"], [
"net",
"ads",
"search",
"-P",
LDAP_FILTER,
"objectGUID",
"sAMAccountName",
"displayName",
"name",
"cn",
],
check=False, check=False,
) )
if result.returncode != 0: if result.returncode != 0:
@@ -270,6 +291,9 @@ def fetch_groups_via_ldap_bind() -> List[Dict[str, str]]:
LDAP_FILTER, LDAP_FILTER,
"objectGUID", "objectGUID",
"sAMAccountName", "sAMAccountName",
"displayName",
"name",
"cn",
] ]
) )
return parse_groups_from_ldap_output(result.stdout) return parse_groups_from_ldap_output(result.stdout)

View File

@@ -37,7 +37,7 @@
logging = file logging = file
log level = 1 auth:5 passdb:5 winbind:3 log level = 1 auth:5 passdb:5 winbind:3
[Privat] [Private]
path = /data/private path = /data/private
read only = no read only = no
browseable = yes browseable = yes

View File

@@ -10,20 +10,27 @@ if [[ ! -d .git ]]; then
exit 1 exit 1
fi fi
log 'Stopping stack' OLD_COMPOSE_FILE="$(mktemp)"
docker compose down
log 'Removing current local compose image(s)' cleanup() {
mapfile -t IMAGE_NAMES < <(docker compose config --images 2>/dev/null | sed '/^$/d' | sort -u) if [[ -f "$OLD_COMPOSE_FILE" ]]; then
if [[ "${#IMAGE_NAMES[@]}" -gt 0 ]]; then rm -f "$OLD_COMPOSE_FILE"
docker image rm -f "${IMAGE_NAMES[@]}" || true fi
else }
log 'No compose-managed local images found to remove.' trap cleanup EXIT
fi
log 'Capturing current compose configuration'
docker compose config > "$OLD_COMPOSE_FILE"
log 'Pulling latest git changes' log 'Pulling latest git changes'
git pull git pull
log 'Building updated images while current stack is running'
docker compose build --no-cache
log 'Stopping previous stack using captured configuration'
docker compose --project-directory "$PWD" -f "$OLD_COMPOSE_FILE" down
log 'Starting stack' log 'Starting stack'
docker compose up -d docker compose up -d

30
setup
View File

@@ -85,6 +85,11 @@ write_env_file() {
local domain_admins_sid="" local domain_admins_sid=""
local fslogix_group_sid="" local fslogix_group_sid=""
local backup_destination="" local backup_destination=""
local backup_start_hour="2"
local backup_retention_daily="3"
local backup_retention_weekly="2"
local backup_retention_monthly="2"
local backup_retention_yearly="1"
local samba_hostname="adsambafsrv" local samba_hostname="adsambafsrv"
local netbios_name="ADSAMBAFSRV" local netbios_name="ADSAMBAFSRV"
local service_password="" local service_password=""
@@ -124,6 +129,16 @@ write_env_file() {
netbios_name="$sanitized_netbios_name" netbios_name="$sanitized_netbios_name"
read -r -p "BACKUP_DESTINATION (optional URL, press Enter to disable): " backup_destination read -r -p "BACKUP_DESTINATION (optional URL, press Enter to disable): " backup_destination
read -r -p "BACKUP_START_HOUR [2]: " backup_start_hour
backup_start_hour="${backup_start_hour:-2}"
read -r -p "BACKUP_RETENTION_DAILY [3]: " backup_retention_daily
backup_retention_daily="${backup_retention_daily:-3}"
read -r -p "BACKUP_RETENTION_WEEKLY [2]: " backup_retention_weekly
backup_retention_weekly="${backup_retention_weekly:-2}"
read -r -p "BACKUP_RETENTION_MONTHLY [2]: " backup_retention_monthly
backup_retention_monthly="${backup_retention_monthly:-2}"
read -r -p "BACKUP_RETENTION_YEARLY [1]: " backup_retention_yearly
backup_retention_yearly="${backup_retention_yearly:-1}"
service_account_sam="$(sanitize_sam_account_name "$SERVICE_ACCOUNT_NAME")" service_account_sam="$(sanitize_sam_account_name "$SERVICE_ACCOUNT_NAME")"
if [[ "$service_account_sam" != "$SERVICE_ACCOUNT_NAME" ]]; then if [[ "$service_account_sam" != "$SERVICE_ACCOUNT_NAME" ]]; then
@@ -162,6 +177,11 @@ DOMAIN_USERS_SID=${domain_users_sid}
DOMAIN_ADMINS_SID=${domain_admins_sid} DOMAIN_ADMINS_SID=${domain_admins_sid}
FSLOGIX_GROUP_SID=${fslogix_group_sid} FSLOGIX_GROUP_SID=${fslogix_group_sid}
BACKUP_DESTINATION=${backup_destination} BACKUP_DESTINATION=${backup_destination}
BACKUP_START_HOUR=${backup_start_hour}
BACKUP_RETENTION_DAILY=${backup_retention_daily}
BACKUP_RETENTION_WEEKLY=${backup_retention_weekly}
BACKUP_RETENTION_MONTHLY=${backup_retention_monthly}
BACKUP_RETENTION_YEARLY=${backup_retention_yearly}
SAMBA_HOSTNAME=${samba_hostname} SAMBA_HOSTNAME=${samba_hostname}
NETBIOS_NAME=${netbios_name} NETBIOS_NAME=${netbios_name}
EOF EOF
@@ -214,6 +234,11 @@ DOMAIN_USERS_SID=${domain_users_sid}
DOMAIN_ADMINS_SID=${domain_admins_sid} DOMAIN_ADMINS_SID=${domain_admins_sid}
FSLOGIX_GROUP_SID=${fslogix_group_sid} FSLOGIX_GROUP_SID=${fslogix_group_sid}
BACKUP_DESTINATION=${backup_destination} BACKUP_DESTINATION=${backup_destination}
BACKUP_START_HOUR=${backup_start_hour}
BACKUP_RETENTION_DAILY=${backup_retention_daily}
BACKUP_RETENTION_WEEKLY=${backup_retention_weekly}
BACKUP_RETENTION_MONTHLY=${backup_retention_monthly}
BACKUP_RETENTION_YEARLY=${backup_retention_yearly}
SAMBA_HOSTNAME=${samba_hostname} SAMBA_HOSTNAME=${samba_hostname}
NETBIOS_NAME=${netbios_name} NETBIOS_NAME=${netbios_name}
# Optional overrides: # Optional overrides:
@@ -225,6 +250,11 @@ NETBIOS_NAME=${netbios_name}
# BACKUP_DESTINATION=smb://DOMAIN%5Cuser:pass@backup.example.com/Backups/samba # BACKUP_DESTINATION=smb://DOMAIN%5Cuser:pass@backup.example.com/Backups/samba
# BACKUP_DESTINATION=davfs://user:pass@webdav.example.com/remote.php/dav/files/backup # BACKUP_DESTINATION=davfs://user:pass@webdav.example.com/remote.php/dav/files/backup
# BACKUP_DESTINATION=sftp://user:pass@sftp.example.com/exports/samba # BACKUP_DESTINATION=sftp://user:pass@sftp.example.com/exports/samba
# BACKUP_START_HOUR=2
# BACKUP_RETENTION_DAILY=3
# BACKUP_RETENTION_WEEKLY=2
# BACKUP_RETENTION_MONTHLY=2
# BACKUP_RETENTION_YEARLY=1
EOF EOF
chmod 600 "$ENV_FILE" chmod 600 "$ENV_FILE"