write /data/metastore/LOCK: no space left on device

[doron-cohen]

Hey, I am running Parca server using your supplied Helm chart with the following configuration:

  server:
    # -- Allows disabling parca server
    enabled: true
    image:
      # -- Overrides the image repository for server
      repository: ghcr.io/parca-dev/parca
      # -- Overrides the image tag for server
      tag: "v0.23.1"
    # -- logging level of parca server
    logLevel: info
    # -- Active memory dedicated for storage.
    storageActiveMemory: 536870912

    extraArgs:
      - --enable-persistence
      - --storage-enable-wal
      - --storage-snapshot-trigger-size=134217728
      - --storage-path=/data
      - --storage-index-on-disk

I get these logs which indicate there is no more storage left on disk:

level=info name=parca ts=2025-09-16T11:33:00.238416702Z caller=factory.go:53 msg="loading bucket configuration"
level=error name=parca ts=2025-09-16T11:33:00.238738041Z caller=parca.go:278 msg="failed to open badger database for metastore" err="Cannot write pid file \"/data/metastore/LOCK\" error: write /data/metastore/LOCK: no space left on device"
level=error name=parca ts=2025-09-16T11:33:00.238759932Z caller=main.go:66 msg="Program exited with error" err="Cannot write pid file \"/data/metastore/LOCK\" error: write /data/metastore/LOCK: no space left on device"

How can I better manage storage retention? We are struggling with this for a while and I couldn't find anything that would help me manage retention on disk. I only found #2829 for retention in S3.

[anarcher]

I have the same concern. How about periodically running a cronjob that deletes old files using find -mtime +n? (like an S3 retention policy)

[doron-cohen]

I could do something like that although I am not sure I can attach the volume to the cronjob while it is attached to the service. I am also unsure of the directory and file structure. I guess S3 is the better solution here. Just as a comment this project is awesome but getting it to work is really hard.