[Bug]: mason build does not rebuild modified submodules

[jmag722]

Summary of Problem

Running mason build will not rebuild a submodule without --force.

Description:
When using submodules in a Chapel library, if I make changes within a submodule and recompile using mason build, it will not update the build. Instead it will say:
Skipping Build... No changes to project
but was expecting:

Compiling [debug] target: Foo
Build Successful

I believe this is because the MasonUtils.chpl:projectModified procedure only iterates over source files defined in the top-level of src/, rather than recursively checking each subdirectory. Indeed when I'd change the top-level file, Mason would rebuild as expected.

Is this issue currently blocking your progress?
No, easy workaround with mason build --force

Steps to Reproduce

Source Code:

// module Foo in top-level src/Foo.chpl
module Foo {
  include module Bar;
  public import this.Bar;
}

// in src/Foo/Bar.chpl
module Bar {
  // run mason build toggling this procedure on and off (or making any other change)
  proc baz(in x: int) {
    return x + 2;
  }
  proc baz(in x: real) {
    return x * 2.0;
  }
}

Mason.toml

[brick]
name="Foo"
version="0.1.0"
type="library"
chplVersion="2.4.0"

[tool.chpl-language-server]
module-dir=["src"]

Compile command:
mason build
also tried things like:
mason build -- -M src/Foo/Bar.chpl

Execution command:
See compile command.

Associated Future Test(s):
N/A

Configuration Information

• Output of chpl --version:

chpl version 2.7.0 pre-release (6f45b6fe6b)
  built with LLVM version 18.1.3
  available LLVM targets: xtensa, m68k, xcore, x86-64, x86, wasm64, wasm32, ve, systemz, sparcel, sparcv9, sparc, riscv64, riscv32, ppc64le, ppc64, ppc32le, ppc32, nvptx64, nvptx, msp430, mips64el, mips64, mipsel, mips, loongarch64, loongarch32, lanai, hexagon, bpfeb, bpfel, bpf, avr, thumbeb, thumb, armeb, arm, amdgcn, r600, aarch64_32, aarch64_be, aarch64, arm64_32, arm64
Copyright 2020-2025 Hewlett Packard Enterprise Development LP
Copyright 2004-2019 Cray Inc.
(See LICENSE file for more details)

• Output of $CHPL_HOME/util/printchplenv --anonymize:

CHPL_TARGET_PLATFORM: linux64
CHPL_TARGET_COMPILER: llvm
CHPL_TARGET_ARCH: x86_64
CHPL_TARGET_CPU: native
CHPL_LOCALE_MODEL: flat
CHPL_COMM: none
CHPL_TASKS: qthreads
CHPL_LAUNCHER: none
CHPL_TIMERS: generic
CHPL_UNWIND: bundled
CHPL_TARGET_MEM: jemalloc
CHPL_ATOMICS: cstdlib
CHPL_GMP: bundled
CHPL_HWLOC: bundled
CHPL_RE2: bundled
CHPL_LLVM: system
CHPL_AUX_FILESYS: none

• Back-end compiler and version, e.g. gcc --version or clang --version:

clang version 21.1.4
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/jared/opt/spack/opt/spack/linux-skylake/llvm-21.1.4-5yzvwku7jb2musecr2rsaos2p4b4aj5p/bin
Configuration file: /home/jared/opt/spack/opt/spack/linux-skylake/llvm-21.1.4-5yzvwku7jb2musecr2rsaos2p4b4aj5p/bin/clang.cfg

[e-kayrakli]

Hmm.. I am unable to reproduce this. If I have the structure as you outlined, when I change Bar.chpl, I see that stat reports the src/Foo directory to have been modified. As such, even though you are right that projectModified only walks the top-level contents, it sees the subdirectory containing Bar.chpl to be modified and it compiles without --force.

Just so that you can check my work, here is what I have. It is based on your reproducer but with some additions to make sure that I can see the behavior when I do mason run:

Mason.toml:

[brick]
name="Foo"
version="0.1.0"
chplVersion="2.7.0"
license="None"
type="application"

[dependencies]

src/Foo.chpl:

module Foo {
  include module Bar;
  public import this.Bar;

  Bar.baz(10);
  Bar.baz(10.0);
}

src/Foo/Bar.chpl:

module Bar {
  // run mason build toggling this procedure on and off (or making any other
  // change)
  proc baz(in x: int) {
    writeln("int overload called 3");
    return x + 2;
  }
  proc baz(in x: real) {
    writeln("real overload called");
    return x * 2.0;
  }
}

[jmag722]

Haven't had a chance to test this yet, but maybe the difference is in my Mason.toml I'm using type="library" without any function calls in Foo.chpl?

…

On Mon, Nov 24, 2025, 09:40 Engin Kayraklioglu ***@***.***> wrote: *e-kayrakli* left a comment (chapel-lang/chapel#28100) <#28100 (comment)> Hmm.. I am unable to reproduce this. If I have the structure as you outlined, when I change Bar.chpl, I see that stat reports the src/Foo directory to have been modified. As such, even though you are right that projectModified only walks the top-level contents, it sees the subdirectory containing Bar.chpl to be modified and it compiles without --force. Just so that you can check my work, here is what I have. It is based on your reproducer but with some additions to make sure that I can see the behavior when I do mason run: Mason.toml: [brick]name="Foo"version="0.1.0"chplVersion="2.7.0"license="None"type="application" [dependencies] src/Foo.chpl: module Foo { include module Bar; public import this.Bar; Bar.baz(10); Bar.baz(10.0); } src/Foo/Bar.chpl: module Bar { // run mason build toggling this procedure on and off (or making any other // change) proc baz(in x: int) { writeln("int overload called 3"); return x + 2; } proc baz(in x: real) { writeln("real overload called"); return x * 2.0; } } — Reply to this email directly, view it on GitHub <#28100 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKI3QHVPQB6NTEUBGGWFLHL36MRFRAVCNFSM6AAAAACNAY4KAKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKNZRGQ2DMNJWHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[e-kayrakli]

Good catch. But alas, I see things behave correctly. I also checked on linux64, where I was using MacOS before. As we rely on system calls, such portability issues could matter, but that doesn't seem to be the case.

I am happy to investigate this further. If you have a full repo or something where I can directly run mason build that could be a better controlled experiment.

[jmag722]

Oh that is strange. My minimum working example is at https://github.com/jmag722/chapel_mwe, commit e7ea9984e08a42f4cc68abb05a818592f1889724

I'll be able to look into this more mid next week after thanksgiving

[jmag722]

I rebuilt chapel with Spack, I'd thought I'd compiled previously with LLVM 21 but looks like it found the default system version first (18.1.3). But even with the new build I'm still getting the same issue ...

Looking at MasonUtils.chpl:projectModified though, I think the current behavior I'm seeing is what's expected from the code? It looks like it won't rebuild unless top-level src file and directory mtimes are newer than those of the target binaries. When you change a submodule file after compiling, does its parent directory mtime change too?

[jabraham17]

I was able to reproduce the issue and I have a working fix. I'll open a PR shortly

[e-kayrakli]

I was able to reproduce the issue and I have a working fix. I'll open a PR shortly

I am curious to hear if you have a guess as to why I wasn't able to repro, beyond potential incompetence on my part. :D

[jabraham17]

No idea 😄, but the fix is #28119