new_local_repository fails when repo contains a BUILD file #24770

davexroth · 2024-12-19T21:24:47Z

Description of the bug:

When using new_local_repository with build_file if the repository being pointed to contains a BUILD.bazel file, bazel will error out with the message below. I would expect the build filed referenced by build_file be used in this case, as happened in bazel 6.5.0.

If build_file_content is used instead of build_file, bazel succeeds.

INFO: Repository +_repo_rules+file instantiated at:
  <builtin>: in <toplevel>
Repository rule new_local_repository defined at:
  /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/bazel_tools/tools/build_defs/repo/local.bzl:103:39: in <toplevel>
ERROR: /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/bazel_tools/tools/build_defs/repo/local.bzl:97:21: An error occurred during the fetch of repository '+_repo_rules+file':
   Traceback (most recent call last):
	File "/usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/bazel_tools/tools/build_defs/repo/local.bzl", line 97, column 21, in _new_local_repository_impl
		rctx.symlink(rctx.attr.build_file, "BUILD.bazel")
Error in symlink: java.io.IOException: Could not create symlink from /usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/file.BUILD.bazel to /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/+_repo_rules+file/BUILD.bazel: /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/+_repo_rules+file/BUILD.bazel (File exists)
ERROR: no such package '@@+_repo_rules+file//': java.io.IOException: Could not create symlink from /usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/file.BUILD.bazel to /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/+_repo_rules+file/BUILD.bazel: /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/+_repo_rules+file/BUILD.bazel (File exists)
ERROR: /usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/BUILD.bazel:9:10: //:file depends on @@+_repo_rules+file//:a in repository @@+_repo_rules+file which failed to fetch. no such package '@@+_repo_rules+file//': java.io.IOException: Could not create symlink from /usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/file.BUILD.bazel to /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/+_repo_rules+file/BUILD.bazel: /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/+_repo_rules+file/BUILD.bazel (File exists)
ERROR: Analysis of target '//:file' failed; build aborted: Analysis failed
INFO: Elapsed time: 5.842s, Critical Path: 0.04s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
FAILED:
    Fetching repository @@rules_cc++cc_configure_extension+local_config_cc; starting

Which category does this issue belong to?

External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Follow the steps in https://github.com/davexroth/bazel_repo_build_overwrite_repro to reproduce.

Which operating system are you running Bazel on?

Linux

What is the output of `bazel info release`?

release 8.0.0

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

Yes. bazel 6.5.0 works, but 7.4.1 and 8.0.0 fail.

Have you found anything relevant by searching the web?

#1697

Any other information, logs, or outputs that you want to share?

INFO: Repository +_repo_rules+file instantiated at:
: in
Repository rule new_local_repository defined at:
/usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/bazel_tools/tools/build_defs/repo/local.bzl:103:39: in
ERROR: /usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/bazel_tools/tools/build_defs/repo/local.bzl:55:13: An error occurred during the fetch of repository '+_repo_rules+file':
Traceback (most recent call last):
File "/usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/bazel_tools/tools/build_defs/repo/local.bzl", line 86, column 29, in _new_local_repository_impl
children = _get_dir_path(rctx).readdir()
File "/usr/local/google/home/davidroth/.cache/bazel/_bazel_davidroth/77517e29f413fe65e1499ca70ade7700/external/bazel_tools/tools/build_defs/repo/local.bzl", line 55, column 13, in _get_dir_path
fail(
Error in fail: The repository's path is "external/file" (absolute: "/usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/external/file") but it does not exist or is not a directory.
ERROR: no such package '@@+_repo_rules+file//': The repository's path is "external/file" (absolute: "/usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/external/file") but it does not exist or is not a directory.
ERROR: /usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/BUILD.bazel:9:10: //:file depends on @@+_repo_rules+file//:a in repository @@+_repo_rules+file which failed to fetch. no such package '@@+_repo_rules+file//': The repository's path is "external/file" (absolute: "/usr/local/google/home/davidroth/GitHub/davexroth/bazel_repo_build_overwrite_repro/external/file") but it does not exist or is not a directory.
ERROR: Analysis of target '//:file' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.205s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

The text was updated successfully, but these errors were encountered:

meteorcloudy · 2025-01-07T14:43:55Z

We should probably delete BUILD.bazel first before creating the file at:
https://cs.opensource.google/bazel/bazel/+/master:tools/build_defs/repo/local.bzl;l=101

Wyverald · 2025-01-08T03:05:29Z

The problem here is that you're specifying path = "external/file",, and that path doesn't exist. The path attribute is for telling Bazel which directory you want to make available as a repo, so it needs to point to something that exists and is a directory.

This should always have been failing, by the way; with your reproducer, I get almost exactly the same failure on 6.5.0 too:

$ USE_BAZEL_VERSION=6.5.0 bazel build :file
2025/01/07 22:02:53 Downloading https://releases.bazel.build/6.5.0/release/bazel-6.5.0-darwin-arm64...
Downloading: 48 MB out of 48 MB (100%) 
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
ERROR: /Users/wyv/test/bazel_repo_build_overwrite_repro/WORKSPACE:18:21: fetching new_local_repository rule //external:file: java.io.IOException: The repository's path is "external/file" (absolute: "/Users/wyv/test/bazel_repo_build_overwrite_repro/external/file") but this directory does not exist.
ERROR: /Users/wyv/test/bazel_repo_build_overwrite_repro/BUILD.bazel:9:10: //:file depends on @file//:a in repository @file which failed to fetch. no such package '@file//': The repository's path is "external/file" (absolute: "/Users/wyv/test/bazel_repo_build_overwrite_repro/external/file") but this directory does not exist.
ERROR: Analysis of target '//:file' failed; build aborted: Analysis failed
INFO: Elapsed time: 10.377s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (32 packages loaded, 165 targets configured)
    Fetching repository @local_config_cc; Restarting.

Wyverald · 2025-01-08T03:07:49Z

We should probably delete BUILD.bazel first before creating the file at: https://cs.opensource.google/bazel/bazel/+/master:tools/build_defs/repo/local.bzl;l=101

This shouldn't be necessary, as rctx.file always overwrites the file: https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/bazel/repository/starlark/StarlarkBaseExternalContext.java;drc=7e7ae8c1b4adcaf2ac11bc76231a10f7f88a9eb4;l=1390

davexroth · 2025-01-08T17:38:41Z

Sorry for the confusion, I somehow didn't include the external path in the repo. I've now updated the reproduction with the missing files to reproduce the problem.

I've verified again, and 6.5.0, but 7.4.1 and 8.0.0 fail.

Wyverald · 2025-01-08T19:04:45Z

Thanks, now the bug is evident -- very close to where Yun linked above, we should delete the existing BUILD.bazel file when we introduce a symlink (so https://cs.opensource.google/bazel/bazel/+/master:tools/build_defs/repo/local.bzl;drc=33775c9e0d78a821aecbed96d3c1dec222a754dc;l=97). This should be very easy to fix.

meteorcloudy · 2025-01-09T09:51:52Z

@davexroth Are you willing to give it a try to fix this? Our bandwidth is pretty limited but happy to review a PR!

…_repository Fixes bazelbuild#24770

davexroth · 2025-01-09T23:21:35Z

Sure thing. See linked PR.

davexroth added type: bug untriaged labels Dec 19, 2024

davexroth assigned iancha1992, satyanandak and sgowroji Dec 19, 2024

github-actions bot added the team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. label Dec 19, 2024

iancha1992 unassigned sgowroji, iancha1992 and satyanandak Dec 19, 2024

Wyverald added P2 We'll consider working on this in future. (Assignee optional) good first issue and removed untriaged labels Jan 8, 2025

davexroth added a commit to davexroth/bazel that referenced this issue Jan 9, 2025

Remove existing BUILD.bazel before symlinking when creating new_local…

e75004f

…_repository Fixes bazelbuild#24770

davexroth linked a pull request Jan 9, 2025 that will close this issue

Remove BUILD.bazel before symlinking when creating new_local_repository #24887

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new_local_repository fails when repo contains a BUILD file #24770

new_local_repository fails when repo contains a BUILD file #24770

davexroth commented Dec 19, 2024 •

edited

Loading

meteorcloudy commented Jan 7, 2025

Wyverald commented Jan 8, 2025

Wyverald commented Jan 8, 2025

davexroth commented Jan 8, 2025

Wyverald commented Jan 8, 2025

meteorcloudy commented Jan 9, 2025

davexroth commented Jan 9, 2025

new_local_repository fails when repo contains a BUILD file #24770

new_local_repository fails when repo contains a BUILD file #24770

Comments

davexroth commented Dec 19, 2024 • edited Loading

Description of the bug:

Which category does this issue belong to?

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Which operating system are you running Bazel on?

What is the output of bazel info release?

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

What's the output of git remote get-url origin; git rev-parse HEAD ?

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

meteorcloudy commented Jan 7, 2025

Wyverald commented Jan 8, 2025

Wyverald commented Jan 8, 2025

davexroth commented Jan 8, 2025

Wyverald commented Jan 8, 2025

meteorcloudy commented Jan 9, 2025

davexroth commented Jan 9, 2025

davexroth commented Dec 19, 2024 •

edited

Loading

What is the output of `bazel info release`?

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

What's the output of `git remote get-url origin; git rev-parse HEAD` ?