cargo release publish --dry-run --workspace fails due to unpublished workspace dependencies

[teor2345]

We're having some trouble doing a dry run of publishing our crates in a workspace.

When we actually publish to crates.io, everything works fine. But we can't test publishing before a release, because we get missing dependency errors during the dry run.

We have a circular path dev-dependency from tower-batch-control to zebra-consensus, but removing it using cargo hack --remove-dev-dependencies does not fix the issue.

It seems like everything that depends on tower-batch-control has this error. But everything outside of that dependency subtree works fine.

Things we've tried

We've tried depending on a published version, which makes the dry run succeed. But then we run into bug #690, where older published versions are overwritten by cargo release version. (It looks like we might just have to set published versions manually.)

Failure Logs

Here's the full error:

   Packaging zebra-consensus v1.0.0-beta.28 (/home/runner/work/zebra/zebra/zebra-consensus)
    Updating crates.io index
error: failed to prepare local package for uploading

Caused by:
  no matching package named `tower-batch-control` found
  location searched: registry `crates-io`
  required by package `zebra-consensus v1.0.0-beta.28 (/home/runner/work/zebra/zebra/zebra-consensus)`

https://github.com/ZcashFoundation/zebra/actions/runs/5449357267/jobs/9913492809?pr=7128#step:7:200

But earlier in the cargo release dry run, that version of tower-batch-control is "published":

  Publishing tower-batch-control
[2023-07-03T23:42:56Z DEBUG cargo_release::steps::publish] skipping verification to avoid unpublished dependencies from dry-run
    Updating crates.io index
   Packaging tower-batch-control v0.2.41-beta.4 (/home/runner/work/zebra/zebra/tower-batch-control)
    Packaged 12 files, 49.7KiB (14.0KiB compressed)
   Uploading tower-batch-control v0.2.41-beta.4 (/home/runner/work/zebra/zebra/tower-batch-control)
warning: aborting upload due to dry run

https://github.com/ZcashFoundation/zebra/actions/runs/5449357267/jobs/9913492809?pr=7128#step:7:161

Success Logs

Other crates that depend on unpublished versions of crates outside the failing subtree work fine:

  Publishing zebra-test
[2023-07-03T23:42:55Z DEBUG cargo_release::steps::publish] skipping verification to avoid unpublished dependencies from dry-run
    Updating crates.io index
   Packaging zebra-test v1.0.0-beta.28 (/home/runner/work/zebra/zebra/zebra-test)
    Packaged 108 files, 3.9MiB (1.5MiB compressed)
   Uploading zebra-test v1.0.0-beta.28 (/home/runner/work/zebra/zebra/zebra-test)
warning: aborting upload due to dry run
  Publishing zebra-chain
[2023-07-03T23:42:56Z DEBUG cargo_release::steps::publish] skipping verification to avoid unpublished dependencies from dry-run
    Updating crates.io index
   Packaging zebra-chain v1.0.0-beta.28 (/home/runner/work/zebra/zebra/zebra-chain)
    Packaged 189 files, 1.2MiB (274.1KiB compressed)
   Uploading zebra-chain v1.0.0-beta.28 (/home/runner/work/zebra/zebra/zebra-chain)
warning: aborting upload due to dry run

[epage]

cargo publish --dry-run requires all dependencies to be in the registry.

If we detect you are publishing multiple packages, we'll give a working but degraded experience by calling cargo publish --no-verify.

You can pass --no-verify to cargo release publish in cases where we aren't auto-detecting that its needed.

In dry-run mode, we could probably add a check for all dependencies to see if they are published and automatically add --no-verify.

[kaleidawave]

Also having this issue. Have a main crate x that locally depends on y. I run publish --dry-run in CI to check that it is readily publishable. Added a new feature to ys manifest and enabled in xs manifest. --dry-run now fails with

...
the package `x` depends on `y`, with features: `feature` but `y` does not have these features.


  failed to select a version for `x` which could resolve this conflict

don't quite understand local dependency versioning but would be good if this could be fixed or add a --allow-local which uses the local version rather than going to crates.io

[epage]

The most we can do for this

• skip the verification step
• warn the user which dependencies are not yet in the registry

[teor2345]

The most we can do for this

• skip the verification step
• warn the user which dependencies are not yet in the registry

Is it possible to use the local dependencies instead?

(Or if it’s not a dry run, publish in dependency order if there are no cycles?)

[epage]

Currently, there is no way to publish a package, even with dry-run, without all of the other packages published, even if done in dependency order. If you do cargo release --workspace, we just skip the verification step because there is nothing we can do. We are blocked on needing features from cargo for this.