Configure failed with "file setup.py not found"

[PatrickSchloesser]

After downloading deltacode and running ./deltacode --help to configure, I get the following error message:

ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /local/worker/testruns/1d3c4735-714d-4292-909b-69854d0698ab/src/LicenseChecker ./deltacode/deltacode: line 114: /local/worker/testruns/1d3c4735-714d-4292-909b-69854d0698ab/src/LicenseChecker/deltacode/bin/deltacode: No such file or directory /local/worker/testruns/1d3c4735-714d-4292-909b-69854d0698ab/src/LicenseChecker/bin/process_delta.rb:59:in read': No such file or directory @ rb_sysopen - /local/worker/testruns/1d3c4735-714d-4292-909b-69854d0698ab/src/LicenseChecker/output.json (Errno::ENOENT)
from /local/worker/testruns/1d3c4735-714d-4292-909b-69854d0698ab/src/LicenseChecker/bin/process_delta.rb:59:in <main>'

I am fairly certain that the "setup.py not found" error is coming from pip install -e .[testing] in the configure script: https://github.com/nexB/deltacode/blob/develop/configure#L38, but I'm not sure how to fix it.

I am running this on linux and have both python3.7 and python2.7 installed at the level of the root directory.

I realize this is likely not an issue specifically with deltacode; however, I haven't found success finding resources to help elsewhere, so I'd appreciate any help you can provide!

[steven-esser]

@PatrickSchloesser Good catch, this is definitely a bug and an easy fix. See the PR I have just opened for the fix: #164

Once all the CI checks have passed, ill merge it in.

[PatrickSchloesser]

Perfect! Thanks for looking into this.

[steven-esser]

@PatrickSchloesser #164 merged. Will close this ticket after your confirmation. Thanks for the bug report! :)

[PatrickSchloesser]

I tried running deltacode again with your new changes incorporated, and I'm still getting the same error. We pull anew directly from github each time the license checker is run, so I'm certain I'm using the newest version. I don't think that your fix is incorrect but rather that my python is setup slightly differently than I thought.

In any case, we have decided to roll back to the last version of deltacode that worked for us since we have decided to pick a particular commit hash for both scancode and deltacode to prevent changes to the tools causing potential negative impacts. However, when I checkout any commit including or prior to January 24th, the main deltacode script is not included in the topmost directory. We've been running ./deltacode to run deltacode from the top directory since last October, so I'm confused why it disappears when I checkout old commits. Do you have any ideas about what I might be doing incorrectly? Currently, when I checkout the commit from January 24th, the topmost directory includes only:

AUTHORS.rst
MANIFEST.in
README.rst
azure-pipelines.yml
configure.bat
etc
setup.cfg
src
tmp
CHANGELOG.rst
NOTICE
apache-2.0.LICENSE
configure
docs
pyproject.toml
setup.py
tests

I realize this is not the response you were looking for; however, I do think this particular issue is continuing to happen due to the complexities of how we're running deltacode (randomly assigned linux machine, builds our license checker, pulls deltacode into the license checker src) and not a bug on your end.

[steven-esser]

@PatrickSchloesser Thanks for the feedback. Let me check into this on my side.

At first glance, I think the missing top-level script come from the way we integrate our repo skeleton: https://github.com/nexB/skeleton. So when you go to check out a "random" commit, it may be files from only this skeleton side. If possible, you may want to revert to using the v1.0.0 tag in your job instead of a commit hash before certain dates.

However, if you are using the very latest from the develop branch, this script should be present and working. What command(s) do you use to fetch the deltacode repo?

[PatrickSchloesser]

I've simply been using the following commands:
git clone https://github.com/nexB/deltacode.git
git checkout commithash

What I've observed is that the when I first pull deltacode, the script is there, but as soon as I checkout the old commit, it disappears from the directory.

I wasn't aware of the v1.0.0 tag. Is that simply including -v1.0.0 in the run command?

[steven-esser]

To checkout the v1.0.0 tag, run git checkout v1.0.0.

What happens if you omit the git checkout commithash command? This should place you at the head of the develop branch.

[PatrickSchloesser]

I see I'll give it a try with the tag. If I do not include the git checkout command, I get the same file "setup.py" not found issue with the same log as in my first post on this issue. I also have the commit hash print with every run, and when I don't specify the git checkout commithash, it shows that it's running with the current head bf767c2.

[PatrickSchloesser]

It looks like deltacode is fixed by using the v1.0.0 tag you recommended; however, when running the entire process (scancode 3.2.3 -> deltacode), I get the same error as was reported by this issue: #122 . I'm aware that this issue was fixed, but since I'm using an earlier v1.0.0, it's not included. I would simply make the code change locally, but because because deltacode is pulled each time the license checker is run, it would be pretty inelegant to change "tool_version" to "scancode_version" each run on the fly. Do you have any ideas on how to get around this issue?

[steven-esser]

@PatrickSchloesser sorry for the delayed response. The only workaround I can think of at the moment would be to run git checkout 31dc82f8d19c1dd5fd4c4d11f7d1076c0dbd0a5c in your build script (as opposed to the v1.0.0 tag).

This commit is immediately after merging the fix to #122, so it should be far enough back in history to not have some of the weirdness you experienced when checking out a commithash previously.

[Pratikrocks]

@MaJuRG how about having a docker image for deltacode?