cabal install haskell-awk no longer works

[gelisam]

The installation instructions don't work anymore. They say to use cabal install haskell-awk, but that's now ambiguous, it could mean v1-install or v2-install.

With cabal v1-install haskell-awk, I now get

runtime/System/Console/Hawk/Runtime.hs:11:1: error:
    Failed to load interface for ‘Data.ByteString.Lazy.Search’
    There are files missing in the ‘stringsearch-0.3.6.6’ package,
    try running 'ghc-pkg check'.
    Use -v to see a list of the files searched for.

Which either means that our upper bounds are not tight enough or that cabal v1-install is not backwards-compatible enough.

With cabal v2-install haskell-awk, I instead get

src/System/Console/Hawk/Lock.hs:29:1: error:
    Failed to load interface for ‘Network.BSD’
    Use -v to see a list of the files searched for.

src/System/Console/Hawk/Lock.hs:30:1: error:
    Failed to load interface for ‘Network’
    Use -v to see a list of the files searched for.

So it seems like the only way to install hawk now is to install it from source. The instructions are again ambiguous (cabal install).

[gelisam]

cabal v1-install complains that a sandbox is required, apparently the bad old approach of globally installing in ~/.cabal is no longer supported. cabal v1-sandbox && cabal v1-install works well.

[gelisam]

cabal v2-install successfully builds the executable and installs it in ~/.cabal/bin/hawk, but since hawk hasn't been updated to work with v2-install yet, running hawk complains with

hawk: No package-db found. Did you install Hawk in an unusual way?

[gelisam]

stack install works fine. That's probably the best way of installing hawk right now: from source, using stack install.

[gelisam]

Fixed by #216; but I'll keep this issue open to remind me to get the hackage version fixed.

[gelisam]

thanks to #242, cabal v1-install, cabal v2-install, cabal v2-run, and cabal-sandbox are all supported in HEAD, but cabal v1-install haskell-awk and cabal v2-install haskell-hawk will continue to fail until I have uploaded a new version to hackage. so I will once again keep this ticket open until I've either updated the hackage version of the documentation or uploaded a new version to hackage.

[andreasabel]

Under GHC 9.0.1, cabal install haskell-awk succeeds, but it crashes when run:

$ hawk -m reverse
warning: directory '.../.hawk' doesn't exist, creating a default one
error: Won't compile:
	<no location info>: error:
    Could not load module ‘System.Console.Hawk.Runtime.Base’
    It is a member of the hidden package ‘haskell-awk-1.2.0.1’.
    You can run ‘:set -package haskell-awk’ to expose it.
    (Note: this unloads all the modules in the current scope.)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.

[gelisam]

Thanks for the report. Hmm, I wonder why my installation-methods monthly test didn't catch this?

Also, given my last comment above yours, I should probably mention for posterity that I forgot to close this ticket when I released the version of hawk which includes the new installation-method detection code. So the actual issue is that there is a regression in the installation-methods detection code.

[gelisam]

Ah, the installation-methods test didn't detect the problem because it uses ghc-8.10.4! I would expect a new version of cabal-install to break it, not a new version of ghc. I think ghc-9.0.1 requires a fairly recent version of cabal, but for completeness, could you please include the output of cabal --version?

[andreasabel]

Could not load module ‘System.Console.Hawk.Runtime.Base’
It is a member of the hidden package ‘haskell-awk-1.2.0.1’.

I got similar errors when a package was missing from .ghc/$ARCH-$VER/environments/default.

Note that I have write-ghc-environment-files: never in my .cabal/config. However, as far as I understood this switch, it only suppresses local environment files .ghc.environment....

Recently, I have started using the master version of cabal-install.

$ cabal --version
cabal-install version 3.5.0.0
compiled using version 3.5.0.0 of the Cabal library 

For the sake of testing the present issue, I installed cabal-install-3.4.0.0. I notice no difference; it works with GHC 8.10.5:

$ cabal install haskell-awk -w ghc-8.10.5
$ hawk -m reverse <<< "1 2 3"
3 2 1

It fails in the described way with GHC 9.0.1.

[gelisam]

Hmm, it works for me! The next step would thus be to figure out which difference between our two setups is the important one.

$ uname -a
Darwin silver 17.2.0 Darwin Kernel Version 17.2.0: Fri Sep 29 18:27:05 PDT 2017; root:xnu-4570.20.62~3/RELEASE_X86_64 x86_64
$ cat ~/.cabal/config | grep write-ghc-environment-files
-- write-ghc-environment-files:
$ cabal --version
cabal-install version 3.4.0.0
compiled using version 3.4.0.0 of the Cabal library
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.0.1
$ git rev-parse HEAD
55f77ef0f76348865b08fddfc8da063803f88c12
$ cabal install
[...]
$  echo "1 2 3" | ~/.cabal/bin/hawk -m reverse
3 2 1

[gelisam]

i can reproduce the problem with cabal install haskell-awk instead of cabal install. One possible explanation is that HEAD has a regression compared to the most recent published version, but I think a more likely explanation is that cabal install and cabal install <package-name> use different package-db paths, and thus the installation-method detection code for cabal install doesn't work for cabal install <package-name>.

I wonder how I can debug this? It would be rather silly to publish a debug version to hackage, especially if it takes me several attempts!

[gelisam]

I realized that I can type cabal install haskell-awk in the repo folder in order to install that version. Doing so does not reproduce the problem, so either that's because doing behaves the same as cabal install, not the same way as when installing from hackage, or that's because it's indeed a regression.

[gelisam]

Good news, I can reproduce via the regular cabal install on the v1.2.0.1 tag, so it's a regression! That will definitely be much easier to debug, since I won't have to publish anything to hackage.

[gelisam]

Wait, no, if I can reproduce on v1.2.0.1 but not in HEAD, that means this is an anti-regression, not a regression; that is, the bug is already fixed in HEAD :)

Let me know if you would like me to publish a new version with the fix, or if you're fine to build from HEAD until the next version is published. We don't really have a release schedule, so I don't know when that will be.

[gelisam]

git-bisect says it was fixed in #267, which is strange since it's a PR about fixing warnings, not about modifying the installation-method detection code!

[gelisam]

Err, #267 is actually HEAD. So that means that while cabal install worked fine for me on HEAD before, now cabal install now fails for me on HEAD. What?

I guess things are not as simple as "it's fixed on HEAD" after all.

[gelisam]

Ok, so the most suspicious thing I did between then and now is a cabal update. This would indicate a regression (not an anti-regression) in one of hawk's dependencies, or at least a change in one of those dependencies which breaks hawk's installation-method detection code.

[gelisam]

Darn, I ran the cabal v2-update 'hackage.haskell.org,2021-06-07T20:24:57Z' command which that cabal update told me to run if I wanted to undo its effects, but cabal install is still broken. So either there is something else which I changed but forgot about since, or that doesn't actually do a full undo.

[gelisam]

Oh wait! I wasn't reading the error message carefully enough. It now fails with a different error message!

$ echo "1 2 3" | ~/.cabal/bin/hawk -m reverse
error: Won't compile:
	<no location info>: error:
    Ambiguous module name ‘System.Console.Hawk.Runtime.Base’:
      it was found in multiple packages:
      haskell-awk-1.2.0.2 haskell-awk-1.2.0.2

That means I did not clean up properly between installations, and that all of my testing above should be ignored.

I was only doing rm ~/.cabal/bin/hawk to remove the executable. That was too naive, I also need to remove the library somehow.

[gelisam]

I don't know of a good way to uninstall a library, but blowing out ~/.cabal like in the bad old days does work.

I can confirm that after a fresh cabal update and a cabal install on HEAD, hawk does work for me, so the difference between the two setups isn't a change in a dependency.

[gelisam]

using cabal install haskell-awk outside the repo to Install the hackage version also works for me, so I have not managed to reproduce the problem locally after all.

[gelisam]

I was able to install cabal-install HEAD by adding

allow-newer:
  hackage-security:template-haskell

allow-newer:
  cabal-install-solver:base

But I was not able to reproduce the problem with it:

$ cabal-3.5.0.0 install
[...]
$ echo "1 2 3" | ~/.cabal/bin/hawk -m reverse
3 2 1

[gelisam]

Okay, I think this is the closest I can get to your setup on my machine, but I still cannot reproduce the problem. Are there any other differences you can think of? Maybe the OS?

$ uname -a
Darwin silver 17.2.0 Darwin Kernel Version 17.2.0: Fri Sep 29 18:27:05 PDT 2017; root:xnu-4570.20.62~3/RELEASE_X86_64 x86_64
$ cat ~/.cabal/config | grep write-ghc-environment-files
write-ghc-environment-files: never
$ cabal-3.5.0.0 --version
cabal-install version 3.5
compiled using version 3.5.0.0 of the Cabal library
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.0.1
$ git rev-parse HEAD  # to show that I'm not in the repo and am thus installing the hackage version
fatal: not a git repository (or any of the parent directories): .git
$ cabal-3.5.0.0 install haskell-awk
[...]
$  echo "1 2 3" | ~/.cabal/bin/hawk -m reverse
3 2 1

[gelisam]

Sorry @andreasabel, I was not able to reproduce the problem on my machine nor on CI, so I won't be able to fix it. Unless you have another idea of what else could possibly be different between our two setups?

Just going off the error message, it looks like something (perhaps some file? I don't have a ~/.ghc/*/environments/default file for ghc-9.0.1) is passing ghc the flags which hides all the packages except those explicitly listed. Neither hawk nor hint is passing those arguments nor explicitly loading a file which might contain them, but hint is delegating to the ghc library, who might.

[MuhammedZakir]

Sorry for asking here. I will open a new issue for this (if needed) after getting andreasbel's reply.

@andreasabel: How did you install ghc-8.10.5? I installed it with GHCup and for me using v8.10.5, I was able to build hawk, but I am getting this error when running:

hawk: /home/username/.ghcup/ghc/8.10.5/lib/ghc-8.10.5/ghci-8.10.5/HSghci-8.10.5.o: unknown symbol `allocateWrite'
hawk: Could not load Object Code /home/username/.ghcup/ghc/8.10.5/lib/ghc-8.10.5/ghci-8.10.5/HSghci-8.10.5.o.

error: GhcException "unable to load package `ghci-8.10.5'"

I am able to use hawk if I build it using v8.10.4.

P.S. I am using Linux. GHCupk used this one when building on mine -- https://downloads.haskell.org/~ghc/8.10.5/ghc-8.10.5-x86_64-fedora27-linux.tar.xz

[juhp]

@MuhammedZakir sorry but that error is quite unrelated: it is a ghc-8.10.5 RTS regression.

[MuhammedZakir]

Thanks for the link! :-) I was confused whether that problem has to do with my setup, especially when I saw compiling with ghc-8.10.5 worked for andreasabel. I forgot to check in GHC repo (silly me).

[andreasabel]

@gelisam

I don't have a ~/.ghc/*/environments/default file for ghc-9.0.1

This could well make the difference.

If I cabal install --lib haskell-awk, then the haskell-awk library is added to the abovementioned default file, and then hawk works:

$ cabal install --lib haskell-awk
...
Up to date

$ cat ~/.ghc/x86_64-darwin-9.0.1/environments/default
clear-package-db
global-package-db
package-db /Users/abel/.cabal/store/ghc-9.0.1/package.db
package-id ghc-9.0.1
package-id bytestring-0.10.12.1
...
package-id crypthsh-0.11.9-573eb819
package-id mtl-2.2.2
package-id hskll-wk-1.2.0.1-e8dedd4f

$ hawk -m reverse <<< "1 2 3" 
3 2 1

So, maybe on some configurations, one also needs to install with the --lib flag.

P.S.: hskll-wk is what v2-cabal makes from haskell-awk on macOS, see haskell/cabal#7209.