-
Notifications
You must be signed in to change notification settings - Fork 133
Troubleshooting
We strongly encourage using conda to maintain the Python libraries on which RAVEN depends. Note that conda can be installed locally without administrative privileges on most systems. However, if conda is not available, the necessary libraries and versions can be obtained by the following call:
cd raven
python scripts/TestHarness/testers/RavenUtils.py --conda-installThis will print to terminal a command that includes a listing of all the libraries, some with specific tagged versions. These libraries may be installed through pip or other Python or OS package managers, after which RAVEN can be run.
If library versions other than the specific tagged versions are installed, RAVEN will exit when run and complain of improper libraries for the version of RAVEN you are trying to run.
If there are missing libraries, (for example if you get errors like ImportError: No module named sklearn) then the Installing RAVEN Libraries page should be used to try and fix the problem.
In the raven directory, the command:
./run_tests --library_report
can be used to see which libraries are being used and which are missing.
Older versions of the MOOSE redistributable package (From http://mooseframework.org/getting-started/osx/ ) contains everything needed to run RAVEN except the scikit-learn package for Python. If this package is missing, running the tests looks like this...
[RAVENUser][~/projects/raven]> ./run_tests
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named sklearn
No raven_libs found and missing modules sklearn
core_example.PWR_CoreChannel_pre_dist.................................... skipped (Broken for unknown reason)
core_example.PWR_CoreChannel_controlled_var_csv......... skipped (Broken, and Josh is not sure what it te...)
core_example.TypPWR_Mult_CoreChannels_control_csv................................. skipped (Relap issue #111)
core_example.TypPWR_Mult_CoreChannels_control..................................... skipped (Relap issue #111)
check_errors.missing_python_file.......................................................................... OK
core_example.simple_branch................................................................................ OK
core_example.pump......................................................................................... OK
DtControlAndProximity.Dt_controlling_test................................................................. OK
DtControlAndProximity.TimeController_test................................................................. OK
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named sklearn
framework.simple_framework skipped (Missing python modules: sklearn PYTHONPATH=/Users/bobk/raven_libs/pylibs/lib/python2.7/site-packages/:/opt/moose/distcc-3.2rc1/lib/python2.6/site-packages)
Fortunately, the redistributable includes the Miniconda package manager which can be used to install it.
[RAVENUser][~]> sudo su
Password:
sh-3.2# conda install scikit-learn
Fetching package metadata: ....
Solving package specifications: .
Package plan for installation in environment /opt/moose/miniconda:
The following packages will be downloaded:
package | build
---------------------------|-----------------
sqlite-3.8.4.1 | 1 824 KB
tk-8.5.18 | 0 1.9 MB
conda-env-2.1.4 | py27_0 15 KB
nose-1.3.4 | py27_1 189 KB
numpy-1.9.2 | py27_0 2.9 MB
requests-2.6.0 | py27_0 594 KB
setuptools-15.0 | py27_0 436 KB
conda-3.10.1 | py27_0 164 KB
pip-6.1.1 | py27_0 1.4 MB
scikit-learn-0.16.0 | np19py27_0 3.3 MB
------------------------------------------------------------
Total: 11.7 MB
The following NEW packages will be INSTALLED:
nose: 1.3.4-py27_1
pip: 6.1.1-py27_0
scikit-learn: 0.16.0-np19py27_0
The following packages will be UPDATED:
conda: 3.10.0-py27_0 --> 3.10.1-py27_0
conda-env: 2.1.3-py27_0 --> 2.1.4-py27_0
numpy: 1.9.1-py27_0 --> 1.9.2-py27_0
requests: 2.5.3-py27_0 --> 2.6.0-py27_0
setuptools: 12.2-py27_0 --> 15.0-py27_0
sqlite: 3.8.4.1-0 --> 3.8.4.1-1
tk: 8.5.15-0 --> 8.5.18-0
Proceed ([y]/n)? y
Fetching packages ...
sqlite-3.8.4.1 100% |################################| Time: 0:00:01 650.17 kB/s
tk-8.5.18-0.ta 100% |################################| Time: 0:00:02 717.50 kB/s
conda-env-2.1. 100% |################################| Time: 0:00:00 203.41 kB/s
nose-1.3.4-py2 100% |################################| Time: 0:00:00 359.41 kB/s
numpy-1.9.2-py 100% |################################| Time: 0:00:05 554.83 kB/s
requests-2.6.0 100% |################################| Time: 0:00:00 639.57 kB/s
setuptools-15. 100% |################################| Time: 0:00:00 597.81 kB/s
conda-3.10.1-p 100% |################################| Time: 0:00:00 466.21 kB/s
pip-6.1.1-py27 100% |################################| Time: 0:00:01 771.29 kB/s
scikit-learn-0 100% |################################| Time: 0:00:04 865.69 kB/s
Extracting packages ...
[ COMPLETE ] |##################################################| 100%
Unlinking packages ...
[ COMPLETE ] |##################################################| 100%
Linking packages ...
[ COMPLETE ] |##################################################| 100%
sh-3.2#
If there are problems with distribution1D or interpolationND these are problems with the crow libraries. These can be built with ./build_raven in the raven directory.
Also, sometimes if the build version of python is different from the run version, the libraries cannot be found. This can generally be fixed by:
./clean_raven
./build_raven
to rebuild the libraries.
If some of the data can not be retrieved from the internet (or git clone or update commands fail) possibly the proxy is not being used. See INL proxy
INL cluster compute nodes cannot access the internet, so ssh to the login node and run them from the login node.
On some systems (for example Ubuntu 16 and Windows 7) the default backend for matplotlib is set to Qt5Agg, which may not be properly set up after installing libraries through conda. A simple fix is to change the default backend to something more generic, for example TkAgg.
The matplotlib defaults file for the raven_libraries conda environment is usually located in ~/miniconda2/envs/raven_libraries/lib/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc. If not, the location can be discovered via the following python commands (from here):
>>> import matplotlib
>>> matplotlib.matplotlib_fname()
'/home/foo/.config/matplotlib/matplotlibrc'Other common paths include ~/miniconda2/envs/raven_libraries/libs/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc.
The keyword to change is "backend".
The problems using QT5Agg seem not to be present if matplotlib is updated to version 2.1.1 or later. RAVEN will likely move to this library version in a future update.
If RAVEN runs in general are failing with errors regarding Qt and Matplotlib, this change often resolves the problem.
It seems we have some proxy problem using conda to install raven libraries:
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/main/osx-64/current_repodata.json>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
If your current network has https://www.anaconda.com blocked, please file
a support request with your network engineering team.
ConnectTimeout(MaxRetryError("HTTPSConnectionPool(host='repo.anaconda.com', port=443): Max retries exceeded with url: /pkgs/main/osx-64/current_repodata.json (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x10be63d68>, 'Connection to webbalance.inl.gov timed out. (connect timeout=9.15)'))"))
I have searched online, and found the following temporary solution:
conda config --set ssl_verify false
After this command, you can use conda to install raven libraries, and then using
conda config --set ssl_verify true
Please be careful to use this option, and if you have a better solution, please advise. some link: (https://support.anaconda.com/hc/en-us/articles/360024043753-SSL-Verification-Error-in-Conda)
- Open Chrome, got to any website, click on the lock icon on the left of the URL. Click on Certificate on the dropdown. Select Zeus-Cluster.inl.gov, and click details, go to the bottom of the details and click the URI to download the certificate.
URI http://certstore.inl.gov/pki/CAINLROOT_CAINLROOT.crt
- Convert this certificate with openssl: openssl x509 -inform der -in ~/Downloads/CAINLROOT_CAINLROOT.crt -out /path/to/converted/certificate.pem
- For a quick check set your shell to acknowledge the certificate: export REQUESTS_CA_BUNDLE=/path/to/converted/certificate.pem
- To set this permanently open your shell profile (.bash_profile) and add this line: export REQUESTS_CA_BUNDLE=/path/to/converted/certificate.pem. Now exit your terminal/shell and reopen. Check again.
Depending on security settings, the python binary libraries might need to be signed on MacOS. If this is the problem, the .so files are created, but they cannot be loaded. A self signed certificate can be created by following instructions in https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html and then the name of it can be added to the .ravenrc file (for example: RAVEN_SIGNATURE = John Doe ) (The .ravenrc file is created in the raven directory when scripts/establish_conda_env.sh --install is run) Then when build_raven is run it will sign the compiled binary files.
If you get a network error like:
$ conda install numpy
Collecting package metadata (current_repodata.json): failed
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/main/osx-64/current_repodata.json>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
If your current network has https://www.anaconda.com blocked, please file
a support request with your network engineering team.
'https://repo.anaconda.com/pkgs/main/osx-64'
try running it with -v:
$ conda install -v numpy
Collecting package metadata (current_repodata.json): ...working... Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1123)'))': /pkgs/r/osx-64/current_repodata.json
(and the output will go on for a while).
Or if you get an error like this when running git:
$ git clone https://github.com/idaholab/raven.git
Cloning into 'raven'...
fatal: unable to access 'https://github.com/idaholab/raven.git/': SSL certificate problem: self signed certificate in certificate chain
This can be caused by a corporate network or a virus scanner imposing it self between your computer and github or conda servers. This can also be caused by someone trying to hack your computer, so make sure you figure out which it is.
The certificate used can be found by using a command like:
openssl s_client -connect github.com:443
or
openssl s_client -connect repo.anaconda.com:443
In order to remove these errors, the first thing you need to do is get the certificate to use. The corporate IT department or the virus scanner vendor should provide this to you.
If it is provided as a .crt file (for example CARoot.crt), it can be converted to a .pem file with the command like:
openssl x509 -inform der -in CARoot.crt -out CARoot.pem
Alternatively, the openssl s_client -connect command can be used to get the root certificate, but if someone is trying to hack your computer, this will work as well.
For conda, export the ```REQUESTS_CA_BUNDLE`` pointing to the pem file (example):
export REQUESTS_CA_BUNDLE=/home/fred/CARoot.pem
For git, set the sslCAInfo variable (example):
git config --global http.sslCAInfo /home/fred/CARoot.pem
(If you ever need to remove this, this will work:
git config --global --unset http.sslcainfo
)
This removes SSL checking which will stop ssl checking errors from happening.
These commands make it easy for anyone that can control the network you connect to to hack your computer, by disabling the ssl checking:
conda config --set ssl_verify false
git config --global http.sslVerify false
To reenable ssl checking:
conda config --set ssl_verify true
git config --global --unset http.sslVerify