[Artix] [py310] Setting up VEnv’s for a legacy Python required by SDXL WebUI
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
[Artix] [py310] Setting up VEnv’s for a legacy Python required by SDXL WebUI
My system’s python is the most recent 3.12.x
But SDiff XL WebUI requires Python 3.10.x which is a legacy Python for my system
Are there guides to set up a virt env for such a legacy Python?
With my SDiff XL WebUI’s webui-user.sh written as below,
################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye)
################################################################
################################################################
Running on hd_scania user
################################################################
################################################################
Experimental support for Renoir: make sure to have at least 4GB of VRAM and 10GB of RAM or enable cpu mode: --use-cpu all --no-half
################################################################
################################################################
ERROR: python3-venv is not installed, aborting...
################################################################
How to redirect the libs to a specific legacy Python version in order to install PyTorch for SDiff XL?
Code:
################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye)
################################################################
################################################################
Running on hd_scania user
################################################################
################################################################
Experimental support for Renoir: make sure to have at least 4GB of VRAM and 10GB of RAM or enable cpu mode: --use-cpu all --no-half
################################################################
################################################################
Create and activate python venv
################################################################
################################################################
Accelerating launch.py...
################################################################
2023-10-11 09:52:26.838501: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/usr/lib/python3.11/site-packages/h5py/__init__.py:36: UserWarning: h5py is running against HDF5 1.14.2 when it was built against 1.14.1, this may cause problems
_warn(("h5py is running against HDF5 {0} when it was built against {1}, "
[09:52:29] WARNING The following values were not passed to `accelerate launch` and had defaults used instead: launch.py:895
`--num_processes` was set to a value of `0`
`--num_machines` was set to a value of `1`
`--mixed_precision` was set to a value of `'no'`
`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Python 3.11.5 (main, Sep 2 2023, 18:29:07) [GCC 13.2.1 20230801]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Installing torch and torchvision
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
python-xyz', where xyz is the package you are trying to
install.
If you wish to install a non-Arch-packaged Python package,
create a virtual environment using 'python -m venv path/to/venv'.
Then use path/to/venv/bin/python and path/to/venv/bin/pip.
If you wish to install a non-Arch packaged Python application,
it may be easiest to use 'pipx install xyz', which will manage a
virtual environment for you. Make sure you have python-pipx
installed via pacman.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
Traceback (most recent call last):
File "/opt/local/.A/SDXL/WebUI/launch.py", line 48, in <module>
main()
File "/opt/local/.A/SDXL/WebUI/launch.py", line 39, in main
prepare_environment()
File "/opt/local/.A/SDXL/WebUI/modules/launch_utils.py", line 352, in prepare_environment
run(f'"{python}" -m {torch_command}', "Installing torch and torchvision", "Couldn't install torch", live=True)
File "/opt/local/.A/SDXL/WebUI/modules/launch_utils.py", line 115, in run
raise RuntimeError("\n".join(error_bits))
RuntimeError: Couldn't install torch.
Command: "/usr/bin/python" -m pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/nightly/rocm5.1.1 python launch.py
Error code: 1
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/bin/accelerate:8 in <module> │
│ │
│ 5 from accelerate.commands.accelerate_cli import main │
│ 6 if __name__ == "__main__": │
│ 7 │ sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) │
│ ❱ 8 │ sys.exit(main()) │
│ 9 │
│ │
│ /usr/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py:45 in main │
│ │
│ 42 │ │ exit(1) │
│ 43 │ │
│ 44 │ # Run │
│ ❱ 45 │ args.func(args) │
│ 46 │
│ 47 │
│ 48 if __name__ == "__main__": │
│ │
│ /usr/lib/python3.11/site-packages/accelerate/commands/launch.py:923 in launch_command │
│ │
│ 920 │ elif defaults is not None and defaults.compute_environment == ComputeEnvironment.AMA │
│ 921 │ │ sagemaker_launcher(defaults, args) │
│ 922 │ else: │
│ ❱ 923 │ │ simple_launcher(args) │
│ 924 │
│ 925 │
│ 926 def main(): │
│ │
│ /usr/lib/python3.11/site-packages/accelerate/commands/launch.py:579 in simple_launcher │
│ │
│ 576 │ process.wait() │
│ 577 │ if process.returncode != 0: │
│ 578 │ │ if not args.quiet: │
│ ❱ 579 │ │ │ raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) │
│ 580 │ │ else: │
│ 581 │ │ │ sys.exit(1) │
│ 582 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CalledProcessError: Command '['/usr/bin/python', 'launch.py']' returned non-zero exit status 1.
And how to install ‘‘Clip’’ Python app for exactly that legacy Python used for the SDiff AI app (instead of the default newer Python used for system wide)?
Code:
Installing clip
Traceback (most recent call last):
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/launch.py", line 48, in <module>
main()
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/launch.py", line 39, in main
prepare_environment()
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/modules/launch_utils.py", line 363, in prepare_environment
run_pip(f"install {clip_package}", "clip")
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/modules/launch_utils.py", line 138, in run_pip
return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/modules/launch_utils.py", line 115, in run
raise RuntimeError("\n".join(error_bits))
RuntimeError: Couldn't install clip.
Command: "/usr/bin/python" -m pip install https://github.com/openai/CLIP/archive/d50d76daa670286dd6cacf3bcd80b5e4823fc8e1.zip --prefer-binary
Error code: 1
stderr: error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
python-xyz', where xyz is the package you are trying to
install.
If you wish to install a non-Arch-packaged Python package,
create a virtual environment using 'python -m venv path/to/venv'.
Then use path/to/venv/bin/python and path/to/venv/bin/pip.
If you wish to install a non-Arch packaged Python application,
it may be easiest to use 'pipx install xyz', which will manage a
virtual environment for you. Make sure you have python-pipx
installed via pacman.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /usr/bin/accelerate:8 in <module> │
│ │
│ 5 from accelerate.commands.accelerate_cli import main │
│ 6 if __name__ == "__main__": │
│ 7 │ sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) │
│ ❱ 8 │ sys.exit(main()) │
│ 9 │
│ │
│ /usr/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py:45 in main │
│ │
│ 42 │ │ exit(1) │
│ 43 │ │
│ 44 │ # Run │
│ ❱ 45 │ args.func(args) │
│ 46 │
│ 47 │
│ 48 if __name__ == "__main__": │
│ │
│ /usr/lib/python3.11/site-packages/accelerate/commands/launch.py:923 in launch_command │
│ │
│ 920 │ elif defaults is not None and defaults.compute_environment == ComputeEnvironment.AMA │
│ 921 │ │ sagemaker_launcher(defaults, args) │
│ 922 │ else: │
│ ❱ 923 │ │ simple_launcher(args) │
│ 924 │
│ 925 │
│ 926 def main(): │
│ │
│ /usr/lib/python3.11/site-packages/accelerate/commands/launch.py:579 in simple_launcher │
│ │
│ 576 │ process.wait() │
│ 577 │ if process.returncode != 0: │
│ 578 │ │ if not args.quiet: │
│ ❱ 579 │ │ │ raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) │
│ 580 │ │ else: │
│ 581 │ │ │ sys.exit(1) │
│ 582 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CalledProcessError: Command '['/usr/bin/python', 'launch.py']' returned non-zero exit status 1.
this is a python vm. I gave you a link about it in post #5. Not that difficult.
Python, perl, java and probably other languages makes their own virtual machines when you start them. These are something like a mini os inside the application.
With everything well set up esp the VEnv i've set up for that legacy Pyhton release, it still has errors with activating and connecting to that SDiff WebUI written by AUTOMATIC1111@GHub
He especially told us PyTorch his Git project uses still requires a legacy Python
But he never has told us what should be used instead of PyTorch for a decent Python release
Code:
asrock-amd-main-os:[hd_scania]:/opt/local/.A/SDXL/WebUI_XL-1.6.0$ l
total 527
-rwxr-xr-x 1 hd_scania root 420577 Aug 31 12:38 screenshot.png
-rwxr-xr-x 1 hd_scania root 35240 Aug 31 12:38 LICENSE.txt
-rwxr-xr-x 1 hd_scania root 33515 Aug 31 12:38 CHANGELOG.md
-rwxr-xr-x 1 hd_scania root 21496 Aug 31 12:38 style.css
-rwxr-xr-x 1 hd_scania root 11860 Aug 31 12:38 README.md
-rwxr-xr-x 1 hd_scania root 8098 Aug 31 12:38 webui.sh
-rwxr-xr-x 1 hd_scania root 5471 Aug 31 12:38 webui.py
-rwxr-xr-x 1 hd_scania root 4736 Aug 31 12:38 script.js
-rwxr-xr-x 1 hd_scania root 3382 Aug 31 12:38 .eslintrc.js
-rwxr-xr-x 1 hd_scania root 1297 Aug 31 12:38 launch.py
-rwxr-xr-x 1 hd_scania root 763 Aug 31 12:38 pyproject.toml
-rwxr-xr-x 1 hd_scania root 657 Aug 31 12:38 CODEOWNERS
-rwxr-xr-x 1 hd_scania hd_scania 603 Oct 21 11:57 webui-user.sh
-rwxr-xr-x 1 hd_scania root 541 Aug 31 12:38 requirements_versions.txt
-rwxr-xr-x 1 hd_scania root 360 Aug 31 12:38 requirements.txt
-rwxr-xr-x 1 hd_scania root 243 Aug 31 12:38 CITATION.cff
-rwxr-xr-x 1 hd_scania root 185 Aug 31 12:38 package.json
-rwxr-xr-x 1 hd_scania root 167 Aug 31 12:38 environment-wsl2.yaml
drwxr-xr-x 9 hd_scania root 125 Oct 21 11:54 modules
-rwxr-xr-x 1 hd_scania root 119 Aug 31 12:38 .pylintrc
-rw-r--r-- 1 hd_scania hd_scania 59 Oct 21 12:26 .src.log
-rwxr-xr-x 1 hd_scania root 49 Aug 31 12:38 requirements-test.txt
-rwxr-xr-x 1 hd_scania root 48 Aug 31 12:38 .eslintignore
drwxr-xr-x 16 hd_scania root 38 Oct 21 13:17 .
drwxr-xr-x 2 hd_scania root 26 Oct 8 12:03 javascript
drwxr-xr-x 2 hd_scania hd_scania 20 Oct 21 11:45 .pyshm
drwxr-xr-x 2 hd_scania root 14 Oct 8 12:03 scripts
drwxr-xr-x 10 hd_scania root 10 Oct 8 12:03 extensions-builtin
drwxr-xr-x 7 hd_scania hd_scania 10 Oct 21 11:58 .git
drwxr-xr-x 3 hd_scania root 9 Oct 8 12:03 test
drwxr-xr-x 7 hd_scania root 8 Oct 21 11:25 ..
drwxr-xr-x 2 hd_scania root 8 Oct 8 12:03 textual_inversion_templates
drwxr-xr-x 2 hd_scania root 7 Oct 8 12:03 html
drwxr-xr-x 7 hd_scania root 7 Oct 8 12:03 models
drwxr-xr-x 2 hd_scania root 6 Oct 8 12:03 configs
drwxr-xr-x 6 hd_scania hd_scania 6 Oct 21 11:16 .py3106
drwxr-xr-x 2 hd_scania root 3 Oct 21 13:19 embeddings
drwxr-xr-x 2 hd_scania root 3 Oct 21 13:19 extensions
asrock-amd-main-os:[hd_scania]:/opt/local/.A/SDXL/WebUI_XL-1.6.0$ bash ./.src.log && bash ./webui.sh
################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye)
################################################################
################################################################
Running on hd_scania user
################################################################
################################################################
Repo already cloned, using it as install directory
################################################################
################################################################
Experimental support for Renoir: make sure to have at least 4GB of VRAM and 10GB of RAM or enable cpu mode: --use-cpu all --no-half
################################################################
################################################################
Create and activate python venv
################################################################
################################################################
ERROR: Cannot activate python venv, aborting...
################################################################
asrock-amd-main-os:[hd_scania]:/opt/local/.A/SDXL/WebUI_XL-1.6.0$
Quote:
Originally Posted by pan64
this is a python vm. I gave you a link about it in post #5. Not that difficult.
Python, perl, java and probably other languages makes their own virtual machines when you start them. These are something like a mini os inside the application.
That Python VEnv was created with the py3106 installation as individual dirs, and that WebUI has been configured for use of that VEnx and here below is my WebUI-user.sh,
And that's an SDiff WebUI Git project by AUTOMATIC1111 on GHub but he never has mentioned anything for the current Python with not requiring PyTprch to function his own SDiff WebUI https://github.com/AUTOMATIC1111/stable-diffusion-webui His works are now unfortunately dormant since this August
Quote:
Originally Posted by pan64
Sorry, I don't know why did it happen. Did you create that python venv? Does it work? Did you configure your webui to use that venv?
Still errors with showing Python version conflicts, SDiff requires a legacy Python but the system's one is most recent
Code:
% bash webui.sh
Traceback (most recent call last):
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/launch.py", line 48, in <module>
main()
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/launch.py", line 44, in main
start()
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/modules/launch_utils.py", line 432, in start
import webui
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/webui.py", line 13, in <module>
initialize.imports()
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/modules/initialize.py", line 21, in imports
import gradio # noqa: F401
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/gradio/__init__.py", line 3, in <module>
import gradio.components as components
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/gradio/components/__init__.py", line 1, in <module>
from gradio.components.annotated_image import AnnotatedImage
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/gradio/components/annotated_image.py", line 12, in <module>
from gradio import utils
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/gradio/utils.py", line 353, in <module>
class AsyncRequest:
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/gradio/utils.py", line 372, in AsyncRequest
client = httpx.AsyncClient()
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/httpx/_client.py", line 1397, in __init__
self._transport = self._init_transport(
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/httpx/_client.py", line 1445, in _init_transport
return AsyncHTTPTransport(
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 275, in __init__
self._pool = httpcore.AsyncConnectionPool(
TypeError: AsyncConnectionPool.__init__() got an unexpected keyword argument 'socket_options'
Traceback (most recent call last):
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/bin/accelerate", line 8, in <module>
sys.exit(main())
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 994, in launch_command
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 636, in simple_launcher
current_env = prepare_multi_gpu_env(args)
subprocess.CalledProcessError: Command '['/opt/local/.A/SDXL/WebUI_XL-1.6.0/venv/bin/python3.10', 'launch.py']' returned non-zero exit status 1.
Traceback (most recent call last):
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/.py3106/.VEnv3106/bin/accelerate", line 8, in <module>
sys.exit(main())
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/.py3106/.VEnv3106/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 45, in main
args.func(args)
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/.py3106/.VEnv3106/lib/python3.10/site-packages/accelerate/commands/launch.py", line 979, in launch_command
simple_launcher(args)
File "/opt/local/.A/SDXL/WebUI_XL-1.6.0/.py3106/.VEnv3106/lib/python3.10/site-packages/accelerate/commands/launch.py", line 628, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/local/.A/SDXL/WebUI_XL-1.6.0/.py3106/.VEnv3106/bin/python3.10', 'launch.py']' returned non-zero exit status 2.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
