[SOLVED] How to Install dependency module locally?

shogun1234 · 03-02-2020, 04:45 PM

I am very new to python (though I have some programming experience), so I am not sure if I understand it correctly or not.

I have two projects A, and B where A depends on B (both contains setup.py files). Usually I will do following operations

Code:

vritualenv -p python3 .env
source .env/bin/activate 
pip3 install <moudule name 1>
pip3 install <moudule name 2>
...

Now I want to modify the dependency B in order to test some functions in the project A. After searching on the internet, it seems that I can simply execute

Code:

python3 install -e .

to install the project. But how can I install module/ project B in the project A? I am confused with this because if I execute in the project A dir, then it simply installs project A by python3 install -e .. Is there any way to achieve my goal?

Thanks for any suggestions.

teckk · 03-03-2020, 02:03 PM

If I understand what you are wanting.
a.py

Code:

from time import sleep

def count():
    for i in range(0, 10)
        print(i)
        sleep(1)

b.py

Code:

import sys

sys.path.append('/dir/where/a.py/is')
from a import count

count()

Code:

>>> import sys
>>> sys.path.append('/your/module/path/')
>>> from a import count
>>> count()
0
1
2
3
4
5
6
7
8
9

shogun1234 · 03-03-2020, 02:28 PM

That's close to what I am looking for. But the projects are more complicated (My explanation could be wrong because I am not familiar with python. Apologize if my explanation is still not clear enough).

Suppose the project A has a setup.py file where the project B is another complicated project (and the project B can be installed by pip as well). Now I want to modify the code in the project B (the project B is downloaded by `git clone https://github.com/... /path/to/projectB`). And I want the change (in the project B) to be reflected in the project A (which is also downloaded by `git clone https://github.com/... /path/to/projectA`). For instance, A call a function in the project B which originally throws an APIError, which extends IOError. Now I want to modify the code in the project B by switching class APIError(IOError) to class APIError(HttpError). After modifying the code in the project B, how can I make this change to be reflected in the project A?

Hope this explanation would be a bit more clear. Thank you again for your help!

Code:

install_requires = [
    "python-dateutil<2.8.1,>=2.1",  
    "ply>=3.9",  
    "colorama>=0.3.9",
    ...
]

...
setup(
    name=...,
    version=version,
    description="...",
    long_description=open("README.rst", "r").read(),
    author="...",
    author_email="...",
    download_url="https://github.com/...",
    license="Apache License 2.0",
    install_requires=install_requires,
    extras_require={
        "all": ["project_B>=1.4.5", ...], # the project B is one of dep of the project A
        ...
    }
    ...
)

Quote:

Originally Posted by teckk

If I understand what you are wanting.
a.py

Code:

from time import sleep

def count():
    for i in range(0, 10)
        print(i)
        sleep(1)

b.py

Code:

import sys

sys.path.append('/dir/where/a.py/is')
from a import count

count()

Code:

>>> import sys
>>> sys.path.append('/your/module/path/')
>>> from a import count
>>> count()
0
1
2
3
4
5
6
7
8
9

teckk · 03-03-2020, 03:19 PM

There is diff and comm to compare file.
See man diff and man comm
Or you could compare them how you want with python.

Example:

Code:

#!/usr/bin/python

#Compare 2 files line by line

def comp(fName1, fName2):
    
    with open(fName1, 'r') as f, open(fName2, 'r') as g:
        
        print("-" * 40)
        print("Comparing files ", " file1 " + fName1, " file2 " +fName2, sep='\n')
        print("-" * 40)

        fLine = f.readline()
        gLine = g.readline()
        lineNo = 1

        while fLine != '' or gLine != '':

            fLine = fLine.rstrip()
            gLine = gLine.rstrip()
            
            if fLine != gLine:
                
                if gLine == '' and fLine != '':
                    print("file1+", "(Line %d)" % lineNo, fLine)

                elif fLine != '':
                    print("file1", "(Line %d)" % lineNo, fLine)
                    
                if fLine == '' and gLine != '':
                    print("file2+", "(Line %d)" % lineNo, gLine)

                elif fLine != '':
                    print("file2", "(Line %d)" %  lineNo, gLine)
                    
                print()

            fLine = f.readline()
            gLine = g.readline()
            lineNo += 1
    
if __name__ == "__main__":
        
    fName1 = input("Enter/Paste first file: ")
    fName2 = input("Enter/Paste second file: ")
    comp(fName1, fName2)

Or...

boughtonp · 03-03-2020, 05:07 PM

Quote:

Originally Posted by shogun1234

After modifying the code in the project B, how can I make this change to be reflected in the project A?

I think this Python documentation is saying you update the version number then run pip install with --upgrade parameter - changing the version alone is not enough. (Seems like a flaw in pip that it can't detect that and upgrade for you.)

shogun1234 · 03-04-2020, 05:08 AM

Really sorry that my explain looks not very clear, or if I misunderstand the meaning for my bad English : (

I don't have privilege to touch the project B's repo. So I can't publish project B's artifact to public remote repo, and then do `pip install ...` operation. What I want to do is more close to publishM2 or publish-local so that I can test if my change looks working or not. I might submit my patch to upstream (i.e. project B) but basically my primary intention is to test my change locally before doing anything else. So the procedure I am looking to is

1. Modify some code in the project B
2. Publish the change of project B locally which can be referenced by the project A.
3. Test the change in the project A to see if the change (in the project B) is working or not.

In addition, the project A uses virtualenv with the param -p python3. So I want to publish the project B artifact to project A's virtualenv in fact. For instance, searching the file I want to modify, I find the module is located at following path

Code:

./.env/lib/python3.7/site-packages/<project B>

Many thanks for all your help.

shogun1234 · 03-04-2020, 08:01 AM

Ok. I suppose I find the solution now, though I am not sure if that's the correct way to do it or not.

* Step1
In project B, build wheel artifact by

Code:

python3 setup.py sdist bdist_wheel

* Step 2
Then in project A (with virtual environment activated), execute

Code:

pip3 install ../path/to/projectB/dist/projectB-<version>-py3-none-any.whl

Before the step 2, you might want to check if the project B's artifact has already been installed or not. If it shows something as below, one can unisntall first by pip3 uninstall projectB

Code:

Requirement already satisfied: projectB==<version> from file:///path/to/projectB/dist/projectB-<version>-py3-none-any.whl in /path/to/projectB (<version>)