How to run python file from another directory

Solved see my answer below for anyone who might find this helpful.

I have two scripts a.py and b.py. In my current directory "C:\Users\MyName\Desktop\MAIN", I run > python a.py.

The first script, a.py runs in my current directory, does something to a bunch of files and creates a new directory (testA) with the edited versions of those files which are simultaneously moved into that new directory. Then I need to run b.py for the files in testA.

As a beginner, I would just copy and paste my b.py script into testA and execute the command again "> python b.py", which runs some commands on those new files and creates another folder (testB) with those edited files.

I am trying to eliminate the hassle of waiting for a.py to finish, move into that new directory, paste b.py, and then run b.py. I am trying to write a bash script that executes these scripts while maintaining my hierarchy of directories.

#!/usr/bin/env bash
 python a.py && python b.py

Script a.py runs smoothly, but b.py does not execute at all. There are no error messages coming up about b.py failing, I just think it cannot execute because once a.py is done, b.py does not exist in that NEW directory. Is there a small script I can add within b.py that moves it into the new directory? I actually tried changing b.py directory paths as well but it did not work.

For example in b.py:

mydir = os.getcwd() # would be the same path as a.py
mydir_new = os.chdir(mydir+"\\testA")

I changed mydirs to mydir_new in all instances within b.py, but that also made no difference...I also don't know how to move a script into a new directory within bash.

As a little flowchart of the folders:

MAIN # main folder with unedited files and both a.py and b.py scripts
|
| (execute a.py)
|
--------testA # first folder created with first edits of files
         |
         | (execute b.py)
         |
         --------------testB # final folder created with final edits of files

TLDR: How do I execute a.py and b.py from the main test folder (bash script style?), if b.py relies on files created and stored in testA. Normally I copy and paste b.py into testA, then run b.py - but now I have 200+ files so copying and pasting is a waste of time.

• Files and Directory Structure
• Executing Python Script on Terminal/Shell
• Running Python Script in another Directory as Regular Script
• Running Python Script in another Directory as a Module
  • Adding a path into the Python environment
  • Method 1: Using the sys package
  • Method 2a: Adding path to an environment variable (on Linux and Mac)
  • Method 2b: Adding path to an environment variable (on Windows)

There are two ways you may want to run a Python script located in another folder – as a regular script or import and run it as a module. This article focuses on how we can do this.

Contents

• 1 Files and Directory Structure
• 2 Executing Python Script on Terminal/Shell
• 3 Running Python Script in another Directory as Regular Script
• 4 Running Python Script in another Directory as a Module
  • 4.1 Adding a path into the Python environment
  • 4.2 Method 1: Using the sys package
  • 4.3 Method 2a: Adding path to an environment variable (on Linux and Mac)
  • 4.4 Method 2b: Adding path to an environment variable (on Windows)

Files and Directory Structure

We will use a file named runScript1.py to run two other files, script1.py and module1.py, located in another directory. We will execute script1.py as a regular Python script and module1.py as a module.

Here is the directory structure (it’s recommended you create these files in the tree structure shown and follow along),

Figure 1: We have two folders in the Desktop – test1 and test2. The folder test1 contains two Python files – script1.py and module1.py, whereas the test2 folder has one script named runScript1.py. Note: In my case, the Desktop is located at /home/kiprono/

We will keep editing runScript1.py, but the content of the other two files will remain the same throughout this post. It is as follows:

script1.py:

def add(a, b):
    return a+b
def multiply(a, b):
    return a*b
x1, x2 = 7, 5
print(f"{x1}+{x2} is equal to",add(x1, x2))
if __name__ == '__main__':
    x1, x2 = 3, 5
    print(f"{x1}*{x2} is equal to",multiply(x1, x2))

This script has two functions to perform simple mathematical operations – add and multiply. We call multiply() function inside __name__ == ‘__main__’ and add() function outside.

Any code inside __name__ == ‘__main__’ is executed only when the script is executed as a regular Python file. If the script is imported as a module, the code under if-block will not be executed.

module1.py:

class BasicOperations():
    def __init__(self, value1, value2):
   	 self.value1 = value1
   	 self.value2 = value2
    def add(self):
   	 return self.value1+self.value2
    def multiply(self):
   	 return self.value1*self.value2
    def power(self):
   	 return self.value1**self.value2
    def integer_division(self):
   	 return self.value1//self.value2
if __name__ == '__main__':
    print("5 plus 7 is equals to: ", 5+7)
    s = BasicOperations(2, 5)
    print(s.multiply())

The module1.py has one class (BasicOperations()) with several functions performing different basic mathematical operations.

Executing Python Script on Terminal/Shell

In this article, we will consistently execute the runScript1.pyfile on the terminal by running the command “python3 script_name”.

You may run into the following error:

Just change “Python3” to “Python” in every command and script, so it looks like this: “python script_name”.

This assumes that you are on the directory containing the script. To change the directory to the folder containing the script, run “cd ” on the terminal. In my case, I will execute runScript1.py as follows,

Note: In this post, we will use the dollar sign ($) to show that we are running the script on the terminal, for example, $ python3 runScript1.py.

Running Python Script in another Directory as Regular Script

Python script can be used to execute another script on a different directory with the help of either the os or subprocess module. Using these modules to run Python files is equivalent to running a Python script on the shell/terminal. The general syntax used is

os.system("python3 /path/to/script/to/execute")
subprocess.run(["python3", "/path/to/script/to/execute"])

Note: If you have Python 2 and Python 3 installed on your computer and want to run Python 2 specify that.

Let’s now have the following code in our runScript1.pyand use it to execute script1.py

runScript1.py:

print("###os module###")
import os
os.system("python3 /home/kiprono/Desktop/test1/script1.py")
print("###subprocess package###")
import subprocess
subprocess.run(["python3", "/home/kiprono/Desktop/test1/script1.py"])

After you run the script, you will get this message:

Output:

###os module###
7+5 is equal to 12
3*5 is equal to 15
###subprocess package###
7+5 is equal to 12
3*5 is equal to 15

Notice that the code under __name__ == ‘__main__’ in script1.py is executed. This is because we run the script from runScript1.py as a regular script. To make the paths shorter, you can also use os.chdir() to change the current working directory (this is equivalent to using cd in the command line) and then pass the script names only as shown below

runScript1.py:

import os
print("####os####")
#change current working directory
os.chdir("/home/kiprono/Desktop/test1/")
# run script1.py using os module
os.system("python script1.py")

# run script1.py using subprocess module
import subprocess
print("####subprocess####")
subprocess.run(["python", "script1.py"])

Run the code.

Output:

####os####
7+5 is equal to 12
3*5 is equal to 15
####subprocess####
7+5 is equal to 12
3*5 is equal to 15

Running Python Script in another Directory as a Module

Whenever you import a module (which is just a Phyton file) in Python by writing “import module1“, for example, the interpreter will look for module1.py in the following places.

• The current working directory – the location of the script importing module1,
• List of folders set on the PYTHONPATH environment variable. We can add folders to this list whenever we want (this will make sense shortly).
• Folders configured at Python installation.

If you attempt to import a module that is not available in the above locations, Python will throw ModuleNotFoundError.

We can use the sys package to check and modify folders to look for. sys.path shows a list of directories the Python interpreter searches whenever a module is being imported.

runScript1.py:

import sys
print(sys.path)

Run the script.

Output:

['/home/kiprono/Desktop/test2', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/kiprono/.local/lib/python3.9/site-packages', '/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages']

Those are the directories Python will be looking for whenever you import modules. By default, the current directory is also searched.

In our case, it is the first element of the list (that is where runScript1.py is located). If we attempt to import module1.py, at this point, we will get ModuleNotFoundError because the folder containing it (‘/home/kiprono/Desktop/test1’) is not part of the sys.path.

Therefore, we must learn how to add a path to the Python environment.

Adding a path into the Python environment

There are two methods – using the sys module or editing Python environment variables explicitly (we will see how to do this in Linux, Mac, and Windows).

Method 1: Using the sys package

The function sys.path.append() allows us to add a path into the environment. Let’s use it to add ‘/home/kiprono/Desktop/test1’, then we can import the module1.py script as a valid Python module.

runScript1.py:

import sys
sys.path.append("/home/kiprono/Desktop/test1")
print(sys.path)
import module1
s = module1.BasicOperations(1, 5)
print(s.add())

Run the script.

Output:

['/home/kiprono/Desktop/test2', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/kiprono/.local/lib/python3.9/site-packages', '/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages', '/home/kiprono/Desktop/test1']
6

As you can see, ‘/home/kiprono/Desktop/test1‘ has been added to the list of directories the interpreter will search, and therefore, we can import module1 and then load the BasicOperations() class. As said earlier, since module1.py is executed as a module, the code under __name__ == ‘__main__’ is not executed.

Every time we start another Python script and need to import modules, we have to append the directories to look for. To avoid having to do this, we can add the path(s) into the environment variable. We will see how to do that in Linux, macOS, and Windows.

Method 2a: Adding path to an environment variable (on Linux and Mac)

On Linux/Mac, you can check the current value of the variable by running  this command on terminal

$ echo $PYTHONPATH

That would be blank unless it has been modified before. To add the path containing our module, we need to:

Open .bash_profile (on Mac) or .profile (on Linux) (the file should be in the home folder(~)) and add the following line to the file:

export PYTHONPATH=$PYTHONPATH:/path/to/add

Alternatively, you can run the following command on the terminal to append the line:

Linux:

echo "export PYTHONPATH=$PYTHONPATH:/path/to/add" >>~/.profile

macOS:

echo "export PYTHONPATH=$PYTHONPATH:/path/to/add" >>~/.bash_profile

If you are running your code on the terminal, you have to restart the terminal (or run source ~/.profile) for the changes to take effect, but if you are running the code on an editor, you might need to reboot the whole system.

runScript1.py:

import sys
print(sys.path)
import module1
s = module1.BasicOperations(1, 5)
print(s.add())

Figure 2: Running the script before and after adding the path into the environment.

Before adding a path into the environment, Python throws ModuleNotFoundError, but after adding the path, the code on runScript1.py was executed with no error.

You can add multiple paths using the line:

export PYTHONPATH=$PYTHONPATH:dir1:dir2:dir3

If you want to remove a path (say dir2), you can run the following command or remove the path on the .profile file:

export PYTHONPATH=$PYTHONPATH:dir1:dir3

Method 2b: Adding path to an environment variable (on Windows)

For Windows (using Windows 11), follow these steps:

1. Open the Run dialog box by Clicking Windows Key + R,
2. Type sysdm.cpl and click Enter. This should open the System Properties window. Go to the Advanced tab and click on Environment Variables at the bottom.

1. To make the path recognizable by system-wise applications, we will add the path into System Variables.

1. Click new or edit. A variable name can be anything but the variable value should be the path you wish to add. In my case: C:\Users\kipro\OneDrive\Desktop\run_file\test1

1. Once done editing, click OK on the New System variable window and OK on the Environment variables window.
2. Now, you may need to restart the Shell/terminal or the code editor you are using for changes to take effect.

How do I run a Python file in another directory?

How do I open a Python file in D drive?