Could someone provide me with a good way of importing a whole directory of modules?
I have a structure like this:
/Foo
bar.py
spam.py
eggs.py
I tried just converting it to a package by adding __init__.py
and doing from Foo import *
but it didn't work the way I had hoped.
martineau
115k25 gold badges160 silver badges284 bronze badges
asked Jun 29, 2009 at 9:38
Evan FosmarkEvan Fosmark
95.9k34 gold badges104 silver badges117 bronze badges
4
List all python [.py
] files in the current folder and put them as __all__
variable in __init__.py
from os.path import dirname, basename, isfile, join
import glob
modules = glob.glob[join[dirname[__file__], "*.py"]]
__all__ = [ basename[f][:-3] for f in modules if isfile[f] and not f.endswith['__init__.py']]
answered Jun 29, 2009 at 10:06
Anurag UniyalAnurag Uniyal
83.2k39 gold badges169 silver badges216 bronze badges
13
Update in 2017: you probably want to use
importlib
instead.
Make the Foo directory a package by adding an __init__.py
. In that __init__.py
add:
import bar
import eggs
import spam
Since you want it dynamic [which may or may not be a good idea], list all py-files with list dir and import them with something like this:
import os
for module in os.listdir[os.path.dirname[__file__]]:
if module == '__init__.py' or module[-3:] != '.py':
continue
__import__[module[:-3], locals[], globals[]]
del module
Then, from your code do this:
import Foo
You can now access the modules with
Foo.bar
Foo.eggs
Foo.spam
etc. from Foo import *
is not a good idea for several
reasons, including name clashes and making it hard to analyze the code.
answered Jun 29, 2009 at 11:10
Lennart RegebroLennart Regebro
162k41 gold badges221 silver badges251 bronze badges
14
Python, include all files under a directory:
For newbies who just can't get it to work who need their hands held.
Make a folder /home/el/foo and make a file
main.py
under /home/el/foo Put this code in there:from hellokitty import * spam.spamfunc[] ham.hamfunc[]
Make a directory
/home/el/foo/hellokitty
Make a file
__init__.py
under/home/el/foo/hellokitty
and put this code in there:__all__ = ["spam", "ham"]
Make two python files:
spam.py
andham.py
under/home/el/foo/hellokitty
Define a function inside spam.py:
def spamfunc[]: print["Spammity spam"]
Define a function inside ham.py:
def hamfunc[]: print["Upgrade from baloney"]
Run it:
el@apollo:/home/el/foo$ python main.py spammity spam Upgrade from baloney
answered Dec 24, 2013 at 0:08
Eric LeschinskiEric Leschinski
139k91 gold badges405 silver badges327 bronze badges
3
Expanding on Mihail's answer, I believe the non-hackish way [as in, not handling the file paths directly] is the following:
- create an empty
__init__.py
file underFoo/
- Execute
import pkgutil
import sys
def load_all_modules_from_dir[dirname]:
for importer, package_name, _ in pkgutil.iter_modules[[dirname]]:
full_package_name = '%s.%s' % [dirname, package_name]
if full_package_name not in sys.modules:
module = importer.find_module[package_name
].load_module[full_package_name]
print module
load_all_modules_from_dir['Foo']
You'll get:
Adrian Mouat
42.8k15 gold badges108 silver badges101 bronze badges
answered Dec 19, 2011 at 2:11
Luca InvernizziLuca Invernizzi
6,3393 gold badges27 silver badges26 bronze badges
5
I got tired of this problem myself, so I wrote a package called automodinit to fix it. You can get it from //pypi.python.org/pypi/automodinit/.
Usage is like this:
- Include the
automodinit
package into yoursetup.py
dependencies. - Replace all __init__.py files like this:
__all__ = ["I will get rewritten"] # Don't modify the line above, or this line! import automodinit automodinit.automodinit[__name__, __file__, globals[]] del automodinit # Anything else you want can go after here, it won't get modified.
That's it! From now on importing a module will set __all__ to a list of .py[co] files in the module and will also import each of those files as though you had typed:
for x in __all__: import x
Therefore the effect of "from M import *" matches exactly "import M".
automodinit
is happy running from inside ZIP archives and is therefore ZIP safe.
Niall
zmo
24k4 gold badges53 silver badges86 bronze badges
answered Mar 5, 2012 at 2:24
Niall DouglasNiall Douglas
8,9142 gold badges43 silver badges52 bronze badges
2
I know I'm updating a quite old post, and I tried using automodinit
, but found out it's setup process is broken for python3. So, based on Luca's answer, I came up with a simpler answer - which might not work with .zip - to this issue, so I figured I should share it
here:
within the __init__.py
module from yourpackage
:
#!/usr/bin/env python
import os, pkgutil
__all__ = list[module for _, module, _ in pkgutil.iter_modules[[os.path.dirname[__file__]]]]
and within another package below yourpackage
:
from yourpackage import *
Then you'll have all the modules that are placed within the package loaded, and if you write a new module, it'll be automagically imported as well. Of course, use that kind of things with care, with great powers comes great responsibilities.
PSL
2281 silver badge9 bronze badges
answered Mar 26, 2016 at 2:54
zmozmo
24k4 gold badges53 silver badges86 bronze badges
import pkgutil
__path__ = pkgutil.extend_path[__path__, __name__]
for imp, module, ispackage in pkgutil.walk_packages[path=__path__, prefix=__name__+'.']:
__import__[module]
answered Jun 7, 2016 at 16:15
Ricky SahuRicky Sahu
22.2k4 gold badges39 silver badges31 bronze badges
I have also encountered this problem and this was my solution:
import os
def loadImports[path]:
files = os.listdir[path]
imps = []
for i in range[len[files]]:
name = files[i].split['.']
if len[name] > 1:
if name[1] == 'py' and name[0] != '__init__':
name = name[0]
imps.append[name]
file = open[path+'__init__.py','w']
toWrite = '__all__ = '+str[imps]
file.write[toWrite]
file.close[]
This function creates a file [in the provided folder] named __init__.py
, which contains an __all__
variable that holds every module in the folder.
For example, I have a folder named Test
which contains:
Foo.py
Bar.py
So in the script I want the modules to be imported into I will write:
loadImports['Test/']
from Test import *
This will import everything from Test
and the __init__.py
file in Test
will now contain:
__all__ = ['Foo','Bar']
Honest Abe
8,1024 gold badges47 silver badges62 bronze badges
answered Jul 30, 2012 at 21:06
0
This is the best way i've found so far:
from os.path import dirname, join, isdir, abspath, basename
from glob import glob
pwd = dirname[__file__]
for x in glob[join[pwd, '*.py']]:
if not x.startswith['__']:
__import__[basename[x][:-3], globals[], locals[]]
answered Jul 2, 2014 at 8:30
Farshid AshouriFarshid Ashouri
14.7k6 gold badges49 silver badges64 bronze badges
1
Anurag Uniyal answer with suggested improvements!
#!/usr/bin/python
# -*- encoding: utf-8 -*-
import os
import glob
all_list = list[]
for f in glob.glob[os.path.dirname[__file__]+"/*.py"]:
if os.path.isfile[f] and not os.path.basename[f].startswith['_']:
all_list.append[os.path.basename[f][:-3]]
__all__ = all_list
answered Jun 13, 2014 at 1:28
Eduardo LucioEduardo Lucio
1,2851 gold badge19 silver badges33 bronze badges
Using importlib
the only thing you've got to add is
from importlib import import_module
from pathlib import Path
__all__ = [
import_module[f".{f.stem}", __package__]
for f in Path[__file__].parent.glob["*.py"]
if "__" not in f.stem
]
del import_module, Path
Asclepius
52.2k15 gold badges151 silver badges131 bronze badges
answered Nov 26, 2019 at 15:56
tedted
12.2k6 gold badges58 silver badges99 bronze badges
1
I've created a module for that, which doesn't rely on __init__.py
[or any other auxiliary file] and makes me type only the following two lines:
import importdir
importdir.do["Foo", globals[]]
Feel free to re-use or contribute: //gitlab.com/aurelien-lourot/importdir
answered Nov 18, 2014 at 23:46
AurelienAurelien
1,6621 gold badge18 silver badges29 bronze badges
Anurag's example with a couple of corrections:
import os, glob
modules = glob.glob[os.path.join[os.path.dirname[__file__], "*.py"]]
__all__ = [os.path.basename[f][:-3] for f in modules if not f.endswith["__init__.py"]]
answered Nov 19, 2014 at 11:13
kzarkzar
2,8412 gold badges16 silver badges13 bronze badges
When from . import *
isn't good enough, this is an improvement over the
answer by ted. Specifically, the use of __all__
is not necessary with this approach.
"""Import all modules that exist in the current directory."""
# Ref //stackoverflow.com/a/60861023/
from importlib import import_module
from pathlib import Path
for f in Path[__file__].parent.glob["*.py"]:
module_name = f.stem
if [not module_name.startswith["_"]] and [module_name not in globals[]]:
import_module[f".{module_name}", __package__]
del f, module_name
del import_module, Path
Note that module_name not in globals[]
is intended to avoid reimporting the module if it's already imported, as this can risk cyclic imports.
answered Mar 26, 2020 at 4:27
AsclepiusAsclepius
52.2k15 gold badges151 silver badges131 bronze badges
See that your __init__.py
defines __all__
. The modules - packages doc says
The
__init__.py
files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case,__init__.py
can just be an empty file, but it can also execute initialization code for the package or set the__all__
variable, described later....
The only solution is for the package author to provide an explicit index of the package. The import statement uses the following convention: if a package’s
__init__.py
code defines a list named__all__
, it is taken to be the list of module names that should be imported when from package import * is encountered. It is up to the package author to keep this list up-to-date when a new version of the package is released. Package authors may also decide not to support it, if they don’t see a use for importing * from their package. For example, the filesounds/effects/__init__.py
could contain the following code:
__all__ = ["echo", "surround", "reverse"]
This would mean that
from sound.effects import *
would import the three named submodules of the sound package.
answered Jun 29, 2009 at 9:47
gimelgimel
80.2k10 gold badges74 silver badges104 bronze badges
I'd like to add to Anurag Uniyal's answer. You can make it even simpler and get rid of a lot of the imports. Contents of the __init__.py file:
from os import listdir
from os.path import dirname
__all__ = [i[:-3] for i in listdir[dirname[__file__]] if not i.startswith['__'] and i.endswith['.py']]
answered Jul 18, 2021 at 15:44
Look at the pkgutil module from the standard library. It will let you do exactly what you want as long as you have an __init__.py
file in the directory. The __init__.py
file can be empty.
answered Feb 8, 2011 at 12:51
Just import them by importlib and add them to __all__
[add
action is optional] in recurse in the __init__.py
of package.
/Foo
bar.py
spam.py
eggs.py
__init__.py
# __init__.py
import os
import importlib
pyfile_extes = ['py', ]
__all__ = [importlib.import_module['.%s' % filename, __package__] for filename in [os.path.splitext[i][0] for i in os.listdir[os.path.dirname[__file__]] if os.path.splitext[i][1] in pyfile_extes] if not filename.startswith['__']]
del os, importlib, pyfile_extes
answered Jan 18, 2018 at 11:07
CheneyCheney
9008 silver badges22 bronze badges
2
I had a nested directory structure i.e. I had multiple directories inside the main directory that contained the python modules.
I added the following script to my __init__.py
file to import all the modules
import glob, re, os
module_parent_directory = "path/to/the/directory/containing/__init__.py/file"
owd = os.getcwd[]
if not owd.endswith[module_parent_directory]: os.chdir[module_parent_directory]
module_paths = glob.glob["**/*.py", recursive = True]
for module_path in module_paths:
if not re.match[ ".*__init__.py$", module_path]:
import_path = module_path[:-3]
import_path = import_path.replace["/", "."]
exec[f"from .{import_path} import *"]
os.chdir[owd]
Probably not the best way to achieve this, but I couldn't make anything else work for me.
answered Jan 15, 2021 at 21:41
Here is a
solution, with which you do not have to write the file name. Just add this code snippet to your __init__.py
from inspect import isclass
from pkgutil import iter_modules
from pathlib import Path
from importlib import import_module
# iterate through the modules in the current package
package_dir = Path[__file__].resolve[].parent
for [_, module_name, _] in iter_modules[[package_dir]]:
# import the module and iterate through its attributes
module = import_module[f"{__name__}.{module_name}"]
for attribute_name in dir[module]:
attribute = getattr[module, attribute_name]
if isclass[attribute]:
# Add the class to this package's variables
globals[][attribute_name] = attribute
Source
answered Sep 5 at 16:01