Can we use structure in python?

Asked 14 years, 1 month ago

Viewed 766k times

Is there a way to conveniently define a C-like structure in Python? I'm tired of writing stuff like:

class MyStruct():
    def __init__(self, field1, field2, field3):
        self.field1 = field1
        self.field2 = field2
        self.field3 = field3

asked Aug 30, 2008 at 14:33

10

Update: Data Classes

With the introduction of Data Classes in Python 3.7 we get very close.

The following example is similar to the NamedTuple example below, but the resulting object is mutable and it allows for default values.

from dataclasses import dataclass


@dataclass
class Point:
    x: float
    y: float
    z: float = 0.0


p = Point(1.5, 2.5)

print(p)  # Point(x=1.5, y=2.5, z=0.0)

This plays nicely with the new typing module in case you want to use more specific type annotations.

I've been waiting desperately for this! If you ask me, Data Classes and the new NamedTuple declaration, combined with the typing module are a godsend!

Improved NamedTuple declaration

Since Python 3.6 it became quite simple and beautiful (IMHO), as long as you can live with immutability.

A new way of declaring NamedTuples was introduced, which allows for type annotations as well:

from typing import NamedTuple


class User(NamedTuple):
    name: str


class MyStruct(NamedTuple):
    foo: str
    bar: int
    baz: list
    qux: User


my_item = MyStruct('foo', 0, ['baz'], User('peter'))

print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))

answered Jul 31, 2017 at 22:54

Can we use structure in python?

RotaretiRotareti

43.6k19 gold badges104 silver badges102 bronze badges

9

Use a named tuple, which was added to the collections module in the standard library in Python 2.6. It's also possible to use Raymond Hettinger's named tuple recipe if you need to support Python 2.4.

It's nice for your basic example, but also covers a bunch of edge cases you might run into later as well. Your fragment above would be written as:

from collections import namedtuple
MyStruct = namedtuple("MyStruct", "field1 field2 field3")

The newly created type can be used like this:

m = MyStruct("foo", "bar", "baz")

You can also use named arguments:

m = MyStruct(field1="foo", field2="bar", field3="baz")

answered Aug 30, 2008 at 15:18

gz.gz.

6,5611 gold badge22 silver badges34 bronze badges

8

You can use a tuple for a lot of things where you would use a struct in C (something like x,y coordinates or RGB colors for example).

For everything else you can use dictionary, or a utility class like this one:

>>> class Bunch:
...     def __init__(self, **kwds):
...         self.__dict__.update(kwds)
...
>>> mystruct = Bunch(field1=value1, field2=value2)

I think the "definitive" discussion is here, in the published version of the Python Cookbook.

jfs

382k180 gold badges944 silver badges1616 bronze badges

answered Aug 30, 2008 at 14:38

dF.dF.

72.2k29 gold badges128 silver badges135 bronze badges

5

Perhaps you are looking for Structs without constructors:

class Sample:
  name = ''
  average = 0.0
  values = None # list cannot be initialized here!


s1 = Sample()
s1.name = "sample 1"
s1.values = []
s1.values.append(1)
s1.values.append(2)
s1.values.append(3)

s2 = Sample()
s2.name = "sample 2"
s2.values = []
s2.values.append(4)

for v in s1.values:   # prints 1,2,3 --> OK.
  print v
print "***"
for v in s2.values:   # prints 4 --> OK.
  print v

answered Sep 21, 2010 at 15:15

8

How about a dictionary?

Something like this:

myStruct = {'field1': 'some val', 'field2': 'some val'}

Then you can use this to manipulate values:

print myStruct['field1']
myStruct['field2'] = 'some other values'

And the values don't have to be strings. They can be pretty much any other object.

answered Aug 30, 2008 at 14:35

Mark BiekMark Biek

143k54 gold badges155 silver badges200 bronze badges

2

dF: that's pretty cool... I didn't know that I could access the fields in a class using dict.

Mark: the situations that I wish I had this are precisely when I want a tuple but nothing as "heavy" as a dictionary.

You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties are stored internally using dicts (at least in CPython).

...Which leads us to your second comment. Believing that Python dicts are "heavy" is an extremely non-pythonistic concept. And reading such comments kills my Python Zen. That's not good.

You see, when you declare a class you are actually creating a pretty complex wrapper around a dictionary - so, if anything, you are adding more overhead than by using a simple dictionary. An overhead which, by the way, is meaningless in any case. If you are working on performance critical applications, use C or something.

Can we use structure in python?

ArtOfWarfare

19.6k17 gold badges130 silver badges188 bronze badges

answered Aug 30, 2008 at 15:20

Vicent MartiVicent Marti

7,1036 gold badges30 silver badges33 bronze badges

1

I would also like to add a solution that uses slots:

class Point:
    __slots__ = ["x", "y"]
    def __init__(self, x, y):
        self.x = x
        self.y = y

Definitely check the documentation for slots but a quick explanation of slots is that it is python's way of saying: "If you can lock these attributes and only these attributes into the class such that you commit that you will not add any new attributes once the class is instantiated (yes you can add new attributes to a class instance, see example below) then I will do away with the large memory allocation that allows for adding new attributes to a class instance and use just what I need for these slotted attributes".

Example of adding attributes to class instance (thus not using slots):

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

p1 = Point(3,5)
p1.z = 8
print(p1.z)

Output: 8

Example of trying to add attributes to class instance where slots was used:

class Point:
    __slots__ = ["x", "y"]
    def __init__(self, x, y):
        self.x = x
        self.y = y

p1 = Point(3,5)
p1.z = 8

Output: AttributeError: 'Point' object has no attribute 'z'

This can effectively works as a struct and uses less memory than a class (like a struct would, although I have not researched exactly how much). It is recommended to use slots if you will be creating a large amount of instances of the object and do not need to add attributes. A point object is a good example of this as it is likely that one may instantiate many points to describe a dataset.

Can we use structure in python?

answered Apr 12, 2018 at 6:17

Can we use structure in python?

Oamar KanjiOamar Kanji

1,4544 gold badges19 silver badges34 bronze badges

1

You can subclass the C structure that is available in the standard library. The ctypes module provides a Structure class. The example from the docs:

>>> from ctypes import *
>>> class POINT(Structure):
...     _fields_ = [("x", c_int),
...                 ("y", c_int)]
...
>>> point = POINT(10, 20)
>>> print point.x, point.y
10 20
>>> point = POINT(y=5)
>>> print point.x, point.y
0 5
>>> POINT(1, 2, 3)
Traceback (most recent call last):
  File "", line 1, in ?
ValueError: too many initializers
>>>
>>> class RECT(Structure):
...     _fields_ = [("upperleft", POINT),
...                 ("lowerright", POINT)]
...
>>> rc = RECT(point)
>>> print rc.upperleft.x, rc.upperleft.y
0 5
>>> print rc.lowerright.x, rc.lowerright.y
0 0
>>>

answered Jun 25, 2015 at 23:50

Can we use structure in python?

Ella RoseElla Rose

4967 silver badges15 bronze badges

0

You can also pass the init parameters to the instance variables by position

# Abstract struct class       
class Struct:
    def __init__ (self, *argv, **argd):
        if len(argd):
            # Update by dictionary
            self.__dict__.update (argd)
        else:
            # Update by position
            attrs = filter (lambda x: x[0:2] != "__", dir(self))
            for n in range(len(argv)):
                setattr(self, attrs[n], argv[n])

# Specific class
class Point3dStruct (Struct):
    x = 0
    y = 0
    z = 0

pt1 = Point3dStruct()
pt1.x = 10

print pt1.x
print "-"*10

pt2 = Point3dStruct(5, 6)

print pt2.x, pt2.y
print "-"*10

pt3 = Point3dStruct (x=1, y=2, z=3)
print pt3.x, pt3.y, pt3.z
print "-"*10

answered Aug 30, 2008 at 15:53

PabloGPabloG

24.8k10 gold badges45 silver badges59 bronze badges

2

Whenever I need an "instant data object that also behaves like a dictionary" (I don't think of C structs!), I think of this cute hack:

class Map(dict):
    def __init__(self, **kwargs):
        super(Map, self).__init__(**kwargs)
        self.__dict__ = self

Now you can just say:

struct = Map(field1='foo', field2='bar', field3=42)

self.assertEquals('bar', struct.field2)
self.assertEquals(42, struct['field3'])

Perfectly handy for those times when you need a "data bag that's NOT a class", and for when namedtuples are incomprehensible...

answered Sep 13, 2013 at 17:40

PhlipPhlip

5,5103 gold badges29 silver badges46 bronze badges

1

Some the answers here are massively elaborate. The simplest option I've found is (from: http://norvig.com/python-iaq.html):

class Struct:
    "A structure that can have any fields defined."
    def __init__(self, **entries): self.__dict__.update(entries)

Initialising:

>>> options = Struct(answer=42, linelen=80, font='courier')
>>> options.answer
42

adding more:

>>> options.cat = "dog"
>>> options.cat
dog

edit: Sorry didn't see this example already further down.

answered Aug 5, 2017 at 0:39

w_jayw_jay

3055 silver badges12 bronze badges

2

You access C-Style struct in python in following way.

class cstruct:
    var_i = 0
    var_f = 0.0
    var_str = ""

if you just want use object of cstruct

obj = cstruct()
obj.var_i = 50
obj.var_f = 50.00
obj.var_str = "fifty"
print "cstruct: obj i=%d f=%f s=%s" %(obj.var_i, obj.var_f, obj.var_str)

if you want to create an array of objects of cstruct

obj_array = [cstruct() for i in range(10)]
obj_array[0].var_i = 10
obj_array[0].var_f = 10.00
obj_array[0].var_str = "ten"

#go ahead and fill rest of array instaces of struct

#print all the value
for i in range(10):
    print "cstruct: obj_array i=%d f=%f s=%s" %(obj_array[i].var_i, obj_array[i].var_f, obj_array[i].var_str)

Note: instead of 'cstruct' name, please use your struct name instead of var_i, var_f, var_str, please define your structure's member variable.

Can we use structure in python?

Trevor Hickey

34.8k28 gold badges150 silver badges258 bronze badges

answered Nov 9, 2014 at 7:37

Sujal ShethSujal Sheth

3744 silver badges10 bronze badges

1

This might be a bit late but I made a solution using Python Meta-Classes (decorator version below too).

When __init__ is called during run time, it grabs each of the arguments and their value and assigns them as instance variables to your class. This way you can make a struct-like class without having to assign every value manually.

My example has no error checking so it is easier to follow.

class MyStruct(type):
    def __call__(cls, *args, **kwargs):
        names = cls.__init__.func_code.co_varnames[1:]

        self = type.__call__(cls, *args, **kwargs)

        for name, value in zip(names, args):
            setattr(self , name, value)

        for name, value in kwargs.iteritems():
            setattr(self , name, value)
        return self 

Here it is in action.

>>> class MyClass(object):
    __metaclass__ = MyStruct
    def __init__(self, a, b, c):
        pass


>>> my_instance = MyClass(1, 2, 3)
>>> my_instance.a
1
>>> 

I posted it on reddit and /u/matchu posted a decorator version which is cleaner. I'd encourage you to use it unless you want to expand the metaclass version.

>>> def init_all_args(fn):
    @wraps(fn)
    def wrapped_init(self, *args, **kwargs):
        names = fn.func_code.co_varnames[1:]

        for name, value in zip(names, args):
            setattr(self, name, value)

        for name, value in kwargs.iteritems():
            setattr(self, name, value)

    return wrapped_init

>>> class Test(object):
    @init_all_args
    def __init__(self, a, b):
        pass


>>> a = Test(1, 2)
>>> a.a
1
>>> 

answered Mar 23, 2015 at 14:32

user124757user124757

792 silver badges5 bronze badges

2

I wrote a decorator which you can use on any method to make it so that all of the arguments passed in, or any defaults, are assigned to the instance.

def argumentsToAttributes(method):
    argumentNames = method.func_code.co_varnames[1:]

    # Generate a dictionary of default values:
    defaultsDict = {}
    defaults = method.func_defaults if method.func_defaults else ()
    for i, default in enumerate(defaults, start = len(argumentNames) - len(defaults)):
        defaultsDict[argumentNames[i]] = default

    def newMethod(self, *args, **kwargs):
        # Use the positional arguments.
        for name, value in zip(argumentNames, args):
            setattr(self, name, value)

        # Add the key word arguments. If anything is missing, use the default.
        for name in argumentNames[len(args):]:
            setattr(self, name, kwargs.get(name, defaultsDict[name]))

        # Run whatever else the method needs to do.
        method(self, *args, **kwargs)

    return newMethod

A quick demonstration. Note that I use a positional argument a, use the default value for b, and a named argument c. I then print all 3 referencing self, to show that they've been properly assigned before the method is entered.

class A(object):
    @argumentsToAttributes
    def __init__(self, a, b = 'Invisible', c = 'Hello'):
        print(self.a)
        print(self.b)
        print(self.c)

A('Why', c = 'Nothing')

Note that my decorator should work with any method, not just __init__.

answered Sep 8, 2015 at 2:38

Can we use structure in python?

ArtOfWarfareArtOfWarfare

19.6k17 gold badges130 silver badges188 bronze badges

I don't see this answer here, so I figure I'll add it since I'm leaning Python right now and just discovered it. The Python tutorial (Python 2 in this case) gives the following simple and effective example:

class Employee:
    pass

john = Employee()  # Create an empty employee record

# Fill the fields of the record
john.name = 'John Doe'
john.dept = 'computer lab'
john.salary = 1000

That is, an empty class object is created, then instantiated, and the fields are added dynamically.

The up-side to this is its really simple. The downside is it isn't particularly self-documenting (the intended members aren't listed anywhere in the class "definition"), and unset fields can cause problems when accessed. Those two problems can be solved by:

class Employee:
    def __init__ (self):
        self.name = None # or whatever
        self.dept = None
        self.salary = None

Now at a glance you can at least see what fields the program will be expecting.

Both are prone to typos, john.slarly = 1000 will succeed. Still, it works.

answered Nov 8, 2016 at 14:06

Can we use structure in python?

Jason CJason C

36.6k14 gold badges116 silver badges166 bronze badges

Here is a solution which uses a class (never instantiated) to hold data. I like that this way involves very little typing and does not require any additional packages etc.

class myStruct:
    field1 = "one"
    field2 = "2"

You can add more fields later, as needed:

myStruct.field3 = 3

To get the values, the fields are accessed as usual:

>>> myStruct.field1
'one'

answered Apr 9, 2019 at 11:10

jochenjochen

3,4932 gold badges37 silver badges47 bronze badges

4

Personally, I like this variant too. It extends @dF's answer.

class struct:
    def __init__(self, *sequential, **named):
        fields = dict(zip(sequential, [None]*len(sequential)), **named)
        self.__dict__.update(fields)
    def __repr__(self):
        return str(self.__dict__)

It supports two modes of initialization (that can be blended):

# Struct with field1, field2, field3 that are initialized to None.
mystruct1 = struct("field1", "field2", "field3") 
# Struct with field1, field2, field3 that are initialized according to arguments.
mystruct2 = struct(field1=1, field2=2, field3=3)

Also, it prints nicer:

print(mystruct2)
# Prints: {'field3': 3, 'field1': 1, 'field2': 2}

answered Nov 6, 2017 at 15:38

normaniusnormanius

7,1904 gold badges44 silver badges77 bronze badges

There is a python package exactly for this purpose. see cstruct2py

cstruct2py is a pure python library for generate python classes from C code and use them to pack and unpack data. The library can parse C headres (structs, unions, enums, and arrays declarations) and emulate them in python. The generated pythonic classes can parse and pack the data.

For example:

typedef struct {
  int x;
  int y;
} Point;

after generating pythonic class...
p = Point(x=0x1234, y=0x5678)
p.packed == "\x34\x12\x00\x00\x78\x56\x00\x00"

How to use

First we need to generate the pythonic structs:

import cstruct2py
parser = cstruct2py.c2py.Parser()
parser.parse_file('examples/example.h')

Now we can import all names from the C code:

parser.update_globals(globals())

We can also do that directly:

A = parser.parse_string('struct A { int x; int y;};')

Using types and defines from the C code

a = A()
a.x = 45
print a
buf = a.packed
b = A(buf)
print b
c = A('aaaa11112222', 2)
print c
print repr(c)

The output will be:

{'x':0x2d, 'y':0x0}
{'x':0x2d, 'y':0x0}
{'x':0x31316161, 'y':0x32323131}
A('aa111122', x=0x31316161, y=0x32323131)

Clone

For clone cstruct2py run:

git clone https://github.com/st0ky/cstruct2py.git --recursive

answered Dec 11, 2018 at 9:34

0

Here is a quick and dirty trick:

>>> ms = Warning()
>>> ms.foo = 123
>>> ms.bar = 'akafrit'

How does it works? It just re-use the builtin class Warning (derived from Exception) and use it as it was you own defined class.

The good points are that you do not need to import or define anything first, that "Warning" is a short name, and that it also makes clear you are doing something dirty which should not be used elsewhere than a small script of yours.

By the way, I tried to find something even simpler like ms = object() but could not (this last exemple is not working). If you have one, I am interested.

answered Sep 26, 2019 at 13:56

calandoacalandoa

5,3382 gold badges25 silver badges24 bronze badges

0

NamedTuple is comfortable. but there no one shares the performance and storage.

from typing import NamedTuple
import guppy  # pip install guppy
import timeit


class User:
    def __init__(self, name: str, uid: int):
        self.name = name
        self.uid = uid


class UserSlot:
    __slots__ = ('name', 'uid')

    def __init__(self, name: str, uid: int):
        self.name = name
        self.uid = uid


class UserTuple(NamedTuple):
    # __slots__ = ()  # AttributeError: Cannot overwrite NamedTuple attribute __slots__
    name: str
    uid: int


def get_fn(obj, attr_name: str):
    def get():
        getattr(obj, attr_name)
    return get
if 'memory test':
    obj = [User('Carson', 1) for _ in range(1000000)]      # Cumulative: 189138883
    obj_slot = [UserSlot('Carson', 1) for _ in range(1000000)]          # 77718299  <-- winner
    obj_namedtuple = [UserTuple('Carson', 1) for _ in range(1000000)]   # 85718297
    print(guppy.hpy().heap())  # Run this function individually. 
    """
    Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0 1000000    24 112000000 34 112000000  34 dict of __main__.User
     1 1000000    24 64000000  19 176000000  53 __main__.UserTuple
     2 1000000    24 56000000  17 232000000  70 __main__.User
     3 1000000    24 56000000  17 288000000  87 __main__.UserSlot
     ...
    """

if 'performance test':
    obj = User('Carson', 1)
    obj_slot = UserSlot('Carson', 1)
    obj_tuple = UserTuple('Carson', 1)

    time_normal = min(timeit.repeat(get_fn(obj, 'name'), repeat=20))
    print(time_normal)  # 0.12550550000000005

    time_slot = min(timeit.repeat(get_fn(obj_slot, 'name'), repeat=20))
    print(time_slot)  # 0.1368690000000008

    time_tuple = min(timeit.repeat(get_fn(obj_tuple, 'name'), repeat=20))
    print(time_tuple)  # 0.16006120000000124

    print(time_tuple/time_slot)  # 1.1694481584580898  # The slot is almost 17% faster than NamedTuple on Windows. (Python 3.7.7)

If your __dict__ is not using, please choose between __slots__ (higher performance and storage) and NamedTuple (clear for reading and use)

You can review this link(Usage of slots) to get more __slots__ information.

answered Jun 5, 2020 at 10:05

Can we use structure in python?

CarsonCarson

4,4242 gold badges27 silver badges36 bronze badges

https://stackoverflow.com/a/32448434/159695 does not work in Python3.

https://stackoverflow.com/a/35993/159695 works in Python3.

And I extends it to add default values.

class myStruct:
    def __init__(self, **kwds):
        self.x=0
        self.__dict__.update(kwds) # Must be last to accept assigned member variable.
    def __repr__(self):
        args = ['%s=%s' % (k, repr(v)) for (k,v) in vars(self).items()]
        return '%s(%s)' % ( self.__class__.__qualname__, ', '.join(args) )

a=myStruct()
b=myStruct(x=3,y='test')
c=myStruct(x='str')

>>> a
myStruct(x=0)
>>> b
myStruct(x=3, y='test')
>>> c
myStruct(x='str')

answered Oct 30, 2017 at 13:40

GalaxyGalaxy

1,7521 gold badge15 silver badges25 bronze badges

1

The following solution to a struct is inspired by the namedtuple implementation and some of the previous answers. However, unlike the namedtuple it is mutable, in it's values, but like the c-style struct immutable in the names/attributes, which a normal class or dict isn't.

_class_template = """\
class {typename}:
def __init__(self, *args, **kwargs):
    fields = {field_names!r}

    for x in fields:
        setattr(self, x, None)            

    for name, value in zip(fields, args):
        setattr(self, name, value)

    for name, value in kwargs.items():
        setattr(self, name, value)            

def __repr__(self):
    return str(vars(self))

def __setattr__(self, name, value):
    if name not in {field_names!r}:
        raise KeyError("invalid name: %s" % name)
    object.__setattr__(self, name, value)            
"""

def struct(typename, field_names):

    class_definition = _class_template.format(
        typename = typename,
        field_names = field_names)

    namespace = dict(__name__='struct_%s' % typename)
    exec(class_definition, namespace)
    result = namespace[typename]
    result._source = class_definition

    return result

Usage:

Person = struct('Person', ['firstname','lastname'])
generic = Person()
michael = Person('Michael')
jones = Person(lastname = 'Jones')


In [168]: michael.middlename = 'ben'
Traceback (most recent call last):

  File "", line 1, in 
michael.middlename = 'ben'

  File "", line 19, in __setattr__

KeyError: 'invalid name: middlename'

answered Mar 29, 2018 at 16:25

If you don't have a 3.7 for @dataclass and need mutability, the following code might work for you. It's quite self-documenting and IDE-friendly (auto-complete), prevents writing things twice, is easily extendable and it is very simple to test that all instance variables are completely initialized:

class Params():
    def __init__(self):
        self.var1 : int = None
        self.var2 : str = None

    def are_all_defined(self):
        for key, value in self.__dict__.items():
            assert (value is not None), "instance variable {} is still None".format(key)
        return True


params = Params()
params.var1 = 2
params.var2 = 'hello'
assert(params.are_all_defined)

answered Jan 28, 2019 at 11:27

Can we use structure in python?

gebbissimogebbissimo

1,6742 gold badges21 silver badges30 bronze badges

The best way I found to do this was to use a custom dictionary class as explained in this post: https://stackoverflow.com/a/14620633/8484485

If iPython autocompletion support is needed, simply define the dir() function like this:

class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self
    def __dir__(self):
        return self.keys()

You then define your pseudo struct like so: (this one is nested)

my_struct=AttrDict ({
    'com1':AttrDict ({
        'inst':[0x05],
        'numbytes':2,
        'canpayload':False,
        'payload':None
    })
})

You can then access the values inside my_struct like this:

print(my_struct.com1.inst)

=>[5]

Can we use structure in python?

halfer

19.6k17 gold badges92 silver badges176 bronze badges

answered May 7, 2020 at 14:23

Can we use structure in python?

TionebTioneb

3262 silver badges5 bronze badges

I think Python structure dictionary is suitable for this requirement.

d = dict{}
d[field1] = field1
d[field2] = field2
d[field2] = field3

answered Jul 8, 2017 at 19:17

Why struct is used in Python?

The module struct is used to convert the native data types of Python into string of bytes and vice versa. We don't have to install it. It's a built-in module available in Python3. The struct module is related to the C languages.

How do you read a structure in Python?

Reading binary data as a numpy array:.
np. frombuffer(bytes_data, dtype=...) : Interpret the given binary data (e.g. a python bytes instance) as a numpy array of the given dtype. ... .
np. fromfile(filename, dtype=...) : Read binary data from filename ..