Python Generator Objects: __sizeof__()

This may be a stupid question but I will ask it anyway. I have a generator object: >>> def gen(): ... for i in range(10): ... yield i ... >>>

Solution 1:

__sizeof__() does not do what you think it does. The method returns the internal size in bytes for the given object, not the number of items a generator is going to return.

Python cannot beforehand know the size of a generator. Take for example the following endless generator (example, there are better ways to create a counter):

def count():
    count = 0while True:
        yield count
        count += 1

That generator is endless; there is no size assignable to it. Yet the generator object itself takes memory:

>>>count.__sizeof__()
88

You don't normally call __sizeof__() you leave that to the sys.getsizeof() function, which also adds garbage collector overhead.

If you know a generator is going to be finite and you have to know how many items it returns, use:

sum(1 for item in generator)

but note that that exhausts the generator.

Solution 2:

As said in other answers, __sizeof__ returns a different thing.

Only some iterators have methods that return the number of not returned elements. For example listiterator has a corresponding __length_hint__ method:

>>>L = [1,2,3,4,5]>>>it = iter(L)>>>it
<listiterator object at 0x00E65350>
>>>it.__length_hint__()
5
>>>help(it.__length_hint__)
Help on built-in function __length_hint__:

__length_hint__(...)
    Private method returning an estimate of len(list(it)).

>>>it.next()
1
>>>it.__length_hint__()
4

Solution 3:

__sizeof__ returns the memory size of an object in bytes, not the length of a generator, which is impossible to determine up front as generators can grow indefinitely.

Solution 4:

If you are certain that the generator you've created is "finite" (has a countable number of elements) and you don't mind waiting a while you can use the following to get what you want:

len(list(gen()))

As the other posters said __sizeof__() is a measure of how much memory something takes up (a much lower level concept that you probably will rarely need), not its length (which is not a feature of generators since there's no guarantee they have a countable length).

Solution 5:

@Martijn Pieters You may also overwirte the sizeof() function and we can achieve this thing that we are trying to do here.But it will not work on some data types such int or float.

classGetLen():
    def__sizeof__(self, x):
        returnlen(x)

so here for int or float we can also put some escape function if the type is int or float like.