# Iterate over list starting from a certain index

1

Here is an example:

``````list_ = [5, 'cat', 0xDEADBEEF, 4.0]

for offset in range(len(list_)):
result = 0
for elem in list_[offset:]:
result = func(result, elem)
return result
``````

where `func` is non-commutative.

In the code above, `list_[offset:]` will create a new list, but all I need is a view to `list_`. How can I optimize this?

python
arrays
list
iterator
iteration

1

To replicate your slicing but in O(1) time each iteration, you can use `collections.deque` with `popleft`:

``````from collections import deque

dq = deque(list_)

for i in range(len(dq)):
print(dq)
dq.popleft()
``````

Result:

``````deque([5, 'cat', 3735928559, 4.0])
deque(['cat', 3735928559, 4.0])
deque([3735928559, 4.0])
deque([4.0])
``````

This should be more efficient than list slicing: see deque.popleft() and list.pop(0). Is there performance difference?. Note also list slicing works in O(k) time where k is the length of the slice.

0

Using `collections.deque`, as suggested by @jpp's answer, is slightly faster sometimes. Both slice and `deque` solutions perform similarly, and better than e.g. using `itertools.islice` or just plain indexes on `list_`.

I've tried to make the versions more or less equivalent overall, and used a dummy `func` that counts the loops:

``````from __future__ import print_function
from collections import deque
from itertools import islice
from timeit import repeat

import numpy as np

list_ = [5, 'cat', 0xDEADBEEF, 4.0]
list_3k = list_ * 3000

def func(x, y):
return x + 1

def f1():
"""list slice"""
result = 0
for offset in range(len(list_)):
for elem in list_[offset:]:
result = func(result, elem)
return result

def f2():
"""deque"""
dq = deque(list_)
result = 0
for i in range(len(dq)):
for elem in dq:
result = func(result, elem)
dq.popleft()
return result

def f3():
"""itertools slice"""
result = 0
for offset in range(len(list_)):
for elem in islice(list_, offset, None):
result = func(result, elem)
return result

def f4():
"""basics"""
result = 0
n = len(list_)
for offset in range(n):
j = offset
while j < n:
result = func(result, list_[j])
j += 1
return result

def timeit(fn, number):
print("{}: {} loops".format(fn.__name__, fn()))
times = repeat(fn, repeat=3, number=number)
print("{:.3f}s ± {:.3f}ms".format(np.mean(times), np.std(times)*1000))

if __name__ == "__main__":
fs = [f1, f2, f3, f4]

for f in fs:
timeit(f, number=1000000)

list_ = list_3k
print()

for f in fs:
timeit(f, number=3)
``````

Results:

``````bash-3.2\$ python3 foo.py
f1: 10 loops
2.161s ± 9.333ms
f2: 10 loops
2.134s ± 5.127ms
f3: 10 loops
2.340s ± 11.928ms
f4: 10 loops
2.315s ± 4.615ms

f1: 72006000 loops
23.073s ± 109.857ms
f2: 72006000 loops
23.495s ± 596.822ms
f3: 72006000 loops
24.432s ± 553.167ms
f4: 72006000 loops
40.509s ± 128.367ms
``````

User contributions licensed under CC BY-SA 3.0