I have a Python app part of which copies files and then runs a checksum check on them to make sure they copied correctly. It's been running fine for months on the internal network at several companies on MacOS and Windows environments with various types of network drives hosting source and destination folders.
However now one user on MacOS Catalina is consistently receiving IOError: [Errno 35] Resource temporarily unavailable
at some point during the call to shutil.copyfileobj()
which I can't figure out.
This is Python 2.7 (Yes, I'd love to be on Python 3, but there are too many dependencies on other products that still don't yet support it).
def _copy_file(self, src, dst, buff_size=10 * 1024 * 1024):
"""
Copy src file into dst destination
:param src: Full path to the source file to copy
:param dst: Full destination path
:param buff_size: Buffer size in bytes to use in optimized copy, -1 means no buffering
"""
# Optimize the buffer for small files
# http://blogs.blumetech.com/blumetechs-tech-blog/2011/05/faster-python-file-copy.html
if buff_size > 0:
buff_size = min(buff_size, os.path.getsize(src))
if(buff_size == 0):
buff_size = 1024
self._logger.debug("Using %d as copy buffer size" % buff_size)
with open(src, "rb") as fin:
with open(dst, "wb") as fout:
shutil.copyfileobj(fin, fout, buff_size)
This returns:
...
File "/xxx/format_copier.py", line 595, in _copy_file
shutil.copyfileobj(fin, fout, buff_size)
IOError: [Errno 35] Resource temporarily unavailable
try/except
looking for IOErrors
that match errno.EAGAIN
retrying up to 5 times still produced the same errors eventually.One other detail is sometimes if the copy does succeed, it occasionally throws the same error while running the checksum.
def crc32checksum(cls, file_path, blocksize=65536):
"""
Return a crc32 checksum for the given file. This is a lot faster than
computing an md5 checksum.
See:
https://docs.python.org/2.7/library/binascii.html?highlight=crc32#binascii.crc32
:param str file_path: Full file path.
:param int blocksize: Block size for reading the file.
:returns: A string with a hexadecimal checksum value.
"""
crc = 0
with open(file_path, "rb") as f:
for block in iter(lambda: f.read(blocksize), ""):
crc = binascii.crc32(block, crc) & 0xffffffff # signed to unsigned
returns:
File "/xxx/format.py", line 314, in crc32checksum
crc = binascii.crc32(block, crc) & 0xffffffff # signed to unsigned
IOError: [Errno 35] Resource temporarily unavailable
I'm at the end of my troubleshooting ability and the IT team verified the network drives are functioning as expected. I'll take any ideas at this point! Thanks.
User contributions licensed under CC BY-SA 3.0