I have a Python app part of which copies files and then runs a checksum check on them to make sure they copied correctly. It's been running fine for months on the internal network at several companies on MacOS and Windows environments with various types of network drives hosting source and destination folders.
However now one user on MacOS Catalina is consistently receiving
IOError: [Errno 35] Resource temporarily unavailable at some point during the call to
shutil.copyfileobj() which I can't figure out.
This is Python 2.7 (Yes, I'd love to be on Python 3, but there are too many dependencies on other products that still don't yet support it).
def _copy_file(self, src, dst, buff_size=10 * 1024 * 1024): """ Copy src file into dst destination :param src: Full path to the source file to copy :param dst: Full destination path :param buff_size: Buffer size in bytes to use in optimized copy, -1 means no buffering """ # Optimize the buffer for small files # http://blogs.blumetech.com/blumetechs-tech-blog/2011/05/faster-python-file-copy.html if buff_size > 0: buff_size = min(buff_size, os.path.getsize(src)) if(buff_size == 0): buff_size = 1024 self._logger.debug("Using %d as copy buffer size" % buff_size) with open(src, "rb") as fin: with open(dst, "wb") as fout: shutil.copyfileobj(fin, fout, buff_size)
... File "/xxx/format_copier.py", line 595, in _copy_file shutil.copyfileobj(fin, fout, buff_size) IOError: [Errno 35] Resource temporarily unavailable
errno.EAGAINretrying up to 5 times still produced the same errors eventually.
One other detail is sometimes if the copy does succeed, it occasionally throws the same error while running the checksum.
def crc32checksum(cls, file_path, blocksize=65536): """ Return a crc32 checksum for the given file. This is a lot faster than computing an md5 checksum. See: https://docs.python.org/2.7/library/binascii.html?highlight=crc32#binascii.crc32 :param str file_path: Full file path. :param int blocksize: Block size for reading the file. :returns: A string with a hexadecimal checksum value. """ crc = 0 with open(file_path, "rb") as f: for block in iter(lambda: f.read(blocksize), ""): crc = binascii.crc32(block, crc) & 0xffffffff # signed to unsigned
File "/xxx/format.py", line 314, in crc32checksum crc = binascii.crc32(block, crc) & 0xffffffff # signed to unsigned IOError: [Errno 35] Resource temporarily unavailable
I'm at the end of my troubleshooting ability and the IT team verified the network drives are functioning as expected. I'll take any ideas at this point! Thanks.
User contributions licensed under CC BY-SA 3.0