shutil.copyfileobj() returns IOError: [Errno 35] Resource temporarily unavailable

0

I have a Python app part of which copies files and then runs a checksum check on them to make sure they copied correctly. It's been running fine for months on the internal network at several companies on MacOS and Windows environments with various types of network drives hosting source and destination folders.

However now one user on MacOS Catalina is consistently receiving IOError: [Errno 35] Resource temporarily unavailable at some point during the call to shutil.copyfileobj() which I can't figure out.

This is Python 2.7 (Yes, I'd love to be on Python 3, but there are too many dependencies on other products that still don't yet support it).

    def _copy_file(self, src, dst, buff_size=10 * 1024 * 1024):
        """
        Copy src file into dst destination

        :param src: Full path to the source file to copy
        :param dst: Full destination path
        :param buff_size: Buffer size in bytes to use in optimized copy, -1 means no buffering
        """
        # Optimize the buffer for small files
        # http://blogs.blumetech.com/blumetechs-tech-blog/2011/05/faster-python-file-copy.html
        if buff_size > 0:
            buff_size = min(buff_size, os.path.getsize(src))
            if(buff_size == 0):
                buff_size = 1024
        self._logger.debug("Using %d as copy buffer size" % buff_size)
        with open(src, "rb") as fin:
            with open(dst, "wb") as fout:
                shutil.copyfileobj(fin, fout, buff_size)

This returns:

  ...
  File "/xxx/format_copier.py", line 595, in _copy_file
    shutil.copyfileobj(fin, fout, buff_size)
IOError: [Errno 35] Resource temporarily unavailable
  • files are being read and written to the same SMB mounted drive
  • it consistently errors on larger files but not always the same ones or at the same moment
  • modifying the buffer settings and using the python default had no effect
  • adding a try/except looking for IOErrors that match errno.EAGAIN retrying up to 5 times still produced the same errors eventually.
  • running this on a different MacOS Catalina machine on the same company network does not produce these issues, it appears specific to this one machine (so far)

One other detail is sometimes if the copy does succeed, it occasionally throws the same error while running the checksum.

    def crc32checksum(cls, file_path, blocksize=65536):
        """
        Return a crc32 checksum for the given file. This is a lot faster than
        computing an md5 checksum.

        See:
        https://docs.python.org/2.7/library/binascii.html?highlight=crc32#binascii.crc32

        :param str file_path: Full file path.
        :param int blocksize: Block size for reading the file.
        :returns: A string with a hexadecimal checksum value.
        """
        crc = 0
        with open(file_path, "rb") as f:
            for block in iter(lambda: f.read(blocksize), ""):
                crc = binascii.crc32(block, crc) & 0xffffffff  # signed to unsigned

returns:

  File "/xxx/format.py", line 314, in crc32checksum
    crc = binascii.crc32(block, crc) & 0xffffffff  # signed to unsigned
IOError: [Errno 35] Resource temporarily unavailable

I'm at the end of my troubleshooting ability and the IT team verified the network drives are functioning as expected. I'll take any ideas at this point! Thanks.

python
python-2.7
io
shutil
asked on Stack Overflow Apr 9, 2021 by kporangehat

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0