Code generation for Python 3 annotation with default value

1

In working over uncompyle6 decompilation bugs I've come across problem of trying to reconcile CPython assembly output with what the Python docs describe for MAKE_FUNCTION.

Python Source:

def foo(x: 'an argument that defaults to 5' = 5):
    return

Disassemby (xdis's version):

 # Argument count:    1
 # Kw-only arguments: 0
 # Number of locals:  1
 # Stack size:        1
 # Flags:             0x00000043 (NOFREE | NEWLOCALS | OPTIMIZED)
 # First Line:        1
 # Constants:
 #    0: 5
 #    1: 'an argument that defaults to 5'
 #    2: ('x',)
 #    3: <code object foo at 0x7f49115938a0, file "exec", line 1>
 #    4: 'foo'
 #    5: None
 # Names:
 #    0: foo
 1 0 LOAD_CONST                0 (5)
   3 LOAD_CONST                1 ('an argument that defaults to 5')
   6 LOAD_CONST                2 (('x',))
   9 LOAD_CONST                3 (<code object foo at 0x7f49115938a0, file "exec", line 1>)
  12 LOAD_CONST                4 ('foo')
  15 EXTENDED_ARG              2 (131072)
  18 MAKE_FUNCTION        131073 (1 positional, 0 name and default, 2 annotations)
  21 STORE_NAME                0 (foo)
  24 LOAD_CONST                5 (None)
  27 RETURN_VALUE

Note that in offset 18 the value is basically the arg value at offset 19 (1) plus the extended arg value.

The interpretation in parenthesis is xdis's and may not be correct. Edit: Not only is it correct, but the additional pair in offset 6 to indicate a tuple is essential.

In https://docs.python.org/3.4/library/dis.html#opcode-MAKE_FUNCTION it says:

Pushes a new function object on the stack. From bottom to top, the consumed stack must consist of

  • argc & 0xFF default argument objects in positional order
  • (argc >> 8) & 0xFF pairs of name and default argument, with the name just below the object on the stack, for keyword-only parameters
  • (argc >> 16) & 0x7FFF parameter annotation objects
  • a tuple listing the parameter names for the annotations (only if there are any annotation objects)

It looks to me like there is one annotation object, not two. And one default argument rather than a positional argument. Also at offset 24 we see an mention of 5, but this is after we have MAKE_FUNCTION. The association of a default value with parameter x is elusive in the code. Some sort of optimization here?

How am I to make sense of the assembly as an accurate representation of the Python source?

Note: I see this code generated in at least Python 3.1 - 3.5

python-3.x
bytecode
cpython
disassembly
asked on Stack Overflow Jun 4, 2017 by rocky • edited Jul 9, 2017 by rocky

1 Answer

1

Below is uncompyle6's deparse of the above. I'm not totally sure it is correct that the first argument of mkfunc_annotate is rightly called a pos_arg rather than say a default value arg.

One of the subtle points in the assembly above is that the LOAD_CONST is of a tuple (with one argument) and that is important in clueing a deparser (or person) in that the function is annotated.

stmts
    sstmt
        stmt
            funcdef_annotate (2)
                 0. mkfunc_annotate (7)
                     0. pos_arg
                        expr
                            L.   1       0  LOAD_CONST            5  5
                     1. annotate_arg
                        expr
                                         3  LOAD_CONST               'an argument that defaults to 5'
                     2. annotate_tuple
                                     6  LOAD_CONST               ('x',)
                     3.              9  LOAD_CONST               '<code_object foo>'
                     4.             12  LOAD_CONST               'foo'
                     5.             15  EXTENDED_ARG     131074  '131072'
                     6.             18  MAKE_FUNCTION_A_2_1        '1 positional, 0 keyword pair, 2 annotated'
                 1. designator
                                21  STORE_NAME               'foo'
answered on Stack Overflow Jul 9, 2017 by rocky • edited Aug 1, 2017 by rocky

User contributions licensed under CC BY-SA 3.0