Dynamically evaluating code PREMIUM

How can you dynamically evaluate Python code?

Evaluating a single Python expression

Let's say we're trying to make a program that accepts a mathematical expression, runs it as Python code, and prints out the result:

import sys

expression = " ".join(sys.argv[1:])
print(f"Expression: {expression}")

result = "???"
print(f"Result: {result}")

Right now our program doesn't print anything out:

$ python3 calc.py 2+3
Expression: 2+3
Result: ???

We need a way to run the given string as Python code.

But we could use the built-in eval function to do that:

import sys

expression = " ".join(sys.argv[1:])
print(f"Expression: {expression}")

result = eval(expression)
print(f"Result: {result}")

The eval function accepts a string that represents a single Python expression and returns the result of that expression:

$ python3 calc.py 2+3
Expression: 2+3
Result: 5

The eval function only accepts one Python expression:

>>> eval("2+3")
5

So we can't pass a string with a semicolon (;) or newlines (\n) in it:

>>> eval("print('hi'); print('hello')")
  File "<string>", line 1
    print('hi'); print('hello')
               ^
SyntaxError: invalid syntax

That would entail passing multiple statements to eval, which isn't allowed.

In fact, we can't even necessarily pass a whole statement to eval. For example we can't assign to a variable with eval:

>>> eval("x = 2+3")
  File "<string>", line 1
    x = 2+3
      ^
SyntaxError: invalid syntax

The eval function will only accept a single expression that will return a result:

>>> eval("2+3")
5

Evaluating multiple Python expressions

What if we wanted to evaluate multiple Python expressions at once? We can't use the eval function to evaluate multiple expressions at once, but we can use the exec function.

If you've ever pasted a block of code into the Python REPL, you might have noticed that the code that you paste can't have any blank lines within it:

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

    def __repr__(self):
        return f"Point({self.x}, {self.y})"

If it does have blank lines within it, the Python REPL will interpret that blank line as the end of a block of code:

    >>> class Point:
...     def __init__(self, x, y):
...         self.x, self.y = x, y
...
>>>     def __repr__(self):
  File "<stdin>", line 1
    def __repr__(self):
IndentationError: unexpected indent
>>>         return f"Point({self.x}, {self.y})"
  File "<stdin>", line 1
    return f"Point({self.x}, {self.y})"
IndentationError: unexpected indent

But we weren't actually trying to end our block of code yet here.

We could fix this problem by pasting our code into a multi-line string, and then use the built-in exec function to execute the code that's in that string:

>>> code = """
... class Point:
...     def __init__(self, x, y):
...         self.x, self.y = x, y
...
...     def __repr__(self):
...         return f"Point({self.x}, {self.y})"
... """
>>> exec(code)

The eval function returns the result of the single expression it runs, but exec doesn't return anything (it always returns None).

The exec function just runs the code we gave it. So, thanks to exec, we now have a class called Point:

>>> Point(2, 3)
Point(2, 3)

Be very cautious running arbitrary code

You should never use eval or exec on Python code that you don't trust. Never put any untrusted user input into eval or exec.

Controlling the environment: not overwriting local variables

What if you do trust the code that you're running in eval or exec, but you don't want to accidentally override your own local variables? Fortunately, both the eval and exec functions accept two optional dictionaries: a globals dictionary, and a locals dictionary.

By default, the globals and locals dictionaries default to the dictionaries you would see if you call the built-in locals() and globals() functions just before eval or exec. But we could customize them by passing in new dictionaries.

Here we have a string that assigns to a variable x and then assigns to a variable y:

>>> code = """
... x = 2
... y = x + 3
... """

Let's use exec by providing an empty globals dictionary and an empty locals dictionary which we'll call variables:

>>> variables = {}
>>> exec(code, {}, variables)

Since this code assigned to some local variables, our variables dictionary should now have two values within it: x and y.

And it does!

>>> variables
{'x': 2, 'y': 5}

It's very common to specify a custom locals dictionary when using exec, because exec is usually used for creating one or more variables.

Better alternatives to eval or exec

Now, I should note that exec and eval are often not the best way to dynamically run some Python code.

Here we have our Point class:

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

    def __repr__(self):
        return f"Point({self.x}, {self.y})"

Let's say we're trying to dynamically look up an attribute on a Point object.

>>> p = Point(1, 2)
>>> coordinate = "x"

We could use eval to do that:

>>> eval(f"p.{coordinate}")
1

But it would probably be better to use the built-in getattr function:

>>> getattr(p, coordinate)
1

The getattr function is intended for dynamically looking up an attribute from a string.

Likewise, if we were trying to dynamically import a module, we could try to use exec for that:

>>> module_name = "math"
>>> modules = {}
>>> exec(f"import {module_name}", modules)
>>> modules[module_name]
<module 'math' from '/usr/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>

But it would probably be better to use the import_module function from Python's importlib module:

>>> import importlib
>>> module_name = "math"
>>> module = importlib.import_module(module_name)
>>> module
<module 'math' from '/usr/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>

The import_module function is designed to dynamically import modules from their name.

Summary

Python's built-in exec and eval functions can dynamically run Python code.

The eval function accepts a single expression and returns the result of that expression. The exec function runs a whole block of Python code.

But keep in mind that these functions are pretty rarely used because there's often a better way to make your Python code a little bit more dynamic.