Decorators, Dataclasses and IDEs¶
I'll talk through some common tips for writing cleaner code and common Python patterns.
Basic Example¶
We define a simple function which adds an exclamation to a string and returns it.
There are often examples which start from functions which simply print to the console but for our purposes we will most likely be returning values, so let's start there.
def exclamation(string):
return string + "!"
exclamation('hello')
'hello!'
Now let's define a decorator function which adds another exclamation mark.
import functools
def emphasis(func):
@functools.wraps(func) # optional
def wrapper(*args, **kwargs):
value = func(*args, **kwargs) # this value is the original "hello!"
return value + "!" # let's add that "!"
# this wrapper_do_twice is a function...
# ...which takes a function and returns it's string return value, with an additional "!"
return wrapper
@emphasis
def exclamation_exclamation(*args, **kwargs):
return exclamation(*args, **kwargs)
exclamation('hello')
'hello!'
exclamation_exclamation(string = 'hello')
'hello!!'
We see the extra exclamation mark!!
Okay, we've had a lot of fun there but now we understand the basic concept let's move on to a more substantive example.
Timing Decorator¶
import datetime
def timing(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start_time = datetime.datetime.now()
result = func(*args, **kwargs)
end_time = datetime.datetime.now()
time_string = \
f"""Function Name:
{func.__name__}
Args:
{args, kwargs}
Start Time:
{start_time}
End Time:
{end_time}
Time Taken:
{(end_time-start_time).total_seconds()} seconds.
"""
print(time_string)
return result
return wrapper
exclamation('hello')
'hello!'
@timing
def exclamation_timed(string):
return string + "!"
exclamation_timed('hello')
Function Name:
exclamation_timed
Args:
(('hello',), {})
Start Time:
2019-12-11 23:23:38.582135
End Time:
2019-12-11 23:23:38.582142
Time Taken:
7e-06 seconds.
'hello!'
Logging Decorator¶
Decorator functions work best when you're centralising some form of complexity, this can increase readability and make subsequent refactoring easier.
Now let's move to a more substantive example... logging!
import logging
logger = logging.getLogger('example_logger')
logging.basicConfig(format='%(asctime)s - %(message)s', level=logging.INFO)
logger.warning('This is an only an example warning - relax')
2019-12-11 23:23:38,593 - This is an only an example warning - relax
def logging_decorator(func):
"""This decorator logs functions.
"""
@functools.wraps(func) # optional but recommended way of preserving func name and doc string.
def logged_function(*args, **kwargs):
"""This function is the logged function.
All it does is add the name of the function to the logging stream.
"""
logger.info(f'running {func.__name__}...')
return func(*args, **kwargs)
return logged_function
@logging_decorator
def exclamation(string):
return string + "!"
exclamation('hello')
2019-12-11 23:23:38,619 - running exclamation...
'hello!'
@logging_decorator
def something_else(string):
"""Adds a question mark to a string and returns it."""
return string + "?"
something_else('hello')
2019-12-11 23:23:38,637 - running something_else...
'hello?'
We can do this because... functions in Python are themselves objects.
Therefore all functions inherit methods from the base function class.
Let's have a look at some of them.
something_else.__class__
function
something_else.__name__
'something_else'
print('You can even look at the docstring...')
print(something_else.__doc__)
You can even look at the docstring... Adds a question mark to a string and returns it.
Let's see if we can take this up a level.
Often you'll be working with classes you have defined, so let's see how we can decorate these.
Clue: we'll be using similar private methods as we used for the function example just now.
class GreetingClass():
def __init__(self, name):
self.name = name
def hello(self):
return f"Hello {self.name}"
test_greeting = GreetingClass('Daniel')
test_greeting.hello()
'Hello Daniel'
Let's look behind the scenes of this class.
GreetingClass.__dict__
mappingproxy({'__module__': '__main__',
'__init__': <function __main__.GreetingClass.__init__(self, name)>,
'hello': <function __main__.GreetingClass.hello(self)>,
'__dict__': <attribute '__dict__' of 'GreetingClass' objects>,
'__weakref__': <attribute '__weakref__' of 'GreetingClass' objects>,
'__doc__': None})
It's got a dictionary* with the keys are methods and values are the functions themselves!
* It's actually a mapping proxy object, which is a lightweight dictionary which doesn't support item assignment, hence we use setattr() below.
def logging_the_class(cls):
for key, func in cls.__dict__.items():
if key.startswith("__") and key.endswith("__") or not callable(func):
# we don't care about decorating __init__ or any private methods
continue
# below is the key line, it sets the function with setattr to the logged function
setattr(cls, key, logging_decorator(func))
print("Wrapped", key)
return cls
@logging_the_class
class GreetingClass():
def __init__(self, name):
self.name = name
def hello(self):
return f"Hello {self.name}"
def goodbye(self):
return f"Goodbye {self.name}"
Wrapped hello Wrapped goodbye
test_greeting = GreetingClass('Daniel')
test_greeting.hello()
2019-12-11 23:23:38,747 - running hello...
'Hello Daniel'
test_greeting.goodbye()
2019-12-11 23:23:38,757 - running goodbye...
'Goodbye Daniel'
Further logging is out of the scope of this talk, but you could do things like dump your logs to a file, or define a decorator which is itself a class (and therefore stores state ) which adds the number of times the function is called to the logging stream.
Decorating a method¶
This is final decorator example and it's a bit more complex.
We're decorating a single method.
The main issue we have, how do we access .self?
First let's see what we do with the loggingdecorator() to make it work for a single method.
def logging_decorator(func):
"""This decorator logs functions.
"""
@functools.wraps(func)
def logged_function(*args, **kwargs):
"""This function is the logged function.
All it does is add the name of the function to the logging stream.
"""
self = args[0]
logger.info(f'running {func.__name__}...')
return func(*args, **kwargs)
return logged_function
def logging_decorator(func):
"""This decorator logs functions.
"""
@functools.wraps(func) # optional but recommended way of preserving func name and doc string.
def logged_function(*args, **kwargs):
"""This function is the logged function.
All it does is add the name of the function to the logging stream.
"""
self = args[0]
logger.info(f'running {func.__name__}...')
return func(*args, **kwargs)
return logged_function
class NewGreetingClass():
def __init__(self, name):
self.name = name
def hello(self):
return f"Hello {self.name}"
@logging_decorator
def goodbye(self):
return f"Goodbye {self.name}"
new_greeting = NewGreetingClass('Dan')
# no logging
new_greeting.hello()
'Hello Dan'
# logged
new_greeting.goodbye()
2019-12-11 23:23:38,814 - running goodbye...
'Goodbye Dan'
Dataclasses¶
Dataclasses are a convenient way to create classes without nearly as much boilerplate code.
Why?¶
from dataclasses import dataclass
class NewGreetingClass():
def __init__(self, name):
self.name: str = name
def hello(self):
return f"Hello {self.name}"
def goodbye(self):
return f"Goodbye {self.name}"
To define the simple class above, we have had to type name three times.
There has to be a better way...
@dataclass
class DCNewGreetingClass:
name: str
def hello(self):
return f"Hello {self.name}"
def goodbye(self):
return f"Goodbye {self.name}"
This pattern allows us to define a class and have its 'init' function generated for us.
This can be especially useful for large classes with many paramaters.
However there is downside...
...as of Python 3.7 Dataclasses do not support properties.
But we can get round this with
@dataclass
class NewGreetingClassStructure:
_name: str
@property
def name(self):
print('Getting name...')
return f"{self._name}"
@name.setter
def name(self, new_name):
print('Setting name...')
self._name = new_name
test = NewGreetingClassStructure('Daniel')
test.name = 'Dan'
Setting name...
Immutable Dataclasses¶
@dataclass(frozen=True)
class ImmutableGreetingClass():
name: str
def hello(self):
return f"Hello {self.name}"
def goodbye(self):
return f"Goodbye {self.name}"
immutable_test = ImmutableGreetingClass(name='Dan')
immutable_test.name = 'Daniel'
--------------------------------------------------------------------------- FrozenInstanceError Traceback (most recent call last) <ipython-input-10-89ee780836ec> in <module> ----> 1 immutable_test.name = 'Daniel' <string> in __setattr__(self, name, value) FrozenInstanceError: cannot assign to field 'name'
Immutability in dataclasses is that simple.
As your class gets bigger, you may want to have a dataclass that you define all parameters in.
Then you can simply inherit from this class to define your methods.
The advantages of IDEs¶
Working with these features in a notebook is possible, but let's close with a look at some of the advantages of an IDE such as Pycharm.
