diff --git a/README.md b/README.md index 2dd2b4e..b4205fe 100644 --- a/README.md +++ b/README.md @@ -3,43 +3,89 @@ Intermediate Python =================== -Python is an amazing language with a strong and friendly community of programmers. However, there is a lack of documentation on what to learn after getting the basics of Python down your throat. Through this book I aim to solve this problem. I would give you bits of information about some interesting topics which you can further explore. +Python is an amazing language with a strong and friendly community of programmers. However, there is a lack of documentation on what to learn after getting the basics of Python down your throat. Through this book I aim to solve this problem. I will give you bits of information about some interesting topics which you can further explore. -The topics which are discussed in this book open up your mind towards some nice corners of Python language. This book is an outcome of my desire to have something like it when I was beginning to learn Python. +The topics which are discussed in this book will open your mind to some nice corners of Python language. This book is an outcome of my desire to have something like this when I was beginning to learn Python. If you are a beginner, intermediate or even an advanced programmer there is something for you in this book. -Please note that this book is not a tutorial and does not teach you Python. The topics are not explained in depth, instead only the minimum required information is given. +Please note that this book is not a tutorial and does not teach you Python. The topics are not explained in-depth and only the minimum required information is given. -I am sure you are as excited as I am so let’s start! +I am sure you are as excited as I am. So, let’s start! Note: This book is a work in progress. If you find anything which you can further improve (I know you will find a lot of stuff) then kindly submit a pull request. :) -Moreover, if you want to add more content to this book then kindly submit a pull request and I would be more than happy to merge it. :+1: +Moreover, if you want to add more content to this book then kindly submit a pull request and I will be more than happy to merge it. :+1: + +------------------- + +**Note:** If you want to tip me for my work then you can buy the donation version of this book from [Gumroad](https://gum.co/intermediate_python). Apart from that, if this book somehow helps you then kindly share your experience with [me](mailto:yasoob.khld@gmail.com). I would really appreciate it. + +------------------- Table of Contents: ------------------ -- [\*args and \*\*kwargs](args_and_kwargs.rst) -- [Debugging](debugging.rst) -- [Generators](generators.rst) -- [Map & Filter](map_&_filter.rst) -- [``set`` Data Structure](set_-_data_structure.rst) -- [Ternary Operators](ternary_operators.rst) -- [Decorators](decorators.rst) -- [Global & Return](global_&_return.rst) -- [Mutation](mutation.rst) -- [\_\_slots\_\_ Magic](__slots__magic.rst) -- [Virtual Environment](virtual_environment.rst) -- [Collections](collections.rst) -- [Enumerate](enumerate.rst) -- [Object introspection](object_introspection.rst) -- [Comprehensions](comprehensions.rst) -- [Exceptions](exceptions.rst) -- [Lambdas](lambdas.rst) -- [One Liners](one_liners.rst) -- [For - Else](for_-_else.rst) -- [Open function](open_function.rst) -- [Targeting Python 2+3](targeting_python_2_3.rst) -- [Coroutines](coroutines.rst) -- [Function caching](function_caching.rst) -- [Context managers](context_managers.rst) +1) Programmer tools + - [Virtual Environment](virtual_environment.rst) + - [Debugging](debugging.rst) + - [Object introspection](object_introspection.rst) +2) Syntax + - [Exceptions](exceptions.rst) + - [For - Else](for_-_else.rst) + - [Ternary Operators](ternary_operators.rst) + - [Global & Return](global_&_return.rst) + - [Open function](open_function.rst) + - [\*args and \*\*kwargs](args_and_kwargs.rst) + - [Context managers](context_managers.rst) +3) Functional programming + - [Enumerate](enumerate.rst) + - [Lambdas](lambdas.rst) + - [``set`` Data Structure](set_-_data_structure.rst) + - [Map & Filter](map_filter.rst) + - [Comprehensions](comprehensions.rst) +4) Data structures + - [Generators](generators.rst) + - [Coroutines](coroutines.rst) + - [Classes](classes.rst) +5) Data types + - [Collections](collections.rst) + - [Mutation](mutation.rst) + - [\_\_slots\_\_ Magic](__slots__magic.rst) +6) Decorators + - [What is a decorator?](decorators.rst) + - [Function caching](function_caching.rst) +7) Extras + - [One Liners](one_liners.rst) + - [Targeting Python 2+3](targeting_python_2_3.rst) + - [Python C extensions](python_c_extension.rst) + +Author: +------ + +- [Muhammad Yasoob Ullah Khalid](https://github.com/yasoob) + +Acknowledgement: +---------------- + +- [Philipp Hagemeister](https://github.com/phihag): + +He wrote the chapter on Open function. Thanks Philipp! :+1: + +Translation: +------------ +If you want to translate this book in any other language then kindly let [me know](mailto:yasoob.khld@gmail.com). I would love your contribution. The currently translated versions are listed below: + +- [Chinese](https://github.com/eastlakeside/interpy-zh) +- [Russian](https://github.com/lancelote/interpy-ru) +- [Korean](https://github.com/DDanggle/interpy-kr) +- [Portuguese](https://github.com/joanasouza/intermediatePython) +- [Spanish](https://github.com/cursospython/LibroPython) + +License: +------- + +This book is released under the [following](http://creativecommons.org/licenses/by-nc-sa/4.0/) CC license (CC BY-NC-SA 4.0). + +If you end up using/recommending this book to someone then kindly [let me know](mailto:yasoob.khld@gmail.com). :smile: + + diff --git a/__slots__magic.rst b/__slots__magic.rst index 3f9e5ab..f6c3407 100644 --- a/__slots__magic.rst +++ b/__slots__magic.rst @@ -19,7 +19,7 @@ of attributes. Here is an example with and without ``__slots__``: .. code:: python class MyClass(object): - def __init__(name, identifier): + def __init__(self, name, identifier): self.name = name self.identifier = identifier self.set_up() @@ -31,7 +31,7 @@ of attributes. Here is an example with and without ``__slots__``: class MyClass(object): __slots__ = ['name', 'identifier'] - def __init__(name, identifier): + def __init__(self, name, identifier): self.name = name self.identifier = identifier self.set_up() @@ -43,3 +43,49 @@ technique. On a sidenote, you might want to give PyPy a try. It does all of these optimizations by default. + + +Below you can see an example showing exact memory usage with and without ``__slots__`` done in IPython thanks to https://github.com/ianozsvald/ipython_memory_usage + +.. code:: python + + Python 3.4.3 (default, Jun 6 2015, 13:32:34) + Type "copyright", "credits" or "license" for more information. + + IPython 4.0.0 -- An enhanced Interactive Python. + ? -> Introduction and overview of IPython's features. + %quickref -> Quick reference. + help -> Python's own help system. + object? -> Details about 'object', use 'object??' for extra details. + + In [1]: import ipython_memory_usage.ipython_memory_usage as imu + + In [2]: imu.start_watching_memory() + In [2] used 0.0000 MiB RAM in 5.31s, peaked 0.00 MiB above current, total RAM usage 15.57 MiB + + In [3]: %cat slots.py + class MyClass(object): + __slots__ = ['name', 'identifier'] + def __init__(self, name, identifier): + self.name = name + self.identifier = identifier + + num = 1024*256 + x = [MyClass(1,1) for i in range(num)] + In [3] used 0.2305 MiB RAM in 0.12s, peaked 0.00 MiB above current, total RAM usage 15.80 MiB + + In [4]: from slots import * + In [4] used 9.3008 MiB RAM in 0.72s, peaked 0.00 MiB above current, total RAM usage 25.10 MiB + + In [5]: %cat noslots.py + class MyClass(object): + def __init__(self, name, identifier): + self.name = name + self.identifier = identifier + + num = 1024*256 + x = [MyClass(1,1) for i in range(num)] + In [5] used 0.1758 MiB RAM in 0.12s, peaked 0.00 MiB above current, total RAM usage 25.28 MiB + + In [6]: from noslots import * + In [6] used 22.6680 MiB RAM in 0.80s, peaked 0.00 MiB above current, total RAM usage 47.95 MiB diff --git a/_static/custom.css b/_static/custom.css new file mode 100644 index 0000000..bcdeef2 --- /dev/null +++ b/_static/custom.css @@ -0,0 +1,6 @@ + +@media screen and (min-width: 1000px){ + iframe{ + margin-left: 330px; + } +} \ No newline at end of file diff --git a/_templates/breadcrumbs.html b/_templates/breadcrumbs.html new file mode 100644 index 0000000..f3f3b94 --- /dev/null +++ b/_templates/breadcrumbs.html @@ -0,0 +1,89 @@ +{# Support for Sphinx 1.3+ page_source_suffix, but don't break old builds. #} + +{% if page_source_suffix %} +{% set suffix = page_source_suffix %} +{% else %} +{% set suffix = source_suffix %} +{% endif %} + +{% if meta is defined and meta is not none %} +{% set check_meta = True %} +{% else %} +{% set check_meta = False %} +{% endif %} + +{% if check_meta and 'github_url' in meta %} +{% set display_github = True %} +{% endif %} + +{% if check_meta and 'bitbucket_url' in meta %} +{% set display_bitbucket = True %} +{% endif %} + +{% if check_meta and 'gitlab_url' in meta %} +{% set display_gitlab = True %} +{% endif %} + +
+
+

New book released!

+

Hi! I just released the alpha version of my new book; Practical Python Projects. + Learn more about it + on my blog. In 325+ pages, I will teach you how to implement 12 end-to-end projects. + You can buy it from Feldroy.com. +

+
+ + + {% if (theme_prev_next_buttons_location == 'top' or theme_prev_next_buttons_location == 'both') and (next or prev) %} + + {% endif %} +
+
diff --git a/args_and_kwargs.rst b/args_and_kwargs.rst index 628956b..85ed174 100644 --- a/args_and_kwargs.rst +++ b/args_and_kwargs.rst @@ -3,39 +3,38 @@ I have come to see that most new python programmers have a hard time figuring out the \*args and \*\*kwargs magic variables. So what are they -? First of all let me tell you that it is not necessary to write \*args +? First of all, let me tell you that it is not necessary to write \*args or \*\*kwargs. Only the ``*`` (asterisk) is necessary. You could have also written \*var and \*\*vars. Writing \*args and \*\*kwargs is just a -convention. So now lets take a look at \*args first. +convention. So now let's take a look at \*args first. Usage of \*args ^^^^^^^^^^^^^^^ \*args and \*\*kwargs are mostly used in function definitions. \*args -and \*\*kwargs allow you to pass a variable number of arguments to a -function. What variable means here is that you do not know beforehand -how many arguments can be passed to your function by the user -so in this case you use these two keywords. \*args is used to send a -**non-keyworded** variable length argument list to the function. Here's -an example to help you get a clear idea: +and \*\*kwargs allow you to pass an unspecified number of arguments to a +function, so when writing the function definition, you do not need to +know how many arguments will be passed to your function. \*args is used to +send a **non-keyworded** variable length argument list to the function. +Here's an example to help you get a clear idea: .. code:: python def test_var_args(f_arg, *argv): - print "first normal arg:", f_arg + print("first normal arg:", f_arg) for arg in argv: - print "another arg through *argv :", arg + print("another arg through *argv:", arg) - test_var_args('yasoob','python','eggs','test') + test_var_args('yasoob', 'python', 'eggs', 'test') This produces the following result: .. code:: python first normal arg: yasoob - another arg through *argv : python - another arg through *argv : eggs - another arg through *argv : test + another arg through *argv: python + another arg through *argv: eggs + another arg through *argv: test I hope this cleared away any confusion that you had. So now let's talk about \*\*kwargs @@ -50,12 +49,11 @@ arguments** in a function. Here is an example to get you going with it: .. code:: python def greet_me(**kwargs): - if kwargs is not None: - for key, value in kwargs.iteritems(): - print "%s == %s" %(key, value) + for key, value in kwargs.items(): + print("{0} = {1}".format(key, value)) >>> greet_me(name="yasoob") - name == yasoob + name = yasoob So you can see how we handled a keyworded argument list in our function. This is just the basics of \*\*kwargs and you can see how useful it is. @@ -71,9 +69,9 @@ Just consider that you have this little function: .. code:: python def test_args_kwargs(arg1, arg2, arg3): - print "arg1:", arg1 - print "arg2:", arg2 - print "arg3:", arg3 + print("arg1:", arg1) + print("arg2:", arg2) + print("arg3:", arg3) Now you can use \*args or \*\*kwargs to pass arguments to this little function. Here's how to do it: diff --git a/classes.rst b/classes.rst index 3e09bb2..9485307 100644 --- a/classes.rst +++ b/classes.rst @@ -53,7 +53,7 @@ Let's take a look at an example: b.pi # Output: 50 -There are not many issues while using mutable class variables. This is +There are not many issues while using immutable class variables. This is the major reason due to which beginners do not try to learn more about this subject because everything works! If you also believe that instance and class variables can not cause any problem if used incorrectly then @@ -159,7 +159,7 @@ its ``__init__`` method is called. For example: # called You can see that ``__init__`` is called immediately after an instance is -created. You can also pass arguments to the class during it's +created. You can also pass arguments to the class during its initialization. Like this: .. code:: python @@ -200,23 +200,24 @@ Implementing **getitem** in a class allows its instances to use the [] def __getitem__(self,i): return self.info[i] - foo = OldClass() - foo['title'] + foo = GetTest() + + foo['name'] # Output: 'Yasoob' foo['number'] - # Output: 36845124 + # Output: 12345812 Without the ``__getitem__`` method we would have got this error: .. code:: python - >>> foo['title'] + >>> foo['name'] Traceback (most recent call last): File "", line 1, in TypeError: 'GetTest' object has no attribute '__getitem__' -Static, Class & Abstract methods -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. Static, Class & Abstract methods +.. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/collections.rst b/collections.rst index 582df6c..f668ca6 100644 --- a/collections.rst +++ b/collections.rst @@ -8,11 +8,13 @@ their usefulness. The ones which we will talk about are: - ``defaultdict`` -- ``counter`` +- ``OrderedDict`` +- ``Counter`` - ``deque`` - ``namedtuple`` +- ``enum.Enum`` (outside of the module; Python 3.4+) -1.\ ``defaultdict`` +``defaultdict`` ^^^^^^^^^^^^^^^^^^^ I personally use defaultdict quite a bit. Unlike ``dict``, with @@ -22,7 +24,7 @@ not. So we can do: .. code:: python from collections import defaultdict - + colours = ( ('Yasoob', 'Yellow'), ('Ali', 'Blue'), @@ -31,23 +33,23 @@ not. So we can do: ('Yasoob', 'Red'), ('Ahmed', 'Silver'), ) - + favourite_colours = defaultdict(list) - + for name, colour in colours: favourite_colours[name].append(colour) - + print(favourite_colours) - - # output - # defaultdict(, - # {'Arham': ['Green'], - # 'Yasoob': ['Yellow', 'Red'], - # 'Ahmed': ['Silver'], + + # output + # defaultdict(, + # {'Arham': ['Green'], + # 'Yasoob': ['Yellow', 'Red'], + # 'Ahmed': ['Silver'], # 'Ali': ['Blue', 'Black'] # }) -One another very important use case is when you are appending to nested +One other very important use case is when you are appending to nested lists inside a dictionary. If a ``key`` is not already present in the dictionary then you are greeted with a ``KeyError``. ``defaultdict`` allows us to circumvent this issue in a clever way. First let me share @@ -66,13 +68,13 @@ share a solution using ``defaultdict``. .. code:: python - import collections - tree = lambda: collections.defaultdict(tree) + from collections import defaultdict + tree = lambda: defaultdict(tree) some_dict = tree() some_dict['colours']['favourite'] = "yellow" # Works fine -You can print the ``some_dict`` using ``json.dumps``. Here is some +You can print ``some_dict`` using ``json.dumps``. Here is some sample code: .. code:: python @@ -81,7 +83,43 @@ sample code: print(json.dumps(some_dict)) # Output: {"colours": {"favourite": "yellow"}} -2.\ ``counter`` +``OrderedDict`` +^^^^^^^^^^^^^^^^^^^ + +``OrderedDict`` keeps its entries sorted as they are initially inserted. +Overwriting a value of an existing key doesn't change the position of +that key. However, deleting and reinserting an entry moves the key to +the end of the dictionary. + +**Problem:** + +.. code:: python + + colours = {"Red" : 198, "Green" : 170, "Blue" : 160} + for key, value in colours.items(): + print(key, value) + # Output: + # Green 170 + # Blue 160 + # Red 198 + # Entries are retrieved in an unpredictable order + +**Solution:** + +.. code:: python + + from collections import OrderedDict + + colours = OrderedDict([("Red", 198), ("Green", 170), ("Blue", 160)]) + for key, value in colours.items(): + print(key, value) + # Output: + # Red 198 + # Green 170 + # Blue 160 + # Insertion order is preserved + +``Counter`` ^^^^^^^^^^^^^^^ Counter allows us to count the occurrences of a particular item. For @@ -91,7 +129,7 @@ colours: .. code:: python from collections import Counter - + colours = ( ('Yasoob', 'Yellow'), ('Ali', 'Blue'), @@ -100,13 +138,13 @@ colours: ('Yasoob', 'Red'), ('Ahmed', 'Silver'), ) - + favs = Counter(name for name, colour in colours) print(favs) # Output: Counter({ - # 'Yasoob': 2, - # 'Ali': 2, - # 'Arham': 1, + # 'Yasoob': 2, + # 'Ali': 2, + # 'Arham': 1, # 'Ahmed': 1 # }) @@ -118,7 +156,7 @@ We can also count the most common lines in a file using it. For example: line_count = Counter(f) print(line_count) -3.\ ``deque`` +``deque`` ^^^^^^^^^^^^^ ``deque`` provides you with a double ended queue which means that you @@ -172,15 +210,21 @@ You can pop values from both sides of the deque: # Output: deque([1, 2, 3]) We can also limit the amount of items a deque can hold. By doing this -when we achieve the maximum limit of out deque it will simply pop out +when we achieve the maximum limit of our deque it will simply pop out the items from the opposite end. It is better to explain it using an example so here you go: .. code:: python - d = deque(maxlen=30) + d = deque([0, 1, 2, 3, 5], maxlen=5) + print(d) + # Output: deque([0, 1, 2, 3, 5], maxlen=5) + + d.extend([6]) + print(d) + #Output: deque([1, 2, 3, 5, 6], maxlen=5) -Now whenever you insert values after 30, the leftmost value will be +Now whenever you insert values after 5, the leftmost value will be popped from the list. You can also expand the list in any direction with new values: @@ -192,18 +236,15 @@ new values: print(d) # Output: deque([0, 1, 2, 3, 4, 5, 6, 7, 8]) -This was just a quick drive through the ``collections`` module. Make -sure you read the official documentation after reading this. - -4.\ ``namedtuple`` +``namedtuple`` ^^^^^^^^^^^^^^^^^^ -You might already be acquainted with tuples. A tuple is a lightweight -object type which allows to store a sequence of immutable Python -objects. They are just like lists but have a few key differences. The -major one is that unlike lists, **you can not change a value in a -tuple**. In order to access the value in a tuple you use integer indexes -like: +You might already be acquainted with tuples. A tuple is basically +a immutable list which allows you to store a sequence of values +separated by commas. They are just like lists but have a few key +differences. The major one is that unlike lists, **you can not +reassign an item in a tuple**. In order to access the value in a +tuple you use integer indexes like: .. code:: python @@ -230,15 +271,15 @@ immutable. print(perry.name) # Output: 'perry' -As you can see that now we can access members of a tuple just by their +You can now see that we can access members of a tuple just by their name using a ``.``. Let's dissect it a little more. A named tuple has two required arguments. They are the tuple name and the tuple field\_names. In the above example our tuple name was 'Animal' and the tuple -field\_names were 'name', 'age' and 'cat'. Namedtuple makes your tuples +field\_names were 'name', 'age' and 'type'. Namedtuple makes your tuples **self-document**. You can easily understand what is going on by having a quick glance at your code. And as you are not bound to use integer indexes to access members of a tuple, it makes it more easy to maintain -your code. Moreover, as **``namedtuple`` instances do not have +your code. Moreover, as **`namedtuple` instances do not have per-instance dictionaries**, they are lightweight and require no more memory than regular tuples. This makes them faster than dictionaries. However, do remember that as with tuples, **attributes in namedtuples @@ -253,7 +294,7 @@ are immutable**. It means that this would not work: perry.age = 42 # Output: Traceback (most recent call last): - # File "", line 1, in + # File "", line 1, in # AttributeError: can't set attribute You should use named tuples to make your code self-documenting. **They @@ -277,7 +318,71 @@ Like this: from collections import namedtuple Animal = namedtuple('Animal', 'name age type') - perry = Animal(name="perry", age=31, type="cat") + perry = Animal(name="Perry", age=31, type="cat") print(perry._asdict()) - # Output: OrderedDict([('name', 'perry'), ('age', 31), ... + # Output: OrderedDict([('name', 'Perry'), ('age', 31), ... + +``enum.Enum`` (Python 3.4+) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Another useful collection is the enum object. It is available in the ``enum`` +module, in Python 3.4 and up (also available as a backport in PyPI named ``enum34``.) +Enums (`enumerated type `_) are +basically a way to organize various things. + +Let’s consider the Animal namedtuple from the last example. It had a ``type`` +field. The problem is, the type was a string. This poses some problems for +us. What if the user types in ``Cat`` because they held the Shift key? Or +``CAT``? Or ``kitten``? + +Enumerations can help us avoid this problem, by not using strings. Consider +this example: + +.. code:: python + + from collections import namedtuple + from enum import Enum + + class Species(Enum): + cat = 1 + dog = 2 + horse = 3 + aardvark = 4 + butterfly = 5 + owl = 6 + platypus = 7 + dragon = 8 + unicorn = 9 + # The list goes on and on... + + # But we don't really care about age, so we can use an alias. + kitten = 1 + puppy = 2 + Animal = namedtuple('Animal', 'name age type') + perry = Animal(name="Perry", age=31, type=Species.cat) + drogon = Animal(name="Drogon", age=4, type=Species.dragon) + tom = Animal(name="Tom", age=75, type=Species.cat) + charlie = Animal(name="Charlie", age=2, type=Species.kitten) + + # And now, some tests. + >>> charlie.type == tom.type + True + >>> charlie.type + + + +This is much less error-prone. We have to be specific, and we should use only +the enumeration to name types. + +There are three ways to access enumeration members. For example, all three +methods will get you the value for ``cat``: + +.. code:: python + + Species(1) + Species['cat'] + Species.cat + +This was just a quick drive through the ``collections`` module. Make +sure you read the official documentation after reading this. diff --git a/comprehensions.rst b/comprehensions.rst index bc441d2..1db2a61 100644 --- a/comprehensions.rst +++ b/comprehensions.rst @@ -3,18 +3,16 @@ Comprehensions Comprehensions are a feature of Python which I would really miss if I ever have to leave it. Comprehensions are constructs that allow -sequences to be built from other sequences. There are three type of -comprehensions in Python: +sequences to be built from other sequences. Several types of +comprehensions are supported in both Python 2 and Python 3: - list comprehensions - dictionary comprehensions - set comprehensions - -list comprehensions were introduced in Python 2. set and -dictionary comprehensions became a part of Python in version 3.0. +- generator comprehensions We will discuss them one by one. Once you get the hang of using ``list`` -comprehensions then you can use anyone of them easily. +comprehensions then you can use any of them easily. ``list`` comprehensions ^^^^^^^^^^^^^^^^^^^^^^^ @@ -36,12 +34,12 @@ Here is a short example: .. code:: python - multiples = [i for i in range(30) if i % 3 is 0] + multiples = [i for i in range(30) if i % 3 == 0] print(multiples) # Output: [0, 3, 6, 9, 12, 15, 18, 21, 24, 27] This can be really useful to make lists quickly. It is even preferred by -some instead of the ``filter`` function. list comprehensions really +some instead of the ``filter`` function. List comprehensions really shine when you want to supply a list to a method or function to make a new list by appending to it in each iteration of the ``for`` loop. For instance you would usually do something like this: @@ -91,6 +89,20 @@ that they use braces ``{}``. Here is an example: .. code:: python - squared = {x for x in [1, 1, 2]} + squared = {x**2 for x in [1, 1, 2]} print(squared) - # Output: {1, 2} + # Output: {1, 4} + +``generator`` comprehensions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +They are also similar to list comprehensions. The only difference is that they don't allocate memory for the whole list but generate one item at a time, thus more memory efficient. + +.. code:: python + + multiples_gen = (i for i in range(30) if i % 3 == 0) + print(multiples_gen) + # Output: at 0x7fdaa8e407d8> + for x in multiples_gen: + print(x) + # Outputs numbers diff --git a/conf.py b/conf.py index 7184904..1de0eaa 100644 --- a/conf.py +++ b/conf.py @@ -42,7 +42,7 @@ # General information about the project. project = u'Python Tips' -copyright = u'2015, Muhammad Yasoob Ullah Khalid' +copyright = u'2017, Muhammad Yasoob Ullah Khalid' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -217,7 +217,7 @@ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - ('index', 'pythontips', u'Python Tips Documentation', + ('index', 'pythontips', u'Python Tips', [u'Muhammad Yasoob Ullah Khalid'], 1) ] @@ -231,7 +231,7 @@ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ - ('index', 'PythonTips', u'Python Tips Documentation', + ('index', 'PythonTips', u'Python Tips', u'Muhammad Yasoob Ullah Khalid', 'PythonTips', 'One line description of project.', 'Miscellaneous'), ] @@ -255,7 +255,7 @@ epub_title = u'Python Tips' epub_author = u'Muhammad Yasoob Ullah Khalid' epub_publisher = u'Muhammad Yasoob Ullah Khalid' -epub_copyright = u'2015, Muhammad Yasoob Ullah Khalid' +epub_copyright = u'2017, Muhammad Yasoob Ullah Khalid' #epub_theme = 'epub' @@ -313,3 +313,4 @@ #html_theme_path = ['_themes'] html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] html_theme = 'sphinx_rtd_theme' + diff --git a/context_managers.rst b/context_managers.rst index f144967..261ebd6 100644 --- a/context_managers.rst +++ b/context_managers.rst @@ -1,4 +1,4 @@ -Context managers +Context Managers ---------------- Context managers allow you to allocate and release resources precisely @@ -30,16 +30,16 @@ advantage of using a ``with`` statement is that it makes sure our file is closed without paying attention to how the nested block exits. A common use case of context managers is locking and unlocking resources -and closing opened files (as I have already showed you). +and closing opened files (as I have already shown you). -Let's see how we can implement our own Context Manager. This would allow +Let's see how we can implement our own Context Manager. This should allow us to understand exactly what's going on behind the scenes. -Implementing Context Manager as a Class: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Implementing a Context Manager as a Class: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ At the very least a context manager has an ``__enter__`` and -``__exit__`` methods defined. Let's make our own file opening Context +``__exit__`` method defined. Let's make our own file-opening Context Manager and learn the basics. .. code:: python @@ -52,7 +52,7 @@ Manager and learn the basics. def __exit__(self, type, value, traceback): self.file_obj.close() -Just by defining ``__enter__`` and ``__exit__`` methods we can use it in +Just by defining ``__enter__`` and ``__exit__`` methods we can use our new class in a ``with`` statement. Let's try: .. code:: python @@ -60,20 +60,20 @@ a ``with`` statement. Let's try: with File('demo.txt', 'w') as opened_file: opened_file.write('Hola!') -Our ``__exit__`` function accepts three arguments. They are required by +Our ``__exit__`` method accepts three arguments. They are required by every ``__exit__`` method which is a part of a Context Manager class. Let's talk about what happens under-the-hood. -1. The ``with`` statement stores the ``__exit__`` method of ``File`` +1. The ``with`` statement stores the ``__exit__`` method of the ``File`` class. -2. It calls the ``__enter__`` method of ``File`` class. -3. ``__enter__`` method opens the file and returns it. -4. the opened file handle is passed to ``opened_file``. -5. we write to the file using ``.write()`` -6. ``with`` statement calls the stored ``__exit__`` method. -7. the ``__exit__`` method closes the file. - -Handling exceptions +2. It calls the ``__enter__`` method of the ``File`` class. +3. The ``__enter__`` method opens the file and returns it. +4. The opened file handle is passed to ``opened_file``. +5. We write to the file using ``.write()``. +6. The ``with`` statement calls the stored ``__exit__`` method. +7. The ``__exit__`` method closes the file. + +Handling Exceptions ^^^^^^^^^^^^^^^^^^^ We did not talk about the ``type``, ``value`` and ``traceback`` @@ -92,20 +92,20 @@ instance: with File('demo.txt', 'w') as opened_file: opened_file.undefined_function('Hola!') -Let's list down the steps which are taken by the ``with`` statement when -an error is encountered. +Let's list the steps which are taken by the ``with`` statement when +an error is encountered: 1. It passes the type, value and traceback of the error to the ``__exit__`` method. 2. It allows the ``__exit__`` method to handle the exception. -3. If ``__exit__`` returns True then the exception was gracefully +3. If ``__exit__`` returns ``True`` then the exception was gracefully handled. -4. If anything else than True is returned by ``__exit__`` method then - the exception is raised by ``with`` statement. +4. If anything other than ``True`` is returned by the ``__exit__`` method then + the exception is raised by the ``with`` statement. In our case the ``__exit__`` method returns ``None`` (when no return statement is encountered then the method returns ``None``). Therefore, -``with`` statement raises the exception. +the ``with`` statement raises the exception: .. code:: python @@ -123,7 +123,7 @@ Let's try handling the exception in the ``__exit__`` method: def __enter__(self): return self.file_obj def __exit__(self, type, value, traceback): - print "Exception has been handled" + print("Exception has been handled") self.file_obj.close() return True @@ -132,11 +132,11 @@ Let's try handling the exception in the ``__exit__`` method: # Output: Exception has been handled -Our ``__exit__`` method returned True, therefore no exception was raised +Our ``__exit__`` method returned ``True``, therefore no exception was raised by the ``with`` statement. -This is not the only way to implement context managers. There is another -way and we will be looking at it in this next section. +This is not the only way to implement Context Managers. There is another +way and we will be looking at it in the next section. Implementing a Context Manager as a Generator ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -153,8 +153,10 @@ Let's see a basic, useless example: @contextmanager def open_file(name): f = open(name, 'w') - yield f - f.close() + try: + yield f + finally: + f.close() Okay! This way of implementing Context Managers appear to be more intuitive and easy. However, this method requires some knowledge about @@ -167,11 +169,11 @@ Let's dissect this method a little. 1. Python encounters the ``yield`` keyword. Due to this it creates a generator instead of a normal function. 2. Due to the decoration, contextmanager is called with the function - name (open\_file) as it's argument. -3. The ``contextmanager`` function returns the generator wrapped by the + name (``open_file``) as its argument. +3. The ``contextmanager`` decorator returns the generator wrapped by the ``GeneratorContextManager`` object. 4. The ``GeneratorContextManager`` is assigned to the ``open_file`` - function. Therefore, when we later call ``open_file`` function, we + function. Therefore, when we later call the ``open_file`` function, we are actually calling the ``GeneratorContextManager`` object. So now that we know all this, we can use the newly generated Context @@ -181,4 +183,3 @@ Manager like this: with open_file('some_file') as f: f.write('hola!') - diff --git a/coroutines.rst b/coroutines.rst index 4763c0f..113cb30 100644 --- a/coroutines.rst +++ b/coroutines.rst @@ -26,42 +26,42 @@ We then commonly use it in a ``for`` loop like this: print(i) It is fast and does not put a lot of pressure on memory because it -**generates** the values on the fly rather then storing them in a list. -Now if we use ``yield`` in the above example more generally we get a +**generates** the values on the fly rather than storing them in a list. +Now, if we use ``yield`` in the above example, more generally, we get a coroutine. Coroutines consume values which are sent to it. A very basic example would be a ``grep`` alternative in Python: .. code:: python def grep(pattern): - print "Searching for %s" % pattern + print("Searching for", pattern) while True: line = (yield) if pattern in line: print(line) Wait! What does ``yield`` return? Well we have turned it into a -coroutine. It does not contain any value initially instead we supply it +coroutine. It does not contain any value initially, instead we supply it values externally. We supply values by using the ``.send()`` method. Here is an example: .. code:: python search = grep('coroutine') - search.next() + next(search) # Output: Searching for coroutine search.send("I love you") search.send("Don't you love me?") search.send("I love coroutines instead!") # Output: I love coroutines instead! -The sent values are accessed by yield. Why did we run ``.next()``? It is -done to start the coroutine. Just like ``generators`` coroutines do not -start the function immediately. Instead they run it in response to -``.next()`` and ``.send()`` methods. Therefore you have to run -``.next()`` so that the execution advances to the ``yield`` expression. +The sent values are accessed by ``yield``. Why did we run ``next()``? It is +required in order to start the coroutine. Just like ``generators``, coroutines do not +start the function immediately. Instead they run it in response to the +``__next__()`` and ``.send()`` methods. Therefore, you have to run +``next()`` so that the execution advances to the ``yield`` expression. -We can close a coroutine by calling the ``.close()`` method. Like: +We can close a coroutine by calling the ``.close()`` method: .. code:: python diff --git a/debugging.rst b/debugging.rst index 58af25a..5e591f8 100644 --- a/debugging.rst +++ b/debugging.rst @@ -2,14 +2,14 @@ Debugging --------- Debugging is also something which once mastered can greatly enhance your -bug hunting skills. Most of the newcomers neglect the importance of the +bug hunting skills. Most newcomers neglect the importance of the Python debugger (``pdb``). In this section I am going to tell you only a few important commands. You can learn more about it from the official documentation. -**Running from commandline** +**Running from the command line** -You can run a script from the commandline using the Python debugger. +You can run a script from the command line using the Python debugger. Here is an example: .. code:: python @@ -58,3 +58,7 @@ function. These are just a few commands. ``pdb`` also supports post mortem. It is also a really handy function. I would highly suggest you to look at the official documentation and learn more about it. + +**Note:** + +It might seem unintuitive to use `pdb.set_trace()` if you are new to this. Fortunately, if you are using Python 3.7+ then you can simply use the `breakpoint()` [built-in function](https://docs.python.org/3/library/functions.html#breakpoint). It automatically imports `pdb` and calls `pdb.set_trace()`. diff --git a/decorators.rst b/decorators.rst index 724acd3..30509fe 100644 --- a/decorators.rst +++ b/decorators.rst @@ -1,21 +1,21 @@ Decorators ---------- -Decorators are a significant part of Python. In simple words they are -functions which modify the functionality of another function. They help -to make our code shorter and more Pythonic. Most of the beginners do not +Decorators are a significant part of Python. In simple words: they are +functions which modify the functionality of other functions. They help +to make our code shorter and more Pythonic. Most beginners do not know where to use them so I am going to share some areas where -decorators can make your code concise. +decorators can make your code more concise. First, let's discuss how to write your own decorator. It is perhaps one of the most difficult concepts to grasp. We will take it one step at a time so that you can fully understand it. -Everything in python is an object: +Everything in Python is an object: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -First of all let's understand functions in python: +First of all let's understand functions in Python: .. code:: python @@ -30,7 +30,7 @@ First of all let's understand functions in python: # We are not using parentheses here because we are not calling the function hi # instead we are just putting it into the greet variable. Let's try to run this - print greet() + print(greet()) # output: 'hi yasoob' # Let's see what happens if we delete the old hi function! @@ -44,14 +44,14 @@ First of all let's understand functions in python: Defining functions within functions: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -So those are the basics when it comes to functions. Lets take your +So those are the basics when it comes to functions. Let's take your knowledge one step further. In Python we can define functions inside other functions: .. code:: python def hi(name="yasoob"): - print "now you are inside the hi() function" + print("now you are inside the hi() function") def greet(): return "now you are in the greet() function" @@ -77,8 +77,8 @@ other functions: #outputs: NameError: name 'greet' is not defined So now we know that we can define functions in other functions. In -simpler words we can make nested functions. Now you need to learn one -more thing that functions can return functions too. +other words: we can make nested functions. Now you need to learn one +more thing, that functions can return functions too. Returning functions from within functions: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -112,13 +112,13 @@ can return it as an output as well: Just take a look at the code again. In the ``if/else`` clause we are returning ``greet`` and ``welcome``, not ``greet()`` and ``welcome()``. -Why is that? It is so because when you put parentheses around it the -function gets executed whereas if you don't put parenthesis around it +Why is that? It's because when you put a pair of parentheses after it, the +function gets executed; whereas if you don't put parenthesis after it, then it can be passed around and can be assigned to other variables -without executing it. Did you get it ? Let me explain it a little bit in +without executing it. Did you get it? Let me explain it in a little bit more detail. When we write ``a = hi()``, ``hi()`` gets executed and -because the name is yasoob by default, the function greet is returned. -If we change the statement to ``a = hi(name = "ali")`` then the welcome +because the name is yasoob by default, the function ``greet`` is returned. +If we change the statement to ``a = hi(name = "ali")`` then the ``welcome`` function will be returned. We can also do print ``hi()()`` which outputs *now you are in the greet() function*. @@ -131,11 +131,11 @@ Giving a function as an argument to another function: return "hi yasoob!" def doSomethingBeforeHi(func): - print("I am doing some boring work before executing hi()") + print("I am doing some boring work before executing hi()") print(func()) doSomethingBeforeHi(hi) - #outputs:I am doing some boring work before executing hi() + #outputs:I am doing some boring work before executing hi() # hi yasoob! Now you have all the required knowledge to learn what decorators really @@ -144,7 +144,7 @@ are. Decorators let you execute code before and after a function. Writing your first decorator: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In the last example we actually made a decorator! Lets modify the +In the last example we actually made a decorator! Let's modify the previous decorator and make a little bit more usable program: .. code:: python @@ -152,7 +152,7 @@ previous decorator and make a little bit more usable program: def a_new_decorator(a_func): def wrapTheFunction(): - print "I am doing some boring work before executing a_func()" + print("I am doing some boring work before executing a_func()") a_func() @@ -170,14 +170,14 @@ previous decorator and make a little bit more usable program: #now a_function_requiring_decoration is wrapped by wrapTheFunction() a_function_requiring_decoration() - #outputs:I am doing some boring work before executing a_function_requiring_decoration() + #outputs:I am doing some boring work before executing a_func() # I am the function which needs some decoration to remove my foul smell - # I am doing some boring work after executing a_function_requiring_decoration() + # I am doing some boring work after executing a_func() Did you get it? We just applied the previously learned principles. This -is exactly what the decorators do in python! They wrap a function and -modify its behaviour in one way or the another. Now you might be -wondering that we did not use the @ anywhere in our code? That is just a +is exactly what the decorators do in Python! They wrap a function and +modify its behaviour in one way or another. Now you might be +wondering why we did not use the @ anywhere in our code? That is just a short way of making up a decorated function. Here is how we could have run the previous code sample using @. @@ -185,14 +185,14 @@ run the previous code sample using @. @a_new_decorator def a_function_requiring_decoration(): - """Hey yo! Decorate me!""" + """Hey you! Decorate me!""" print("I am the function which needs some decoration to " - " remove my foul smell") + "remove my foul smell") a_function_requiring_decoration() - #outputs: I am doing some boring work before executing a_function_requiring_decoration() + #outputs: I am doing some boring work before executing a_func() # I am the function which needs some decoration to remove my foul smell - # I am doing some boring work after executing a_function_requiring_decoration() + # I am doing some boring work after executing a_func() #the @a_new_decorator is just a short way of saying: a_function_requiring_decoration = a_new_decorator(a_function_requiring_decoration) @@ -206,9 +206,9 @@ Python. Now there is one problem with our code. If we run: # Output: wrapTheFunction That's not what we expected! Its name is -"a\_function\_requiring\_decoration". Well our function was replaced by +"a\_function\_requiring\_decoration". Well, our function was replaced by wrapTheFunction. It overrode the name and docstring of our function. -Luckily Python provides us a simple function to solve this problem and +Luckily, Python provides us a simple function to solve this problem and that is ``functools.wraps``. Let's modify our previous example to use ``functools.wraps``: @@ -219,7 +219,7 @@ that is ``functools.wraps``. Let's modify our previous example to use def a_new_decorator(a_func): @wraps(a_func) def wrapTheFunction(): - print("I am doing some boring work before executing a_func()") + print("I am doing some boring work before executing a_func()") a_func() print("I am doing some boring work after executing a_func()") return wrapTheFunction @@ -236,7 +236,7 @@ that is ``functools.wraps``. Let's modify our previous example to use Now that is much better. Let's move on and learn some use-cases of decorators. -**Blueprint :** +**Blueprint:** .. code:: python @@ -251,19 +251,19 @@ decorators. @decorator_name def func(): - print("Function is running") + return("Function is running") can_run = True print(func()) # Output: Function is running - can_run=False + can_run = False print(func()) # Output: Function will not run Note: ``@wraps`` takes a function to be decorated and adds the functionality of copying over the function name, docstring, arguments -list, etc. This allows to access the pre-decorated function's properties +list, etc. This allows us to access the pre-decorated function's properties in the decorator. Use-cases: @@ -272,8 +272,8 @@ Use-cases: Now let's take a look at the areas where decorators really shine and their usage makes something really easy to manage. -1. Authorization -^^^^^^^^^^^^^^^^ +Authorization +~~~~~~~~~~~~~ Decorators can help to check whether someone is authorized to use an endpoint in a web application. They are extensively used in Flask web @@ -295,8 +295,8 @@ authentication: return f(*args, **kwargs) return decorated -2. Logging -^^^^^^^^^^ +Logging +~~~~~~~ Logging is another area where the decorators shine. Here is an example: @@ -313,7 +313,7 @@ Logging is another area where the decorators shine. Here is an example: @logit def addition_func(x): - """does some math""" + """Do some math.""" return x + x @@ -321,3 +321,133 @@ Logging is another area where the decorators shine. Here is an example: # Output: addition_func was called I am sure you are already thinking about some clever uses of decorators. + +Decorators with Arguments +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Come to think of it, isn't ``@wraps`` also a decorator? But, it takes an +argument like any normal function can do. So, why can't we do that too? + +This is because when you use the ``@my_decorator`` syntax, you are +applying a wrapper function with a single function as a parameter. +Remember, everything in Python is an object, and this includes +functions! With that in mind, we can write a function that returns +a wrapper function. + +Nesting a Decorator Within a Function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Let's go back to our logging example, and create a wrapper which lets +us specify a logfile to output to. + +.. code:: python + + from functools import wraps + + def logit(logfile='out.log'): + def logging_decorator(func): + @wraps(func) + def wrapped_function(*args, **kwargs): + log_string = func.__name__ + " was called" + print(log_string) + # Open the logfile and append + with open(logfile, 'a') as opened_file: + # Now we log to the specified logfile + opened_file.write(log_string + '\n') + return func(*args, **kwargs) + return wrapped_function + return logging_decorator + + @logit() + def myfunc1(): + pass + + myfunc1() + # Output: myfunc1 was called + # A file called out.log now exists, with the above string + + @logit(logfile='func2.log') + def myfunc2(): + pass + + myfunc2() + # Output: myfunc2 was called + # A file called func2.log now exists, with the above string + +Decorator Classes +~~~~~~~~~~~~~~~~~ + +Now we have our logit decorator in production, but when some parts +of our application are considered critical, failure might be +something that needs more immediate attention. Let's say sometimes +you want to just log to a file. Other times you want an email sent, +so the problem is brought to your attention, and still keep a log +for your own records. This is a case for using inheritence, but +so far we've only seen functions being used to build decorators. + +Luckily, classes can also be used to build decorators. So, let's +rebuild logit as a class instead of a function. + +.. code:: python + + class logit(object): + +        _logfile = 'out.log' +     + def __init__(self, func): + self.func = func + +        def __call__(self, *args): +            log_string = self.func.__name__ + " was called" + print(log_string) + # Open the logfile and append +            with open(self._logfile, 'a') as opened_file: + # Now we log to the specified logfile + opened_file.write(log_string + '\n') + # Now, send a notification + self.notify() + +            # return base func +            return self.func(*args) + +             + + def notify(self): + # logit only logs, no more + pass + +This implementation has an additional advantage of being much cleaner than +the nested function approach, and wrapping a function still will use +the same syntax as before: + +.. code:: python + +    logit._logfile = 'out2.log' # if change log file +    @logit +    def myfunc1(): + pass + + myfunc1() + # Output: myfunc1 was called + +Now, let's subclass logit to add email functionality (though this topic +will not be covered here). + +.. code:: python + + class email_logit(logit): + ''' + A logit implementation for sending emails to admins + when the function is called. + ''' + def __init__(self, email='admin@myproject.com', *args, **kwargs): + self.email = email + super(email_logit, self).__init__(*args, **kwargs) + + def notify(self): + # Send an email to self.email + # Will not be implemented here + pass + +From here, ``@email_logit`` works just like ``@logit`` but sends an email +to the admin in addition to logging. diff --git a/enumerate.rst b/enumerate.rst index a087c94..3bcd2f6 100644 --- a/enumerate.rst +++ b/enumerate.rst @@ -1,18 +1,25 @@ Enumerate --------- -Enumerate is a built-in function of Python. It's usefulness can not be +Enumerate is a built-in function of Python. Its usefulness can not be summarized in a single line. Yet most of the newcomers and even some advanced programmers are unaware of it. It allows us to loop over something and have an automatic counter. Here is an example: .. code:: python + + my_list = ['apple', 'banana', 'grapes', 'pear'] + for counter, value in enumerate(my_list): + print counter, value - for counter, value in enumerate(some_list): - print(counter, value) + # Output: + # 0 apple + # 1 banana + # 2 grapes + # 3 pear -This is not it. ``enumerate`` also accepts some optional arguments which -make it even more useful. +And there is more! ``enumerate`` also accepts an optional argument that +allows us to specify the starting index of the counter. .. code:: python @@ -26,9 +33,9 @@ make it even more useful. # 3 grapes # 4 pear -The optional argument allows us to tell ``enumerate`` from where to -start the index. You can also create tuples containing the index and -list item using a list. Here is an example: +An example of where the optional argument of ``enumerate`` comes in handy +is creating tuples containing the index and list item using a list. Here +is an example: .. code:: python diff --git a/exceptions.rst b/exceptions.rst index 9ec5aa6..ebeb0d5 100644 --- a/exceptions.rst +++ b/exceptions.rst @@ -5,9 +5,11 @@ Exception handling is an art which once you master grants you immense powers. I am going to show you some of the ways in which we can handle exceptions. -In basic terminology we are aware of ``try/except`` clause. The code -which can cause an exception to occur is put in the ``try`` block and +In basic terminology we are aware of the ``try/except`` structure. The code +that can cause an exception to occur is put in the ``try`` block and the handling of the exception is implemented in the ``except`` block. +The code in the ``except`` block will only execute if the ``try`` block +runs into an exception. Here is a simple example: .. code:: python @@ -34,8 +36,8 @@ tuple. Like so: except (IOError, EOFError) as e: print("An error occurred. {}".format(e.args[-1])) -Another method is to handle individual exception in a separate except -block. We can have as many except blocks as we want. Here is an example: +Another method is to handle individual exceptions in separate ``except`` +blocks. We can have as many ``except`` blocks as we want. Here is an example: .. code:: python @@ -48,30 +50,38 @@ block. We can have as many except blocks as we want. Here is an example: print("An error occurred.") raise e -This way if the exception is not handled by the first except block then -it is passed on to the second block. Now the last method involves +This way if the exception is not handled by the first ``except`` block then +it may be handled by a following block, or none at all. Now the last method involves trapping ALL exceptions: .. code:: python try: file = open('test.txt', 'rb') - except Exception: + except Exception as e: # Some logging if you want - raise + raise e + +This can be helpful when you have no idea about the exceptions that may +be thrown by your program. If you are just looking to catch all execptions, +but don't actually care about what they are, you can even exclude the +``Exception as e`` part. -This can be helpful when you have no idea about the exception which can -be thrown by your program. +Note:: catching all exceptions may have unintended consequences because catching +all exceptions may also catch the ones you want to occur; for example, in +many command-line based programs, pressing control+c will terminate the program, +but if you catch all excepts, the ``KeyboardInterrupt`` will be caught as an +exception, so pressing control+c will NOT terminate the program. -``Finally`` clause +``finally`` clause ~~~~~~~~~~~~~~~~~~ -We wrap our main code in the try clause. After that we wrap some code in -except clause which gets executed if an exception occurs in the code -wrapped in try clause. But in this example we will use a third clause as +We wrap our main code in the ``try`` clause. After that we wrap some code in +an ``except`` clause which gets executed if an exception occurs in the code +wrapped in the ``try`` clause. In this example we will use a third clause as well which is the ``finally`` clause. The code which is wrapped in the -finally clause will run even if no exception occurs. It might be used -for cleaning up after a script. Here is a simple example: +``finally`` clause will run whether or not an exception occurred. It might be used +to perform clean-up after a script. Here is a simple example: .. code:: python @@ -80,17 +90,20 @@ for cleaning up after a script. Here is a simple example: except IOError as e: print('An IOError occurred. {}'.format(e.args[-1])) finally: - print("This would be printed even if no exception occurs!") + print("This would be printed whether or not an exception occurred!") # Output: An IOError occurred. No such file or directory - # This would be printed even if no exception occurs! + # This would be printed whether or not an exception occurred! ``try/else`` clause ~~~~~~~~~~~~~~~~~~~ Often times we might want some code to run if **no** exception occurs. This -can easily be achieved by using an ``else`` clause. Most people don't -use it and honestly I have myself not used it widely. Here is an +can easily be achieved by using an ``else`` clause. One might ask: why, if +you only want some code to run if no exception occurs, wouldn't you simply +put that code inside the ``try``? The answer is that then any exceptions in +that code will be caught by the ``try``, and you might not want that. Most +people don't use it and honestly I have myself not used it widely. Here is an example: .. code:: python @@ -100,13 +113,16 @@ example: except Exception: print('exception') else: - print('This would only run if no exception occurs.') + # any code that should only run if no exception occurs in the try, + # but for which exceptions should NOT be caught + print('This would only run if no exception occurs. And an error here ' + 'would NOT be caught.') finally: print('This would be printed in every case.') # Output: I am sure no exception is going to occur! - # This would only run if no exception occurs. + # This would only run if no exception occurs. And an error here would NOT be caught # This would be printed in every case. -The else clause would only run if no exception occurs and it would run +The ``else`` clause would only run if no exception occurs and it would run before the ``finally`` clause. diff --git a/for_-_else.rst b/for_-_else.rst index c6980c9..f7fbe82 100644 --- a/for_-_else.rst +++ b/for_-_else.rst @@ -1,42 +1,41 @@ -For - Else ----------- +``for/else`` +------------ Loops are an integral part of any language. Likewise ``for`` loops are an important part of Python. However there are a few things which most -beginners do not know about them. We will discuss a few of them one by -one. +beginners do not know about them. We will discuss a few of them one-by-one. -Let's first start of by what we know. We know that we can use for loops +Let's first start off with what we know. We know that we can use ``for`` loops like this: .. code:: python fruits = ['apple', 'banana', 'mango'] for fruit in fruits: - print fruit.capitalize() + print(fruit.capitalize()) # Output: Apple # Banana # Mango -That is the very basic structure of a for loop. Now let's move on to +That is the very basic structure of a ``for`` loop. Now let's move on to some of the lesser known features of ``for`` loops in Python. -1.\ ``else`` clause: -^^^^^^^^^^^^^^^^^^^^ +``else`` Clause +^^^^^^^^^^^^^^^ -For loops also have an ``else`` clause which most of us are unfamiliar -with. The ``else`` clause executes when the loop completes normally. -This means that the loop did not encounter any ``break``. They are -really useful once you understand where to use them. I myself came to +``for`` loops also have an ``else`` clause which most of us are unfamiliar +with. The ``else`` clause executes after the loop completes normally. +This means that the loop did not encounter a ``break`` statement. They are +really useful once you understand where to use them. I, myself, came to know about them a lot later. The common construct is to run a loop and search for an item. If the -item is found, we break the loop using ``break``. There are two +item is found, we break out of the loop using the ``break`` statement. There are two scenarios in which the loop may end. The first one is when the item is found and ``break`` is encountered. The second scenario is that the loop -ends. Now we may want to know which one of these is the reason for a -loops completion. One method is to set a flag and then check it once the +ends without encountering a ``break`` statement. Now we may want to know which one of these is the reason for a +loop's completion. One method is to set a flag and then check it once the loop ends. Another is to use the ``else`` clause. This is the basic structure of a ``for/else`` loop: @@ -60,21 +59,19 @@ documentation: for n in range(2, 10): for x in range(2, n): if n % x == 0: - print n, 'equals', x, '*', n/x + print(n, 'equals', x, '*', n/x) break -It outputs the prime numbers between 2 to 10. Now for the fun part. We -can add an additional ``else`` block which catches the numbers which are -not prime and tells us so: +It finds factors for numbers between 2 to 10. Now for the fun part. We +can add an additional ``else`` block which catches the numbers which have no factors and are therefore prime numbers: .. code:: python for n in range(2, 10): for x in range(2, n): if n % x == 0: - print n, 'equals', x, '*', n/x + print( n, 'equals', x, '*', n/x) break else: # loop fell through without finding a factor - print n, 'is a prime number' - + print(n, 'is a prime number') diff --git a/function_caching.rst b/function_caching.rst index 865a866..7100aad 100644 --- a/function_caching.rst +++ b/function_caching.rst @@ -52,9 +52,9 @@ is a generic cache: memo = {} @wraps(function) def wrapper(*args): - if args in memo: + try: return memo[args] - else: + except KeyError: rv = function(*args) memo[args] = rv return rv @@ -67,6 +67,8 @@ is a generic cache: fibonacci(25) +**Note:** memoize won't cache unhashable types (dict, lists, etc...) but only the immutable types. Keep that in mind when using it. + `Here `__ is a fine article by Caktus Group in which they caught a bug in Django which occurred due to ``lru_cache``. It's an interesting read. Do check it out. diff --git a/generators.rst b/generators.rst index 45220ee..23cd1b0 100644 --- a/generators.rst +++ b/generators.rst @@ -20,8 +20,8 @@ Iterable An ``iterable`` is any object in Python which has an ``__iter__`` or a ``__getitem__`` method defined which returns an **iterator** or can take -indexes (Both of these dunder methods are fully explained in a previous -chapter). In short an ``iterable`` is any object which can provide us +indexes (You can read more about them `here `_). +In short an ``iterable`` is any object which can provide us with an **iterator**. So what is an **iterator**? Iterator @@ -73,10 +73,11 @@ simple example of a ``generator`` function: It is not really useful in this case. Generators are best for calculating large sets of results (particularly calculations involving loops themselves) where you don't want to allocate the memory for all -results at the same time. Python modified a lot of Python 2 functions -which returned ``lists`` to return ``generators`` in Python 3. It is -because ``generators`` are not resource intensive. Here is an example -which calculates fibonacci numbers: +results at the same time. Many Standard Library functions that return +``lists`` in Python 2 have been modified to return ``generators`` in +Python 3 because ``generators`` require fewer resources. + +Here is an example ``generator`` which calculates fibonacci numbers: .. code:: python @@ -133,10 +134,10 @@ element of a sequence. So let's test out our understanding: As we can see that after yielding all the values ``next()`` caused a ``StopIteration`` error. Basically this error informs us that all the -values have been yielded. You might be wondering that why don't we get -this error while using a ``for`` loop? Well the answer is simple. The +values have been yielded. You might be wondering why we don't get +this error when using a ``for`` loop? Well the answer is simple. The ``for`` loop automatically catches this error and stops calling -``next``. Do you know that a few built-in data types in Python also +``next``. Did you know that a few built-in data types in Python also support iteration? Let's check it out: .. code:: python @@ -148,17 +149,24 @@ support iteration? Let's check it out: # TypeError: str object is not an iterator Well that's not what we expected. The error says that ``str`` is not an -iterator. Well it is right! It is an iterable but not an iterator. This -means that it supports iteration but we can not directly iterate over -it. How can we then iterate over it? It's time to learn about one more +iterator. Well it's right! It's an iterable but not an iterator. This +means that it supports iteration but we can't iterate over +it directly. So how would we iterate over it? It's time to learn about one more built-in function, ``iter``. It returns an ``iterator`` object from an -iterable. Here is how we can use it: +iterable. While an ``int`` isn't an iterable, we can use it on string! .. code:: python + int_var = 1779 + iter(int_var) + # Output: Traceback (most recent call last): + # File "", line 1, in + # TypeError: 'int' object is not iterable + # This is because int is not iterable + my_string = "Yasoob" my_iter = iter(my_string) - next(my_iter) + print(next(my_iter)) # Output: 'Y' Now that is much better. I am sure that you loved learning about diff --git a/global_&_return.rst b/global_&_return.rst index c8ef0ca..0661752 100644 --- a/global_&_return.rst +++ b/global_&_return.rst @@ -1,9 +1,9 @@ -Global & Return +Global and Return --------------- -You might have encountered some functions written in python which have a +You might have encountered some functions written in Python which have a ``return`` keyword in the end of the function. Do you know what it does? It -is similar to return in other languages. Lets examine this little +is similar to return in other languages. Let's examine this little function: .. code:: python @@ -15,7 +15,7 @@ function: print(result) # Output: 8 -The function above takes two values as input and then output their +The function above takes two values as input and then outputs their addition. We could have also done: .. code:: python @@ -28,15 +28,15 @@ addition. We could have also done: print(result) # Output: 8 -So first lets talk about the first bit of code which involves the +So first let's talk about the first bit of code which involves the ``return`` keyword. What that function is doing is that it is assigning the value to the variable which is calling that function which in our -case is ``result``. In most cases and you won't need to use the -``global`` keyword. However lets examine the other bit of code as well +case is ``result``. In most cases you won't need to use the +``global`` keyword. However, let's examine the other bit of code as well which includes the ``global`` keyword. So what that function is doing is that it is making a global variable ``result``. What does global mean here? Global variable means that we can access that variable outside the -scope of the function as well. Let me demonstrate it with an example : +scope of the function as well. Let me demonstrate it with an example: .. code:: python @@ -47,17 +47,17 @@ scope of the function as well. Let me demonstrate it with an example : add(2, 4) print(result) - # Oh crap we encountered an exception. Why is it so ? - # the python interpreter is telling us that we do not - # have any variable with the name of result. It is so - # because the result variable is only accessible inside + # Oh crap, we encountered an exception. Why is it so? + # the python interpreter is telling us that we do not + # have any variable with the name of result. It is so + # because the result variable is only accessible inside # the function in which it is created if it is not global. Traceback (most recent call last): - File "", line 1, in + File "", line 1, in result NameError: name 'result' is not defined - # Now lets run the same code but after making the result + # Now lets run the same code but after making the result # variable global def add(value1, value2): global result @@ -71,4 +71,90 @@ So hopefully there are no errors in the second run as expected. In practical programming you should try to stay away from ``global`` keyword as it only makes life difficult by introducing unwanted variables to the global scope. - + +Multiple return values +^^^^^^^^^^^^^^^^^^^^^^^ + +So what if you want to return two variables from a function instead of one? There are a couple of approaches which new programmers take. The most famous approach is to use ``global`` keyword. Let's take a look at a useless example: + +.. code:: python + + def profile(): + global name + global age + name = "Danny" + age = 30 + + profile() + print(name) + # Output: Danny + + print(age) + # Output: 30 + +**Note:**Don't try to use the above mentioned method. I repeat, don't try to use the above mentioned method! + +Some try to solve this problem by *returning* a ``tuple``, ``list`` or ``dict`` with the required values after the function terminates. It is one way to do it and works like a charm: + +.. code:: python + + def profile(): + name = "Danny" + age = 30 + return (name, age) + + profile_data = profile() + print(profile_data[0]) + # Output: Danny + + print(profile_data[1]) + # Output: 30 + +Or by more common convention: + +.. code:: python + + def profile(): + name = "Danny" + age = 30 + return name, age + + profile_name, profile_age = profile() + print(profile_name) + # Output: Danny + print(profile_age) + # Output: 30 + +Keep in mind that even in the above example we are returning a tuple (despite the lack of parenthesis) and not separate multiple values. If you want to take it one step further, you can also make use of `namedtuple `_. Here is an example: + +.. code:: python + + from collections import namedtuple + def profile(): + Person = namedtuple('Person', 'name age') + return Person(name="Danny", age=31) + + # Use as namedtuple + p = profile() + print(p, type(p)) + # Person(name='Danny', age=31) + print(p.name) + # Danny + print(p.age) + #31 + + # Use as plain tuple + p = profile() + print(p[0]) + # Danny + print(p[1]) + #31 + + # Unpack it immediatly + name, age = profile() + print(name) + # Danny + print(age) + #31 + +This is a better way to do it, along with returning ``list`` and ``dict``. Don't use ``global`` keyword unless you know what you are doing. ``global`` might be a better option in a few cases but is not in most of them. diff --git a/index.rst b/index.rst index 3ae92a1..b45de64 100644 --- a/index.rst +++ b/index.rst @@ -1,6 +1,8 @@ :orphan: -**Note:** If you want to tip me then kindly signup at my `mailing list `__ and I will send you an email once I have a tip receiving system in place! :) +.. note:: + + You can sign up to my `mailing list `__ so that you remain in sync with any major updates to this book or my future projects! Intermediate Python =================== @@ -11,7 +13,7 @@ Preface Python is an amazing language with a strong and friendly community of programmers. However, there is a lack of documentation on what to learn after getting the basics of Python down your throat. Through this book I aim to solve this problem. I would give you bits of information about some interesting topics which you can further explore. -The topics which are discussed in this book open up your mind towards some nice corners of Python language. This book is an outcome of my desire to have something like it when I was beginning to learn Python. +The topics which are discussed in this book open up your mind towards some nice corners of Python language. This book is an outcome of my desire to have something like this when I was beginning to learn Python. If you are a beginner, intermediate or even an advanced programmer there is something for you in this book. @@ -19,21 +21,26 @@ Please note that this book is not a tutorial and does not teach you Python. The I am sure you are as excited as I am so let’s start! -Note: This book is a work in progress. If you find anything which you can further improve (I know you will find a lot of stuff) then kindly submit a pull request. :) +**Note:** This book is a continuous work in progress. If you find anything which you can further improve (I know you will find a lot of stuff) then kindly submit a pull request! -============ +================== Author -============ +================== + +I am Muhammad Yasoob Ullah Khalid. I have been programming extensively in Python for over 3 years now. I have been involved in a lot of Open Source projects. I regularly blog about interesting Python topics over at my `blog `_ . In 2014 I also spoke at EuroPython which was held in Berlin. It is the biggest Python conference in Europe. If you have an interesting Internship opportunity for me then I would definitely like to hear from you! -I am Muhammad Yasoob Ullah Khalid. I have been programming extensively in Python for over 3 years now. I have been involved in a lot of Open Source projects. I regularly blog about interesting Python topics over at my `blog `_ . In 2014 I also spoke at EuroPython which was held in Berlin. It is the biggest Python conference in Europe. +================== +Table of Contents +================== .. toctree:: :maxdepth: 2 + :numbered: args_and_kwargs debugging generators - map_&_filter + map_filter set_-_data_structure ternary_operators decorators @@ -43,12 +50,15 @@ I am Muhammad Yasoob Ullah Khalid. I have been programming extensively in Python virtual_environment collections enumerate + zip object_introspection comprehensions exceptions + classes lambdas one_liners for_-_else + python_c_extension open_function targeting_python_2_3 coroutines diff --git a/lambdas.rst b/lambdas.rst index 1223dca..756e260 100644 --- a/lambdas.rst +++ b/lambdas.rst @@ -21,7 +21,7 @@ normal functions and even behave like them. print(add(3, 5)) # Output: 8 -Here are a few useful use cases for lambdas and just a few way in which +Here are a few useful use cases for lambdas and just a few ways in which they are used in the wild: **List sorting** @@ -39,5 +39,5 @@ they are used in the wild: .. code:: python data = zip(list1, list2) - data.sort() + data = sorted(data) list1, list2 = map(lambda t: list(t), zip(*data)) diff --git a/map_&_filter.rst b/map_filter.rst similarity index 59% rename from map_&_filter.rst rename to map_filter.rst index d7404bb..9552cf8 100644 --- a/map_&_filter.rst +++ b/map_filter.rst @@ -1,11 +1,11 @@ -Map & Filter +Map, Filter and Reduce ------------ -These are two functions which facilitate a functional approach to +These are three functions which facilitate a functional approach to programming. We will discuss them one by one and understand their use cases. -1. Map +Map ^^^^^^ ``Map`` applies a function to all the items in an input\_list. Here is @@ -33,7 +33,7 @@ Here you go: .. code:: python items = [1, 2, 3, 4, 5] - squared = map(lambda x: x**2, items) + squared = list(map(lambda x: x**2, items)) Most of the times we use lambdas with ``map`` so I did the same. Instead of a list of inputs we can even have a list of functions! @@ -41,15 +41,15 @@ of a list of inputs we can even have a list of functions! .. code:: python def multiply(x): - return (x*x) + return (x*x) def add(x): - return (x+x) + return (x+x) funcs = [multiply, add] for i in range(5): - value = map(lambda x: x(i), funcs) + value = list(map(lambda x: x(i), funcs)) print(value) - + # Output: # [0, 0] # [1, 2] @@ -57,10 +57,10 @@ of a list of inputs we can even have a list of functions! # [9, 6] # [16, 8] -2. Filter +Filter ^^^^^^^^^ -As the name suggests, filter creates a list of elements for which a +As the name suggests, ``filter`` creates a list of elements for which a function returns true. Here is a short and concise example: .. code:: python @@ -75,3 +75,33 @@ The filter resembles a for loop but it is a builtin function and faster. **Note:** If map & filter do not appear beautiful to you then you can read about ``list/dict/tuple`` comprehensions. + +Reduce +^^^^^^^^^ + +``Reduce`` is a really useful function for performing some computation on +a list and returning the result. It applies a rolling computation to sequential +pairs of values in a list. For example, if you wanted to compute the product +of a list of integers. + +So the normal way you might go about doing this task in python is using +a basic for loop: + +.. code:: python + + product = 1 + list = [1, 2, 3, 4] + for num in list: + product = product * num + + # product = 24 + + +Now let's try it with reduce: + +.. code:: python + + from functools import reduce + product = reduce((lambda x, y: x * y), [1, 2, 3, 4]) + + # Output: 24 diff --git a/mutation.rst b/mutation.rst index 440222a..493579c 100644 --- a/mutation.rst +++ b/mutation.rst @@ -30,7 +30,8 @@ something like this: bar += ['bye'] print(foo) - # Output: ['hi'] + # Expected Output: ['hi'] + # Output: ['hi', 'bye'] print(bar) # Output: ['hi', 'bye'] diff --git a/object_introspection.rst b/object_introspection.rst index 790097a..fa3a351 100644 --- a/object_introspection.rst +++ b/object_introspection.rst @@ -6,7 +6,7 @@ type of an object at runtime. It is one of Python's strengths. Everything in Python is an object and we can examine those objects. Python ships with a few built-in functions and modules to help us. -1.\ ``dir`` +``dir`` ^^^^^^^^^^^ In this section we will learn about ``dir`` and how it facilitates us @@ -20,13 +20,13 @@ example: my_list = [1, 2, 3] dir(my_list) - # Output: ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', - # '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', - # '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', - # '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', + # Output: ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', + # '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', + # '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', + # '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', # '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', - # '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', - # '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', + # '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', + # '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', # 'remove', 'reverse', 'sort'] Our introspection gave us the names of all the methods of a list. This @@ -34,7 +34,7 @@ can be handy when you are not able to recall a method name. If we run ``dir()`` without any argument then it returns all names in the current scope. -2.\ ``type`` and ``id`` +``type`` and ``id`` ^^^^^^^^^^^^^^^^^^^^^^^ The ``type`` function returns the type of an object. For example: @@ -64,7 +64,7 @@ The ``type`` function returns the type of an object. For example: print(id(name)) # Output: 139972439030304 -3.\ ``inspect`` module +``inspect`` module ^^^^^^^^^^^^^^^^^^^^^^ The inspect module also provides several useful functions to get diff --git a/one_liners.rst b/one_liners.rst index 0f2abf1..17dc776 100644 --- a/one_liners.rst +++ b/one_liners.rst @@ -1,15 +1,15 @@ -One Liners +One-Liners ---------- -In this chapter I will show you some one liner Python commands which can -be really helpful sometimes. +In this chapter I will show you some one-liner Python commands which can +be really helpful. -**Simple Webserver** +**Simple Web Server** Ever wanted to quickly share a file over a network? Well you are in -luck. Python has a similar feature just for you. Go to the directory -which you want to serve over network and write the following code in -terminal: +luck. Python has a feature just for you. Go to the directory +which you want to serve over the network and write the following code in +your terminal: .. code:: python @@ -19,9 +19,9 @@ terminal: # Python 3 python -m http.server -**Pretty printing** +**Pretty Printing** -You can print a list and dictionary in a beautiful format in Python +You can print a list and dictionary in a beautiful format in the Python repl. Here is the relevant code: .. code:: python @@ -29,19 +29,69 @@ repl. Here is the relevant code: from pprint import pprint my_dict = {'name': 'Yasoob', 'age': 'undefined', 'personality': 'awesome'} - pprint(my_dict) - -This is more effective on dicts. Moreover, if you want to pretty print + print(dir(my_dict)) + # ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] + + pprint(dir(my_dict)) + # ['__add__', + # '__class__', + # '__contains__', + # '__delattr__', + # '__delitem__', + # '__dir__', + # '__doc__', + # '__eq__', + # '__format__', + # '__ge__', + # '__getattribute__', + # '__getitem__', + # '__gt__', + # '__hash__', + # '__iadd__', + # '__imul__', + # '__init__', + # '__init_subclass__', + # '__iter__', + # '__le__', + # '__len__', + # '__lt__', + # '__mul__', + # '__ne__', + # '__new__', + # '__reduce__', + # '__reduce_ex__', + # '__repr__', + # '__reversed__', + # '__rmul__', + # '__setattr__', + # '__setitem__', + # '__sizeof__', + # '__str__', + # '__subclasshook__', + # 'append', + # 'clear', + # 'copy', + # 'count', + # 'extend', + # 'index', + # 'insert', + # 'pop', + # 'remove', + # 'reverse', + # 'sort'] + + +This is more effective on nested ``dict`` s. Moreover, if you want to pretty print json quickly from a file then you can simply do: .. code:: python - cat file.json | python -m json.tools + cat file.json | python -m json.tool **Profiling a script** -This can be extremely helpful in pin pointing the bottlenecks in your -scripts. +This can be extremely helpful in pinpointing the bottlenecks in your +scripts: .. code:: python @@ -71,9 +121,13 @@ is a simple example: a_list = [[1, 2], [3, 4], [5, 6]] print(list(itertools.chain.from_iterable(a_list))) # Output: [1, 2, 3, 4, 5, 6] + + # or + print(list(itertools.chain(*a_list))) + # Output: [1, 2, 3, 4, 5, 6] -**One Line Constructors** +**One-Line Constructors** Avoid a lot of boilerplate assignments when initializing a class @@ -84,5 +138,5 @@ Avoid a lot of boilerplate assignments when initializing a class self.__dict__.update({k: v for k, v in locals().items() if k != 'self'}) -A couple of more one liners can be found on the `Python +Additional one-liners can be found on the `Python website `__. diff --git a/open_function.rst b/open_function.rst index bd5b702..3683bb3 100644 --- a/open_function.rst +++ b/open_function.rst @@ -1,4 +1,4 @@ -Open function +``open`` Function ------------- `open `__ opens @@ -17,10 +17,10 @@ you spot them all? If not, read on. By the end of this article, you'll know what's wrong in the above code, and, more importantly, be able to avoid these mistakes in your own code. Let's start with the basics: -The return of open is a file handle, given out from the operating system +The return value from ``open`` is a file handle, given out from the operating system to your Python application. You will want to return this file handle once you're finished with the file, if only so that your application -won't reach the limit of the number of open file handle it can have at +won't reach the limit of the number of open file handles it can have at once. Explicitly calling ``close`` closes the file handle, but only if the @@ -28,7 +28,7 @@ read was successful. If there is any error just after ``f = open(...)``, ``f.close()`` will not be called (depending on the Python interpreter, the file handle may still be returned, but that's another story). To make sure that the file gets closed whether an exception occurs or not, -pack it into a ```with`` statement: +pack it into a ``with`` statement: .. code:: python @@ -53,11 +53,11 @@ or text mode (a string of characters). In general, if the format is written by humans, it tends to be text mode. ``jpg`` image files are not generally written by humans (and are -indeed not readable to humans), and you should therefore open them in -binary mode by adding a ``b`` to the text string (if you're following +indeed not readable by humans), and you should therefore open them in +binary mode by adding a ``b`` to the mode string (if you're following the opening example, the correct mode would be ``rb``). If you open something in text mode (i.e. add a ``t``, or nothing apart from -``r/r+/w/a``), you must also know which encoding to use - for a +``r/r+/w/a``), you must also know which encoding to use. For a computer, all files are just bytes, not characters. Unfortunately, ``open`` does not allow explicit encoding specification @@ -65,16 +65,16 @@ in Python 2.x. However, the function `io.open `__ is available in both Python 2.x and 3.x (where it is an alias of ``open``), and does the right thing. You can pass in the encoding with the -``encoding`` keyword. If you don't pass in any encoding, a system- (and -Python-) specific default will be picked. You may be tempted to rely on +``encoding`` keyword. If you don't pass in any encoding, a system -- and +Python -- specific default will be picked. You may be tempted to rely on these defaults, but the defaults are often wrong, or the default -encoding cannot actually express all characters (this will happen on -Python 2.x and/or Windows). So go ahead and pick an encoding. ``utf-8`` -is a terrific one. When you write a file, you can just pick the encoding +encoding cannot actually express all characters in the file (this will happen often on +Python 2.x and/or Windows). So go ahead and pick an encoding. Encoding is the way to instruct computers about how the numbers should be stored as bytes in memory. ``utf-8`` +is a terrific one and is supported by major browsers and programming languages. When you write a file, you can just pick the encoding to your liking (or the liking of the program that will eventually read your file). -How do you find out which encoding a file you read has? Well, +How do you find out which encoding a file you're reading was written in? Well, unfortunately, there is no foolproof way to detect the encoding - the same bytes can represent different, but equally valid characters in different encodings. Therefore, you must rely on metadata (for example, @@ -93,11 +93,11 @@ determines whether it's JPG (hint: These files start with the bytes jpgdata = inf.read() if jpgdata.startswith(b'\xff\xd8'): - text = u'This is a jpeg file (%d bytes long)\n' + text = u'This is a JPEG file (%d bytes long)\n' else: text = u'This is a random file (%d bytes long)\n' with io.open('summary.txt', 'w', encoding='utf-8') as outf: outf.write(text % len(jpgdata)) -I am sure that now you would use ``open`` correctly! +I am sure that now you will use ``open`` correctly! diff --git a/python_c_extension.rst b/python_c_extension.rst new file mode 100644 index 0000000..f4cf5d4 --- /dev/null +++ b/python_c_extension.rst @@ -0,0 +1,377 @@ +Python C extensions +=================== + +An interesting feature offered to developers by the CPython +implementation is the ease of interfacing C code to Python. + +There are three key methods developers use to call C functions from +their python code - ``ctypes``, ``SWIG`` and ``Python/C API``. Each +method comes with its own merits and demerits. + +Firstly, why would you want to interface C with Python? + +A few common reasons are : + +- You want speed and you know C is about 50x faster than Python. +- Certain legacy C libraries work just as well as you want them to, so you don't want to rewrite them in python. +- Certain low level resource access - from memory to file interfaces. +- Just because you want to. + +CTypes +--------- + +The Python `ctypes +module `__ is probably +the easiest way to call C functions from Python. The ctypes module +provides C compatible data types and functions to load DLLs so that +calls can be made to C shared libraries without having to modify them. +The fact that the C side needn't be touched adds to the simplicity of +this method. + +**Example** + +Simple C code to add two numbers, save it as ``add.c`` + +.. code:: c + + //sample C file to add 2 numbers - int and floats + + int add_int(int, int); + float add_float(float, float); + + int add_int(int num1, int num2){ + return num1 + num2; + } + + float add_float(float num1, float num2){ + return num1 + num2; + } + +Next compile the C file to a ``.so`` file (DLL in windows) This will +generate an adder.so file. + +.. code:: bash + + #For Linux + $ gcc -shared -Wl,-soname,adder -o adder.so -fPIC add.c + + #For Mac + $ gcc -shared -Wl,-install_name,adder.so -o adder.so -fPIC add.c + +Now in your python code - + +.. code:: python + + from ctypes import * + + #load the shared object file + adder = CDLL('./adder.so') + + #Find sum of integers + res_int = adder.add_int(4,5) + print "Sum of 4 and 5 = " + str(res_int) + + #Find sum of floats + a = c_float(5.5) + b = c_float(4.1) + + add_float = adder.add_float + add_float.restype = c_float + print "Sum of 5.5 and 4.1 = ", str(add_float(a, b)) + +And the output is as follows + +:: + + Sum of 4 and 5 = 9 + Sum of 5.5 and 4.1 = 9.60000038147 + +In this example the C file is self explanatory - it contains two +functions, one to add two integers and another to add two floats. + +In the python file, first the ctypes module is imported. Then the CDLL +function of the ctypes module is used to load the shared lib file we +created. The functions defined in the C lib are now available to us via +the ``adder`` variable. When ``adder.add_int()`` is called, internally a +call is made to the ``add_int`` C function. The ctypes interface allows +us to use native python integers and strings by default while calling +the C functions. + +For other types such as boolean or float, we have to use the correct +ctypes. This is seen while passing parameters to the +``adder.add_float()``. We first create the required c\_float types from +python decimal values, and then use them as arguments to the C code. +This method is simple and clean, but limited. For example it's not +possible to manipulate objects on the C side. + +SWIG +------- + +Simplified Wrapper and Interface Generator, or SWIG for short is another +way to interface C code to Python. In this method, the developer must +develop an extra interface file which is an input to SWIG (the command +line utility). + +Python developers generally don't use this method, because it is in most +cases unnecessarily complex. This is a great method when you have a +C/C++ code base, and you want to interface it to many different +languages. + +**Example** (from the `SWIG website `__ ) + +The C code, ``example.c`` that has a variety of functions and variables + +.. code:: c + + #include + double My_variable = 3.0; + + int fact(int n) { + if (n <= 1) return 1; + else return n*fact(n-1); + } + + int my_mod(int x, int y) { + return (x%y); + } + + char *get_time() + { + time_t ltime; + time(<ime); + return ctime(<ime); + } + +The interface file - this will remain the same irrespective of the +language you want to port your C code to : + +:: + + /* example.i */ + %module example + %{ + /* Put header files here or function declarations like below */ + extern double My_variable; + extern int fact(int n); + extern int my_mod(int x, int y); + extern char *get_time(); + %} + + extern double My_variable; + extern int fact(int n); + extern int my_mod(int x, int y); + extern char *get_time(); + +And now to compile it + +:: + + unix % swig -python example.i + unix % gcc -c example.c example_wrap.c \ + -I/usr/local/include/python2.1 + unix % ld -shared example.o example_wrap.o -o _example.so + +Finally, the Python output + +.. code:: python + + >>> import example + >>> example.fact(5) + 120 + >>> example.my_mod(7,3) + 1 + >>> example.get_time() + 'Sun Feb 11 23:01:07 1996' + >>> + +As we can see, SWIG achieves the same result, but requires a slightly +more involved effort. But it's worth it if you are targeting multiple +languages. + +Python/C API +--------------- + +The `C/Python API `__ is probably the +most widely used method - not for its simplicity but for the fact that +you can manipulate python objects in your C code. + +This method requires your C code to be specifically written for +interfacing with Python code. All Python objects are represented as a +PyObject struct and the ``Python.h`` header file provides various +functions to manipulate it. For example if the PyObject is also a +PyListType (basically a list), then we can use the ``PyList_Size()`` +function on the struct to get the length of the list. This is equivalent +to calling ``len(list)`` in python. Most of the basic +functions/opertions that are there for native Python objects are made +available in C via the ``Python.h`` header. + +**Example** + +To write a C extension that adds all the elements in a python list. (all elements are numbers) + +Let's start with the final interface we'd like to have, here is the +python file that uses the C extension : + +.. code:: python + + #Though it looks like an ordinary python import, the addList module is implemented in C + import addList + + l = [1,2,3,4,5] + print "Sum of List - " + str(l) + " = " + str(addList.add(l)) + +The above looks like any ordinary python file, which imports and uses +another python module called ``addList``. The only difference is that +the addList module is not written in Python at all, but rather in C. + +Next we'll have a look at the C code that get's built into the +``addList`` Python module. This may seem a bit daunting at first, but +once you understand the various components that go into writing the C +file, it's pretty straightforward. + +*adder.c* + +.. code:: c + + //Python.h has all the required function definitions to manipulate the Python objects + #include + + //This is the function that is called from your python code + static PyObject* addList_add(PyObject* self, PyObject* args){ + + PyObject * listObj; + + //The input arguments come as a tuple, we parse the args to get the various variables + //In this case it's only one list variable, which will now be referenced by listObj + if (! PyArg_ParseTuple( args, "O", &listObj)) + return NULL; + + //length of the list + long length = PyList_Size(listObj); + + //iterate over all the elements + long i, sum =0; + for(i = 0; i < length; i++){ + //get an element out of the list - the element is also a python objects + PyObject* temp = PyList_GetItem(listObj, i); + //we know that object represents an integer - so convert it into C long + long elem = PyInt_AsLong(temp); + sum += elem; + } + + //value returned back to python code - another python object + //build value here converts the C long to a python integer + return Py_BuildValue("i", sum); + } + + //This is the docstring that corresponds to our 'add' function. + static char addList_docs[] = + "add( ): add all elements of the list\n"; + + /* This table contains the relavent info mapping - + , , + , + */ + static PyMethodDef addList_funcs[] = { + {"add", (PyCFunction)addList_add, METH_VARARGS, addList_docs}, + {NULL, NULL, 0, NULL} + }; + + /* + addList is the module name, and this is the initialization block of the module. + , , + */ + PyMODINIT_FUNC initaddList(void){ + Py_InitModule3("addList", addList_funcs, + "Add all ze lists"); + } + +A step by step explanation : + +- The ```` file consists of all the required types (to represent Python object types) and function definitions (to operate on the python objects). +- Next we write the function which we plan to call from python. Conventionally the function names are {module-name}\_{function-name}, which in this case is ``addList_add``. More about the function later. +- Then fill in the info table - which contains all the relevant info of the functions we desire to have in the module. Every row corresponds to a function, with the last one being a sentinel value (row of null elements). +- Finally the module initialization block which is of the signature ``PyMODINIT_FUNC init{module-name}``. + +The function ``addList_add`` accepts arguments as a PyObject type struct +(args is also a tuple type - but since everything in python is an +object, we use the generic PyObject notion). The incoming arguments is +parsed (basically split the tuple into individual elements) by +``PyArg_ParseTuple()``. The first parameter is the argument variable to +be parsed. The second argument is a string that tells us how to parse +each element in the args tuple. The character in the Nth position of the +string tells us the type of the Nth element in the args tuple, example - +'i' would mean integer, 's' would mean string and 'O' would mean a +Python object. Next multiple arguments follow, these are where you would +like the ``PyArg_ParseTuple()`` function to store all the elements that +it has parsed. The number of such arguments is equal to the number of +arguments which the module function expects to receive, and positional +integrity is maintained. For example if we expected a string, integer +and a python list in that order, the function signature would be + +.. code:: c + + int n; + char *s; + PyObject* list; + PyArg_ParseTuple(args, "siO", &s, &n, &list); + +In this case we only have to extract a list object, and store it in the +variable ``listObj``. We then use the ``PyList_Size()`` function on our +list object and get the length. This is similar to how you would call +``len(list)`` in python. + +Now we loop through the list, get each element using the +``PyList_GetItem(list, index)`` function. This returns a PyObject\*. But +since we know that the Python objects are also ``PyIntType``, we just +use the ``PyInt_AsLong(PyObj *)`` function to get the required value. We +do this for every element and finally get the sum. + +The sum is converted to a python object and is returned to the Python +code with the help of ``Py_BuildValue()``. Here the "i" indicates that +the value we want to build is a python integer object. + +Now we build the C module. Save the following code as ``setup.py`` + +.. code:: python + + #build the modules + + from distutils.core import setup, Extension + + setup(name='addList', version='1.0', \ + ext_modules=[Extension('addList', ['adder.c'])]) + +and run + +.. code:: sh + + python setup.py install + +This should now build and install the C file into the python module we +desire. + +After all this hard work, we'll now test if the module works - + +.. code:: python + + #module that talks to the C code + import addList + + l = [1,2,3,4,5] + print "Sum of List - " + str(l) + " = " + str(addList.add(l)) + +And here is the output + +:: + + Sum of List - [1, 2, 3, 4, 5] = 15 + +So as you can see, we have developed our first successful C Python +extension using the Python.h API. This method does seem complex at +first, but once you get used to it it can prove to be quite useful. + +Other ways to interface C code to Python is to use an alternative and +faster build of python - `Cython `__. But Cython is +a slightly different language than the main stream python we see. Hence +that method is not covered here. diff --git a/set_-_data_structure.rst b/set_-_data_structure.rst index 22182ee..7fa9878 100644 --- a/set_-_data_structure.rst +++ b/set_-_data_structure.rst @@ -40,8 +40,8 @@ You can intersect two sets. For instance: .. code:: python valid = set(['yellow', 'red', 'blue', 'green', 'black']) - input = set(['red', 'brown']) - print(input.intersection(valid)) + input_set = set(['red', 'brown']) + print(input_set.intersection(valid)) # Output: set(['red']) **Difference** @@ -52,8 +52,8 @@ difference method. For example: .. code:: python valid = set(['yellow', 'red', 'blue', 'green', 'black']) - input = set(['red', 'brown']) - print(input.difference(valid)) + input_set = set(['red', 'brown']) + print(input_set.difference(valid)) # Output: set(['brown']) You can also create sets using the new notation: diff --git a/targeting_python_2_3.rst b/targeting_python_2_3.rst index e6c779d..caa0c1e 100644 --- a/targeting_python_2_3.rst +++ b/targeting_python_2_3.rst @@ -4,11 +4,11 @@ Targeting Python 2+3 In a lot of cases you might want to develop programs which can be run in both Python 2+ and 3+. -Just imagine that you have a very popular Python module which is use by -hundreds of people but not all of them have Python 2 or 3. In that case +Just imagine that you have a very popular Python module which is used by +hundreds of people but not all of them have the same version of Python (2 or 3). In that case you have two choices. The first one is to distribute 2 modules, one for Python 2 and the other for Python 3. The other choice is to modify your -current code and make is compatible with both Python 2 and 3. +current code and make it compatible with both Python 2 and 3. In this section I am going to highlight some of the tricks which you can employ to make a script compatible with both of them. @@ -16,20 +16,17 @@ employ to make a script compatible with both of them. **Future imports** The first and most important method is to use ``__future__`` imports. It -allows you to import Python 3 functionality in Python 2. Here is an -example: +allows you to import Python 3 functionality in Python 2. Here are a couple +examples: -- Context managers were new in Python 2.6+. For using them in Python 2.5 - you can use: +Context managers were new in Python 2.6+. For using them in Python 2.5 you can use: .. code:: python from __future__ import with_statement -- ``print`` function - ``print`` was changed to a function in Python 3. If you want to use it -in Python 2 you can import it from ``__future__``. +in Python 2 you can import it from ``__future__``: .. code:: python @@ -42,7 +39,7 @@ in Python 2 you can import it from ``__future__``. **Dealing with module renaming** -First tell me how you import packages in your script ? Most of us do +First, tell me how you import packages in your script ? Most of us do this : .. code:: python @@ -69,31 +66,31 @@ code below : import urllib2 as urllib_request # for Python 2 So let me explain the above code a little. We are wrapping our importing -code in a try except clause. We are doing it because in Python 2 there is -no ``urllib.request`` module and will result in an ImportError. The -functionality of ``urllib.request`` is provided by ``urllib2`` module in -Python 2. So now when in Python 2 we try to import ``urllib.request`` and -get an ``ImportError`` we tell Python to import ``urllib2`` instead. +code in a ``try/except`` clause. We are doing it because in Python 2 there is +no ``urllib.request`` module so this would result in an ``ImportError``. The +functionality of ``urllib.request`` is provided by the ``urllib2`` module in +Python 2. So, when using Python 2, we try to import ``urllib.request`` and +if we get an ``ImportError`` then we tell Python to import ``urllib2`` instead. The final thing you need to know about is the ``as`` keyword. It is -mapping the imported module to ``urllib_request``. So that now all of -the Classes and methods of ``urllib2`` are available to us by +mapping the imported module to ``urllib_request``. So that all of +the classes and methods within ``urllib2`` are available to us via the alias ``urllib_request``. **Obsolete Python 2 builtins** Another thing to keep in mind is that there are 12 Python 2 builtins which have been removed from Python 3. Make sure that you don't use them -in Python 2 as well in order to make your code compatible with Python 3. -Here is a way to enforce you to abandon these 12 builtins in Python 2 as -well. +in Python 2 in order to make your code compatible with Python 3. +Here is a way to enforce that you abandon these 12 builtins in Python 2 as +well: .. code:: python from future.builtins.disabled import * Now whenever you try to use the modules which are abandoned in Python 3, -it raises a NameError like this: +it raises a ``NameError`` like this: .. code:: python @@ -105,7 +102,7 @@ it raises a NameError like this: **External standard-library backports** There are a few packages in the wild which provide Python 3 -functionality in Python 2. For instance we have: +functionality in Python 2. For instance, we have: - enum ``pip install enum34`` - singledispatch ``pip install singledispatch`` diff --git a/ternary_operators.rst b/ternary_operators.rst index 47e6336..6f9cbbc 100644 --- a/ternary_operators.rst +++ b/ternary_operators.rst @@ -12,14 +12,14 @@ expressions. .. code:: python - condition_is_true if condition else condition_is_false + value_if_true if condition else value_if_false **Example:** .. code:: python - is_fat = True - state = "fat" if is_fat else "not fat" + is_nice = True + state = "nice" if is_nice else "not nice" It allows to quickly test a condition instead of a multiline if statement. Often times it can be immensely helpful and can make your @@ -38,11 +38,76 @@ is some sample code: .. code:: python - fat = True - fitness = ("skinny","fat")[fat] - print("Ali is " + fitness) - # Output: Ali is fat + nice = True + personality = ("mean", "nice")[nice] + print("The cat is ", personality) + # Output: The cat is nice + +This works simply because True == 1 and False == 0, and so can be done +with lists in addition to tuples. The above example is not widely used and is generally disliked by Pythonistas for not being Pythonic. It is also easy to confuse where to put the true value and where to put the false value in the tuple. + +Another reason to avoid using a tupled ternery is that it results in +both elements of the tuple being evaluated, whereas the if-else +ternary operator does not. + +**Example:** + +.. code:: python + + condition = True + print(2 if condition else 1/0) + #Output is 2 + + print((1/0, 2)[condition]) + #ZeroDivisionError is raised + +This happens because with the tupled ternary technique, the tuple is +first built, then an index is found. For the if-else ternary operator, +it follows the normal if-else logic tree. Thus, if one case could +raise an exception based on the condition, or if either case is a +computation-heavy method, using tuples is best avoided. + + +**ShortHand Ternary** + +In python there is also the shorthand ternary tag which is a shorter version of the +normal ternary operator you have seen above. + +Syntax was introduced in Python 2.5 and can be used in python 2.5 or greater. + +**Example** + +.. code:: python + + >>> True or "Some" + True + >>> + >>> False or "Some" + 'Some' + +The first statement (`True or "Some"`) will return `True` and the second statement (`False or "Some"`) will return `Some`. + +This is helpful in case where you quickly want to check for the output of a function and give a useful message if the output is empty: + +.. code:: python + + >>> output = None + >>> msg = output or "No data returned" + >>> print(msg) + No data returned + +Or as a simple way to define function parameters with dynamic default values: + +.. code:: python + + >>> def my_function(real_name, optional_display_name=None): + >>> optional_display_name = optional_display_name or real_name + >>> print(optional_display_name) + >>> my_function("John") + John + >>> my_function("Mike", "anonymous123") + anonymous123 diff --git a/virtual_environment.rst b/virtual_environment.rst index 8da1426..28b119f 100644 --- a/virtual_environment.rst +++ b/virtual_environment.rst @@ -1,48 +1,50 @@ Virtual Environment ------------------- -Have you ever heard of ``virtualenv``? The chances are that if you are a -beginner then you might not have heard about it but if you are a -seasoned programmer than it's a vital part of your toolset. So what -``virtualenv`` really is? ``Virtualenv`` is a tool which allows us to +Have you ever heard of ``virtualenv``? If you are a beginner, +then you might not have heard about it but if you are a +seasoned programmer then it may well be a vital part of your toolset. + +So what is ``virtualenv``? ``Virtualenv`` is a tool which allows us to make isolated python environments. Imagine you have an application that -needs version 2 of a LibraryBar, but another application requires +needs version 2 of a library, but another application requires version 3. How can you use and develop both these applications? If you install everything into ``/usr/lib/python2.7/site-packages`` (or whatever your platform's standard location is), it's easy to end up in a -situation where you unintentionally upgrade a package that shouldn't be -upgraded. In another case just imagine that you have an application -which is fully developed and you do not want to make any change to the -libraries it is using but at the same time you start developing another -application which requires the updated versions of those libraries. What -will you do? It is where ``virtualenv`` comes into play. It creates -isolated environments for you python application and allows you to -install Python libraries in that isolated environment instead of -installing them globally. - -In order to install it just type this command in the shell: +situation where you unintentionally upgrade a package. + +In another case, imagine that you have an application which is fully +developed and you do not want to make any change to the libraries it is +using but at the same time you start developing another application +which requires the updated versions of those libraries. + +What will you do? Use ``virtualenv``! It creates isolated environments +for your python application and allows you to install Python libraries +in that isolated environment instead of installing them globally. + +To install it, just type this command in the shell: .. code:: python $ pip install virtualenv -Now i am going to list some of it's commands. The most important ones -are: +The most important commands are: - ``$ virtualenv myproject`` -- ``$ source bin/activate`` +- ``$ source myproject/bin/activate`` This first one makes an isolated virtualenv environment in the ``myproject`` folder and the second command activates that isolated -environment. While running the first command you have to make a -decision. +environment. + +While creating the virtualenv you have to make a decision. Do you +want this virtualenv to use packages from your system ``site-packages`` +or install them in the virtualenv’s site-packages? By default, +virtualenv will not give access to the global ``site-packages``. -Do you want this virtualenv to use packages from your system -``site-packages`` or install them in the virtualenv’s site-packages? By -default, virtualenv will not give access to the global ``site-packages``. If you want your ``virtualenv`` to have access to your systems -``site-packages`` use the ``--system-site-packages`` switch when creating +``site-packages``, use the ``--system-site-packages`` switch when creating your virtualenv like this: .. code:: python @@ -55,6 +57,9 @@ You can turn off the ``env`` by typing: $ deactivate +Running `python` after deactivating will use your system installation +of Python again. + **Bonus** You can use ``smartcd`` which is a library for bash and zsh and allows @@ -63,7 +68,6 @@ helpful to activate and deactivate a ``virtualenv`` when you change directories. I have used it quite a lot and love it. You can read more about it on `GitHub `__ -This was just a short intro to virtualenv. There's a lot more to it. For -further study i recommend `this -link. `__ -It will remove all of your confusions about virtualenv. +This was just a short intro to virtualenv. There's a lot more to it; `this +link `__ has more +information. diff --git a/zip.rst b/zip.rst new file mode 100644 index 0000000..947fe2a --- /dev/null +++ b/zip.rst @@ -0,0 +1,71 @@ +Zip and unzip +------------- + +**Zip** + +Zip is a useful function that allows you to combine two lists easily. + +After calling zip, an iterator is returned. In order to see the content wrapped inside, we need to first convert it to a list. + +Example: + +.. code:: python + + first_name = ['Joe','Earnst','Thomas','Martin','Charles'] + + last_name = ['Schmoe','Ehlmann','Fischer','Walter','Rogan','Green'] + + age = [23, 65, 11, 36, 83] + + print(list(zip(first_name,last_name, age))) + + # Output + # + # [('Joe', 'Schmoe', 23), ('Earnst', 'Ehlmann', 65), ('Thomas', 'Fischer', 11), ('Martin', 'Walter', 36), ('Charles', 'Rogan', 83)] + +One advantage of zip is that it improves readability of for loops. + +For example, instead of needing multiple inputs, you only need one zipped list for the following for loop: + +.. code:: python + + first_name = ['Joe','Earnst','Thomas','Martin','Charles'] + last_name = ['Schmoe','Ehlmann','Fischer','Walter','Rogan','Green'] + age = [23, 65, 11, 36, 83] + + for first_name, last_name, age in zip(first_name, last_name, age): + print(f"{first_name} {last_name} is {age} years old") + + # Output + # + # Joe Schmoe is 23 years old + # Earnst Ehlmann is 65 years old + # Thomas Fischer is 11 years old + # Martin Walter is 36 years old + # Charles Rogan is 83 years old + +**Unzip** + +We can use the `zip` function to unzip a list as well. This time, we need an input of a list with an asterisk before it. + +The outputs are the separated lists. + +Example: + +.. code:: python + + full_name_list = [('Joe', 'Schmoe', 23), + ('Earnst', 'Ehlmann', 65), + ('Thomas', 'Fischer', 11), + ('Martin', 'Walter', 36), + ('Charles', 'Rogan', 83)] + + first_name, last_name, age = list(zip(*full_name_list)) + print(f"first name: {first_name}\nlast name: {last_name} \nage: {age}") + + # Output + + # first name: ('Joe', 'Earnst', 'Thomas', 'Martin', 'Charles') + # last name: ('Schmoe', 'Ehlmann', 'Fischer', 'Walter', 'Rogan') + # age: (23, 65, 11, 36, 83) +