{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Module 2, Practical 2\n", "\n", "In this practical we will learn how to test a Python program and how to measure its runtime and memory usage (i.e., compexity in time and space).\n", "\n", "## Testing\n", "\n", "Testing a software product allows to check whether the written code actually yields the expected results. In particular, testing the execution of code is called *dynamic* testing.\n", "\n", "In Python, we can use `assert` to quickly test that functions we have written behave as they should: \n", "\n", "```\n", "assert(condition)\n", "```\n", "\n", "```condition``` must be ```True``` for program execution to continue without errors. \n", "If it is ```False```, an ```AssertionError``` exception will be raised." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "ename": "AssertionError", "evalue": "", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mAssertionError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[1], line 11\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m(sumOfTwo(\u001b[38;5;241m3\u001b[39m,\u001b[38;5;241m5\u001b[39m) \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m8\u001b[39m)\n\u001b[1;32m 10\u001b[0m \u001b[38;5;66;03m# what happens here?\u001b[39;00m\n\u001b[0;32m---> 11\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m(sumOfTwo(\u001b[38;5;241m3\u001b[39m,\u001b[38;5;241m5\u001b[39m) \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m1\u001b[39m)\n", "\u001b[0;31mAssertionError\u001b[0m: " ] } ], "source": [ "def sumOfTwo(x, y):\n", " return x + y\n", "\n", "# this should return 8\n", "sumOfTwo(3,5)\n", "\n", "# check that the function is working properly\n", "assert(sumOfTwo(3,5) == 8)\n", "\n", "# what happens here?\n", "assert(sumOfTwo(3,5) == 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Always remember that these tests are not exhaustive, they check only specific aspects of the function and a subset of possible inputs. \n", "For example, *what happens if the user provides one integer and one string as parameters?*\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "ename": "TypeError", "evalue": "unsupported operand type(s) for +: 'int' and 'str'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[2], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43msumOfTwo\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m3\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mbaduser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", "Cell \u001b[0;32mIn[1], line 2\u001b[0m, in \u001b[0;36msumOfTwo\u001b[0;34m(x, y)\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21msumOfTwo\u001b[39m(x, y):\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mx\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m+\u001b[39;49m\u001b[43m \u001b[49m\u001b[43my\u001b[49m\n", "\u001b[0;31mTypeError\u001b[0m: unsupported operand type(s) for +: 'int' and 'str'" ] } ], "source": [ "sumOfTwo(3, \"baduser\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Unit testing\n", "\n", "Unit testing refers to tests that verify the functionality of a code unit, typically a function. Asserts are a quick way to test a piece of code but they don't provide us with much detail (e.g. how different is the result we obtained from the expected one?). \n", "Luckily, Python provides us with a specific module for unit testing: *unittest* (see [here the doc](https://docs.python.org/3/library/unittest.html)). \n", "\n", "*unittest* is an OOP-based module - you need to subclass the ```unittest.TestCase``` class:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fibonacci(5): [1, 1, 2, 3, 5]\n" ] } ], "source": [ "import unittest\n", "\n", "def fibonacci(n):\n", " lst = []\n", " for i in range(0, n):\n", " if(i <= 1):\n", " lst.append(1)\n", " else:\n", " lst.append(lst[i-2] + lst[i-1])\n", " return(lst)\n", "\n", "print(\"Fibonacci(5):\", fibonacci(5))\n", "\n", "class FibonacciTest(unittest.TestCase):\n", "\n", " def test_listLength(self):\n", " self.assertEqual(len(fibonacci(4)), 4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The test methods name begins with ```test_```. \n", "\n", "```unittest``` offers many types of assertions: ```assertTrue```, ```assertEqual```, ```assertListEquals``` and more. \n", "\n", "In a command-line environment, the unit test woud be launched by typing:\n", "```python3 -m unittest file_test``` \n", "\n", "So to run these code chunks, save the code in a test.py file and run it from the terminal" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Example**\n", "\n", "Try yourself to complete the ```FibonacciTest``` class by adding the following asserts:\n", "\n", "1. check that the last (highest) number is correct\n", "2. check that a negative ```n``` provided as input does not break the function\n", "3. check that a floating point ```n``` as input does not break the function \n", "\n", "
Show/Hide solution" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fibonacci(5): [1, 1, 2, 3, 5]\n" ] } ], "source": [ "import unittest\n", "\n", "def fibonacci(n):\n", " lst = []\n", " for i in range(0, n):\n", " if(i <= 1):\n", " lst.append(1)\n", " else:\n", " lst.append(lst[i-2] + lst[i-1])\n", " return(lst)\n", "\n", "print(\"Fibonacci(5):\", fibonacci(5))\n", "\n", "class FibonacciTest(unittest.TestCase):\n", "\n", " def test_listLength(self):\n", " self.assertEqual(len(fibonacci(4)), 4) # fibonacci(4)) returns 4 numbers\n", "\n", " def test_checkLast(self):\n", " self.assertEqual(max(fibonacci(5)), 5) # last number of fibonacci(5)) is 5\n", "\n", " def test_checkNegative(self):\n", " self.assertIsNotNone(fibonacci(-10)) # fibonacci(-10) returns something\n", "\n", " def test_checkFloatingPoint(self):\n", " self.assertIsNotNone(fibonacci(4.5)) # what happens here?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "The ```checkFloatingPoint``` test fails. Let's fix the ```fibonacci``` function to solve the issue and run the unit tests again! \n", " \n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What if we decide to *not* accept a floating point input? We can raise a ```TypeError``` exception and modify the unit test accordingly.\n", "\n", "For a preliminary guide to Python exceptions, please refer to the official Python tutorial: [https://docs.python.org/3/tutorial/errors.html](https://docs.python.org/3/tutorial/errors.html)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fibonacci(5): [1, 1, 2, 3, 5]\n" ] } ], "source": [ "import unittest\n", "\n", "def fibonacci(n):\n", " if isinstance(n, int):\n", " lst = []\n", " for i in range(0, n):\n", " if(i <= 1):\n", " lst.append(1)\n", " else:\n", " lst.append(lst[i-2] + lst[i-1])\n", " return(lst)\n", " else:\n", " raise(TypeError) # we raise an exception!\n", "\n", "print(\"Fibonacci(5):\", fibonacci(5))\n", "\n", "class FibonacciTest(unittest.TestCase):\n", "\n", " def test_listLength(self):\n", " self.assertEqual(len(fibonacci(4)), 4) # fibonacci(4)) returns 4 numbers\n", "\n", " def test_checkLast(self):\n", " self.assertEqual(max(fibonacci(5)), 5) # last number of fibonacci(5)) is 5\n", "\n", " def test_checkNegative(self):\n", " self.assertIsNotNone(fibonacci(-10)) # fibonacci(-10) returns something\n", " \n", " def test_checkFloatingPoint(self):\n", " with self.assertRaises(TypeError): # fibonacci(4.5) has to raise an exception\n", " fibonacci(4.5)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise\n", "\n", "Modify the ```FibonacciTest``` class to:\n", "\n", "1. raise a ```ValueError``` exception if the provided ```n``` is negative\n", "2. check that a big ```n``` (>>100, for example 100,000) does not raise a ```MemoryError``` exception\n", "\n", "
Show/Hide solution" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fibonacci(5): [1, 1, 2, 3, 5]\n" ] } ], "source": [ "import unittest\n", "\n", "def fibonacci(n):\n", " if isinstance(n, int):\n", " if n > 0:\n", " lst = []\n", " for i in range(0, n):\n", " if(i <= 1):\n", " lst.append(1)\n", " else:\n", " lst.append(lst[i-2] + lst[i-1])\n", " return(lst)\n", " else:\n", " raise(ValueError) # we raise an exception!\n", " else:\n", " raise(TypeError) # we raise another exception!\n", "\n", "print(\"Fibonacci(5):\", fibonacci(5))\n", "\n", "class FibonacciTest(unittest.TestCase):\n", "\n", " def test_listLength(self):\n", " self.assertEqual(len(fibonacci(4)), 4) # fibonacci(4)) returns 4 numbers\n", "\n", " def test_checkLast(self):\n", " self.assertEqual(max(fibonacci(5)), 5) # last number of fibonacci(5)) is 5\n", "\n", " def test_checkNegative(self):\n", " with self.assertRaises(ValueError):\n", " fibonacci(-10) # fibonacci(-10) has to raise an exception this time...\n", " \n", " def test_checkFloatingPoint(self):\n", " with self.assertRaises(TypeError): # fibonacci(4.5) has to raise an exception\n", " fibonacci(4.5)\n", " \n", " def test_checkBig(self):\n", " self.assertIsNotNone(fibonacci(100000)) # test fibonacci(100000) for memory issues\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " \n", "## Measuring time\n", "\n", "Another aspect of testing is to probe the time and memory required by your algorithms to be executed under different inputs and parameters. \n", " \n", "Python provides the ```timeit``` package to measure the execution time of small snippets of code. \n", "Execution of the snippet is repeated multiple times (```number``` parameter) to allow obtaining an estimate also for the quickest snippets :\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import timeit\n", "\n", "def fibonacci(n):\n", " if isinstance(n, int):\n", " if n > 0:\n", " lst = []\n", " for i in range(0, n):\n", " if(i <= 1):\n", " lst.append(1)\n", " else:\n", " lst.append(lst[i-2] + lst[i-1])\n", " return(lst)\n", " else:\n", " raise(ValueError)\n", " else:\n", " raise(TypeError)\n", "\n", "# setup ensures that timeit can access specific functions from your local environment\n", "timeit.timeit('fibonacci(5)', number=10000, setup=\"from __main__ import fibonacci\")\n", "\n", "times = []\n", "times.append(timeit.timeit('fibonacci(5)', number=10000, setup=\"from __main__ import fibonacci\"))\n", "times.append(timeit.timeit('fibonacci(10)', number=10000, setup=\"from __main__ import fibonacci\"))\n", "times.append(timeit.timeit('fibonacci(15)', number=10000, setup=\"from __main__ import fibonacci\"))\n", "\n", "timeSeries = pd.Series(times)\n", "timeSeries.plot()\n", "# to change x labels...\n", "plt.xticks([0, 1, 2], ['fibonacci(5)', 'fibonacci(10)', 'fibonacci(15)'])\n", "plt.show()\n", "plt.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![fibonacciTimes](images/fibonacciTimes.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise\n", "\n", "Write the ```recursiveFibonacci``` function that computes the n-th Fibonacci number recursively. \n", "Test the execution times of the two fibonacci functions with ```n``` ranging from 1 to 20 and plot the two distributions\n", "\n", "
Show/Hide solution" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import timeit\n", "\n", "def fibonacci(n):\n", " if isinstance(n, int):\n", " if n > 0:\n", " lst = []\n", " for i in range(0, n):\n", " if(i <= 1):\n", " lst.append(1)\n", " else:\n", " lst.append(lst[i-2] + lst[i-1])\n", " return(lst)\n", " else:\n", " raise(ValueError)\n", " else:\n", " raise(TypeError)\n", "\n", "def recursiveFibonacci(n):\n", " if n == 1:\n", " return 0\n", " elif n == 2:\n", " return 1\n", " else:\n", " return recursiveFibonacci(n-1) + recursiveFibonacci(n-2)\n", "\n", "timesIterative = []\n", "for n in range(1, 20):\n", " timesIterative.append(timeit.timeit('fibonacci(+' + str(n) + ')', number=1000, setup=\"from __main__ import fibonacci\"))\n", "\n", "timesRecursive = []\n", "for n in range(1, 20):\n", " timesRecursive.append(timeit.timeit('recursiveFibonacci(+' + str(n) + ')', number=1000, setup=\"from __main__ import recursiveFibonacci\"))\n", "\n", "timesDict = {'iterative': timesIterative, 'recursive': timesRecursive}\n", "timesDf = pd.DataFrame(timesDict)\n", "timesDf.plot(logy=True) # plot times in logarithmic scale to highlight differences\n", "plt.show()\n", "plt.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![fibonacciTimesRec](images/fibonacciTimesRec.png)\n", "\n", "
\n", "\n", "## Measuring memory\n", "\n", "Memory used by an object, function or full Python scripts can be measured by different means. \n", "\n", "The ```sys.getsizeof``` method is one of them: \n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1656\n", "3256\n" ] } ], "source": [ "import sys\n", "\n", "l1 = [1]*200\n", "l2 = [2]*400\n", "\n", "print(sys.getsizeof(l1))\n", "print(sys.getsizeof(l2))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, measuring the memory footprint of a whole function (rather than single objects), would be much more useful. \n", "We can achieve that, for instance, with the ```memory-profiler``` package.\n", "\n", "You can install it with pip: ```pip install -U memory_profiler```" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Save this file as memProfSample.py\n", "\n", "from memory_profiler import profile\n", "\n", "@profile\n", "def allocatingFunction():\n", " a = [1] * (10 ** 6)\n", " b = [2] * (2 * 10 ** 7)\n", " del b\n", " return a\n", "\n", "if __name__ == '__main__':\n", " allocatingFunction()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "Filename: memProfSample.py\n", "\n", "Line # Mem usage Increment Occurences Line Contents\n", "============================================================\n", " 5 39.9 MiB 39.9 MiB 1 @profile\n", " 6 def allocatingFunction():\n", " 7 47.5 MiB 7.6 MiB 1 a = [1] * (10 ** 6)\n", " 8 200.1 MiB 152.6 MiB 1 b = [2] * (2 * 10 ** 7)\n", " 9 47.5 MiB -152.6 MiB 1 del b\n", " 10 47.5 MiB 0.0 MiB 1 return a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise\n", "\n", "Test the functionalities introduced in this practical to analyze some of the programs developed during the past lessons.\n" ] } ], "metadata": { "kernelspec": { "display_name": "3.10.14", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.14" }, "orig_nbformat": 2 }, "nbformat": 4, "nbformat_minor": 2 }