{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Quick Start Guide\n", "\n", "TurboGraph is a Python library for defining\n", "and computing **dependent computations** using a directed acyclic graph (DAG).\n", "\n", "πŸ“Œ **[Complete Notebook](https://gitlab.cern.ch/particlepredatorinvasion/turbograph/-/blob/master/docs/source/quickstart.ipynb)** \n", "πŸ“Œ **[Documentation Page](https://turbograph.docs.cern.ch/master/quickstart.html)**\n", "\n", "\n", "## πŸš€ Why Use TurboGraph?\n", "\n", "βœ” **Automatically determines dependencies & execution order** \n", "βœ” **Computes only required values** \n", "βœ” **Allows dynamic overrides of values/functions** \n", "βœ” **Reuses and modifies graphs without rebuilding**\n", "\n", "\n", "---\n", "## ➊ Setting Up: Logging & Debugging" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import logging\n", "\n", "logging.basicConfig(\n", " level=logging.INFO, # Change to DEBUG for more detailed logs\n", " format=\"%(name)s - %(levelname)s - %(message)s\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## βž‹ Understanding the Computation Graph (DAG)\n", "\n", "TurboGraph structures computations as a **directed acyclic graph (DAG)**:\n", "\n", "βœ” **Vertices (Nodes):** Represent computations or values \n", "βœ” **Edges (Connections):** Define dependencies between computations \n", "\n", "Each **vertex** in the graph can hold:\n", "- A **fixed value**\n", "- A **function** depending on other values\n", "- **Predecessors** (dependencies)\n", "\n", "### πŸ”Ή Example Computation Graph\n", "\n", "```python\n", "specs = {\n", " \"a\": 5, \n", " \"b\": 3, \n", " \"add\": lambda a, b: a + b, # Depends on \"a\" and \"b\"\n", " \"double\": lambda add: add * 2, # Depends on \"add\"\n", "}\n", "```\n", "\n", "#### Graph Structure\n", "```\n", " a b\n", " \\ /\n", " \\ /\n", " (add)\n", " |\n", " (double)\n", "```\n", "\n", "#### Execution Order\n", "1️⃣ Compute `a` and `b` first (as they have no dependencies). \n", "2️⃣ Compute `add` using `a` and `b`. \n", "3️⃣ Compute `double` using `add`. \n", "\n", "βœ… TurboGraph ensures computations are executed in the correct order **automatically**.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## ➌ Running Computations with [compute()](api/main.rst#turbograph.compute)\n", "\n", "The [compute()](api/main.rst#turbograph.compute) function:\n", "\n", "βœ” **Builds the computation graph (DAG)** \n", "βœ” **Determines execution order** \n", "βœ” **Computes only necessary values** \n", "βœ” **Allows flexible overrides** \n", "\n", "### πŸ”Ή Basic Computation Example" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from turbograph import compute\n", "\n", "specs = {\n", " \"a\": 5, # Constant value.\n", " \"b\": 3, # Constant value.\n", " \"add\": lambda a, b: a + b, # TurboGraph infers dependencies \"a\" and \"b\".\n", " \"double\": lambda add: add * 2, # Depends on \"add\".\n", "}\n", "\n", "compute(specs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… **TurboGraph detects dependencies and computes in the correct order!**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή Overriding Values and Functions" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "compute(\n", " specs,\n", " values={\"a\": 10}, # Override \"a\".\n", " funcs={\"double\": lambda x: x + 1}, # Modify \"double\".\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"x\"` is overridden to `10`, and `\"double\"` now computes `add + 1`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή Computing Only a Subset" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "compute(specs, vertices=[\"add\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… Computes only `\"add\"`, retrieving required dependencies automatically.\n", " `\"final\"` is not computed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## ➍ Defining Computations (Vertex Specifications)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "TurboGraph provides **four** ways to specify computations.\n", "\n", "### πŸ”ΉMethod 1: Constant Values" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "specs = {\"a\": 1, \"b\": 2}\n", "compute(specs, vertices=[\"b\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"b\"` is directly returned **without computation**.\n", "\n", "### πŸ”Ή Method 2: Functions (Automatic Dependency Inference)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "specs = {\n", " \"a\": lambda: 7,\n", " \"b\": lambda: 8,\n", " \"add\": lambda a, b: a + b,\n", "}\n", "compute(specs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"add\"` is computed only after `\"a\"` and `\"b\"` are evaluated.\n", "\n", "### πŸ”Ή Method 3: Dictionary Specification (Manual Dependencies)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "specs = {\n", " # `\"value\"` takes precedence over `\"func\"` during computation.\n", " \"a\": {\"func\": lambda: 0, \"value\": 7},\n", " \"b\": {\"value\": 8},\n", " # Dependencies: \"a\" and \"b\" explicitly specified.\n", " \"add\": {\"func\": lambda x, y: x + y, \"predecessors\": (\"a\", \"b\")},\n", "}\n", "compute(specs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"add\"` is computed after `\"a\"` and `\"b\"`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή Method 4: Sequence Specification\n", "\n", "Explicitly define **function**, **dependencies**, and **values** in a list or tuple." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "specs = {\n", " \"a\": [None, (), 3],\n", " \"b\": [], # Uses defaults.\n", " \"diff\": [lambda x, y: x - y, [\"a\", \"b\"]],\n", "}\n", "\n", "compute(specs, values={\"b\": 5})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"diff\"` is computed using `\"a\"` and `\"b\"` as dependencies." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## ➎ Controlling Function Execution with `call_mode`\n", "\n", "The `call_mode` parameter **controls how dependency values are passed** to functions.\n", "\n", "### Available `call_mode` Options\n", "\n", "| Mode | Description | Example Function Signature |\n", "|---------|-----------------------------|----------------------------|\n", "| `args` | Positional Arguments | `def func(a, b): ...` |\n", "| `kwargs`| Keyword Arguments | `def func(a=1, b=2): ...` |\n", "| `arg` | Single dictionary argument | `def func(inputs): ...` |\n", "\n", "The default `call_mode` is `args`.\n", "\n", "### πŸ”Ή `args` (Positional Arguments)\n", "\n", "Dependencies are passed as **positional arguments**, matching the function’s parameter order." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def add_args(a: int, b: int, *, c: int = 0) -> int:\n", " \"\"\"Add three numbers. `\"c\"` is a keyword-only argument and is therefore ignored.\"\"\"\n", " return a + b + c\n", "\n", "\n", "specs = {\"a\": lambda: 2, \"b\": lambda: 3, \"add\": add_args}\n", "compute(specs, call_mode=\"args\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"add\"` is called as `add_args(a, b)`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή `kwargs` (Keyword Arguments)\n", "\n", "Dependencies are passed as **named arguments**." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def add_kwargs(c: int = 1, /, a: int = 2, *, b: int = 0) -> int:\n", " \"\"\"Add three numbers. `c` is positional-only, and is therefore ignored.\"\"\"\n", " return a + b + c\n", "\n", "\n", "specs = {\"a\": lambda: 2, \"b\": lambda: 3, \"add\": add_kwargs}\n", "compute(specs, call_mode=\"kwargs\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"add\"` is called as `add_kwargs(a=2, b=3)`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή `arg` (Single Dictionary Argument)\n", "\n", "All values are passed as a dictionary." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def add_arg(inputs: dict[str, int]) -> int:\n", " \"\"\"Add two numbers from a dictionary.\"\"\"\n", " return inputs[\"a\"] + inputs[\"b\"]\n", "\n", "\n", "specs = {\n", " \"a\": lambda _: 2,\n", " \"b\": lambda _: 3,\n", " \"add\": {\"func\": add_arg, \"predecessors\": [\"a\", \"b\"]},\n", "}\n", "compute(specs, call_mode=\"arg\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… Function receives all dependencies as a dictionary." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## ➏ Reusing Graphs\n", "\n", "TurboGraph allows for building, reusing, modifying computation graphs.\n", "\n", "Instead of recomputing everything from scratch, you can **reuse** and **modify** existing graphs.\n", "\n", "### πŸ”Ή Building & Computing a Graph with [build_graph()](api/main.rst#turbograph.build_graph) and [compute_from_graph()](api/main.rst#turbograph.compute_from_graph)\n", "\n", "Instead of directly using [compute()](api/main.rst#turbograph.compute),\n", "you can manually build a computation graph with [build_graph()](api/main.rst#turbograph.build_graph)\n", "and compute values later using [compute_from_graph()](api/main.rst#turbograph.compute_from_graph).\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from turbograph import build_graph, compute_from_graph\n", "\n", "specs = {\n", " \"input1\": lambda: 5,\n", " \"input2\": lambda: 3,\n", " \"add\": lambda input1, input2: input1 + input2,\n", " \"output\": lambda add: add * 2,\n", "}\n", "graph = build_graph(specs)\n", "\n", "compute_from_graph(graph, vertices=[\"output\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… The graph is **built once** and **reused multiple times**.\n", "\n", "### πŸ”Ή Computing with Different Values\n", "\n", "You can override **specific values** without modifying the underlying graph structure." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "compute_from_graph(graph, [\"add\"], values={\"input1\": 10})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… The graph remains unchanged, but `\"add\"` is recomputed using the new `\"input1\"` value.\n", "\n", "\n", "### πŸ”Ή The [Graph](api/turbograph.core.rst#turbograph.core.graphwrapper.GraphWrapper) Object\n", "\n", "The [Graph](api/turbograph.core.rst#turbograph.core.graphwrapper.GraphWrapper) object\n", "serves as a wrapper around a NetworkX or iGraph graph, depending on the selected backend.\n", "\n", "It provides a simple interface for constructing, modifying, and querying graphs\n", "within TurboGraph computations. This object is mainly intended for internal use." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "internal_graph = graph.graph # Access the underlying graph object\n", "\n", "print(\"Internal graph:\", internal_graph)\n", "print(\"Vertices:\", graph.vertices)\n", "print(\"Edges:\", graph.edges)\n", "print(\"Vertex attributes:\", graph.get_all_vertex_attributes())\n", "print(\"Call mode:\", graph.call_mode)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή Use the [.compute()](api/turbograph.core.rst#turbograph.core.graphwrapper.GraphWrapper.compute) Method\n", "\n", "For convenience, the `.compute()` method method is available on the graph object.\n", "It serves as an alias for [compute_from_graph()](api/main.rst#turbograph.compute_from_graph)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "graph.compute([\"add\"], {\"input1\": 10})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ➐ Modifying Graphs\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή Rebuilding a Graph ([rebuild_graph()](api/main.rst#turbograph.rebuild_graph))\n", "\n", "To modify specific computations while **preserving dependencies**,\n", "use [rebuild_graph()](api/main.rst#turbograph.rebuild_graph)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from turbograph import compute_from_graph, rebuild_graph\n", "\n", "updated_graph = rebuild_graph(\n", " graph,\n", " vertices=[\"add\"],\n", " funcs={\"add\": lambda input1, input2: input1 - input2},\n", ")\n", "\n", "# Compute using the updated graph.\n", "compute_from_graph(updated_graph)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"add\"` now computes subtraction instead of addition, while keeping the rest of the graph intact. \n", "βœ… The node `\"output\"` is removed since it does not depend on `\"add\"`.\n", "\n", "You can clear precomputed values using `turbograph.NA`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from turbograph import NA\n", "\n", "graph = build_graph(\n", " {\n", " \"input1\": lambda: 5,\n", " \"input2\": lambda: 3,\n", " \"add\": {\n", " \"func\": lambda input1, input2: input1 + input2,\n", " \"predecessors\": [\"input1\", \"input2\"],\n", " \"value\": 3, # Precomputed value.\n", " },\n", " }\n", ")\n", "\n", "updated_graph = rebuild_graph(\n", " graph,\n", " values={\n", " \"input1\": 10,\n", " \"input2\": 3,\n", " \"add\": NA, # Reset the value of \"add\".\n", " },\n", ")\n", "\n", "compute_from_graph(updated_graph)\n", "# equivalent to updated_graph.compute()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "βœ… `\"add\"` is now recalculated instead of using a previous precomputed value `3`.\n", "\n", "### πŸ”Ή Use the [.rebuild()](api/turbograph.core.rst#turbograph.core.graphwrapper.GraphWrapper.rebuild) Method\n", "\n", "The `.rebuild()` method allows you to rebuild the graph directly.\n", "It's an alias for [rebuild_graph()](api/main.rst#turbograph.rebuild_graph)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Equivalent to the previous example but using the `rebuild` method.\n", "other_updated_graph = graph.rebuild(\n", " values={\n", " \"input1\": 10,\n", " \"input2\": 3,\n", " \"add\": NA,\n", " }\n", ")\n", "\n", "# Both updated graphs should be identical.\n", "assert updated_graph == other_updated_graph" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### πŸ”Ή Resetting the Graph with the [.reset()](api/turbograph.core.rst#turbograph.core.graphwrapper.GraphWrapper.reset) Method\n", "\n", "The [.reset()](api/turbograph.core.rst#turbograph.core.graphwrapper.GraphWrapper.reset)\n", "removes all precomputed values, functions, and the `call_mode`\n", "in the graph, while preserving the graph structure.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from pprint import pprint\n", "\n", "graph = build_graph(\n", " {\n", " \"input1\": 2,\n", " \"add\": lambda input1, input2: input1 + input2,\n", " },\n", " call_mode=\"args\",\n", ")\n", "print(\"Before reset:\")\n", "pprint(graph.get_all_vertex_attributes())\n", "print(\"Call mode:\", graph.call_mode, \"\\n\")\n", "\n", "graph.reset()\n", "print(\"After reset:\")\n", "pprint(graph.get_all_vertex_attributes())\n", "print(\"Call mode:\", graph.call_mode)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## ✨ Takeaways\n", "\n", "TurboGraph provides a flexible approach to defining and executing dependent\n", "computations. It enables you to:\n", "\n", "- **Define Computations:** Use functions, dictionaries, or sequences to declare computation rules and dependencies.\n", "- **Execute & Modify Computations:**\n", " - Compute full or partial graphs dynamically using [compute()](api/main.rst#turbograph.compute).\n", " - Control function argument handling with `call_mode`.\n", " - Override values and functions at runtime via `values` and `funcs`.\n", "- **Reuse & Update Graphs:**\n", " - Construct graphs with [build_graph()](api/main.rst#turbograph.build_graph) and execute computations using [compute_from_graph()](api/main.rst#turbograph.compute_from_graph).\n", " - Modify graphs selectively with [rebuild_graph()](api/main.rst#turbograph.rebuild_graph).\n" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.6" } }, "nbformat": 4, "nbformat_minor": 2 }