{ "cells": [ { "cell_type": "markdown", "id": "251dc4e3-05a5-4c16-947b-629a0ff0dea4", "metadata": {}, "source": [ "# 🚀 Quick Start Guide\n", "\n", "## Transform Your boto3 Experience with Type-Safe Dataclasses\n", "\n", "Welcome to **boto3-dataclass**! This library transforms boring boto3 dictionaries into beautiful, type-safe dataclasses with full autocomplete support. Say goodbye to `response['Key']['SubKey']` and hello to `response.Key.SubKey` with full IDE support!" ] }, { "cell_type": "markdown", "id": "c3df0cdc-a3fc-47cd-a267-526f07deb1cd", "metadata": {}, "source": [ "## 💡 What You'll Get\n", "\n", "- ✅ **Full autocomplete** in your IDE\n", "- ✅ **Type safety** with mypy\n", "- ✅ **Easy dot notation** instead of dictionary access\n", "- ✅ **Zero performance overhead** with lazy loading\n", "- ✅ **100% compatible** with existing boto3 code" ] }, { "cell_type": "markdown", "id": "56c5bb36-b4d0-4f7e-a51e-d3d83142fd31", "metadata": {}, "source": [ "## 📦 Installation\n", "\n", "Install the service you need (e.g., for IAM, S3):\n", "\n", "```bash\n", "pip install boto3-dataclass[iam,s3]\n", "```\n", "\n", "Or match with your boto3 version\n", "\n", "```bash\n", "pip install \"boto3-dataclass[iam,s3]>=1.40.0,<1.41.0\"\n", "```\n", "\n", "Or install everything at once:\n", "\n", "```bash\n", "pip install boto3-dataclass[all] # Installs all AWS services\n", "```" ] }, { "cell_type": "markdown", "id": "05682963-bd7c-498f-830f-6d3e1ba492bc", "metadata": {}, "source": [ "## ⚡ 30-Second Demo\n", "\n", "Let's see the magic in action with AWS IAM:\n", "\n", "### Before (Standard boto3)\n", "\n", "```python\n", "import boto3\n", "\n", "# ❌ No type hints, no autocomplete\n", "client = boto3.client(\"iam\")\n", "response = client.get_role(RoleName=\"my-role\")\n", "\n", "# ❌ Dictionary access - error-prone and no autocomplete\n", "role_name = response[\"Role\"][\"RoleName\"]\n", "request_id = response[\"ResponseMetadata\"][\"RequestId\"]\n", "```\n", "\n", "### After (With boto3-dataclass)\n", "```python\n", "from boto3_dataclass_iam import iam_caster\n", "import boto3\n", "\n", "# ✅ Get your response as usual\n", "client = boto3.client(\"iam\")\n", "response = client.get_role(RoleName=\"my-role\")\n", "\n", "# ✅ Transform it to a type-safe dataclass\n", "typed_response = iam_caster.get_role(response)\n", "\n", "# ✅ Enjoy autocomplete and type safety!\n", "role_name = typed_response.Role.RoleName\n", "request_id = typed_response.ResponseMetadata.RequestId\n", "```" ] }, { "cell_type": "markdown", "id": "6afc7162-88c3-4faa-bbc5-fe9c790bf3c2", "metadata": {}, "source": [ "## 🛠️ Complete Working Example\n", "\n", "Here's a complete example showing common patterns:" ] }, { "cell_type": "code", "execution_count": 15, "id": "699901cd-e3e7-4afc-9115-fefee6dbf02e", "metadata": {}, "outputs": [], "source": [ "aws_profile = \"bmt_app_dev_us_east_1\"\n", "aws_region = \"us-east-1\"" ] }, { "cell_type": "code", "execution_count": 17, "id": "3f00b06a-0027-457f-8d65-d3b4a06e768d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Role Name: lambda-power-user-role\n", "Created: 2023-05-11 04:07:16+00:00\n", "Request ID: 95dd9d22-e4ad-471b-8682-02bf8c465fff\n" ] } ], "source": [ "# Step 1: Import the caster for your service\n", "from boto3_dataclass_iam import iam_caster\n", "import boto3\n", "\n", "# Step 2: Create your boto3 client as usual\n", "boto_ses = boto3.Session(profile_name=aws_profile, region_name=aws_region)\n", "client = boto_ses.client(\"iam\")\n", "\n", "# Step 3: Make your API calls as normal\n", "response = client.get_role(RoleName=\"lambda-power-user-role\")\n", "\n", "# Step 4: Convert to dataclass for better experience\n", "role_info = iam_caster.get_role(response)\n", "\n", "# Step 5: Enjoy type-safe access with autocomplete!\n", "print(f\"Role Name: {role_info.Role.RoleName}\")\n", "print(f\"Created: {role_info.Role.CreateDate}\")\n", "print(f\"Request ID: {role_info.ResponseMetadata.RequestId}\")\n", "\n", "# =============================================================================\n", "# AVAILABLE ATTRIBUTES WITH AUTOCOMPLETION\n", "# =============================================================================\n", "# Top-level response attributes:\n", "# response.ResponseMetadata - AWS response metadata\n", "# response.Role - The IAM role details\n", "# response.boto3_raw_data - Original raw response (for debugging)\n", "\n", "# Role-specific attributes (all with IDE autocompletion):\n", "# response.Role.Path - Role path\n", "# response.Role.RoleName - Name of the role\n", "# response.Role.RoleId - Unique role identifier\n", "# response.Role.Arn - Amazon Resource Name\n", "# response.Role.CreateDate - When the role was created\n", "# response.Role.AssumeRolePolicyDocument - Trust policy document\n", "# response.Role.Description - Role description (if set)\n", "# response.Role.MaxSessionDuration - Maximum session duration\n", "# response.Role.PermissionsBoundary - Permissions boundary (if set)\n", "# response.Role.Tags - Role tags\n", "# response.Role.RoleLastUsed - Last usage information\n", "\n", "# =============================================================================\n", "# BENEFITS\n", "# =============================================================================\n", "# ✓ IDE autocompletion and IntelliSense\n", "# ✓ Type safety and error prevention\n", "# ✓ Better code readability\n", "# ✓ Reduced runtime errors from typos\n", "# ✓ Access to original raw data when needed" ] }, { "cell_type": "markdown", "id": "3d3382ca-e245-4e20-8714-8b5dbd5fb157", "metadata": {}, "source": [ "### Pro Tip: Using with boto-session-manager\n", "\n", "For even better type hints on your boto3 clients, combine with [boto-session-manager](https://pypi.org/project/boto-session-manager/):" ] }, { "cell_type": "code", "execution_count": 19, "id": "21e72800-0520-45e7-85eb-5e0b9ca9437c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Role Name: lambda-power-user-role\n", "Created: 2023-05-11 04:07:16+00:00\n", "Request ID: ab7910de-e09f-46bb-bb32-4252ce4f5afb\n" ] } ], "source": [ "from boto_session_manager import BotoSesManager\n", "\n", "# Get fully typed boto3 clients\n", "bsm = BotoSesManager(profile_name=aws_profile, region_name=aws_region)\n", "iam_client = bsm.iam_client # ✅ This has full type hints!\n", "\n", "# Make calls and convert responses\n", "response = iam_client.get_role(RoleName=\"lambda-power-user-role\")\n", "role_info = iam_caster.get_role(response) # ✅ Full autocomplete here too!\n", "\n", "# Enjoy type-safe access with autocomplete!\n", "print(f\"Role Name: {role_info.Role.RoleName}\")\n", "print(f\"Created: {role_info.Role.CreateDate}\")\n", "print(f\"Request ID: {role_info.ResponseMetadata.RequestId}\")" ] }, { "cell_type": "markdown", "id": "9dc15304-33bb-47d5-a9c5-70d9eee3d36e", "metadata": {}, "source": [ "## 🎯 How It Works - The Complete Picture\n", "\n", "Understanding the boto3-dataclass ecosystem is key to using it effectively. Here's how all the pieces fit together:\n", "\n", "### 🧩 The Three-Layer Architecture\n", "\n", "**Layer 1: Standard boto3**\n", "\n", "- Each AWS Service has a service name (the string you use to create clients)\n", "- For example, IAM uses: `iam_client = boto3.client('iam')`\n", "- See [boto3 IAM documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html)\n", "\n", "**Layer 2: mypy-boto3 Type Stubs**\n", "\n", "- Each AWS Service has a [Python stub-only package](https://peps.python.org/pep-0561/) like `mypy_boto3_{service_name}`\n", "- For example: [mypy-boto3-iam](https://pypi.org/project/mypy-boto3-iam/) provides TypedDict-style type hints\n", "- **Limitation**: These stubs give you type checking but **NO autocomplete** for attribute access\n", "\n", "**Layer 3: boto3-dataclass (This Library!)**\n", "\n", "- Each AWS Service gets a dataclass package: `boto3_dataclass_{service_name}`\n", "- For example: [boto3-dataclass-iam](https://pypi.org/project/boto3-dataclass-iam/)\n", "- **Benefit**: Converts response dictionaries to **user-friendly dataclasses** with full autocomplete and type hints\n", "\n", "### 🔄 The Conversion Pattern\n", "\n", "The magic happens with a simple, predictable pattern:\n", "\n", "**When you have:**\n", "\n", "- A service client: `{service_name}_client` from boto3\n", "- A response dictionary from: `client.method_name(...)`\n", "\n", "**You get:**\n", "\n", "- A corresponding caster: `{service_name}_caster` from `boto3_dataclass_{service_name}`\n", "- A conversion method: `caster.method_name(response)` that transforms dictionaries to dataclasses\n", "\n", "### 📋 Quick Reference\n", "\n", "```python\n", "# The Universal Pattern for ANY AWS Service:\n", "\n", "# 1. Create boto3 client\n", "client = boto3.client(\"{service_name}\")\n", "\n", "# 2. Import corresponding caster\n", "from boto3_dataclass_{service_name} import {service_name}_caster\n", "\n", "# 3. Make API calls as normal\n", "response = client.method_name(...)\n", "\n", "# 4. Convert to dataclass\n", "typed_response = {service_name}_caster.method_name(response)\n", "\n", "# 5. Enjoy autocomplete and type safety!\n", "```" ] }, { "cell_type": "markdown", "id": "35cdac15-2d36-44e4-9605-4d923822e79d", "metadata": {}, "source": [ "### 🎯 Why This Design Works\n", "\n", "- **Predictable**: Same pattern across all 200+ AWS services\n", "- **Optional**: Use dataclass conversion only when you need it\n", "- **Compatible**: Works with existing boto3 code without changes\n", "- **Performance**: Zero overhead until you convert responses" ] }, { "cell_type": "markdown", "id": "c7f64e2c-2dff-4c0a-abd6-f7357e848fc7", "metadata": {}, "source": [ "## 🏗️ Available Services\n", "\n", "We support **all AWS services** that have mypy-boto3 stubs!" ] }, { "cell_type": "markdown", "id": "5d15d795-9c9e-4f5c-852d-36817fc1e255", "metadata": {}, "source": [ "## 🔍 Behind the Scenes (Optional Reading)\n", "\n", "*Curious how the magic works? Here's the technical overview:*\n", "\n", "### The Architecture\n", "\n", "Our system automatically generates dataclasses from mypy-boto3 type stubs:\n", "\n", "1. **Parser**: Reads `mypy_boto3_{service_name}/type_defs.pyi` files\n", "2. **Generator**: Creates dataclass definitions in `boto3_dataclass_{service_name}/type_defs.py`\n", "3. **Caster**: Provides conversion methods in `boto3_dataclass_{service_name}/caster.py`\n", "\n", "### Smart Design Choices\n", "\n", "- **Lazy Loading**: Attributes are loaded only when accessed (zero overhead)\n", "- **Raw Data Access**: Original dict always available via `.boto3_raw_data`\n", "- **Type Safety**: Full mypy compatibility with proper type annotations\n", "- **Zero Magic**: No monkey patching or runtime modifications\n", "\n", "### Example Generated Code\n", "\n", "**1. Dataclass Definitions (`type_defs.py`)**\n", "Here's what gets generated for an IAM Role:" ] }, { "cell_type": "code", "execution_count": 20, "id": "fc845fce-aea4-4f6a-9ca5-3923762a92d1", "metadata": {}, "outputs": [], "source": [ "# content of boto3_dataclass_iam/type_defs.py\n", "# -*- coding: utf-8 -*-\n", "\n", "import typing as T\n", "import dataclasses\n", "from functools import cached_property\n", "\n", "if T.TYPE_CHECKING: # pragma: no cover\n", " from mypy_boto3_iam import type_defs\n", "\n", "\n", "def field(name: str):\n", " def getter(self):\n", " return self.boto3_raw_data[name]\n", "\n", " return cached_property(getter)\n", "\n", "\n", "@dataclasses.dataclass(frozen=True)\n", "class ResponseMetadata:\n", " boto3_raw_data: \"type_defs.ResponseMetadataTypeDef\" = dataclasses.field()\n", "\n", " RequestId = field(\"RequestId\")\n", " HTTPStatusCode = field(\"HTTPStatusCode\")\n", " HTTPHeaders = field(\"HTTPHeaders\")\n", " RetryAttempts = field(\"RetryAttempts\")\n", " HostId = field(\"HostId\")\n", "\n", " @classmethod\n", " def make_one(cls, boto3_raw_data: T.Optional[\"type_defs.ResponseMetadataTypeDef\"]):\n", " if boto3_raw_data is None:\n", " return None\n", " return cls(boto3_raw_data=boto3_raw_data)\n", "\n", " @classmethod\n", " def make_many(\n", " cls,\n", " boto3_raw_data_list: T.Optional[\n", " T.Iterable[\"type_defs.ResponseMetadataTypeDef\"]\n", " ],\n", " ):\n", " if boto3_raw_data_list is None:\n", " return None\n", " return [\n", " cls(boto3_raw_data=boto3_raw_data) for boto3_raw_data in boto3_raw_data_list\n", " ]\n", "\n", "# More dataclass model ...\n", "\n", "@dataclasses.dataclass(frozen=True)\n", "class Role:\n", " boto3_raw_data: \"type_defs.RoleTypeDef\" = dataclasses.field()\n", "\n", " Path = field(\"Path\")\n", " RoleName = field(\"RoleName\")\n", " RoleId = field(\"RoleId\")\n", " Arn = field(\"Arn\")\n", " CreateDate = field(\"CreateDate\")\n", " AssumeRolePolicyDocument = field(\"AssumeRolePolicyDocument\")\n", " Description = field(\"Description\")\n", " MaxSessionDuration = field(\"MaxSessionDuration\")\n", "\n", " @cached_property\n", " def PermissionsBoundary(self): # pragma: no cover\n", " return AttachedPermissionsBoundary.make_one(\n", " self.boto3_raw_data[\"PermissionsBoundary\"]\n", " )\n", "\n", " @cached_property\n", " def Tags(self): # pragma: no cover\n", " return Tag.make_many(self.boto3_raw_data[\"Tags\"])\n", "\n", " @cached_property\n", " def RoleLastUsed(self): # pragma: no cover\n", " return RoleLastUsed.make_one(self.boto3_raw_data[\"RoleLastUsed\"])\n", "\n", " @classmethod\n", " def make_one(cls, boto3_raw_data: T.Optional[\"type_defs.RoleTypeDef\"]):\n", " if boto3_raw_data is None:\n", " return None\n", " return cls(boto3_raw_data=boto3_raw_data)\n", "\n", " @classmethod\n", " def make_many(\n", " cls, boto3_raw_data_list: T.Optional[T.Iterable[\"type_defs.RoleTypeDef\"]]\n", " ):\n", " if boto3_raw_data_list is None:\n", " return None\n", " return [\n", " cls(boto3_raw_data=boto3_raw_data) for boto3_raw_data in boto3_raw_data_list\n", " ]\n", " \n", "\n", "@dataclasses.dataclass(frozen=True)\n", "class GetRoleResponse:\n", " boto3_raw_data: \"type_defs.GetRoleResponseTypeDef\" = dataclasses.field()\n", "\n", " @cached_property\n", " def Role(self): # pragma: no cover\n", " return Role.make_one(self.boto3_raw_data[\"Role\"])\n", "\n", " @cached_property\n", " def ResponseMetadata(self): # pragma: no cover\n", " return ResponseMetadata.make_one(self.boto3_raw_data[\"ResponseMetadata\"])\n", "\n", " @classmethod\n", " def make_one(cls, boto3_raw_data: T.Optional[\"type_defs.GetRoleResponseTypeDef\"]):\n", " if boto3_raw_data is None:\n", " return None\n", " return cls(boto3_raw_data=boto3_raw_data)\n", "\n", " @classmethod\n", " def make_many(\n", " cls,\n", " boto3_raw_data_list: T.Optional[T.Iterable[\"type_defs.GetRoleResponseTypeDef\"]],\n", " ):\n", " if boto3_raw_data_list is None:\n", " return None\n", " return [\n", " cls(boto3_raw_data=boto3_raw_data) for boto3_raw_data in boto3_raw_data_list\n", " ]" ] }, { "cell_type": "markdown", "id": "ded50da4-849a-41c2-a03a-cca41a7a4b94", "metadata": {}, "source": [ "**2. Caster Module (`caster.py`)**\n", "\n", "Then we have a `boto3_dataclass_{service_name}/caster.py` module for each service like this:" ] }, { "cell_type": "code", "execution_count": 14, "id": "f5f101e8-2f26-491d-9669-7597d4126320", "metadata": {}, "outputs": [], "source": [ "# -*- coding: utf-8 -*-\n", "\n", "import typing as T\n", "\n", "from . import type_defs as dc_td\n", "\n", "if T.TYPE_CHECKING: # pragma: no cover\n", " from mypy_boto3_iam import type_defs as bs_td\n", "\n", "\n", "class IAMCaster:\n", " def get_role(\n", " self,\n", " res: \"bs_td.GetRoleResponseTypeDef\",\n", " ) -> \"dc_td.GetRoleResponse\":\n", " return dc_td.GetRoleResponse.make_one(res)\n", "\n", " # More client method ...\n", "\n", "iam_caster = IAMCaster()" ] }, { "cell_type": "markdown", "id": "3cefb202-6533-458c-9bb0-c12263350f47", "metadata": {}, "source": [ "**The Caster Magic:**\n", " \n", "- The caster class **mimics the exact same method names** as the boto3 client\n", "- When you call `client.get_role(...)`, you use `caster.get_role(response)`\n", "- **No memorization needed** - if the client has the method, so does the caster!\n", "- Each method converts the raw response dictionary to a type-safe dataclass" ] }, { "cell_type": "markdown", "id": "b8f6c981-2fbf-4802-a645-a47280447656", "metadata": {}, "source": [ "## 🎉 Next Steps\n", "\n", "**Ready to upgrade your boto3 experience?**\n", "\n", "1. **Install** the service package you need: `pip install boto3-dataclass[{service}]`\n", "2. **Import** the caster: `from boto3_dataclass_{service} import {service}_caster`\n", "3. **Convert** your responses: `typed_response = caster.method_name(response)`\n", "4. **Enjoy** the autocomplete and type safety!\n", "\n", "**Questions or issues?** Check out our [GitHub repository](https://github.com/MacHu-GWU/boto3_dataclass-project) or [documentation](https://boto3-dataclass.readthedocs.io/).\n", "\n", "*Happy coding! 🐍✨*" ] }, { "cell_type": "code", "execution_count": null, "id": "9b8137be-cadb-44a6-9a16-3345b5b4204c", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.8" } }, "nbformat": 4, "nbformat_minor": 5 }