quickstart.qmd

---
title: "LLM Quick Start"
format:
  revealjs:
    theme: simple
    transition: slide
    slide-number: true
    chalkboard: true
editor:
  render-on-save: true
---

```{r include=FALSE}
slack_channel <- "#hackathon-22"
```

## Setup {.smaller}

#### Option 1: Cloud

- Log in to [https://workbench.posit.it/](https://workbench.posit.it/) and start a session in your IDE of choice
  - We recommend RStudio for R, and Positron for R or Python
- Clone https://github.com/jcheng5/llm-quickstart
- Open `llm-quickstart` as a Project (RStudio) or Folder (Positron) in your IDE
- Grab your OpenAI/Anthropic API keys; see the thread in <code>`r slack_channel`</code>

#### Option 2: Local

- Clone https://github.com/jcheng5/llm-quickstart
- Grab your OpenAI/Anthropic API keys; see the thread in <code>`r slack_channel`</code>

- For R: `install.packages(c("ellmer", "shinychat", "dotenv", "shiny", "paws.common", "magick", "beepr"))`
- For Python: `pip install -r requirements.txt`

# Introduction

## Framing LLMs

::: {.incremental}
- Our focus: Practical, actionable information
  - Often, _just_ enough knowledge so you know what to search for (or better yet, what to ask an LLM)
- We will treat LLMs as black boxes
- Don't focus on how they work (yet)
  - Leads to bad intuition about their capabilities
  - Better to start with a highly empirical approach
:::

# Anatomy of a Conversation

## LLM Conversations are HTTP Requests

::: {.incremental}
- Each interaction is a separate HTTP API request
- The API server is entirely stateless (despite conversations being inherently stateful!)
:::

## Example Conversation

::: {style="text-align: right;"}
"What's the capital of the moon?"
:::

`"There isn't one."`

::: {style="text-align: right;"}
"Are you sure?"
:::

`"Yes, I am sure."`

## Example Request

```{.bash code-line-numbers="|5|6-9|7|8"}
curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4.1",
    "messages": [
        {"role": "system", "content": "You are a terse assistant."},
        {"role": "user", "content": "What is the capital of the moon?"}
    ]
}'
```

- System prompt: behind-the-scenes instructions and information for the model
- User prompt: a question or statement for the model to respond to

## Example Response (abridged)

```{.json code-line-numbers="|3-6|7|12"}
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The moon does not have a capital. It is not inhabited or governed.",
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21,
    "completion_tokens_details": {
      "reasoning_tokens": 0
    }
  }
}
```

## Example Request

```{.bash code-line-numbers="|9|10"}
curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4.1",
    "messages": [
      {"role": "system", "content": "You are a terse assistant."},
      {"role": "user", "content": "What is the capital of the moon?"},
      {"role": "assistant", "content": "The moon does not have a capital. It is not inhabited or governed."},
      {"role": "user", "content": "Are you sure?"}
    ]
}'
```

## Example Response (abridged)

```{.json code-line-numbers="|3-6|10-12"}
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "Yes, I am sure. The moon has no capital or formal governance."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 52,
    "completion_tokens": 15,
    "total_tokens": 67,
    "completion_tokens_details": {
      "reasoning_tokens": 0
    }
  }
}
```

## Tokens

::: {.incremental}
- Fundamental units of information for LLMs
- Words, parts of words, or individual characters
  - "hello" → 1 token
  - "unconventional" → 3 tokens: `un|con|ventional`
  - 4K video frame at full res → 6885 tokens
- Example with [OpenAI Tokenizer](https://platform.openai.com/tokenizer)
- Important for:
  - Model input/output limits
  - API pricing is usually by token (see [comparison](https://pricepertoken.com/))
:::

# Choose a Package {.smaller}

- R:
  - [`ellmer`](https://ellmer.tidyverse.org/) high-level, easy, much less ambitious than langchain
    - OpenAI, Anthropic, Google are well supported
    - Several other providers are supported but may not be as well tested
    - Get help from [Ellmer Assistant](https://jcheng.shinyapps.io/ellmer-assistant/)
- Python:
  - [`openai`](https://platform.openai.com/docs/overview?lang=python), [`anthropic`](https://docs.anthropic.com/en/home) - from LLM providers, low-level but solid 
  - [`langchain`](https://python.langchain.com/docs/introduction/) - high-level, all models, sprawling scope... but polarizing architecture, steep learning curve, and supposedly questionable code quality
  - [`chatlas`](https://posit-dev.github.io/chatlas/) - high-level, basically a port of R's `ellmer`
  - Many, many other options are available

# Your Turn

## Instructions {.smaller}

Open and run one of these options:

- `01-basics.R`
  - or `01-basics-bedrock.R` for cloud
- `01-basics-openai.py` (low level library)
- `01-basics-langchain.py` (high level framework)
- `01-basics-chatlas.py` (high level framework, similar to R)
  - or `01-basics-chatlas-bedrock.py` for cloud

If it errors, now is the time to debug.

If it works, study the code and try to understand how it maps to the low-level HTTP descriptions we just went through.

## Summary

- A message is an object with a `role` ("system", "user", "assistant") and a `content` string
- A chat conversation is a growing list of messages
- The OpenAI chat API is a stateless HTTP endpoint: takes a list of messages as input, returns a new message as output

# Creating chatbot UIs

## Shiny for R

{`shinychat`} package\
  [https://github.com/posit-dev/shinychat](https://github.com/posit-dev/shinychat)

- Designed to be used with ellmer
- [Ellmer Assistant](https://jcheng.shinyapps.io/ellmer-assistant) is quite good for getting started

## Shiny for Python

[Creating chatbots](https://shiny.posit.co/py/docs/genai-chatbots.html) in Shiny for Python

- [Shiny Assistant](https://gallery.shinyapps.io/assistant/) on the web can't help you with `ui.Chat` for data privacy reasons, so instead...

## Shiny Assistant for VS Code and Positron

![](shiny-assistant-extension-1.png)

## Installation

![](shiny-assistant-extension-2.png)

## Shiny Assistant requirements

- In **VS Code**, requires Copilot subscription
- In **Positron**, requires:
  - Anthropic or OpenAI API key
  - Enable Positron Assistant. [Instructions](https://connect.posit.it/positron-wiki/positron-assistant.html#using-positron-assistant)

## {querychat}

![](https://github.com/posit-dev/querychat/raw/main/animation.gif)

[https://github.com/posit-dev/querychat](https://github.com/posit-dev/querychat) (R and Python)

## Other Python frameworks

- Streamlit has an excellent [chat component](https://docs.streamlit.io/develop/tutorials/llms/build-conversational-apps) with a [nice LangChain integration](https://docs.streamlit.io/develop/tutorials/llms/llm-quickstart)
- Gradio has a [chat component](https://www.gradio.app/guides/creating-a-chatbot-fast) that is extremely easy to use

# Tool Calling

## What is Tool Calling?

::: {.incremental}

- Allows LLMs to interact with other systems
- Sounds complicated? It isn't!
- Supported by most of the newest LLMs, but not all

:::

## How It Works

::: {.incremental}
- [Not like this](https://sequencediagram.org/index.html?presentationMode=readOnly#initialData=C4S2BsFMAIAkHsDu1j3uaBjAhucIA7Ac2gBEB5AUQGUA5QTAIAVaReAJwGsAoL7TVNtACqAZ0hsuAW2zBxIXABNIIkEQIhM8AiOgAxAIwA2AOwBBaKZEqRwbAWBcADtjahMIZ-ejxHkAokgZAAtxADp2Ih5RcQBaAD5La1t7AC5oAG8AHQJoaDZ0SDSAIgBXMTYigBps3M17P2BigHUgmQByHWAQlkCu8TzVIOBoAiRoGT0-RGwAT2gABRcOAH4q7IBfLkSQGztgeJ8p3pC2cLYiNIBxSmYAegDg8VvMErY2Btuy24AGACZf-QAVlCACsRFouId-McwhF4ttdqkMkVZJJHEU0oZAZVoEU6gowCAtCIMbiRCUCAQZkVNgjkvs4tE2Gksjk8gVitgrDt6Ws2XVZEiigBJYAdaBY6BKIjvZTjAgKaDkykzQAoBNhoAAjQIlUAAMxKGAUswAhEUNkA) - with the assistant executing stuff
- [Yes like this](https://sequencediagram.org/index.html?presentationMode=readOnly#initialData=C4S2BsFMAIAkHsDu1j3uaBjAhucIA7Ac2gBEB5AUQGVpF4AnAawCgWAHbB0TETg4NHjtIBRJGzAAFpAYA6RkRbZMqBtACqAZ1ksAtpNkhcAE0haQRAiEzwCW6ADEAjADYA7AEFonrRa3A2AJs2rIAtAB8vv6BAgBc0ADeADoE0NAM6JAJAEQArjoMOQA0qem2AqLAuQDqUpIA5A7SMOKSMuoMllKCBEjQkk6iiNgAntAAClxMAPwlZSho4FoJANpEkMAA+ph5DAxVW20tDAC6qQC+LNEgAUHAkaEMCQDCngAy79Ab27v7h8cOgAKHIABgATODnABWHIAShYT0iwmGEhOCgYRASAHFKAAVaAAekBskJfwOAkJBUJEKh0LkACstHYWCixGiOhiiI9CglEjlgJA9OwcglXNDitAchUTGAQHYtKKpVo8gQCKMclckVE-LdYtVoAAlfEaQ0AOSSAqFIrFEqlMrlCqVORVao1VxudwEPNkfIWmSguWwuq9wHmaSwdkF8SlAElgE1oOLoGYiAdzAMCCZoK71YAUAmw0AARhI8qAAGZ5DAmMYAQhylyAA)
  - User asks assistant a question; includes metadata for available tools
  - Assistant asks the user to invoke a tool, passing its desired arguments
  - User invokes the tool, and returns the output to the assistant
  - Assistant incorporates the tool's output as additional context for formulating a response
:::

## How It Works

Another way to think of it:

- The client can perform tasks that the assistant can't do
- Tools put control into the hands of the assistant—it decides when to use them, and what arguments to pass in, and what to do with the results
- Having an "intelligent-ish" coordinator of tools is a surprisingly general, powerful capability!

# Your Turn {.smaller}

Take a minute to look at _one of_ the following docs. See if you can get them to run, and try to understand the code.

- R: [`ellmer` docs](https://ellmer.tidyverse.org/articles/tool-calling.html) (anticlimactically easy), or example `02-tools.R` in `llm-quickstart` repo
- Python
  - `openai` example: `02-tools-openai.py` (tedious, low-level, but understandable)
  - `langchain` example: `02-tools-langchain.py` (not bad)
  - `chatlas` example: `02-tools-chatlas.py` (easy, like `ellmer`)

# Model Context Protocol {.smaller}

- A standardized way for tools to make themselves available to LLM apps, without writing more code
- RPC protocol, so tools can be written in any language
- The application that uses the tool is an MCP client
- MCP Servers provide the tools. [Examples](https://github.com/modelcontextprotocol/servers):
  - [Google Maps](https://github.com/modelcontextprotocol/servers/blob/main/src/google-maps)
  - [Filesystem](https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem) - access to files on disk
  - [Playwright MCP](https://github.com/microsoft/playwright-mcp) - control a web browser
- Clients use the tools. [Examples](https://modelcontextprotocol.io/clients): Claude Desktop app, Claude Code, Continue, Cursor, many others

# Choosing a model

- OpenAI
- Anthropic Claude
- Google Gemini
- Open weights models (can run locally)

## OpenAI models {.smaller}

- **GPT-4.1**: good general purpose model, 1 million token context length
  - **GPT-4.1-mini** and **GPT-4.1-nano** are faster, cheaper, and dumber versions
- **o3**: reasoning model; better for complex math and coding, but much slower ~~and more expensive~~
- **o4-mini**: faster and cheaper reasoning model, not as good as o3 but cheaper than GPT-4.1
- API access via [OpenAI](https://platform.openai.com/docs/overview/) or [Azure](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
- **Takeaway**: Good models for general purpose use
- [All OpenAI models](https://platform.openai.com/docs/models)

## Anthropic models {.smaller}

- **Claude Sonnet 4**: good general purpose model, best for code generation. Has thinking mode.
  - **Claude Sonnet 3.7** and **3.5** are both still excellent
- **Claude Opus 4**: even stronger than Sonnet 4 (supposedly), but more expensive and slower. Has thinking mode.
- **Claude Haiku 3.5**: Faster, cheaper, but not cheap enough
- API access via [Anthropic](https://docs.anthropic.com/) or [AWS Bedrock](https://aws.amazon.com/bedrock/claude/) ([instructions for using Bedrock at Posit](https://positpbc.atlassian.net/wiki/spaces/SE/pages/1376550915/Using+Claude+via+Bedrock+at+Posit))
- **Takeaway**: Best model for code generation
- [All Anthropic models](https://docs.anthropic.com/en/docs/about-claude/models/overview)

## Google models

- **Gemini 2.5 Pro**: 1 million token context length. Has thinking mode.
- **Gemini 2.5 Flash**: 1 million token context length, very fast. Has thinking mode.
- **Takeaway**: Competitive with OpenAI and Anthropic
- [All Gemini models](https://ai.google.dev/gemini-api/docs/models)

## Llama models {.smaller}

- Open weights: you can download the model
- Can run locally, for example with [Ollama](https://ollama.com/)
- **Llama 3.1 405b**: text, 229GB. Not quite as smart as best closed models.
- **Llama 3.2 90b**: text+vision, 55GB
- **Llama 3.2 11b**: text+vision, 7.9GB (can run comfortably on Macbook)
- API access via [OpenRouter](https://openrouter.ai/), [Groq](https://groq.com/), [AWS Bedrock](https://aws.amazon.com/bedrock/llama/), others
- **Takeaway**: OK models if you want to keep all information on premises.

## Other open weights model families

- [🇺🇸 Gemma 3](https://deepmind.google/models/gemma/) (Google)
- [🇨🇳 Qwen3](https://qwen.ai/home) (Alibaba)
- [🇨🇳 DeepSeek R1](https://www.deepseek.com/en)
- [🇫🇷 Mistral 3.2 Small](https://mistral.ai/models)
- [🇨🇳 Kimi K2](https://www.kimi.ai/) (MoonshotAI)
- [🇨🇳 GLM-4.5 Air](https://github.com/zai-org/GLM-4.5) (Z.ai)

# Customizing behavior and knowledge {.smaller}

- You want to customize how the LLM responds
- LLM doesn't know your specific information

**Some solutions**

  - Prompt engineering
  - Retrieval-Augmented Generation
  - Agentic search
  - Fine tuning

## Prompt engineering: Directing behavior

::: {.incremental}

- "Respond with just the minimal information necessary."
- "Think through this step-by-step."
- "If the user asks a data-related question, produce R code to accomplish that task."
- "Only provide answers that you are sure about. If you are uncertain about an answer, say so."

:::

## Prompt engineering: Using examples to guide behavior

- **Goal:** Extract ingredient list from recipe and return in a structured format.

- Example user input:

  ```
  In a large bowl, cream together 1 cup of softened unsalted butter and ½ cup of white sugar until smooth. Beat in 1 egg and 1 teaspoon of vanilla extract. Gradually stir in 2 cups of all-purpose flour until the dough forms. Finally, fold in 1 cup of semisweet chocolate chips. Drop spoonfuls of dough onto an ungreased baking sheet and bake at 350°F (175°C) for 10-12 minutes, or until the edges are lightly browned. Let the cookies cool on the baking sheet for a few minutes before transferring to a wire rack to cool completely. Enjoy!
  ```


## System prompt 1: just instructions

```
The user input contains a recipe. Extract a list of ingredients and return it in JSON format.
```

Assistant response:

```{.json}
{
  "ingredients": [
    "1 cup of softened unsalted butter",
    "½ cup of white sugar",
    "1 egg",
    "1 teaspoon of vanilla extract",
    "2 cups of all-purpose flour",
    "1 cup of semisweet chocolate chips"
  ]
}
```

## System prompt 2: detailed description of output

```
The user input contains a recipe. Extract a list of ingredients and return it in JSON format. It should be an array of objects, where each object has keys `ingredient`, `quantity`, and `unit`. Put each object on one line of output.
```

Assistant response:

```{.json}
[
    {"ingredient": "unsalted butter", "quantity": 1, "unit": "cup"},
    {"ingredient": "white sugar", "quantity": 1/2, "unit": "cup"},
    {"ingredient": "egg", "quantity": 1, "unit": "large"},
    {"ingredient": "vanilla extract", "quantity": 1, "unit": "teaspoon"},
    {"ingredient": "all-purpose flour", "quantity": 2, "unit": "cups"},
    {"ingredient": "semisweet chocolate chips", "quantity": 1, "unit": "cup"}
]
```

## System prompt 3: one-shot example

````
The user input contains a recipe. Extract a list of ingredients and return it in JSON format.

Example Output:

```json
[
    { "ingredient": "Flour", "quantity": 1, "unit": "cup" },
    { "ingredient": "Vegetable oil", "quantity": 0.5, "unit": "tsp" },
    { "ingredient": "Onion", "quantity": 1, "unit": null },
]
```
````

Assistant response:

```{.json}
[
    { "ingredient": "Unsalted butter", "quantity": 1, "unit": "cup" },
    { "ingredient": "White sugar", "quantity": 0.5, "unit": "cup" },
    { "ingredient": "Egg", "quantity": 1, "unit": null },
    { "ingredient": "Vanilla extract", "quantity": 1, "unit": "teaspoon" },
    { "ingredient": "All-purpose flour", "quantity": 2, "unit": "cups" },
    { "ingredient": "Semisweet chocolate chips", "quantity": 1, "unit": "cup" }
]
```

## Adding context/knowledge to prompt

::: {.incremental}

- Add documentation files to prompt
- Examples
  - [Ellmer assistant](https://github.com/jcheng5/ellmer-assistant/blob/main/prompt.generated.md) uses README files in prompt
  - [Querychat](https://github.com/posit-dev/querychat/blob/main/pkg-r/inst/prompt/prompt.md?plain=1)
  - [FastHTML LLM prompt](https://www.fastht.ml/docs/llms-ctx.txt)
- Docs must fit in context window
- Add positive examples (negative examples don't work well)

:::

## RAG: Retrieval Augmented Generation {.smaller}

- Useful when documents don't fit into context window
- Steps:
  - User sends query to app: _"How do I ...?"_
  - App **retrieves** relevant chunks of text via search
  - App sends text and query to LLM
    - _<chunk 1>, <chunk 2>, <chunk 3>. How do I ...?_
  - LLM responds with answer
- Search method typically a semantic instead of keyword search, using vector DB
- LLM will only know about chunks that were retrieved; does not "know" entire corpus
- In general, prompt stuffing works better, if docs fit in context window

## Agentic search

- Similar to RAG:
  - Extra information is provided to LLM
- Different from RAG:
  - Application does *not* search documents and send to LLM along with user prompt
  - User prompt is sent to LLM, then LLM uses a _tool_ to search for relevant documents

## Fine tuning

- Update weights for an existing model with new information
- Not all models can be fine-tuned
- Data must be provided in chat conversation format, with query and response
  - Can't just feed it documents -- this makes fine-tuning more difficult in practice
- Supposedly not very effective unless you have a lot of training data

## Takeaways

- **First try prompting, then RAG or agentic search, and then fine tuning.**
- Other resources
  - OpenAI's [prompt engineering guide](https://platform.openai.com/docs/guides/prompt-engineering)
  - Anthropic's [prompt engineering guide](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview)
  - [Fine-tuning vs. RAG article](https://finetunedb.com/blog/fine-tuning-vs-rag/)


# Getting structured output

## Going beyond chat

- **Structured output** can be easily consumed by code: JSON, YAML, CSV, etc.
- **Unstructured output** cannot: text, images, etc.

LLMs are good at generating unstructured output, but with a little effort, you can get structured output as well.

## Several techniques (choose one) {.smaller}

- **Post-processing:** Use a regular expression to extract structured data from the unstructured output (e.g. <code>/&grave;&grave;&grave;json\\n(.*?)\\n&grave;&grave;&grave;/</code>)
- **System prompt:** Simply ask the LLM to output structured data. Be clear about what specific format you want, and provide examples---it _really_ helps!
- **Structured Output:** GPT-4.1 and GPT-4.1-mini have a first-class Structured Output feature: outputs strictly adhere to a JSON schema you write. (Docs: [openai](https://platform.openai.com/docs/guides/structured-outputs), [LangChain](https://python.langchain.com/docs/how_to/structured_output/))
- **Tool calling:** Create a tool to receive your output, e.g., `set_result(object)`, where its implementation sets some variable. (Works great for ellmer.)
- **LangChain:** Has its own [abstractions](https://python.langchain.com/docs/how_to/#output-parsers) for parsing unstructured output

Ask <code>`r slack_channel`</code> for help if you're stuck! (Or ask ChatGPT/Claude to make an example.)

# Vision

## Using images as input

- Modern models are pretty good at this—but this frontier is _especially_ jagged
- Can understand both photographs and plots
- Examples for R and Chatlas are in your repo as `05-vision*`
- See docs for [LangChain multimodal](https://python.langchain.com/docs/how_to/multimodal_inputs/), [OpenAI vision](https://platform.openai.com/docs/guides/vision)

# Brainstorming

<!--
## Sample of past cohort projects {.smaller}

- **Photography:**
  - Evaluate photos for focus issues.
  - Auto-generate photo title, tags, and description.
  - Advanced OCR on specific document types.
- **Gaming:**
  - Dungeon master for D&D campaign.
  - Agent for navigating simple ASCII maze.
- **Data:**
  - Tool for auto-generating ALT tags for plots.
  - Tool for making realistic-looking mock datasets.
  - Code linter for R and PyData code.
  - Dataset exploring assistant.
- **API integrations:** GitHub, Posit Connect, Zendesk, Fusion360, Disney theme park wait times.
-->

<!--
## Past cohort projects

- [Cohort 1](https://docs.google.com/presentation/d/1wJKis5xUvce2PS2ZeUOp3h-L6Ul9NQ2CgNeZ9LsF5W8/edit?usp=sharing)
- [Cohort 2](https://docs.google.com/presentation/d/1j15NfbrRDfHIbnO7Jt3gS63CafArboe70AUJw4hFn8E/edit?usp=sharing)
- [Cohort 3](https://docs.google.com/presentation/d/1NmgeTgzs31r8g8-b2liOzKXDmYU_iXSIA8Ih8Z4ZzMM/edit?usp=sharing)
- [Cohort 4](https://docs.google.com/presentation/d/157q63jIrrTLHwO5SSN9RINs5FxN-9o1YcffKnefs_wg/edit#slide=id.p)
- [Cohort 5](https://docs.google.com/presentation/d/1taHrsRcjHcFI0bTXEZjP3fNs5beiDkaiKTcMHonWGfQ/edit#slide=id.p)
- [Cohort 6](https://docs.google.com/presentation/d/1e7oGe_I6yp69m111b5BjGnguo7AQRua6pXBS3WaP7FY/edit?usp=sharing)
- [Cohort 7](https://docs.google.com/presentation/d/1DZVC7d-ycbnwh6nbDyNZEGwdtPMD3FB4NoIf0dd915o/edit?usp=sharing)
- [Cohort 8](https://docs.google.com/presentation/d/1OMj8S4-nH-bM_1QZ-1aztZw0hc5gMuxO4vDKngrUYxQ/edit?usp=sharing)
- [Cohort 9](https://docs.google.com/presentation/d/1MUiSctRtiMF9N8tGLRvtbWrNl49Ur0ty8soX0NO1v_Y/edit?usp=sharing)
- [Cohort 10](https://docs.google.com/presentation/d/1ArrYMvHCIDrgfj4xFsUkBNE1T0kt2mCPUtLJ4kiPYIY/edit?usp=sharing)
- [Cohort 11](https://docs.google.com/presentation/d/1nPCWVG1jKgmxvXQsvWgkOlZsgWrHawD5Dp3qF2j0Rpg/edit?usp=sharing)
- [Cohort 12](https://docs.google.com/presentation/d/17RRsGauC7TfiRLCJh40VnEsawwv727eK5np9F0SzmN8/edit?usp=sharing)
- [Cohort 13](https://docs.google.com/presentation/d/1JnViUBV_AzkSU35vAOklg60OLPsL2BsAk8iy6H_lBbM/edit?usp=sharing)
- [Cohort 14](https://docs.google.com/presentation/d/1Svl7XZOHD_g21NZ0jX8hICmIKUHRyoCEirzDMDeh3UM/edit?usp=sharing)
- [Cohort 15](https://docs.google.com/presentation/d/1TfGWfH2ck9s5Y71XrLc-7hIn7m9evFTEtkVg-GEs6G8/edit?usp=sharing)
- [Cohort 16](https://docs.google.com/presentation/d/1ZR5klcr20WSuJEI2EmYIQan9DffUNMBMp_L6Ik0sBl0/edit?usp=sharing)

## Interesting tools that don't require coding

- [Google NotebookLM](https://notebooklm.google/)
- [Claude Projects](https://claude.ai/projects) (requires Claude subscription)
-->