Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions generate_book_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def main():
"children": [
{
"file": "tutorials/Schedule/schedule_intro.md",
"short_title": "Overview",
"title": "Overview",
"children": [
{"file": "tutorials/Schedule/daily_schedules.md"},
{"file": "tutorials/Schedule/shared_calendars.md"},
Expand All @@ -62,11 +62,11 @@ def main():
"children": [
{
"file": "tutorials/TechnicalHelp/tech_intro.md",
"short_title": "Overview",
"title": "Overview",
"children": [
{
"file": "tutorials/TechnicalHelp/Jupyterbook.md",
"short_title": "Using Jupyterbook",
"title": "Using Jupyterbook",
"children": [
{"file": "tutorials/TechnicalHelp/Tutorial_colab.md"},
{"file": "tutorials/TechnicalHelp/Tutorial_kaggle.md"},
Expand Down Expand Up @@ -235,7 +235,7 @@ def convert_sections_to_children(entries):
return result


# ---- Pre-processing helpers (ported verbatim from nmaci generate_book.py) ----
# ---- Pre-processing helpers (ported verbatim from nmaci generate_book.py) ----


def pre_process_notebook(file_path):
Expand Down
14 changes: 4 additions & 10 deletions parse_build_for_errors_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
└── children[K] (type='output')
jupyter_data: {output_type: 'error', ename: 'NotImplementedError', ...}

We walk every .json file, find 'output' nodes whose jupyter_data.ename matches
our error list, remove them from their parent 'outputs' node, and also remove
We walk every .json file, find 'output' nodes whose jupyter_data.output_type
is 'error', remove them from their parent 'outputs' node, and also remove
the 'outputs' node entirely if it becomes empty.

Run as: python parse_html_for_errors_v2.py student
Expand All @@ -27,8 +27,6 @@

sys.argv[1] # "student" or "instructor" — accepted but not used (kept for compat)

ERROR_NAMES = {"NotImplementedError", "NameError"}

HTML_ROOT = "book/_build/html"


Expand Down Expand Up @@ -80,7 +78,7 @@ def strip_error_outputs(node):
"""Recursively walk the mdast tree and remove error output nodes.

Targets 'outputs' nodes (type='outputs') that contain one or more
'output' children with jupyter_data.ename in ERROR_NAMES.
'output' children with jupyter_data.output_type == 'error'.

Returns count of individual error output nodes removed.
"""
Expand Down Expand Up @@ -125,11 +123,7 @@ def filter_error_outputs(outputs_node):
new_children.append(child)
continue
jd = child.get("jupyter_data", {})
if (
isinstance(jd, dict)
and jd.get("output_type") == "error"
and jd.get("ename") in ERROR_NAMES
):
if jd.get("output_type") == "error":
removed += 1
else:
new_children.append(child)
Expand Down
8 changes: 4 additions & 4 deletions tutorials/W2D3_Microlearning/W2D3_Tutorial1.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -721,11 +721,11 @@
"\\newcommand{\\weight}{\\mathbf{W}}\n",
"\\newcommand{\\loss}{\\mathcal{L}}\n",
"\\newcommand{\\derivative}[2]{\\frac{d#1}{d#2}}\n",
"\\newcommand{\\pderivative}[2]{\\frac{\\partial#1}{\\partial#2}}\n",
"\\newcommand{\\pderivative}[2]{\\frac{\\partial#1}{\\partial #2}}\n",
"\\newcommand{\\rate}{\\mathbf{r}}\n",
"\\newcommand{\\T}{^{\\top}}\n",
"\\newcommand{\\RR}{\\mathbb{R}}\n",
"\\newcommand{\\EE}{\\mathbb{E}\\,}\n",
"\\newcommand{\\EE}{\\mathbb{E}}\n",
"\\newcommand{\\brackets}[1]{\\left(#1\\right)}\n",
"\\newcommand{\\sqbrackets}[1]{\\left[#1\\right]}\n",
"\\newcommand{\\var}[1]{\\mathbb{V}\\mathrm{ar}\\brackets{#1}}$\n",
Expand Down Expand Up @@ -1229,14 +1229,14 @@
"$\\newcommand{\\rate}{\\mathbf{r}}$\n",
"$\\newcommand{\\T}{^{\\top}}$\n",
"$\\newcommand{\\RR}{\\mathbb{R}}$\n",
"$\\newcommand{\\EE}{\\mathbb{E}\\,}$\n",
"$\\newcommand{\\EE}{\\mathbb{E}}$\n",
"$\\newcommand{\\brackets}[1]{\\left(#1\\right)}$\n",
"$\\newcommand{\\sqbrackets}[1]{\\left[#1\\right]}$\n",
"$\\newcommand{\\var}[1]{\\mathbb{V}\\mathrm{ar}\\brackets{#1}}$\n",
"\n",
"The main issue of perturbation methods is noise, meaning that across many samples of input stimuli and network perturbations, the gradient estimates will be much more variable than would be the case for backpropagation. This means that many, many more perturbations/training samples will be required to obtain an accurate gradient estimate: the consequence will be either very slow or much less effective learning. \n",
"\n",
"Here, we will demonstrate the noisiness of these learning algorithms analytically for a simplified loss and network. This derivation is principally to satisfy your curiosity: no subsequent exercises will depend on your understanding of the mathematics here, and we will subsequently provide empirical evidence based on network simulations as well. First, we will work with a linear network so $\\widehat\\targetdim =\\weight\\stim$, where $\\widehat\\targetdim\\in\\RR^M$, $\\weight\\in\\RR^{M\\times N}$ and $\\stim\\in\\RR^N$. Second, we will assume that the target output is zero $\\targetdim=0$, so the loss becomes $\\loss(\\weight)=\\frac{1}{2}\\|\\weight\\stim\\|^2_2$. (This is equivalent to saying that $\\targetdim=\\weight^*\\stim$ and then shifting the actual weights to be $\\weight - \\weight^*$; notice that here we treat the loss as a function of $\\weight$, rather than $\\Delta \\weight$.)\n",
"Here, we will demonstrate the noisiness of these learning algorithms analytically for a simplified loss and network. This derivation is principally to satisfy your curiosity: no subsequent exercises will depend on your understanding of the mathematics here, and we will subsequently provide empirical evidence based on network simulations as well. First, we will work with a linear network so $\\mathbf{\\widehat{y}} =\\weight\\stim$, where $\\mathbf{\\widehat{y}}\\in\\RR^M$, $\\weight\\in\\RR^{M\\times N}$ and $\\stim\\in\\RR^N$. Second, we will assume that the target output is zero $\\targetdim=0$, so the loss becomes $\\loss(\\weight)=\\frac{1}{2}\\|\\weight\\stim\\|^2_2$. (This is equivalent to saying that $\\targetdim=\\weight^*\\stim$ and then shifting the actual weights to be $\\weight - \\weight^*$; notice that here we treat the loss as a function of $\\weight$, rather than $\\Delta \\weight$.)\n",
"\n",
"\n",
"With these changes, we will compute the variance of weight updates for a given input $\\stim$, i.e.\n",
Expand Down
8 changes: 4 additions & 4 deletions tutorials/W2D3_Microlearning/student/W2D3_Tutorial1.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -725,7 +725,7 @@
"\\newcommand{\\rate}{\\mathbf{r}}\n",
"\\newcommand{\\T}{^{\\top}}\n",
"\\newcommand{\\RR}{\\mathbb{R}}\n",
"\\newcommand{\\EE}{\\mathbb{E}\\,}\n",
"\\newcommand{\\EE}{\\mathbb{E}}\n",
"\\newcommand{\\brackets}[1]{\\left(#1\\right)}\n",
"\\newcommand{\\sqbrackets}[1]{\\left[#1\\right]}\n",
"\\newcommand{\\var}[1]{\\mathbb{V}\\mathrm{ar}\\brackets{#1}}$\n",
Expand Down Expand Up @@ -1205,14 +1205,14 @@
"$\\newcommand{\\rate}{\\mathbf{r}}$\n",
"$\\newcommand{\\T}{^{\\top}}$\n",
"$\\newcommand{\\RR}{\\mathbb{R}}$\n",
"$\\newcommand{\\EE}{\\mathbb{E}\\,}$\n",
"$\\newcommand{\\EE}{\\mathbb{E}}$\n",
"$\\newcommand{\\brackets}[1]{\\left(#1\\right)}$\n",
"$\\newcommand{\\sqbrackets}[1]{\\left[#1\\right]}$\n",
"$\\newcommand{\\var}[1]{\\mathbb{V}\\mathrm{ar}\\brackets{#1}}$\n",
"\n",
"The main issue of perturbation methods is noise, meaning that across many samples of input stimuli and network perturbations, the gradient estimates will be much more variable than would be the case for backpropagation. This means that many, many more perturbations/training samples will be required to obtain an accurate gradient estimate: the consequence will be either very slow or much less effective learning. \n",
"\n",
"Here, we will demonstrate the noisiness of these learning algorithms analytically for a simplified loss and network. This derivation is principally to satisfy your curiosity: no subsequent exercises will depend on your understanding of the mathematics here, and we will subsequently provide empirical evidence based on network simulations as well. First, we will work with a linear network so $\\widehat\\targetdim =\\weight\\stim$, where $\\widehat\\targetdim\\in\\RR^M$, $\\weight\\in\\RR^{M\\times N}$ and $\\stim\\in\\RR^N$. Second, we will assume that the target output is zero $\\targetdim=0$, so the loss becomes $\\loss(\\weight)=\\frac{1}{2}\\|\\weight\\stim\\|^2_2$. (This is equivalent to saying that $\\targetdim=\\weight^*\\stim$ and then shifting the actual weights to be $\\weight - \\weight^*$; notice that here we treat the loss as a function of $\\weight$, rather than $\\Delta \\weight$.)\n",
"Here, we will demonstrate the noisiness of these learning algorithms analytically for a simplified loss and network. This derivation is principally to satisfy your curiosity: no subsequent exercises will depend on your understanding of the mathematics here, and we will subsequently provide empirical evidence based on network simulations as well. First, we will work with a linear network so $\\widehat{\\targetdim} =\\weight\\stim$, where $\\widehat{\\targetdim}\\in\\RR^M$, $\\weight\\in\\RR^{M\\times N}$ and $\\stim\\in\\RR^N$. Second, we will assume that the target output is zero $\\targetdim=0$, so the loss becomes $\\loss(\\weight)=\\frac{1}{2}\\|\\weight\\stim\\|^2_2$. (This is equivalent to saying that $\\targetdim=\\weight^*\\stim$ and then shifting the actual weights to be $\\weight - \\weight^*$; notice that here we treat the loss as a function of $\\weight$, rather than $\\Delta \\weight$.)\n",
"\n",
"\n",
"With these changes, we will compute the variance of weight updates for a given input $\\stim$, i.e.\n",
Expand Down Expand Up @@ -2183,4 +2183,4 @@
},
"nbformat": 4,
"nbformat_minor": 4
}
}