Extending EngineeringPaper.xyz with Python
Extending EngineeringPaper.xyz with Python
A live version of this calculation is available at EngineeringPaper.xyz.
Introduction
EngineeringPaper.xyz is designed to make make it easy to build, document, and share powerful calculations without ever needing to write a line of code. However, you may run into situations where you would like to utilize some of Python’s powerful computational libraries, such as NumPy, SciPy, or scikit-learn, to augment your calculations. Since EP already uses the SymPy Python library to perform your calculations, it is natural to extend its capabilities by writing your own Python functions. EP uses the Pyodide project to run Python in your browser, so you don’t need to install anything in order to make use of the Python libraries and all of the calculations run locally on your computer. You can click the info icon ⓘ at the top right of the code cells below to see a list of all of the available Python modules and their versions. Code cells allow you to define custom numerical functions that accept one or more scalar and/or matrix inputs and output a scalar or matrix result. Additionally, you can have your custom Python function output text, HTML, or SVG images to create custom visualizations based on the values in your EP sheet. This code cell tutorial will demonstrate the capabilities of EP code cells through several examples.
Code cells are created by clicking the Insert Code Cell button </> at the bottom of the sheet (or between cells). When you first create a code cell, it will have a basic Python function called _calculate _with one input argument and a return value. There will also be a code cell function definition expression at the top of the code cell that will default to $\mathrm{CodeFunc1}\left(\left\lbrack any\right\rbrack\right)=\left\lbrack none\right\rbrack$. The function definition tells EP how many inputs your function will have and how to handle the units for the inputs and outputs of your function. $\left\lbrack any\right\rbrack$ indicates that the function accepts any units for the input argument. If the input argument needs to have specific units, they can be specified directly in the function definition. For example, use $\mathrm{CodeFunc1}\left(\left\lbrack m\right\rbrack\right)=\left\lbrack none\right\rbrack$ to indicate that the input units need to be a distance. If the input or output arguments must have no units, $\left\lbrack none\right\rbrack$ is used. If multiple inputs are required for a code function, their unit specifications are separated by commas as in $\mathrm{CodeFunc1}\left(\left\lbrack m\right\rbrack, \left\lbrack s\right\rbrack\right)=\left\lbrack m\cdot s\right\rbrack$, which defines a function where the first argument is distance, the second argument is time, and the result is distance times time.
You may notice in the code cells below that there is a check box labeled Use SymPy Mode. The default is to have this box unchecked, in this case the inputs to your function will be floats for scalar values and 2D NumPy arrays for matrix or vector values. Your function will return a number (int or float) for a scalar result or a NumPy array, a list, or a nested list for a vector or matrix result. In SymPy mode, your function will operate on SymPy expressions directly, as will be discussed in the examples below. For matrix or vector inputs and/or outputs, the units specified in the function definition will be applied to all elements of the matrix. You can also specify the units specification as a matrix to allow different values in the matrix to have different units as in $\mathrm{CodeFunc1}\left(\left\lbrack none\right\rbrack\right)=\begin{bmatrix}\left\lbrack m\right\rbrack & \left\lbrack s\right\rbrack\ \left\lbrack kg\right\rbrack & \left\lbrack none\right\rbrack\end{bmatrix}$. Instead of individually specifying the units of each element of the array, the units can be specified on a per row or per column basis. This example specifies the units for each column of the result matrix where the result matrix can have any number of rows: $\mathrm{CodeFunc1}\left(\left\lbrack none\right\rbrack\right)=\begin{bmatrix}\left\lbrack m\right\rbrack & \left\lbrack s\right\rbrack & \left\lbrack none\right\rbrack\end{bmatrix}$. This is useful when the number of rows or columns in the result is not known in advance or for cases with a large number of rows or columns.
These concepts, and more, will be illustrated through a series of examples. Each of the examples aims to use code cells to expand the capabilities of EP beyond what is currently possible to illustrate the types of situations in which you would consider extending EP with Python.
Example 1: Scalar Input and Output
In the first example, we implement the Gamma function, which is not included as a built-in function in EP. Python provides a Gamma function in the standard library through the math module. The first thing we do in our script is import the math module so that it can be used in our calculate function. For this case, the calculate function is about as simple as it can get, it just outputs the result of the Gamma function applied to the input value. See the code cell below for the implementation. Note that in the function definition, we can give the function any valid variable name. In this case, we name the function with the Greek letter for Gamma, $\Gamma$. Note that the name of the actual Python function is always _calculate _since this is the function that EP is looking for within each code cell. Below the code cell, we see the result of calling the Gamma function for an input of 10.
import math
def calculate(value):
return math.gamma(value)
$\mathrm{\Gamma}\left(10\right)= 3.6288\times 10^{5}$
Example 2: Scalar Input and Output in SymPy Mode
Whenever SymPy provides a function or capability, such as the Gamma function, it’s usually best to use SymPy’s implementation of that function or capability and to define our code cell function in SymPy mode. When using SymPy Mode, the values passed into our function are maintained as SymPy expressions or matrices and we return SymPy expressions or matrices as results. Internally, EP effectively uses SymPy mode for everything. This has many benefits including being able to work with symbolic expressions in addition to numeric expressions. Additionally, SymPy uses arbitrary precision math when converting expressions into numbers, so more precision is maintained in calculations performed in SymPy mode. The following code cell reimplements the Gamma function in SymPy mode using SymPy’s built in Gamma function implementation:
from sympy import gamma
def calculate(value):
return gamma(value)
$\mathrm{\Gamma^{\prime}}\left(10\right)= 3.6288\times 10^{5}$
One benefit of this approach is that the Gamma function is now able to handle larger input values without causing an error. For example, the following Gamma function call will cause an error if used with the non-SymPy version:
$\mathrm{\Gamma^{\prime}}\left(10000\right)= 2.84625968091705\times 10^{35655}$
In addition to implementing the SymPy versions of functions, Sympy Mode also opens up SymPy’s powerful tools for symbolic expression manipulation. For example, polynomial factoring can be performed with the code cell function below. Note that the Automatically Simplify Symbolic Expressions sheet setting needs to be disabled for such manipulations to not be lost by EP’s default simplification strategy. Additionally, note that we need to import SymPy in every code cell where SymPy is needed since each code cell runs in its own isolated namespace. However, because of how Python imports modules, there is no performance impact to this since Python only ever imports a module once and keeps a reference to it for any future imports of the same module.
import sympy
def calculate(value):
return sympy.factor(value)
$x^2+5\cdot x+6= x^{2} + 5 \cdot x + 6$
$\mathrm{factor}\left(x^2+5\cdot x+6\right)= \left(x + 2\right) \cdot \left(x + 3\right)$
Example 3: Inputs and Output with Units
Now let’s explore the case where our inputs and outputs have units. Another function that is not included in EP by default is atan2, which takes two inputs and returns an angle (note that EP currently provides $angle\left(x+y\cdot i\right)$ which is equivalent to $atan2\left(y,x\right)$). If we would like to provide $x$ and $y$ in length units, we define the code cell function below. Note that since there are two inputs, two units specifications need to be provided to the function, both as $\left\lbrack m\right\rbrack$. The output units are set to $\left\lbrack rad\right\rbrack$ to force the results to have angle units. Not that, in general, it’s best to use SI units for inputs and outputs for code cell functions since the user’s values will be converted to whatever units are specified in the code cell function definition and the output will be converted from whatever units are specified in the function definition ($\left\lbrack rad\right\rbrack$ in this case) to the user’s units. In general, it’s best to do your calculations directly in SI units and let EP convert the user’s values to SI for the calculation and to convert the result back to the user’s units after the calculation. Since SymPy has a built-in atan2 implementation, this code cell is also be defined in SymPy mode.
import sympy
def calculate(y, x):
return sympy.atan2(y, x)
$\mathrm{atan2}\left(\frac{\sqrt3}{2}\cdot1\left\lbrack m\right\rbrack,\frac12\cdot1\left\lbrack m\right\rbrack\right)=\left\lbrack deg\right\rbrack =60 \left\lbrack deg\right\rbrack$
Example 4: Defining a Custom Units Function
One obvious limitation of the atan2 implementation above is that it it only allows distance units for the $x$ and $y$ inputs. However, it should work for any set of consistent units, or inputs without any units at all. We could specify both of the input units as $\left\lbrack any\right\rbrack$, and it will work with either distance units, or no units for the inputs. However, one limitation of this approach is that it would also work if $x$ has distance units and $y$ has time units. Cases like this, where more than one set of units needs to be handled by the same code cell function, can be handled by defining a custom_dims function that takes as many input arguments as the _calculate _function and returns the result dimensions. Using a custom_dims function requires that the unit specification of all of the inputs and outputs be set to $\left\lbrack any\right\rbrack$. Note that two utility functions are provided to the code cell environment, ensure_all_equivalent and ensure_all_unitless. Both of these functions take any number of inputs and return the units of the first argument if the unit check passes, either all equivalent or all unitless. If the unit check fails, a TypeError exception is raised, which a signal to the EP units system that there is a dimension error for the calculation. Additionally, note that the custom dimensions function makes use of the dimension_definitions object from sympy.physics.units.definitions to access the angle dimension. dimension_definitions also includes the mass, length, time, current, temperature, luminous_intensity, amount_of_substance, angle, and information dimensions which can be multiplied, divided, and raised to powers to construct whatever dimensions are needed for the output of your custom_dims function.
from sympy.physics.units.definitions import dimension_definitions
import sympy
def calculate(y, x):
return sympy.atan2(y, x)
def custom_dims(y_dims, x_dims):
ensure_all_equivalent(y_dims, x_dims)
return dimension_definitions.angle
$\mathrm{atan2^{\prime}}\left(\frac{\sqrt3}{2},\frac12\right)=\left\lbrack deg\right\rbrack =60 \left\lbrack deg\right\rbrack$
The following example implements a product function that works for any units by defining a custom_dims function that multiplies the dimensions of all of the input arguments. This example also shows how to handle the case of a variable number of inputs to code cell functions by placing a asterisk in front of the input parameter name, which represents a variable number of inputs as a list. This is the standard syntax for a variable number of input arguments in Python.
import sympy as sp
def calculate(*values):
return sp.Mul(*values)
def custom_dims(*dims):
return sp.Mul(*dims)
$\mathrm{prod}\left(1\left\lbrack s\right\rbrack,:2\left\lbrack m\right\rbrack,:3\left\lbrack kg\right\rbrack\right)= 6 \left\lbrack kg\cdot m\cdot s\right\rbrack$
Occasionally, the actual numerical values of the inputs and/or outputs to the calculate function are needed by the custom_dims function. To facilitate this, the custom_dims function can accept a final input argument called dim_values, which is a dictionary with two entries. dim_values[“args”] is a list containing the input argument numerical values and dim_values[“result”] is the numerical result value. This is an advanced feature and is not needed in most situations.
Example 5: Accepting and Returning Vectors and Matrices
Python code cell functions are designed to transparently handle passing in and/or returning vectors and matrices. When not in SymPy Mode, matrices are represented as 2D NumPy arrays. When in SymPy mode, SymPy Matrix objects are used to represent vectors and matrices.
Let’s first look at an example of a code cell function that returns a matrix. The following function creates a random matrix without using SymPy mode. Note that when calling NumPy’s rand function, the number of rows and columns need to be integer values. When not in SymPy mode, all of the input arguments will be float or NumPy float arrays. Python’s int function is used to convert the rows and cols values to integers as required by NumPy’s rand function.
import numpy as np
def calculate(rows, cols):
return np.random.rand(int(rows), int(cols))
$\mathrm{rand}\left(2,3\right)= \begin{bmatrix} 0.0153523353792465 & 0.67149744717596 & 0.312647353883221 \ 0.248132261289886 & 0.317296447991159 & 0.499470809930004 \end{bmatrix}$
A similar function can be made using SymPy mode. In SymPy mode, the input arguments are not converted to float so the int function does not need to be used:
import sympy as sp
def calculate(rows, cols):
return sp.randMatrix(rows, cols)
$\mathrm{rand^{\prime}}\left(2,3\right)= \begin{bmatrix} 42 & 97 & 44 \ 87 & 63 & 30 \end{bmatrix}$
Code cell functions can also accept one or more matrices as inputs. For example, let’s create a function that returns the mean, median, and mode of an input vector. Note that if a list or tuple is returned, it will be automatically converted into a column vector. Additionally, note the use of Python’s _print _function within the _calculate _function. The printed values are displayed below code cell and can be useful when debugging a code cell.
import statistics
def calculate(vector):
print(vector)
mean = statistics.mean(*vector)
median = statistics.median(*vector)
mode = statistics.mode(*vector)
return mean, median, mode
$\mathrm{stats}\left(\begin{bmatrix}1 & 1 & 2 & 5 & 6 & 9\end{bmatrix}\right)= \begin{bmatrix} 4 \ 3.5 \ 1 \end{bmatrix}$
Example 6: Using Code Cell Functions to Display Custom Text and Graphics
A powerful feature of code cells is the ability to display results as plain text, HTML, or markdown. HTML is a very powerful option since it can include text formatting, such as colors, and images and graphics, such as SVG vector images. To make use of this functionality, you need to tell EP what format your result is in by defining the output as $\left\lbrack text\right\rbrack$, $\left\lbrack html\right\rbrack$, or $\left\lbrack markdown\right\rbrack$ and your calculate function needs to return a string the defines the desired output. The following example uses the $\left\lbrack text\right\rbrack$ output type:
def calculate(value, threshold):
if value > threshold:
return f"\n🚫Fail: Stress of {value} MPa exceeds the allowable stress of {threshold} MPa"
else:
return f"\n✅Pass: Stress of {value} MPa is below the allowable stress of {threshold} MPa"
✅Pass: Stress of 10.0 MPa is below the allowable stress of 20.0 MPa
The following example uses the $\left\lbrack html\right\rbrack$ output type which enables the definition of more expressive rendering results and also allows the inclusion of figures and images. Note that, for security reasons, the HTML content is sanitized before rendering using the DOMPurify library which strips out any script or style tags, amongst other things, to prevent the rendered HTML from impacting the rest of the page.
def calculate(value, threshold):
fail_text_style = '"color: red; font-weight: bold;"'
fail_div_style ='"border: 3px solid red; border-radius: 5px; width: fit-content; padding: 10px"'
pass_text_style = '"color: green; font-weight: bold;"'
pass_div_style ='"border: 3px solid green; border-radius: 5px; width: fit-content; padding: 10px"'
if value > threshold:
return f"""
<h4>Result Summary</h4>
<div style={fail_div_style}>
🚫<span style={fail_text_style}>Fail:</span> Stress of
<span style={fail_text_style}>{value} MPa</span> exceeds allowable stress of {threshold} MPa
</div>
"""
else:
return f"""
<h4>Result Summary</h4>
<div style={pass_div_style}>
✅<span style={pass_text_style}>Pass:</span> Stress of
<span style={pass_text_style}>{value} MPa</span> is below the allowable stress of {threshold} MPa
</div>
"""
Result Summary
Result Summary
The ability to render to HTML opens up many possibilities, including the ability to include images and graphics. In particular, the ability to include SVG vector images can be used to create figures that are driven by values in the sheet. The following example uses the drawsvg library to draw the work envelope for a SCARA robot based on its arm lengths and joint angle ranges:
import drawsvg as dw
import numpy as np
from numpy import pi
def calculate(lengths, min_angles, max_angles):
l1, l2 = lengths[:,0]
theta1_min, theta2_min = min_angles[:,0]
theta1_max, theta2_max = max_angles[:,0]
d = dw.Drawing(2.2*(l1+l2), 2.2*(l1+l2), origin='center')
# assumes symmetric range for outer arm and that outer arm is shorter than the inner arm
inner_radius = np.sqrt(l1**2+l2**2-2*l1*l2*np.cos((180-theta2_max)*pi/180))
alpha = np.asin((l2*np.sin((180-theta2_max)*pi/180))/inner_radius)*180/pi + \
theta1_max
path = dw.Path(stroke='royalblue', fill='lightblue', fill_rule='nonzero')
path.arc(0, 0, l1+l2, theta1_min, theta1_max)
path.arc(l1*np.cos(theta1_min*pi/180), -l1*np.sin(theta1_min*pi/180),
l2, theta1_max, theta1_max+theta2_max, include_m=False)
path.arc(0, 0, inner_radius, alpha, -alpha, cw=False, include_m=False)
path.arc(l1*np.cos(theta1_min*pi/180), l1*np.sin(theta1_min*pi/180),
l2, theta1_min+theta2_min, theta1_min, include_m=False)
group = dw.Group(transform='scale(1,-1)')
group.append(path)
group.append(dw.Rectangle(-.2*l1,-.2*l1, l1+.4*l1,l1*.4, rx=.2*l1, ry=.2*l1, fill='grey', stroke='black'))
group.append(dw.Circle(0, 0,.05*l1))
group.append(dw.Rectangle(-.1*l1+l1,-.1*l1, l2+.3*l2, l1*.2, rx=.1*l1, ry=.1*l1, fill='grey', stroke='black'))
group.append(dw.Circle(l1, 0,.05*l1))
group.append(dw.Circle(l1+l2, 0,.05*l1))
group1 = dw.Group(transform=f'rotate({theta1_max})', opacity=0.3)
group1.append(dw.Rectangle(-.2*l1,-.2*l1, l1+.4*l1,l1*.4, rx=.2*l1, ry=.2*l1, fill='grey', stroke='black'))
group1.append(dw.Circle(0, 0,.05*l1))
group1.append(dw.Rectangle(-.1*l1+l1,-.1*l1, l2+.3*l2, l1*.2, rx=.1*l1, ry=.1*l1, fill='grey', stroke='black'))
group1.append(dw.Circle(l1, 0,.05*l1))
group1.append(dw.Circle(l1+l2, 0,.05*l1))
group2 = dw.Group(transform=f'rotate({theta2_max}, {l1}, 0)')
group2.append(dw.Rectangle(-.1*l1+l1,-.1*l1, l2+.3*l2, l1*.2, rx=.1*l1, ry=.1*l1, fill='grey', stroke='black'))
group2.append(dw.Circle(l1, 0,.05*l1))
group2.append(dw.Circle(l1+l2, 0,.05*l1))
group1.append(group2)
group.append(group1)
d.append(group)
return f"<h4>SCARA Robot Work Envelope</h4>{d.as_svg()}"
SCARA Robot Work Envelope
Example 7: Creating Matplotlib Plots
There are two ways to include a Matplotlib plots. Both cases require a few extra steps to convert the plot into HTML. The first example renders the plot as a PNG image. Both examples create a polar plot since EP doesn’t natively support polar plots. Note that it is required to set the Matplotlib rendering backend to “AGG” or “SVG” using the matplotlib.use() function call. Additionally, it’s important to call plot.close(fig) before your function exits to avoid excessive memory usage since Matplotlib doesn’t free up the memory for these plots automatically.
import matplotlib.pyplot as plt
import matplotlib
matplotlib.use("AGG") # must choose an image backend to avoid error
import numpy as np
import io
import base64
def calculate(num_turns):
r = np.arange(0, num_turns, num_turns/200)
theta = 2 * np.pi * r
# Create a figure
fig, ax = plt.subplots(subplot_kw={'projection': 'polar'})
ax.plot(theta, r)
ax.set_rmax(2)
ax.set_rticks([0.5, 1, 1.5, 2]) # Less radial ticks
ax.set_rlabel_position(-22.5) # Move radial labels away from plotted line
ax.grid(True)
ax.set_title("A line plot on a polar axis", va='bottom')
# Save to a BytesIO buffer
buf = io.BytesIO()
fig.savefig(buf, format='png')
buf.seek(0)
# Encode to base64 (required to embed into HTML)
img_base64 = base64.b64encode(buf.read()).decode('utf-8')
# Embed in HTML
html_img = f'<img src="data:image/png;base64,{img_base64}"/>'
# Important to call plt.close to avoid a memory leak
plt.close(fig)
return html_img
Finally, Matplotlib figures can be also be rendered as SVG vector images. However, instead of using the HTML option, this time we’re going to use the Markdown output option. In browsers, there’s no difference between the HTML and the Markdown approach. The reason to use the Markdown approach for images is that they will appear in exported Word docx and PDF files when rendered as Markdown but will not appear when rendered as HTML. To display an SVG using Markdown, it needs to be converted to a Base64 data URI (similar to what as done for the PNG image in the above example). In the code cell below, a utility function is added for this purpose, illustrating how additional functions may be added to help organize your code.
import matplotlib.pyplot as plt
import matplotlib
matplotlib.use("AGG") # must choose an image backend to avoid error
import numpy as np
import io
import base64
def svg_to_base64_data_uri(svg_string: str) -> str:
encoded_bytes = base64.b64encode(svg_string.encode('utf-8'))
encoded_str = encoded_bytes.decode('ascii')
return f"data:image/svg+xml;base64,{encoded_str}"
def calculate(num_turns):
r = np.arange(0, num_turns, num_turns/200)
theta = 2 * np.pi * r
# Create a figure
fig, ax = plt.subplots(subplot_kw={'projection': 'polar'})
ax.plot(theta, r)
ax.set_rmax(2)
ax.set_rticks([0.5, 1, 1.5, 2]) # Less radial ticks
ax.set_rlabel_position(-22.5) # Move radial labels away from plotted line
ax.grid(True)
ax.set_title("A line plot on a polar axis", va='bottom')
# Save to an in-memory string buffer as SVG
svg_buffer = io.StringIO()
fig.savefig(svg_buffer, format='svg')
svg_string = svg_buffer.getvalue()
svg_buffer.close()
plt.close(fig)
return f"})"
