Engineering Math

Gradient

Gradient

The gradient \(\grad\) of a scalar-valued function \(f: \mathbb{R}^3 \rightarrow \mathbb{R}\) is a vector field \(\bm{F}:\mathbb{R}^3 \rightarrow \mathbb{R}^3\); that is, \(\grad f\) is a vector-valued function on \(\mathbb{R}^3\). The gradient’s local direction and magnitude are those of the local maximum rate of increase of \(f\). This makes it useful in optimization (e.g., in the method of gradient descent).

In classical mechanics, quantum mechanics, relativity, string theory, thermodynamics, and continuum mechanics (and elsewhere) the principle of least action is taken as fundamental (Feynman, Leighton, and Sands 2010). This principle tells us that nature’s laws quite frequently seem to be derivable by assuming a certain quantity—called action—is minimized. Considering, then, that the gradient supplies us with a tool for optimizing functions, it is unsurprising that the gradient enters into the equations of motion of many physical quantities.

The gradient is coordinate-independent, but its coordinate-free definitions don’t add much to our intuition.

Vector fields from gradients are special

Although all gradients are vector fields, not all vector fields are gradients. That is, given a vector field \(\bm{F}\), it may or may not be equal to the gradient of any scalar-valued function \(f\). Let’s say of a vector field that is a gradient that it has gradience.1 Those vector fields that are gradients have special properties. Surprisingly, those properties are connected to path independence and curl. It can be shown that iff a field is a gradient, line integrals of the field are path independent. That is, for a vector field, \[\begin{align} \text{gradience} &\Leftrightarrow \text{path independence}. \end{align}\]

Considering what we know from section 5.2 about path independence we can expand fig. ¿fig:curl-independence-circulation-connectedness? to obtain fig. ¿fig:curl-independence-circulation-connectedness2?.

 Figure 5.10
Figure 5.10: An implication graph relating gradience, zero curl, zero circulation, path independence, and connectedness. Green edges represent implication (a implies b) and black edges represent logical conjunctions.

One implication is that gradients have zero curl! Many important fields that describe physical interactions (e.g., static electric fields, Newtonian gravitational fields) are gradients of scalar fields called potentials.

Exploring gradient

Gradient is perhaps best explored by considering it for a scalar field on \(\mathbb{R}^2\). Such a field in cartesian coordinates \(f(x,y)\) has gradient \[\begin{align} \grad f &= \begin{bmatrix} \partial_x f & \partial_y f \end{bmatrix}^\top \end{align}\] That is, \(\grad f = \bm{F} = \partial_x f\ \bm{\hat{i}} + \partial_y f\ \bm{\hat{j}}\). If we overlay a quiver plot of \(\bm{F}\) over a “color density” plot representing the \(f\), we can increase our intuition about the gradient.

First, load some Python packages.

import numpy as np
import sympy as sp
import matplotlib.pyplot as plt
from matplotlib.ticker import LogLocator
from matplotlib.colors import *

Now we define some symbolic variables and functions.

x, y = sp.symbols('x y', real=True)

Rather than repeat code, let’s write a single function grad_plotter_2D() to make several of these plots.

def grad_plotter_2D(
  field=x*y, grid_width=3, grid_decimate_x=8, grid_decimate_y=8,
  norm=None, # Density plot normalization
  scale=None, # Arrow length scale (auto)
  print_vector=True, mask=False, # Mask vector lengths
):
  # Define symbolics
  x, y = sp.symbols('x y', real=True)
  field = sp.sympify(field)
  # Compute vector field
  F_x = field.diff(x).simplify()
  F_y = field.diff(y).simplify()
  if field.is_constant():
    print('Warning: field is constant (no plot)')
  if print_vector:
    print(f'The gradient is:')
    print(sp.Array([F_x, F_y]))
  # Lambdify for numerics
  F_x_fun = sp.lambdify((x, y), F_x, 'numpy')
  F_y_fun = sp.lambdify((x, y), F_y, 'numpy')
  if F_x.is_constant:
    F_x_fun1 = F_x_fun # Dummy
    F_x_fun = lambda x, y: F_x_fun1(x, y) * np.ones(x.shape)
  if F_y.is_constant:
    F_y_fun1 = F_y_fun # Dummy
    F_y_fun = lambda x, y: F_y_fun1(x, y) * np.ones(x.shape)
  if not field.is_constant():
    den_fun = sp.lambdify((x, y), field, 'numpy') 
  # Create grid
  w = grid_width
  Y, X = np.mgrid[-w:w:100j, -w:w:100j]
  # Evaluate numerically
  F_x_num = F_x_fun(X, Y)
  F_y_num = F_y_fun(X, Y)
  if not field.is_constant():
    den_num = den_fun(X, Y)
  # Mask F_x and F_y
  if mask:
    masking_a = np.sqrt(np.square(F_x_num) + np.square(F_y_num))
    F_x_num = np.ma.masked_where(masking_a > w / 5., F_x_num)
    F_y_num = np.ma.masked_where(masking_a > w / 5., F_y_num)
  # Plot
  if not field.is_constant():
    fig, ax = plt.subplots()
    cmap = plt.get_cmap('coolwarm')
    im = plt.pcolormesh(X, Y, den_num, cmap=cmap, norm=norm)
    plt.colorbar()
    dx = grid_decimate_y
    dy = grid_decimate_x
    plt.quiver(
      X[::dx, ::dy], Y[::dx, ::dy], 
      F_x_num[::dx, ::dy], F_y_num[::dx, ::dy], 
      units='xy', scale=scale
    )
    plt.title(f'$f(x,y) = {sp.latex(field)}$')
    return fig, ax
  return 1, 1

Let’s inspect several cases. While considering the following plots, remember that they all have zero curl!

fig, ax = grad_plotter_2D(field=x)
plt.draw()
 Figure 5.11
Figure 5.11: Gradient of f(x,y) = x
fig, ax = grad_plotter_2D(field=x+y)
plt.draw()
 Figure 5.12
Figure 5.12: Gradient of f(x,y) = x + y
fig, ax = grad_plotter_2D(field=1)

Gravitational potential

Gravitational potentials have the form of \(1/\text{distance}\). Let’s check out the gradient.

fig, ax = grad_plotter_2D(
  field=1/sp.sqrt(x**2+y**2),
  norm=SymLogNorm(linthresh=.3, linscale=.3), mask=True,
)
plt.draw()
 Figure 5.13
Figure 5.13: Gradient of $f(x, y) = 1/\sqrt{x^2+y^2}$

Conic section fields

Gradients of conic section fields can be explored. The following is called a parabolic field.

fig, ax = grad_plotter_2D(field=x**2)
plt.draw()

The following are called elliptic fields.

fig, ax = grad_plotter_2D(field=x**2 + y**2)
plt.draw()
 Figure 5.14
Figure 5.14: Gradient of f(x,y) = x2 + y2
fig, ax = grad_plotter_2D(field=-x**2 - y**2)
plt.draw()
 Figure 5.15
Figure 5.15: Gradient of f(x,y) =  − x2 − y2

The following is called a hyperbolic field.

fig, ax = grad_plotter_2D(field=x**2 - y**2)
plt.show()
 Figure 5.16
Figure 5.16: Gradient of f(x,y) = x2 − y2

Conservative Force Fields

A force field is conservative if it is the gradient of a potential function. This means no energy is lost by a body acted on by a conservative force field, and that if the field is solely responsible for the body’s trajectory through the field, if the body returns to its starting position, it has the same potential energy and kinetic energy with which it started. The two most common conservative force fields encountered in engineering are gravitational fields and static electric fields.

A conservative force field \(\bm{f}\) can be derived from its potential energy function \(U\) via the gradient \[ \bm{f} = -\grad U. \] This means the force field always points in the direction of greatest decrease of potential energy, known colloquially as “downhill.”

Example 5.1

In many engineering mechanics problems, a body experiences gravitational forcing from the Earth (or another planet). However, in many cases, the centers of mass of the body and the Earth do not change significantly, relative to their overall distance. In such cases, we approximate the gravitational potential energy to be linear. For a body with mass m, the gravitational potential energy is approximately Ug = mgh, where g is a constant acceleration (g ≈ 9.81 m/s2 on Earth’s surface) and h is the vertical (i.e., with Earth downward) distance to some arbitrary reference position. Computing the force field fg, in Cartesian coordinates with y the vertical coordinate pointed upward, $$ \bm{f}_g = -\grad{U_g} = -m g \hat{\bm{j}}. $$ This constant force is the familiar “force of gravity” we encounter in so many mechanics problems. This is a uniform field always pointing downward. Note that we not only ignore small changes in magnitude here, we also ignore small changes in direction because a more complete model of the gravitational field always points toward the center of the other mass.

With reference to figure 5.17, suppose a bead with mass m = 1 kg is connected to a rigid circular hoop of radius R = 1 m, oriented in the xy plane, with a constant gravitational force with acceleration g = 9.81 m/s2 in the negative y direction. Parameterize the curve by its arclength and compute the trajectory of the bead from an initial position, assuming the hoop is frictionless. When the bead returns to its initial point, does it have the same potential energy? Now include a viscous damping force with damping coefficient b. Recompute the trajectory and comment on the nonconservative nature of the viscous damping force. As t → ∞, how much work have each of the gravitational force field and damping force performed?

 Figure 5.17
Figure 5.17: A bead on a hoop.

Our approach is to apply Newton’s second law in a tangential and normal basis (TNB basis or Serret–Frenet frame) with unit vectors ut and un, as shown in figure 5.17 [@hibbeler2015, § 13.5]. The advantage of using this coordinate system is that the variable (normal) force applied to the bead by the hoop is decoupled from the force along the (tangential) trajectory of the bead.

The position vector r has its tail at the origin O of the Cartesian coordinate system and its head at the center of the bead. The easiest parameter to describe the circular curve with is the angle θ from the positive x axis. This parametric representation of r is, in Cartesian coordinates, $$ \bm{r}(\theta) = \begin{bmatrix} R \cos\theta \\ R \sin\theta \end{bmatrix}. $$ However, when working with a TNB basis, we should parameterize the position vector with the arclength s [@oreilly2019,§ 3.1.2]. In general, the process of changing parameters from parameter θ to arclength s generally proceeds as follows. Compute the arclength s from its definition [@kreyszig2011,p. 385], for some a ∈ ℝ, $$ s = \int_a^\theta \sqrt{\bm{r}'(\tilde{\theta}) \cdot \bm{r}'(\tilde{\theta})}\ d\tilde{\theta},$$ then solve for θ(s). The arclength parameter s can now replace θ by substitution, r(θ) → r(θ(s)) → r(s).

However in our case the (circular) arclength has the well-known relationship with the corresponding angle, s = Rθ. Solving for θ(s), θ(s) = s/R. So the arclength parameterization of the position vector is $$ \bm{r}(s) = \begin{bmatrix} R \cos(s/R) \\ R \sin(s/R) \end{bmatrix}. $$

From here, we proceed in Python. Begin by loading the necessary packages.

import numpy as np
import sympy as sp
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp, cumulative_trapezoid

Define a symbolic parametric representation of the position vector r(s) in Cartesian coordinates. In our code, we will use T instead of θ.

R = sp.symbols("R", positive=True)  # Radius of the hoop
s = sp.symbols("s", real=True)  # Arc length
r_s = sp.Matrix([R * sp.cos(s/R), R * sp.sin(s/R)])

Now we can compute the gravitational force in the tangential direction. The gravitational force is $\bm{F}_g = -m g \hat{\bm{j}}$, where m is the mass of the bead. The unit tangent vector ut can be conveniently computed from the arclength parameterization of r as $$\bm{u}_t = \frac{d\bm{r}}{ds} = \begin{bmatrix}-\sin(s/R) \\ \cos(s/R)\end{bmatrix}.$$ The gravitational force in the tangential direction is then $$\bm{F}_g(s) \cdot \bm{u}_t = \bm{F}_g \cdot \frac{d\bm{r}}{ds} = \begin{bmatrix}0 \\ -m g\end{bmatrix} \cdot \begin{bmatrix}-\sin(s/R) \\ \cos(s/R)\end{bmatrix} = -m g \cos(s/R).$$ Computing this expression symbolically:

m, g = sp.symbols("m, g", positive=True) # Mass, grav. acceleration
F_g = sp.Matrix([0, -m * g])  # Grav. force (Cartesian coordinates)
u_t = r_s.diff(s)  # Unit tangent vector (Cartesian coordinates)
    # This has unit length because r_s is parameterized by arc length
F_g_t = F_g.dot(u_t)  # Grav. force in the tangential direction

Convert the symbolic expression to a Python function for numerical evaluation.

params = {R: 1, m: 1, g: 9.81}  # Parameters
F_g_t_fun = sp.lambdify((s), F_g_t.subs(params), "numpy")

Apply Newton’s second law to the mass to form a dynamic system state space model. The state vector is $\begin{bmatrix}s & v\end{bmatrix}^\top$, where s is the arc length and v is the velocity in the tangential direction. To define the state space model, we need to compute the time derivatives of the state variables. By definition, the velocity is the rate of change of arc length with respect to time, so $$\frac{ds}{dt} = v.$$ So this is the first equation in our system of ODEs comprising the state space model. The tangential acceleration is defined as the rate of change of velocity with respect to time, $$a = \frac{dv}{dt}.$$ Applying Newton’s second law to the bead in the tangential direction, we have ma = Fg(s) ⋅ ut =  − mgcos (s/R). Substituting the definition of acceleration and the gravitational force, we have $$\frac{dv}{dt} = -g \cos(s/R).$$ This is the second equation in our state space model. Define the state space model in Python:

def system(t, X, params):
    """Bead on a hoop dynamic system, sans damping."""
    s, v = X  # Unpack the state vector
    ds_dt = v  # Velocity in the tangential direction
    dv_dt = 1/m.subs(params) * F_g_t_fun(s)
        # Acceleration in the tangential direction
    return [ds_dt, dv_dt]

Numerically solve the system of ODEs for an initial state vector [s0,v0].

X0 = [sp.pi/3, 0]  # Initial state vector [s0, v0]
t = np.linspace(0, 10, 201)  # Time array
X_t = solve_ivp(
    system, (t[0], t[-1]), 
    X0, t_eval=t, args=(params,)
).y  # Solve the system

Now write a function to plot the trajectory of the bead on the hoop through time.

def plot_trajectory(t, X_t):
    """Plot the trajectory of the bead on the hoop through time."""
    fig, ax = plt.subplots(2, 1, sharex=True)
    ax[0].plot(t, X_t[0])
    ax[0].set_ylabel("Arc length $s$ (m)")
    ax[1].plot(t, X_t[1])
    ax[1].set_ylabel("Velocity $v$ (m/s)")
    ax[1].set_xlabel("Time $t$ (s)")
    return fig

Plot the trajectory of the bead on the hoop through time.

fig = plot_trajectory(t, X_t)
plt.draw()
 Figure 5.18
Figure 5.18: Arclength s and velocity v of the bead on the hoop through time without damping.

Now write a function to plot the trajectory of the bead on the hoop through space and time.

def plot_trajectory_space_time(t, X_t, r_s):
    """Plot the trajectory of the bead through space and time"""
    # First define a function to compute the position vector r(s) 
    # from the state variable arclength s 
    r_s_fun = sp.lambdify((s), r_s.subs(params), "numpy")
    r_s_t = r_s_fun(X_t[0])  # Position vector r(s) for each s in X_t
    fig = plt.figure()
    ax = fig.add_subplot(projection="3d")
    t_normalized = (t - np.min(t)) / (1.2*np.max(t) - np.min(t))
    for i in range(len(t) - 1): # Each segment with a different color
        ax.plot(
            r_s_t[0][0][i:i+2], 
            r_s_t[1][0][i:i+2], 
            [t[i], t[i+1]], 
            color=plt.cm.viridis(t_normalized[i])
        )
    ticks = np.array([-1, 0, 1])*params[R]
    ax.set_xticks(ticks)
    ax.set_yticks(ticks)
    ax.set_xlabel("$x$ (m)")
    ax.set_ylabel("$y$ (m)")
    ax.set_zlabel("Time $t$ (s)")
    ax.view_init(elev=150, azim=0, roll=90)
    return fig

Plot the trajectory of the bead on the hoop through space and time.

fig = plot_trajectory_space_time(t, X_t, r_s)
plt.draw()
 Figure 5.19
Figure 5.19: Trajectory of the bead on the hoop through space and time without damping.

Now add a viscous damping force in the anti-tangential direction.

b = sp.symbols("b", positive=True)  # Damping coefficient
v = sp.symbols("v", real=True)  # Velocity
F_d_t = -b * v  # Damping force in the tangential direction
params[b] = 0.1  # Damping coefficient
F_d_t_fun = sp.lambdify((v), F_d_t.subs(params), "numpy")  # Numerical

Apply Newton’s second law to the mass to form a dynamic system state space model.

def system_damped(t, X, params):
    """Bead on a hoop dynamic system, with damping."""
    s, v = X  # Unpack the state vector
    ds_dt = v  # Velocity in the tangential direction
    dv_dt = 1/m.subs(params) * (F_g_t_fun(s) + F_d_t_fun(v))
        # Acceleration in the tangential direction
    return [ds_dt, dv_dt]

Numerically solve the system of ODEs for the same initial state.

X_t_damped = solve_ivp(
    system_damped, (t[0], t[-1]), 
    X0, t_eval=t, args=(params,)
).y  # Solve the system

Plot the trajectory of the bead on the hoop through time with damping.

fig = plot_trajectory(t, X_t_damped)
plt.draw()
 Figure 5.20
Figure 5.20: Arclength s and velocity v of the bead on the hoop through time with damping.

Plot the trajectory of the bead on the hoop through space and time with damping.

fig = plot_trajectory_space_time(t, X_t_damped, r_s)
plt.draw()
 Figure 5.21
Figure 5.21: Trajectory of the bead on the hoop through space and time with damping.

Now consider the work done by the gravitational and damping forces on the bead as t → ∞. From the state space model, in steady state, a = v = 0. This corresponds to the bead at rest at the bottom of the hoop, with s =  − Rπ/2.

As we have already established, the gravitational force is conservative, so the work done by it is independent of the path taken. The work done by the damping force is given by Wd = ∫s0s1Fd ⋅ utds. Substituting the expression for the damping force and the unit tangent vector, we have Wd = ∫s0s1 − bvds =  − bs0s1vds. Or, changing the variable of integration to time via the substitution ds = vdt, Wd =  − bt0t1v2dt. We can evaluate this integral numerically by simulating the system for a relatively long time to get v(t).

t_long = np.linspace(0, 100, 3001)  # Long time array
X_t_long = solve_ivp(
    system_damped, (t_long[0], t_long[-1]), 
    X0, t_eval=t_long, args=(params,)
).y  # Solve the system

Compute the work done by the damping force.

W_d = -params[b] * \
    cumulative_trapezoid(X_t_long[1]**2, t_long, initial=0)
print(f"The work done by the damping force is approximately "
      f"{W_d[-1]:.2f} J.")
The work done by the damping force is approximately -17.91 J.

Plot the work done by the damping force through time.

fig, ax = plt.subplots()
ax.plot(t_long, W_d)
ax.set_xlabel("Time $t$ (s)")
ax.set_ylabel("Damping Force Work $W_d$ (J)")
plt.show()
 Figure 5.22
Figure 5.22: Work done by the damping force on the bead through time.

Now compute the work done by the gravitational force. Begin by computing the height of the bead at the initial and final states as follows:

h0 = R * sp.sin(X0[0]/R)  # Height, initial state
h1 = R * sp.sin(-sp.pi/2)  # Height, final state

If the damping force was not present and the bead started from rest (T0 = 0), in the downward position the kinetic energy would be equal to the difference in potential energy from the initial position; that is, T1 = U0 − U1. The difference in kinetic energy from the initial position would be equal to the work done by the gravitational force, Wg = T1 − T0 = U0 − U1 = mgh0 − mgh1 = mg(h0h1). Regardless of the presence of the damping force or the path taken, by the work-energy theorem the work done by the gravitational force is equal to this change in kinetic energy of the bead. Let’s compute this work done by the gravitational force.

W_g = (m * g * (h0 - h1)).evalf(subs=params)
18.3057092111253

We expect that over time the work done by the damping force will be equal in magnitude to the work done by the gravitational force in moving the bead from its initial position to the bottom of the hoop.

Feynman, Richard P., Robert B. Leighton, and Matthew Sands. 2010. The Feynman Lectures on Physics. New Millennium. Perseus Basic Books. http://www.feynmanlectures.caltech.edu/index.html.

  1. This is nonstandard terminology, but we’re bold.↩︎

Online Resources for Section 5.3

No online resources.