Tensor & Autograd

Tensor is the central data structure of simplegrad. It wraps a NumPy array and records operations in a computation graph so that gradients can be computed via reverse-mode automatic differentiation. Every mathematical operation on a Tensor returns a new Tensor and, when comp_grad=True, wires itself into the graph for backpropagation.

import simplegrad as sg

x = sg.Tensor([1.0, 2.0, 3.0], requires_grad=True)
y = (x * x).sum()
y.backward()
print(x.grad)  # [2. 4. 6.]

Tensor

N-dimensional array with automatic differentiation support.

Wraps a numpy or cupy array and records operations into a dynamic computation graph. Call .backward() on a scalar output to propagate gradients back to all leaf tensors with comp_grad=True.

Attributes

Attribute	Type	Description
.values	ndarray \| None	The underlying NumPy or CuPy array. None until .realize() is called in lazy mode.
.shape	tuple	Shape of the tensor. Available even before .realize().
.dtype	str	Data type string, e.g. "float32".
.device	str	Device string, e.g. "cpu" or "cuda:0".
.comp_grad	bool	Whether gradient tracking is enabled for this tensor.
.is_leaf	bool	True if the tensor was not produced by an operation (e.g. a parameter or input).
.grad	ndarray \| None	Accumulated gradient array after .backward(). None until backprop runs.
.label	str \| None	Optional name used in computation graph visualizations.

Methods

Method	Description
.backward()	Run backpropagation from this tensor, computing gradients for all leaves.
.realize()	Execute all pending forward computations in lazy mode.
.zero_grad()	Zero gradients on all leaf tensors in the computation graph.
.convert_to()	Convert tensor values to a different dtype.
.to_device()	Copy this tensor to a target device, returning a new leaf tensor.
.deferred()	Class method — create an unrealized tensor that defers computation.
.T	Property — transpose of the tensor.

Arithmetic operators

Operator	Expression	Description
__add__	a + b	Element-wise addition with a scalar or tensor.
__sub__	a - b	Element-wise subtraction.
__mul__	a * b	Element-wise multiplication with a scalar or tensor.
__truediv__	a / b	Element-wise division.
__pow__	a ** n	Element-wise power with a scalar exponent.
__matmul__	a @ b	Matrix multiplication.
__neg__	-a	Element-wise negation.

---

Execution mode

Simplegrad supports two execution modes — eager (default) and lazy. In eager mode every operation runs NumPy immediately. In lazy mode operations are recorded into a graph and only executed when .realize() or .backward() is called, enabling whole-graph optimisations. The no_grad context manager disables gradient tracking entirely, which is useful during inference to save memory and computation.

import simplegrad as sg

# Disable gradient tracking during inference
with sg.no_grad():
    out = model(x)

# Defer execution until .realize()
with sg.lazy():
    out = model(x)
out.realize()

no_grad()

Context manager that disables gradient computation globally.

lazy()

Context manager that activates lazy execution mode for the enclosed block.

Inside the block, tensor operations record their computation without running numpy. Call .realize() on the output tensor (or call .backward()) to execute the full graph.

mode(mode: str) -> None

Set the global execution mode persistently.

Parameters:

• mode (str) –

  Either "eager" (default, execute numpy immediately) or "lazy" (defer execution until .realize() is called).

Raises:

• ValueError –

  If mode is not "eager" or "lazy".

is_lazy() -> bool

Return True if lazy execution mode is currently active.