What is a Linear Operator?

An operator takes operands and returns a result based on the value of its operands. A function takes arguments and returns a result based on the value of its arguments. Thus an operator is really the same thing as a function.

For example:

add_operator(operand1,operand2) -> adds operand1 and operand2.

multiply_operator(operand1,operand2) -> multiplies operand1 and operand2.

differentiate_operator(operand1,operand2) -> differentiate an expression. One of the operands must be the expression to differentiate and the other the variable to differentiate with respect to.

Some operators have some special properties that make them very useful.

Let’s consider the differentiation operator. Let’s say we have two functions, $f(x)$ and $g(x)$, and we want to differentiate with respect to $x$. If we add the two functions ($f$ and $g$), and then apply the operator, we would get the same result as if we applied the operator to each function first, and then added the results! So in other words:

$$\frac{d}{dx}(f + g) = \frac{d}{dx}(f) + \frac{d}{dx}(g)$$

Additionally, if we have just one function, say $f(x)$ and some constant $c$, if we multiply the function by the constant and then apply the operator, we would get the same results as if we first applied the operator to $f$ and then multiplied that result with $c$. So in other words:

$$\frac{d}{dx}(cf) = \frac{d}{dx}(f) c$$

We call the first property additivity and the second property multiplicitive scaleability. Not all operators have these properties, but the ones that do, we call them linear operators.

Linear operators are a special type of operators, we know more about them, we can make more assumptions when we are working with them, thus we have more knowns - which is always a good thing!

This is analogous to knowing the exact type of function you have (e.g. polynomial of degree 3) rather than knowing that we just have some function. Knowing the specific type of something you have gives you more knowns!

All the operators above (add, multiply, and differentiate) are linear operators. Go ahead, check for yourself if they have the above 2 properties. You’ll find out that they in fact do!

Now, let’s take a look at an operator that is not linear. Consider an operator that raises something to a power:

power(operand1, operand2) -> raises operand1 to the value of operand2.

Let’s say that we have some functions $f$ and $g$. Let’s say that for now we are simply interested in squaring things, i.e. our power will always be 2. Is the following true?

$$(f + g)^2 = f^2 + g^2$$

No, it is not! Thus power is not a linear operator. For fun, let’s check if power violates multiplicitive scaelability too. Is the following true?

$$(cf)^2 = c f^2$$

It is not true! Thus power violated both required conditions for being a linear operator.

• an operator operates on its operand(s) and returns some results, it’s just like a function really
• operators (functions) that satisfy additivity and multiplicative scaleability properties are called linear operators
  • additivity: $O(f + g) = O(f) + O(g)$
  • multiplicitive scaeability: $O(cf) = c O(f)$
  • $O$ is the operator, $f$ and $g$ are functions