The White Rabbit put on his spectacles. “Where shall I begin, please your Majesty?” he asked. “Begin at the beginning,” the King said gravely, “and go on till you come to the end: then stop.”
– Lewis Carroll, Alice’s Adventures in Wonderland
A discrete-time signal is, essentially, a sequence of numbers or symbols. A temperature time series can be viewed as a real-valued discrete-time signal, whereas the output of an A/D converter can be thought of as a symbol-valued discrete-time signal if we view each binary word as a symbol. For simplicity, in this post we will consider sequences of real numbers only. More formally, a discrete-time signal is a function from a set of indices to . Again for simplicity, we will only consider the case where , i.e., we will focus on doubly-infinite sequences.
A discrete-time system transforms input discrete-time signals into output discrete-time signals. For the time being, we will consider single-input, single-output (SISO) systems only. Let and be the real-valued input and output signals of a discrete-time system. Then, we can think of the system as an operator that transforms signal into signal , as depicted below
The output of the system is thus . We will focus on the class of linear time-invariant (LTI) systems. But, what is a linear system? Or a time-invariant one? Let us “begin at the beginning”.
A linear system is one “that possesses the important property of superposition” , i.e.,
In words: “if an input consists of the weighted sum of several signals, then the output is the superposition–that is, the weighted sum–of the responses of the system to each of those signals” . Note that if is linear, then
i.e., “for linear systems, an input which is zero for all time results in an output which is zero for all time” . Pictorially, if is linear, then the following block diagram
This is not a mere academic curiosity. Note that the former block diagram contains multipliers, one -input adder, and one block; by contrast, the latter block diagram contains multipliers, one -input adder, and blocks. Hence, the former block diagram can be implemented using less hardware, i.e., it is cheaper to implement.
A system is time-invariant “if a time-shift in the input signal results in an identical time-shift in the output signal” . Let us introduce the time-shift operator , where represents:
- signal delayed by samples if .
- signal “anticipated” by samples if .
Note that , i.e., is the identity operator. From the definition, if a system is time-invariant, then
and, since , we obtain
For the sake of clarity, let us write , and . Then, for time-invariant systems, the following equality
holds for every input signal , i.e., for time-invariant systems the operators and do commute. Pictorially, we have that the following block diagrams
are equivalent. Discrete-time LTI systems are sometimes called linear shift-invariant (LSI) systems .
Response of LTI Systems
Let the unit impulse signal be defined by
and let be the unit impulse signal shifted by samples. An arbitrary discrete-time signal can thus be represented as a linear combination of shifted unit impulses
which is known as the sifting property of the discrete-time unit impulse signal . If the input signal is , the output signal will be
If the system is linear (we demand no time-invariance at this point), then
i.e., the response of a linear system to a discrete-time signal is “the superposition of the scaled responses of the system to each of these shifted impulses” . Let denote the system’s response to the shifted unit impulse signal . Then we can write the output signal as
Hence, a linear system is characterized by the set of responses to shifted unit impulses, , an infinite set. Note that
If the system is also time-invariant, then and commute and, thus
In words: time-invariance means that “the responses of a time-invariant system to the time-shifted unit impulses are simply time-shifted versions of one another” . The system’s output is thus
i.e., the system’s output signal is the superposition of scaled, time-shifted versions of the response to the unit impulse signal . Let , which we call the system’s impulse response. Then we have that
Hence, from one single discrete-time signal we can generate the entire set . Therefore, the LTI system can be characterized by one signal, , or by one transfer function (which is the Z-transform of the impulse response).
Finally, we have that the output of a discrete-time LTI system is
where the -th sample of the output signal is, thus, given by the following convolution sum
 Alan V. Oppenheim, Alan S. Willsky, S. Hamid Nawab, Signals & Systems, 2nd edition, Prentice-Hall, 1996.
 Alan V. Oppenheim, Ronald W. Schafer, Digital Signal Processing, Prentice-Hall, 1975.