强化学习（Reinforcement Learning,RL）中有智能体（Agent）和环境（Environment）。智能体与环境进行交互（Interaction），智能体观察到环境的一个状态（State），对环境执行一个动作（Action），得到环境的一个奖励（Reward），环境进入下一个状态，不断重复，见强化学习交互图。环境最初处于初始状态，过程中状态不断发生变化，直到环境进入一个终止状态（Terminal State）。整个过程是智能体与环境交互的序列（Sequence）决策过程。强化学习的目标是最大化智能体在环境中获得的最终累积奖励，称为回报（Return）。这里只考虑交互是有限的情况，称为有限期（Finite Horizon）。

马尔可夫决策过程

强化学习假设上述序列决策过程是一个随机过程，由马尔可夫决策过程（Markov Decision Process,MDP）描述。

$$\begin{equation} MDP = \{\mathcal{S},\mathcal{A},P,R,\gamma\}\end{equation}$$

其中：

• $\mathcal{S}$：状态空间，环境中所有可能的状态集合。
• $\mathcal{A}$：动作空间，智能体在环境中可以执行的所有可能的动作集合。
• $P$：状态转移概率函数，描述在状态$s$下执行动作$a$后转移到下一个状态$s'$的概率，即$P(s'|s,a)$，其满足马尔可夫性。
• $R$：奖励函数，描述在状态$s$下执行动作$a$后获得的奖励，即$R(s,a)$。
• $\gamma$：折扣因子（衰减系数）。

马尔可夫过程中产生的序列可以表示为：

$$S_0, A_0, R_1, S_1, A_1, \ldots ,S_{t} , A_{t}, R_{t+1}, \ldots R_{T}, S_{T}$$

称为轨迹（Trajectory）。其中，$S_t$表示时间步$t$的状态，$A_t$表示时间步$t$的动作，$R_{t+1}$表示在时间步$t$执行动作后获得的奖励。

轨迹数据中的累积的奖励或回报为：

$$\begin{equation} G_t = R_{t+1} + \gamma R_{t+2} + \gamma^2 R_{t+3} + \cdots\end{equation}$$

环境决定了状态转移概率$P(s'|s,a)$和奖励函数$R(s,a)$。状态转移概率和奖励函数称为模型（Model）。智能体决定的是策略（Policy），即策略函数$\pi:\mathcal{S} \times \mathcal{A} \rightarrow [0,1]$，定义为条件概率分布$P(A=a \mid S=s)$。

状态价值函数（State Value Function）：$V: \mathcal{S} \mapsto \mathbb{R}$，其中$V_{\pi}(s) = \mathbb{E}_{\pi}[G_t \mid S_t = s]$表示智能体在时刻$t$ 处于状态$s$ 时，按照策略$\pi$ 行动所获得的回报的期望。价值函数衡量了某个状态的好坏程度，反映了智能体从当前状态出发能够为目标完成带来多大“好处”。

动作价值函数（Action-value Function）：$q: \mathcal{S} \times \mathcal{A} \mapsto \mathbb{R}$，其中 $q_{\pi}(s, a) = \mathbb{E}_{\pi}[G_t \mid S_t = s, A_t = a]$表示智能体在时刻$t$处于状态$s$并选择动作$a$后，按照策略$\pi$ 行动所获得的回报的期望。通过最大化动作价值函数，智能体可得到最佳奖励。

于是我们给出强化学习的问题：

给定一个马尔可夫决策过程$MDP = \{\mathcal{S},\mathcal{A},P,R,\gamma\}$，学习一个最优策略$\pi^*$，使得对于所有状态$s \in \mathcal{S}$，$V_{\pi^*}(s) \geq V_{\pi}(s)$，其中$\pi$是任意其他策略。

贝尔曼方程

贝尔曼方程（Bellman Equation）也被称作动态规划方程（Dynamic Programming Equation），是强化学习中的一个重要工具，用于描述状态价值函数和动作价值函数之间的关系.

我们已知

$$\begin{align} & \begin{aligned} V_{\pi}(s) & = \mathbb{E}_{\pi}[R_{t+1} + \gamma G_{t+2} + \gamma^2 G_{t+3} + \cdots \mid S_t = s]\\ & = \sum_{a \in \mathcal{A}} \pi(s,a)q_{\pi}(s,a) \end{aligned}\\ & \begin{aligned} q_{\pi}(s, a) = \mathbb{E}_{\pi}[R_{t+1} + \gamma G_{t+2} + \gamma^2 G_{t+3} + \cdots \mid S_t = s, A_t = a]\\ = R(s' \mid s,a) + \gamma \sum_{s' \in \mathcal{S}} P(s,a,s')V_{\pi}(s') \end{aligned}\end{align}$$

则代入可得

$$\begin{align} V_{\pi}(s) & = \sum_{a \in \mathcal{A}} \pi(s,a) \sum_{s' \in \mathcal{S}}P(s' \mid s,a)[R(s,a,s') + \gamma V_{\pi}(s')]\\ q_{\pi}(s, a) & = \sum_{s' \in \mathcal{S}}P(s' \mid s,a) \left[R(s,a,s') + \gamma \sum_{a' \in \mathcal{A}} \pi(s',a')q_{\pi}(s',a')\right]\end{align}$$

可知两个价值函数取值与时间都没有关系。状态价值函数只与策略$\pi$、在策略$\pi$下的从某个状态转移到其后续状态所获得的回报以及之后所得的回报有关；动作价值函数只与瞬时奖励和下一步的状态和动作有关。

贝尔曼方程描述了价值函数的递归关系。在实际中，需要计算得到最优策略以指导智能体在当前状态如何选择一个可获得最大回报的动作。求解最优策略的一种方法就是去求解最优的状态价值函数或最优的动作价值函数（即基于价值方法，value-based approach）。一旦找到了最优的状态价值函数或动作价值函数，自然而然也就找到了最优策略。当然，在强化学习中还有基于策略（policy-based）和基于模型（model-based）等不同方法。具体的来说，强化学习分为模型无关（model-free）和模型相关（model-based）两大类方法。而上述所说的基于价值的方法和基于策略的方法都属于模型无关方法。

Backtracking

The General Backtracking Method

\begin{algorithm}\caption{BACKTRACKREC}\begin{algorithmic}\Input  Explicit or implicit description of the sets $X_1,X_2,\dots,X_n$.\Output A solution vector $v = (x_1,x_2,\dots,x_i),0 \leq i \leq n$.\State $v \gets ()$\State $flag \gets$ \False\State \Call{advance}{1}\If{$flag$}\State output $v$\Else\State output "no solution"\EndIf\Procedure{advance}{$k$}\For{each $x \in X_k$}\State $x_k \gets x$; append $x_k$ to $v$\If{$v$ is a final solution}\State set $flag \gets$ \True and exit\Elif{$v$ is partial}\State \Call{advance}{$k+1$}\EndIf\EndFor\EndProcedure\end{algorithmic}\end{algorithm}

\begin{algorithm}\caption{BACKTRACKINER}\begin{algorithmic}\Input  Explicit or implicit description of the sets $X_1,X_2,\dots,X_n$.\Output A solution vector $v = (x_1,x_2,\dots,x_i),0 \leq i \leq n$.\State $v \gets ()$\State $flag \gets$ \False\State $k \gets 1$\While{$k \geq 1$}\While{$X_k$ is not exhausted}\State $x_k \gets$ next element in $X_K$; append $x_k$ to $v$\If{$v$ is a final solution}\State set $flag \gets$ \True and exit from the two while loops.\Elif{$v$ is partial}\State $k \gets k + 1$ \Comment{advance}\EndIf\EndWhile\State Reset $X_k$ so that the next element is the first.\State $k \gets k-1$ \Comment{backtrack}\EndWhile\If{$flag$}\State output $v$\Else\State output "no solution"\EndIf\end{algorithmic}\end{algorithm}

The 3-Coloring Problem

Consider the problem 3-coloring: Given an undirected graph $G = (V, E)$, it is required to color each vertex in $V$ with one of three colors, say $1$, $2$, and $3$, such that no two adjacent vertices have the same color. We call such a coloring legal; otherwise, if two adjacent vertices have the same color, it is illegal. A coloring can be represented by an $n$-tuple $(c_1, c_2, \ldots, c_n)$ such that $c_i \in \{1, 2, 3\}$, $1 \leq i \leq n$.

\begin{algorithm}\caption{3-COLORREC}\begin{algorithmic}\Input An undirected graph $G = (V,E)$.\Output A $3$-coloring $c[1..n]$ of the vertices of $G$, where each $c[j]$ is $1$, $2$, and $3$.\For{$k \gets 1$ \To $n$}\State $c[k] \gets 0$\EndFor\State $flag \gets$ \False\State \Call{graphcolor}{1}\If{$flag$}\State output $c$\Else\State output "no solution"\EndIf\Procedure{graphcolor}{$k$}\For{$color = 1$ \To $3$}\State $c[k] \gets color$\If{$c$ is a legal coloring}\State set $flag \gets$ \True and exit\Elif{$c$ is partial}\State \Call{graphcolor}{$k+1$}\EndIf\EndFor\EndProcedure\end{algorithmic}\end{algorithm}

\begin{algorithm}\caption{3-COLORITER}\begin{algorithmic}\Input An undirected graph $G = (V,E)$.\Output A $3$-coloring $c[1..n]$ of the vertices of $G$, where each $c[j]$ is $1$, $2$, or $3$.\For{$k \gets 1$ \To $n$}\State $c[k] \gets 0$\EndFor\State $flag \gets$ \False\State $k \gets 1$\While{$k \ge 1$}    \While{$c[k] \le 2$}        \State $c[k] \gets c[k] + 1$        \If{$c$ is a legal coloring}             \State set $flag \gets$ \True and exit from the two while loops.        \ElsIf{$c$ is partial}             \State $k \gets k + 1$ \Comment{advance}        \EndIf    \EndWhile    \State $c[k] \gets 0$    \State $k \gets k - 1$ \Comment{backtrack}\EndWhile\If{$flag$} \State output $c$\Else \State output "no solution"\EndIf\end{algorithmic}\end{algorithm}

Below is the implementation of these pseudocodes in Python(Adjacency list):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

G = {
    'a': ['b','c'],
    'b': ['a','d','e'],
    'c': ['a','d','e'],
    'd': ['b','c'],
    'e': ['c','d']
}

V = list(G.keys())


def three_color_rec(G):
    c = {v: 0 for v in V}
    flag = False
    n = len(V)

    def is_legal(v, color):
        for neighbor in G[v]:
            if c[neighbor] == color:
                return False
        return True

    def graphcolor(k):
        nonlocal flag
        if k > n:
            flag = True
            return
        
        for color in range(1,4):
            if is_legal(V[k-1], color):
                c[V[k-1]] = color
                graphcolor(k + 1)
                if flag:
                    return
                c[V[k-1]] = 0

    graphcolor(1)
    if flag:
        print(c)
    else:
        print("no solution")

def three_color_iter(G):
    n = len(V)
    c = {v: 0 for v in V}
    flag = False
    k = 1

    def is_legal():
        for v in V:
            if c[v] == 0:
                return False
        for v in V:
            for neighbor in G[v]:
                if c[v] == c[neighbor]:
                    return False
        return True

    def is_partial():
        for i in range(1, k+1):
            v = V[i-1]
            for neighbor in G[v]:
                if neighbor in V[:k] and c[neighbor] == c[v]:
                    return False
        return True

    while k >= 1:
        if k > n:
            if is_legal():
                flag = True
                break
            k -= 1
            continue
            
        found_next_step = False
        while c[V[k-1]] <= 2 and not found_next_step:
            c[V[k-1]] += 1
            
            if is_legal():
                flag = True
                break
            elif is_partial():
                k += 1
                found_next_step = True
        
        if flag:
            break
            
        if not found_next_step:
            c[V[k-1]] = 0
            k -= 1

    if flag:
        print(c)
    else:
        print("no solution")

The 8-Queens Problem

The classical 8-queens can be stated as follows. How can we arrange eight queens on an $8 \times 8$ chessboard so that no two queens can attack each other? Two queens can attack each other if they are in the same row, column, or diagonal.
Consider a chessboard of size $4 \times 4$. Since no two queens can be put in the same row, each queen is in a different row. Since there are four positions in each row, there are $4^4$ possible configurations. Each possible configuration can be described by a vector with four components $x = (x_1, x_2, x_3, x_4)$.

\begin{algorithm}\caption{4-QUEENSITER}\begin{algorithmic}\Input None.\Output Vector $x[1..4]$ corresponding to the solution of the $4$-queens problem.\For{$k \gets 1$ \To $4$}\State $x[k] \gets 0$ \Comment{no queens are placed on the chessboard}\EndFor\State $flag \gets$ \False\State $k \gets 1$\While{$k \ge 1$}    \While{$x[k] \le 3$}        \State $x[k] \gets x[k] + 1$        \If{$x$ is a legal placement}            \State set $flag \gets$ \True and exit from the two while loops.        \ElsIf{$x$ is partial}            \State $k \gets k + 1$ \Comment{advance}        \EndIf    \EndWhile    \State $x[k] \gets 0$    \State $k \gets k - 1$ \Comment{backtrack}\EndWhile\If{$flag$}\State output $x$\Else\State output "no solution"\EndIf\end{algorithmic}\end{algorithm}

Below is the implementation of the above pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

def n_queens(n):
    x = [0] * n
    flag = False

    def is_legal(x,k):
        for i in range(k):
            if x[i] == x[k] or abs(x[i] - x[k]) == abs(i - k):
                return False
        return True


    k = 0
    while k >= 0:
        while x[k] < n:
            if is_legal(x,k):
                if k == n - 1:
                    flag = True
                    break
                else:
                    k += 1
                    x[k] = 0
            else:
                x[k] += 1
        if flag:
            break

        x[k] = 0
        k -= 1
        if k >= 0:
            x[k] += 1
    if flag:
        print(x)

    else:
        print("No solution")

Branch and Bound

7.3 旅行商问题 - 分支界限法_哔哩哔哩_bilibili 可以听听印度老哥的超绝口音

Randomized Algorithms

Randomized algorithms can be classified into two categories. The first category is referred to as Las Vegas algorithms. It constitutes those ran- domized algorithms that always give a correct answer or do not give an answer at all. This is to be contrasted with the other category of random- ized algorithms, which are referred to as Monte Carlo algorithms. A Monte Carlo algorithm always gives an answer, but may occasionally produce an answer that is incorrect. However, the probability of producing an incorrect answer can be made arbitrarily small by running the algorithm repeatedly with independent random choices in each run.

Randomized Quicksort

\begin{algorithm}\caption{Randomized Quicksort}\begin{algorithmic}\Input  An array $A[1..n]$ of $n$ elements.\Output The elements in $A$ sorted in nondecreasing order.\State \Call{rquicksort}{$1, n$}\Procedure{rquicksort}{$low,high$}\If{$low < high$}\State $v \gets$ \Call{random}{$low,high$}\State interchange $A[low]$ and $A[v]$\State \Call{split}{$A[low..high]$,$w$}\State \Call{rquicksort}{$low, w-1$}\State \Call{rquicksort}{$w+1, high$}\EndIf\EndProcedure\end{algorithmic}\end{algorithm}

\begin{algorithm}\caption{SPLIT}\begin{algorithmic}\Input An array of elements $A[low..high]$.\Output (1) $A$ with its elements rearranged as described; (2) $w$, the new position of the splitting element $A[low]$.    \State $i \gets low$    \State $x \gets A[low]$    \For{$j \gets low+1$ \textbf{to} $high$}        \If{$A[j] \leq x$}            \State $i \gets i + 1$            \If{$i \neq j$}                \State interchange $A[i]$ and $A[j]$            \EndIf        \EndIf    \EndFor    \State interchange $A[low]$ and $A[i]$    \State $w \gets i$    \Return $A$ and $w$\end{algorithmic}\end{algorithm}

Below is the implementation of the above pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import random

def split(A, low, high):
    i = low
    x = A[low]

    for j in range(low + 1, high + 1):
        if A[j] <= x:
            i += 1
            if i != j:
                A[i], A[j] = A[j], A[i]

    A[low], A[i] = A[i], A[low]
    w = i
    return (A,w)

def rquicksort(A, low, high):
    if low < high:
        v = random.randint(low, high-1)
        A[low], A[v] = A[v], A[low]
        A, w = split(A, low, high)
        rquicksort(A, low, w - 1)
        rquicksort(A, w + 1, high)

The algorithm is Las Vegas algorithm since it always produces a correctly sorted array. The expected running time of randomized quicksort is $\Theta(n \log n)$, and the worst-case running time is $\Theta(n^2)$.

Randomized Selection

\begin{algorithm}\caption{Randomized Selection}\begin{algorithmic}\Input An array $A[1..n]$ of $n$ elements and an integer $k$, $1 \leq k \leq n$.\Output The $k$th smallest element in $A$.\State \Return \Call{rselect}{$A, 1, n, k$}\Procedure{rselect}{$A, low, high, k$}\State $v \gets$ \Call{random}{$low, high$}\State $x \gets A[v]$\State Partition $A$ into three arrays $A_1 = \{a \mid a < x\}$, $A_2 = \{a \mid a = x\}$, $A_3 = \{a \mid a > x\}$\If{$|A_1| \ge k$}    \Return \Call{rselect}{$A_1, 1, |A_1|, k$}\ElsIf{$|A_1| + |A_2| \ge k$}    \Return $x$\Else    \Return \Call{rselect}{$A_3, 1, |A_3|, k - |A_1| - |A_2|$}\EndIf\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of the above pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import random

def rselect(A,low,high,k):
    v = random.randint(low,high)
    x = A[v]

    A_1 = []
    A_2 = []
    A_3 = []

    for a in A:
        if a < x:
            A_1.append(a)
        elif a == x:
            A_2.append(a)
        else:
            A_3.append(a)

    if k <= len(A_1):
        return rselect(A_1,0,len(A_1)-1,k)
    elif k <= len(A_1) + len(A_2):
        return x
    else:
        return rselect(A_3,0,len(A_3)-1,k - len(A_1) - len(A_2))

The expected running time of randomized selection is $\Theta(n)$, and in the worst case, it is $\Theta(n^2)$.

Testing String Equality

Another alternative would be for $A$ to derive from $x$ a much shorter string that could serve as a “fingerprint” of $x$ and send it to $B$. $B$ then would use the same derivation to obtain a fingerprint for $y$, and then compare the two fingerprints. If they are equal, then $B$ would assume that $x=y$; otherwise, he would conclude that $x\neq y$. $B$ then notifies $A$ of the outcome of the test. This method requires the transmission of a much shorter string across the channel. For a string $w$, let $I(w)$ be the integer represented by the bit string $w$. One method of fingerprinting is to choose a prime number $p$ and then use the fingerprint function

$$I_{p}(x)=I(x)\ (\text{mod}\ p).$$

If $p$ is not too large, then the fingerprint $I_{p}(x)$ can be sent as a short string. The number of bits to be transmitted is thus $O(\log p)$. If $I_{p}(x)\neq I_{p}(y)$, then obviously $x\neq y$. However, the converse is not true. That is, if $I_{p}(x)=I_{p}(y)$, then it is not necessarily the case that $x=y$. We refer to this phenomenon as a false match. In general, a false match occurs if $x\neq y$, but $I_{p}(x)=I_{p}(y)$, i.e., $p$ divides $I(x)-I(y)$. We will later bound the probability of a false match.
The weakness of this method is that, for fixed $p$, there are certain pairs of strings $x$ and $y$ on which the method will always fail. We get around the problem of the existence of these pairs $x$ and $y$ by choosing $p$ at random every time the equality of two strings is to be checked, rather than agreeing on $p$ in advance. Moreover, choosing $p$ at random allows for resending another fingerprint and thus increasing the confidence in the case $x=y$. This method is described in Algorithm Stringequalitytest shown below (the value of $M$ will be determined later).

\begin{algorithm}\caption{String Equality Test}\begin{algorithmic}\State $A$ chooses $p$ at random from the set of primes less than $M$.\State $A$ sends $p$ and $I_p(x)$ to $B$.\State $B$ checks whether $I_p(x) = I_p(y)$ and confirms the equality or inequality of the two strings $x$ and $y$.\end{algorithmic}\end{algorithm}

Below is the implementation of the above pseudocode in Python:

1
2
3
4
5
6
7
8
9
10

import random
from sympy import primerange
def string_equality_test(x, y, M):
    primes = list(primerange(1, M))
    p = random.choice(primes)

    I_p_x = int(x, 2) % p
    I_p_y = int(y, 2) % p

    return I_p_x == I_p_y

Pattern Matching

Given a string of text $X = x_1 x_2 \ldots x_n$ of length $n$ and a pattern $P = p_1 p_2 \ldots p_m$ of length $m$, where $m \leq n$, determine whether or not the pattern appears in the text.

$$\begin{aligned}I_p(X_j) = (x_j \cdot 2^{m-1} + \dots + x_{j+m-1}) \bmod p\\I_p(Y) = (y_1 \cdot 2^{m-1} + \dots + y_m) \bmod p\end{aligned}$$

\begin{algorithm}\caption{Pattern Matching}\begin{algorithmic}\Input A string of text $X$ and a pattern $Y$ of lengths $n$ and $m$, respectively, where $m \leq n$.\Output The first position of $Y$ in $X$ if $Y$ occurs in $X$; otherwise $0$.\State Choose $p$ at random from the set of primes less than $M$\State $j \gets 1$\State Compute $W_p \gets 2^m \ (\text{mod}\ p)$, $I_p(Y)$, and $I_p(X_j)$\While{$j \leq n - m + 1$}    \If{$I_p(X_j) = I_p(Y)$}        \Return $j$ \Comment{A match is found (probably)}    \EndIf    \State $j \gets j + 1$    \State Compute $I_p(X_j)$ using $I_p(X(j+1)) = (2I_p(X(j)) - 2^m x_j + x_{j+m})(\bmod \ p)$\EndWhile\Return $0$ \Comment{$Y$ does not occur in $X$ (definitely)}\end{algorithmic}\end{algorithm}

The expected running time of the above algorithm is $O(n + m)$.

Approximation Algorithms

Basic Definitions

A combinatorial optimization problem $\Pi$ is either a minimization problem or a maximization problem. It consists of three components:

• A set $D_{\Pi}$ of instances.
• For each instance $I\in D_{\Pi}$, there is a finite set $S_{\Pi}(I)$ of candidate solutions for $I$.
• Associated with each solution $\sigma\in S_{\Pi}(I)$ to an instance $I$ in $D_{\Pi}$, there is a value $f_{\Pi}(\sigma)$ called the solution value for $\sigma$.
  If $\Pi$ is a minimization problem, then an optimal solution $\sigma^{*}$ for an instance $I\in D_{\Pi}$ has the property that for all $\sigma\in S_{\Pi}(I)$, $f_{\Pi}(\sigma^{*})\leq f_{\Pi}(\sigma)$. An optimal solution for a maximization problem is defined similarly. Throughout this chapter, we will denote by $OPT(I)$ the value $f_{\Pi}(\sigma^{*})$.
  An approximation algorithm $A$ for an optimization problem $\Pi$ is a (polynomial time) algorithm such that given an instance $I\in D_{\Pi}$, it outputs some solution $\sigma\in S_{\Pi}(I)$. We will denote by $A(I)$ the value $f_{\Pi}(\sigma)$.

Difference Bounds

For all instances $I$ of the problem, the most desirable solution that can be obtained by an approximation algorithm $A$ is such that $|A(I) − OPT (I)| \leq K$, for some constant $K$.
Let $G = (V, E)$ be a planar graph. In solving the $n$-coloring problem for $G$, it holds that $|A(I) − OPT (I)| \leq 1$.
There is no approximation algorithm with difference bound that solves the knapsack problem. Since the knapsack problem is known to be $NP$-complete, it is highly unlikely that the approximation algorithm A exists unless $NP = P$.

Relative Performance Bounds

Let II be a minimization problem and $I$ an instance of II. Let $A$ be an approximation algorithm to solve II. We define the approximation ratio $R_A(I)$ to be

$$R_A(I) = \frac{A(I)}{OPT(I)}.$$

If II is a maximization problem, then we define $R_A(I)$ to be

$$R_A(I) = \frac{OPT(I)}{A(I)}.$$

Thus, the approximation ratio is always greater than or equal to 1. This has been done so that we will have a uniform measure for the quality of the solution produced by $A$.

The bin packing problem

The optimization version of the BIN PACKING problem can be stated as follows. Given a collection of items $u_1, u_2, \ldots, u_n$ of sizes $s_1, s_2, \ldots, s_n$, where each $s_j$ is between 0 and 1, we are required to pack these items into the minimum number of bins of unit capacity. We list here one heuristic for the BIN PACKING problem.
First Fit (FF). In this method, the bins are indexed as 1, 2, \ldots All bins are initially empty. The items are considered for packing in the order $u_1, u_2, \ldots, u_n$. To pack item $u_i$, find the least index $j$ such that bin $j$ contains at most $1 - s_i$, and add item $u_i$ to the items packed in bin $j$.

$$R_{FF}(I) = \frac{FF(I)}{OPT(I)} < 2$$

The Greedy Approach

Dijkstra’s Algorithm

We define $\lambda [y]$ as the shortest known distance from the source vertex (vertex 1) to vertex $y$. Initially, $\lambda [y]$ is set to the weight of the edge connecting vertex 1 to vertex $y$ if such an edge exists. If no such edge exists, $\lambda [y]$ is set to infinity ($\infty$), indicating that vertex $y$ is not yet reachable from the source.

\begin{algorithm}\caption{DIJKSTRA}\begin{algorithmic}\Input A weight directed graph $G = (V,E)$, where $V = \{ 1,2,\dots ,n \}$.\Output The distance from vertex 1 to every other vertex in $G$.\State $X = \{ 1 \}; \quad Y \gets V - \{ 1 \}; \quad \lambda [1] \gets 0$\For{$y \gets 2$ \To $n$}\If{$y$ is adjacent to $1$ then $\lambda[y] \gets length[1,y]$}    \Else        \State $\lambda [y] \gets \infty$    \EndIf\EndFor\For{$j \gets 1$ \To $n - 1$}    \State Let $y \in Y$ be such that $\lambda[y]$ is minimum    \State $X \gets X \cup \{y\}$ \Comment{add vertex $y$ to $X$}    \State $Y \gets Y - \{y\}$ \Comment{delete vertex $y$ from $Y$}    \For{each edge $(y, w)$}        \If{$w \in Y$ and $\lambda[y] + length[y, w] < \lambda[w]$}            \State $\lambda[w] \gets \lambda[y] + length[y, w]$        \EndIf    \EndFor\EndFor\end{algorithmic}\end{algorithm}

In the figure , those vertices to the left of the dashed line belong to $X$, and the others belong to $Y$.
The time complexity of Dijkstra’s algorithm is $\Theta (n^2)$.
Below is the implementation of the pseudocode in Python(Adjacency matrix):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

def Dijkstra(length):
    n = len(length)
    X = [0]
    Y = [i for i in range(1,n)]
    Lambda = [0 for i in range(0,n)]

    for y in range(1,n):
        if length[0][y]:
            Lambda[y] = length[0][y]
        else:
            Lambda[y] = float("inf")
        
    for j in range(0, n-1):
        min_value = float("inf")
        y_min = -1
        
        # Linear scan through Y to find vertex with smallest distance estimate
        for vertex in Y:
            if Lambda[vertex] < min_value:
                min_value = Lambda[vertex]
                y_min = vertex

        # If no valid vertex found, remaining vertices are unreachable
        if y_min == -1:
            break

        X.append(y_min)
        Y.remove(y_min)

        for w in range(n):
            if w in Y and length[y_min][w] != 0:
                if Lambda[y_min] + length[y_min][w] < Lambda[w]:
                    Lambda[w] = Lambda[y_min] + length[y_min][w]
    
    return Lambda

Minimum Cost Spanning Trees

Let $G = (V, E)$ be a connected undirected graph with weights on its edges. A spanning tree $(V, T)$ of $G$ is a subgraph of $G$ that is a tree. If $G$ is weighted and the sum of the weights of the edges in $T$ is minimum, then $(V, T)$ is called a minimum cost spanning tree or simply a minimum spanning tree.

Kruskal’s Algorithm

\begin{algorithm}\caption{KRUSKAL}\begin{algorithmic}\Input A weighted connected undirected graph $G = (V,E)$ with $n$ vertices.\Output The set of edges $T$ of a minimum cost spanning tree for $G$.\State Sort the edges in $E$ by nondecreasing weight.\For{each vertex $v \in V$}    \State \Call{makeset}{$\{v\}$}\EndFor\State $T \gets \{\}$\While{$|T| < n - 1$}    \State Let $(x, y)$ be the next edge in $E$.    \If{\Call{find}{$x$} $\neq$ \Call{find}{$y$}}        \State \Call{add}{$x, y$} to $T$        \State \Call{union}{$x, y$}    \EndIf\EndWhile\end{algorithmic}\end{algorithm}

Prim’s Algorithm

\begin{algorithm}\caption{PRIM}\begin{algorithmic}\Input A weighted connected undirected graph $G = (V, E)$, where $V = \{1, 2, \dots, n\}$.\Output The set of edges $T$ of a minimum cost spanning tree for $G$.\State $T \gets \{\}; \quad X \gets \{1\}; \quad Y \gets V - \{1\}$\For{$y \gets 2$ \To $n$}    \If{$y$ is adjacent to $1$}        \State $N[y] \gets 1$        \State $C[y] \gets c[1, y]$    \Else        \State $C[y] \gets \infty$    \EndIf\EndFor\For{$j \gets 1$ \To $n - 1$} \Comment{find $n - 1$ edges}    \State Let $y \in Y$ be such that $C[y]$ is minimum    \State $T \gets T \cup \{(y, N[y])\}$ \Comment{add edge $(y, N[y])$ to $T$}    \State $X \gets X \cup \{y\}$ \Comment{add vertex $y$ to $X$}    \State $Y \gets Y - \{y\}$ \Comment{delete vertex $y$ from $Y$}    \For{each vertex $w \in Y$ that is adjacent to $y$}        \If{$c[y, w] < C[w]$}            \State $N[w] \gets y$            \State $C[w] \gets c[y, w]$        \EndIf    \EndFor\EndFor\end{algorithmic}\end{algorithm}

Huffman tree

\begin{algorithm}\caption{HUFFMAN}\begin{algorithmic}\Input A set $C = \{c_1, c_2, \dots, c_n\}$ of $n$ characters and their frequencies $\{f(c_1), f(c_2), \dots, f(c_n)\}$.\Output A Huffman tree $(V, T)$ for $C$.\State Insert all characters into a min-heap $H$ according to their frequencies.\State $V \gets C; \quad T \gets \{\}$\For{$j \gets 1$ \To $n - 1$}    \State $c \gets $\Call{deletemin}{H}    \State $c' \gets $\Call{deletemin}{H}    \State $f(v) \gets f(c) + f(c')$ \Comment{$v$ is a new node}    \State \Call{insert}{$H, v$}    \State $V \gets V \cup \{v\}$ \Comment{Add $v$ to $V$}    \State $T \gets T \cup \{(v, c), (v, c')\}$ \Comment{Make $c$ and $c'$ children of $v$ in $T$}\EndFor\end{algorithmic}\end{algorithm}

The time complexity of the algorithm is $O(n \log n)$.

Graph Traversal

Depth-First Search

\begin{algorithm}\caption{DFS}\begin{algorithmic}\Input A (directed or undirected) graph $G = (V, E)$.\Output Preordering and postordering of the vertices in the corresponding depth-first search tree.\State $predfn \gets 0; \quad postdfn \gets 0$\For{each vertex $v \in V$}    \State mark $v$ unvisited\EndFor\For{each vertex $v \in V$}    \If{$v$ is marked unvisited}        \State \Call{dfs}{$v$}    \EndIf\EndFor\Procedure{dfs}{$v$}    \State mark $v$ visited    \State $predfn \gets predfn + 1$    \For{each edge $(v, w) \in E$}        \If{$w$ is marked unvisited}            \State \Call{dfs}{$w$}        \EndIf    \EndFor    \State $postdfn \gets postdfn + 1$\EndProcedure\end{algorithmic}\end{algorithm}

Breadth-First Search

\begin{algorithm}\caption{BFS}\begin{algorithmic}\Input A (directed or undirected) graph $G = (V, E)$.\Output Numbering of the vertices in breadth-first search order.\State $bfn \gets 0$\For{each vertex $v \in V$}    \State mark $v$ unvisited\EndFor\For{each vertex $v \in V$}    \If{$v$ is marked unvisited}        \State \Call{bfs}{$v$}    \EndIf\EndFor\Procedure{bfs}{$v$}    \State $Q \gets \{v\}$    \State mark $v$ visited    \While{$Q \neq \{\}$}        \State $v \gets$ \Call{Pop}{$Q$}        \State $bfn \gets bfn + 1$        \For{each edge $(v, w) \in E$}            \If{$w$ is marked unvisited}                \State \Call{Push}{$w, Q$}                \State mark $w$ visited            \EndIf        \EndFor    \EndWhile\EndProcedure\end{algorithmic}\end{algorithm}

Finding Articulation Points in a Graph

A vertex $v$ in an undirected graph $G$ with more than two vertices is called an articulation point if there exist two vertices $u$ and $w$ different from $v$ such that any path between $u$ and $w$ must pass through $v$. Thus, if $G$ is connected, the removal of $v$ and its incident edges will result in a disconnected subgraph of $G$. A graph is called biconnected if it is connected and has no articulation points.
To find the set of articulation points, we perform a depth-first search traversal on $G$. During the traversal, we maintain two labels with each vertex $v \in V$: $\alpha[v]$ and $\beta[v]$. $\alpha[v]$ is simply predfn in the depth-first search algorithm, which is incremented at each call to the depth-first search procedure. $\beta[v]$ is initialized to $\alpha[v]$, but may change later on during the traversal. For each vertex $v$ visited, we let $\beta[v]$ be the minimum of the following:

• $\alpha[v]$.
• $\alpha[u]$ for each vertex $u$ such that $(v, u)$ is a back edge.
• $\beta[w]$ for each edge $(v, w)$ in the depth-first search tree.
  The articulation points are determined as follows:
• The root is an articulation point if and only if it has two or more children in the depth-first search tree.
• A vertex $v$ other than the root is an articulation point if and only if $v$ has a child $w$ with $\beta[w] \geq \alpha[v]$.
  The formal algorithm for finding the articulation points is given as Algorithm ARTICPOINTS.

\begin{algorithm}\caption{ARTICPOINTS}\begin{algorithmic}\Input A connected undirected graph $G = (V, E)$.\Output Array $A[1..count]$ containing the articulation points of $G$, if any.\State Let $s$ be the start vertex.\For{each vertex $v \in V$}    \State mark $v$ unvisited\EndFor\State $predfn \gets 0; \quad count \gets 0; \quad rootdegree \gets 0$\State \Call{dfs}{$s$}\Procedure{dfs}{$v$}    \State mark $v$ visited; $artpoint \gets \textbf{false}$; $predfn \gets predfn + 1$    \State $\alpha[v] \gets predfn; \quad \beta[v] \gets predfn$ \Comment{Initialize $\alpha[v]$ and $\beta[v]$}    \For{each edge $(v, w) \in E$}        \If{$(v, w)$ is a tree edge}            \State \Call{dfs}{$w$}            \If{$v = s$}                \State $rootdegree \gets rootdegree + 1$                \If{$rootdegree = 2$}                    \State $artpoint \gets \textbf{true}$                \EndIf            \Else                \State $\beta[v] \gets \min\{\beta[v], \beta[w]\}$                \If{$\beta[w] \geq \alpha[v]$}                    \State $artpoint \gets \textbf{true}$                \EndIf            \EndIf        \ElsIf{$(v, w)$ is a back edge}            \State $\beta[v] \gets \min\{\beta[v], \alpha[w]\}$        \Else            \State \textbf{do nothing} \Comment{$w$ is the parent of $v$}        \EndIf    \EndFor    \If{$artpoint$}        \State $count \gets count + 1$        \State $A[count] \gets v$    \EndIf\EndProcedure\end{algorithmic}\end{algorithm}

We illustrate the action of Algorithm ARTICPOINTS by finding the articulation points of the graph shown in the figure. See the figure. Each vertex $v$ in the depth-first search tree is labeled with $\alpha[v]$ and $\beta[v]$. The depth-first search starts at vertex $a$ and proceeds to vertex $e$. A back edge $(e, c)$ is discovered, and hence $\beta[e]$ is assigned the value $\alpha[c] = 3$. Now, when the search backs up to vertex $d$, $\beta[d]$ is assigned $\beta[e] = 3$. Similarly, when the search backs up to vertex $c$, its label $\beta[c]$ is assigned the value $\beta[d] = 3$. Now, since $\beta[d] \geq \alpha[c]$, vertex $c$ is marked as an articulation point. When the search backs up to $b$, it is also found that $\beta[c] \geq \alpha[b]$, and hence $b$ is also marked as an articulation point. At vertex $b$, the search branches to a new vertex $f$ and proceeds, as illustrated in the figure, until it reaches vertex $j$. The back edge $(j, h)$ is detected and hence $\beta[j]$ is set to $\alpha[h] = 8$. Now, as described before, the search backs up to $i$ and then $h$ and sets $\beta[i]$ and $\beta[h]$ to $\beta[j] = 8$. Again, since $\beta[i] \geq \alpha[h]$, vertex $h$ is marked as an articulation point. For the same reason, vertex $g$ is marked as an articulation point. At vertex $g$, the back edge $(g, a)$ is detected, and hence $\beta[g]$ is set to $\alpha[a] = 1$. Finally, $\beta[f]$ and then $\beta[b]$ are set to $1$ and the search terminates at the start vertex. The root $a$ is not an articulation point since it has only one child in the depth-first search tree.
Below is the implementation of the pseudocode in Python(Adjacency list):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

graph = {
    'a': ['b','g'],
    'b': ['a','c','f'],
    'c': ['b','d','e'],
    'd': ['c','e'],
    'e': ['c','d'],
    'f': ['b','g'],
    'g': ['a','f','h'],
    'h': ['g','i','j'],
    'i': ['h','j'],
    'j': ['h','i']
}

V = list(graph.keys())

is_visit = {v: False for v in V}

predfn = 0
count = 0
rootdegree = 0

alpha = {v: 0 for v in V}
beta = {v: 0 for v in V}

# 节点的父节点，判断是否为“回边”
parent = {v: None for v in V}

root = V[0]  # 转换为列表再取第一个元素

A = []

def dfs(v):
    global predfn,count, rootdegree
    is_visit[v] = True
    artpoint = False
    predfn = predfn + 1

    alpha[v] = predfn
    beta[v] = predfn

    for w in graph[v]:
        if not is_visit[w]:
            # 标注为树边
            parent[w] = v
            dfs(w)

            if v == root:
                rootdegree += 1
                if rootdegree == 2:
                    artpoint = True
            else:
                beta[v] = min(beta[v],beta[w])
                if beta[w] >= alpha[v]:
                    artpoint = True
        
        elif w != parent[v] and is_visit[w]:
            beta[v] = min(beta[v],alpha[w])

        else:
            pass # do nothing
        

    if artpoint:
        count += 1
        A.append(v)

dfs(root)
print(A)

Induction

Sort#2 Radix Sort

\begin{algorithm}\caption{RADIXSORT}\begin{algorithmic}\Input A linked list of numbers $L = \{a_1 , a_2, . . . , a_n\}$ and $k$, the number of digits.\Output $L$ sorted in nondecreasing order.\For{$j \gets 1$ \To $k$}         \State Prepare 10 empty lists $L_0,L_1,\cdots,L_9$        \While{$L$ is \Not empty}            \State $a \gets$ next element in $L$.Delect $a$ from $L$.            \State $i \gets j$th digit in $a$.Append $a$ to list $L_i$.            \EndWhile            \State $L \gets L_0$            \For{$i \gets 1$ \To 9}            \State $L \gets L,L_i$ \Comment{append list $L_i$ to $L$}            \EndFor        \EndFor        \Return $L$    \end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

def RadixSort(arr, k):
    for j in range(k):
        # Prepare 10 empty lists L_0, L_1, ..., L_9
        lists = [[] for _ in range(10)]
        
        # Distribute elements into the corresponding lists based on the j-th digit
        while arr:
            a = arr.pop(0)  # Delect a from L
            digit = (a // (10 ** j)) % 10  # Get the j-th digit
            lists[digit].append(a)  # Append a to list L_i
        
        # Concatenate all lists back into arr
        arr = []
        for sublist in lists:
            arr.extend(sublist)
    
    return arr

Integer Exponentiation

An efficient method can be deduced as follows. Let $m = \lfloor n/2 \rfloor$, and suppose we know how to compute $x^m$. Then, we have two cases: If $n$ is even, then $x^n = (x^m)^2$; otherwise, $x^n = x \cdot (x^m)^2$.

\begin{algorithm}\caption{EXPREC}\begin{algorithmic}\Input A real number $x$ and a nonnegative integet $n$.\Output $x^n$\State power($x$,$n$)\Procedure{power}{$x,m$} \Comment{Compute $x^{m}$}\If{$m$ = 0}\State $y \gets 1$\Else\State $y \gets \text{power}(x, \lfloor m/2 \rfloor)$\State $y \gets y^{2}$\If{$m$ is odd}\State $y \gets x \cdot y$\EndIf\EndIf\Return $y$\EndProcedure\end{algorithmic}\end{algorithm}

\begin{algorithm}\caption{EXP}\begin{algorithmic}\Input A real number $x$ and a nonnegative integet $n$.\Output $x^n$\STATE $y \gets 1$\STATE Let $n$ be $d_k d_{k-1} \dots d_0$ in binary notation.\FOR{$j \gets k$ \DOWNTO $0$}    \STATE $y \gets y^{2}$    \IF{$d_j = 1$}        \STATE $y \gets x \cdot y$    \ENDIF\ENDFOR\Return $y$\end{algorithmic}\end{algorithm}

Below is the implementation of these pseudocodes in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

def exprec(x,n):
    if n == 0:
        return 1
    else:
        y = exprec(x, n // 2)
        y = y * y
        if n % 2 == 1:
            y = x * y
        return y

def exp(x,n):
    y = 1
    binary_n = bin(n)[2:]
    for bit in binary_n:   
        y = y * y
        if bit == '1':
            y = x * y
    return y

Evaluating Polynomials (Horner’s Rule)

Suppose we have a sequence of $n+1$ real numbers $a_0, a_1, \ldots, a_n$ and a real number $x$, and we want to evaluate the polynomial

$$P_n(x) = a_n x^n + a_{n-1} x^{n-1} + \cdots + a_1 x + a_0.$$

The straightforward approach would be to evaluate each term separately. This approach is very inefficient since it requires $n + n - 1 + \cdots + 1 = n(n+1)/2$ multiplications. A much faster method can be derived by induction as follows. We observe that

$$\begin{aligned} P_n(x) &= a_n x^n + a_{n-1} x^{n-1} + \cdots + a_1 x + a_0 \\ &= ((\cdots (((a_n x + a_{n-1})x + a_{n-2})x + a_{n-3})\cdots)x + a_1)x + a_0. \end{aligned}$$

This evaluation scheme is known as Horner’s rule. Use of this scheme leads to the following more efficient method. Suppose we know how to evaluate

$$P_{n-1}(x) = a_n x^{n-1} + a_{n-1} x^{n-2} + \cdots + a_2 x + a_1.$$

Then, using one more multiplication and one more addition, we have

$$P_n(x) = x P_{n-1}(x) + a_0.$$

This implies Algorithm HORNER.

\begin{algorithm}\caption{HORNER}\begin{algorithmic}\Input A sequence of $n+1$ real numbers $a_0, a_1, \dots, a_n$ and a real number $x$.\Output $P_n(x) = a_n x^n + a_{n-1} x^{n-1} + \dots + a_1 x + a_0$.\State $p \gets a_n$\For{$j \gets 1$ \to $n$}    \State $p \gets x \cdot p + a_{n-j}$\EndFor\Return $p$\end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8

def horner(A, x):
    n = len(A)
    p = A[n - 1]

    for j in range(0, n - 1):
        p = x * p + A[n - j - 2]
        
    return p

or

1
2
3
4
5
6
7
8

def horner(A, x):
    n = len(A)
    p = A[-1]
    
    for j in range(1, n):
        p = x * p + A[-j-1]

    return p

Generating Permutations

In this section, we study the problem of generating all permutations of the numbers $1, 2,\dots, n$.

The first algorithm

The algorithm uses a recursive backtracking approach to generate all permutations of an array by systematically swapping elements at different positions and exploring each arrangement through depth-first traversal. Note that interchanging back is necessary to restore the original state of the array for proper backtracking, ensuring all permutations are generated correctly without interference between recursive branches.

\begin{algorithm}\caption{Permutations1}\begin{algorithmic}\Input A positive integer $n$.\Output All permutations of the numbers $1,2,\cdots,n$.\For{$j \gets 1$ \to $n$}\State $P[j] \gets j$\EndFor\State \Call{perm1}{$1$}\Procedure{perm1}{m}\If{$m = n$}\State \textbf{output} $P[1 \cdots n]$\Else\For{$j \gets m$ \to $n$}\State interchange $P[j]$ and $P[m]$\State \Call{perm1}{$m+1$}\State interchange $P[j]$ and $P[m]$\EndFor\EndIf\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

def perm1(m):
    if m == n:
        print(P)
    else:
        for j in range(m, n):
            P[j], P[m] = P[m], P[j]
            perm1(m + 1)
            P[j], P[m] = P[m], P[j]

n = 3
P = [0] * n
for i in range(n):
    P[i] = i+1
perm1(0)

Time complexity: $\Theta(n \cdot n!)$.

The Second Algorithm

The algorithm uses backtracking to generate all permutations of the numbers 1 to n by recursively placing each number from n down to 1 in every available position.

\begin{algorithm}\caption{Permutations2}\begin{algorithmic}\Input A positive integer $n$.\Output All permutations of the numbers $1,2,\cdots,n$.\For{$j \gets 1$ \to $n$}\State $P[j] \gets 0$\EndFor\State \Call{perm2}{$n$}\Procedure{perm2}{m}\If{$m = 0$}\State \textbf{output} $P[1 \cdots n]$\Else\For{$j \gets 1$ \to $n$}\If{$P[j] = 0$}\State $P[j] \gets m$\State \Call{perm2}{$m-1$}\State $P[j] \gets 0$\EndIf\EndFor\EndIf\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

n = 3
P = [0] * n

def perm2(m):
    if m == 0:
        print(P)
    else:
        for j in range(n):
            if P[j] == 0:
                P[j] = m
                perm2(m - 1)
                P[j] = 0

perm2(n)

Finding the Majority Element

Let $A[1..n]$ be a sequence of integers. An integer a in $A$ is called the majority if it appears more than $\lfloor n/2 \rfloor$ times in $A$.
Observation: If two different elements in the original sequence are removed, then the majority in the original sequence remains the majority in the new sequence.
This observation suggests the following procedure for finding an element that is a candidate for being the majority.

\begin{algorithm}\caption{MAJORITY}\begin{algorithmic}\Input An array $A[1 \dots n]$ of $n$ elements.\Output The majority element if it exists; otherwise none.\State $c \gets$ \Call{candidate}{$1$}\State $count \gets 0$\For{$j \gets 1$ \to $n$}\If{$A[j] = c$}\State $count \gets count + 1$\EndIf\EndFor\If{$count > \lfloor n/2 \rfloor$}\Return $c$\Else\Return none\EndIf\Procedure{candidate}{$m$}\State $j \gets m$\State $c \gets A[m]$\State $\textit{count} \gets 1$\While{$j < n$ \and $\textit{count} > 0$}    \State $j \gets j + 1$    \If{$A[j] = c$}        \State $\textit{count} \gets \textit{count} + 1$    \Else        \State $\textit{count} \gets \textit{count} - 1$    \EndIf\EndWhile\If{$j = n$}    \Return $c$\Else    \Return \Call{candidate}{$j + 1$}\EndIf\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

def candidate(m):
    j = m
    c = A[m]
    count = 1
    while j < n - 1 and count > 0:
        j += 1
        if A[j] == c:
            count += 1
        else:
            count -= 1
    if j == n - 1:
        return c
    else:
        return candidate(j + 1)

def majority(A):
    n = len(A)
    c = candidate(0)
    print("Candidate:", c)
    count = 0
    for j in range(n):
        if A[j] == c:
            count += 1
    if count > n // 2:
        return c
    else:
        return None

Divide and Conquer

Binary Search

The binary search algorithm is a classic example of the divide-and-conquer paradigm. It works on a sorted array by repeatedly dividing the search interval in half. If the value of the search key is less than the item in the middle of the interval, narrow the interval to the lower half. Otherwise, narrow it to the upper half. Repeatedly check until the value is found or the interval is empty.

\begin{algorithm}\caption{BINARYSEARCHREC}\begin{algorithmic}\Input An array $A[1..n]$ of $n$ elements sorted in nondecreasing order and an element $x$.\Output $j$ if $x=A[j],1\leq j\leq n$, and $0$ otherwise.\State binarysearch($1$, $n$)\Procedure{binarysearch}{$low, high$}    \If{$low > high$}        \Return $0$    \Else        \State $mid \gets \lfloor (low + high) / 2 \rfloor$        \If{$x = A[mid]$}            \Return $mid$        \ElsIf{$x < A[mid]$}            \Return \Call{binarysearch}{$low$, $mid-1$}        \Else            \Return \Call{binarysearch}{$mid+1$, $high$}        \EndIf    \EndIf\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11

def binarysearch(low, high):
    if low > high:
        return 0
    else:
        mid = (low + high) // 2
        if x == A[mid]:
            return mid
        elif x < A[mid]:
            return binarysearch(low, mid - 1)
        else:
            return binarysearch(mid + 1, high)

Sort#3 Mergesort

\begin{algorithm}\caption{MERGESORT}\begin{algorithmic}\Input An array $A[1 \dots n]$ of $n$ elements.\Output $A[1 \dots n]$ sorted in nondecreasing order.\State \Call{mergesort}{$A$,$1$,$n$}\Procedure{mergesort}{$A$,$low$,$high$}\If{$low < high$}\State $mid \gets \lfloor (low + high) / 2 \rfloor$\State \Call{mergesort}{$A$,$low$,$mid$}\State \Call{mergesort}{$A$,$mid+1$,$high$}\State \Call{merge}{$A$,$low$,$mid$,$high$}\EndIf\EndProcedure\end{algorithmic}\end{algorithm}

The MERGE algorithm is discussed in Algo#1 Basic Concepts in Algorithmic Analysis | TheValuePoint’s Blog.

\begin{algorithm}\caption{MERGE}\begin{algorithmic}\Input An array $A[1..m]$ of elements and three indices $p,q$, and $r$, with $1 \leq p \leq q < r \leq m$, such that both the subarrays $A[p..q]$ and $A[q+1..r]$ are sorted individually in nondecreasing order.\Output $A[p..r]$ contains the result of merging the two subarrays $A[p..q]$ and $A[q+1..r]$.\State \textbf{comment:} $B[p..r]$ is an auxiliary array.\State $s \gets p$; $t \gets q+1$; $k \gets p$\While{$s \leq q$ and $t \leq r$}    \If{$A[s] \leq A[t]$}        \State $B[k] \gets A[s]$        \State $s \gets s + 1$    \Else        \State $B[k] \gets A[t]$        \State $t \gets t + 1$    \EndIf    \State $k \gets k + 1$\EndWhile\If{$s = q + 1$}    \State $B[k..r] \gets A[t..r]$\Else    \State $B[k..r] \gets A[s..q]$\EndIf\State $A[p..r] \gets B[p..r]$\end{algorithmic}\end{algorithm}

Below is the implementation of the MERGESORT pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

def merge(arr,p,q,r):
    B = [0] * (r - p + 1)
    s = p
    t = q + 1
    k = p
    while s <= q and t <= r:
        if arr[s] <= arr[t]:
            B[k - p] = arr[s]
            s += 1
        else:
            B[k - p] = arr[t]
            t += 1
        k += 1

    if s == q+1:
        while t <= r:
            B[k - p] = arr[t]
            t += 1
            k += 1
    else:
        while s <= q:
            B[k - p] = arr[s]
            s += 1
            k += 1
    
    for i in range(r - p + 1):
        arr[p + i] = B[i]
    return arr

def mergesort(arr, low, high):
    if low < high:
        mid = (low + high) // 2
        mergesort(arr, low, mid)
        mergesort(arr, mid + 1, high)
        merge(arr, low, mid, high)

Compared to the bottom-up sorting approach, merge sort is top-down.

\begin{algorithm}\caption{BOTTOMUPSORT}\begin{algorithmic}\Input  An array $A[1..n]$ of $n$ elements.\Output $A[1..n]$ sorted in nondecreasing order.\State $t \gets 1$\While{$t < n$}\State $s \gets t; \quad t \gets 2s; \quad i \gets 0$\While{$i+t \leq n$}\State MERGE($A$, $i + 1$, $i + s$, $i + t$)\State $i \gets i+t$    \EndWhile    \If{$i+s < n$}    \State MERGE($A$, $i + 1$, $i + s$, n)    \EndIf    \EndWhile    \end{algorithmic}\end{algorithm}

Selection: Finding the Median and the $k$th Smallest Element

\begin{algorithm}\caption{SELECT}\begin{algorithmic}\Input An array $A[1..n]$ of $n$ elements and an integer $k$, $1 \leq k \leq n$.\Output The $k$th smallest element in $A$.\Procedure{select}{$A,k$}    \State $n \gets |A|$    \If{$n < 44$}        \State sort $A$        \Return $A[k]$    \EndIf    \State $q \gets \lfloor n/5 \rfloor$    \State Divide $A$ into $q$ groups of $5$ elements each. If $5$ does not divide $n$, discard the remaining elements.    \State Sort each of the $q$ groups individually and extract its median. Let $M$ be the set of medians.    \State $mm \gets \text{select}(M,\lceil q/2 \rceil)$ \Comment{median of medians}    \State Partition $A$ into three arrays:    \State $A_1 = \{ a \mid a < mm \}$    \State $A_2 = \{ a \mid a = mm \}$    \State $A_3 = \{ a \mid a > mm \}$    \If{$|A_1| \geq k$}        \Return $\text{select}(A_1,k)$    \ElsIf{$|A_1| + |A_2| \geq k$}        \Return $mm$    \Else        \Return $\text{select}(A_3,k - |A_1| - |A_2|)$    \EndIf\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

def select(A,low,high,k):
    p = high - low + 1
    if p < 44:
        sub = A[low:high+1]
        sub.sort()
        return sub[k]
    
    q = p // 5

    M = []

    for i in range(q):
        temp = []
        for j in range(5):
            temp.append(A[low + i*5 + j])
        temp.sort()
        M.append(temp[2])

    M.sort()
    mm = select(M, 0, len(M)-1, (q+1)//2 - 1)

    A_1 = []
    A_2 = []
    A_3 = []

    for i in range(low,high+1):
        if A[i] < mm:
            A_1.append(A[i])
        elif A[i] > mm:
            A_3.append(A[i])
        else:
            A_2.append(A[i])

    if k < len(A_1):
        return select(A_1,0,len(A_1)-1,k)
    elif k < len(A_1) + len(A_2):
        return mm
    else:
        return select(A_3,0,len(A_3)-1,k - len(A_1) - len(A_2))

Analysis of the selection algorithm

Assuming $A_1'$ is the set of elements less than the median of medians and $A_3'$ is the set of elements greater than the median of medians, we have

$$|A_1'| \geq 3 \lceil \lfloor n/5 \rfloor /2 \rceil \geq \frac{3}{2}\lfloor n/5 \rfloor \qquad \text{and} \qquad |A_3'| \leq n - \frac{3}{2}\lfloor n/5 \rfloor \leq n-\frac{3}{2} \left(\frac{n-4}{5}\right) = 0.7n+1.2$$

also

$$|A_3'| \geq \frac{3}{2}\lfloor n/5 \rfloor \qquad \text{and} \qquad |A_1'| \leq 0.7n+1.2$$

Now we are in a position to estimate $T(n)$, the running time of the algorithm. Steps 1 and 2 of procedure select in the algorithm cost $\Theta(1)$ time each. Step 3 costs $\Theta(n)$ time. Step 4 costs $\Theta(n)$ time, as sorting each group costs a constant amount of time. In fact, sorting each group costs no more than seven comparisons. The cost of Step 5 is $T(\lfloor n/5\rfloor)$. Step 6 takes $\Theta(n)$ time. By the above analysis, the cost of Step 7 is at most $T(0.7n+1.2)$. Now we wish to express this ratio in terms of the floor function and get rid of the constant $1.2$. For this purpose, let us assume that $0.7n+1.2\leq\lfloor 0.75n\rfloor$. Then, this inequality will be satisfied if $0.7n+1.2\leq 0.75n-1$, i.e., if $n\geq 44$. This is why we have set the threshold in the algorithm to $44$. We conclude that the cost of this step is at most $T(\lfloor 0.75n\rfloor)$ for $n\geq 44$. This analysis implies the following recurrence for the running time of Algorithm select.

$$T(n)\leq\begin{cases}c&\text{if }n<44,\\ T(\lfloor n/5\rfloor)+T(\lfloor 3n/4\rfloor)+cn&\text{if }n\geq 44,\end{cases}$$

for some constant $c$ that is sufficiently large. Since $(1/5)+(3/4)<1$, it follows by Theorem 1.5, that the solution to this recurrence is $T(n)=\Theta(n)$. In fact, by Example 1.39,

$$T(n)\leq\frac{cn}{1-1/5-3/4}=20cn.$$

Note that each ratio $>0.7n$ results in a different threshold. For instance, choosing $0.7n+1.2\leq\lfloor 0.71n\rfloor$ results in a threshold of about $220$. The following theorem summarizes the main result of this section.
The time complexity of Algorithm select is $\Theta(n)$.

Sort#4 Quicksort

A partitioning algorithm

\begin{algorithm}\caption{split}\begin{algorithmic}\Input An array of elements $A[low..high]$.\Output (1) $A$ with its elements rearranged as described; (2) $w$, the new position of the splitting element $A[low]$.    \State $i \gets low$    \State $x \gets A[low]$    \For{$j \gets low+1$ \textbf{to} $high$}        \If{$A[j] \leq x$}            \State $i \gets i + 1$            \If{$i \neq j$}                \State interchange $A[i]$ and $A[j]$            \EndIf        \EndIf    \EndFor    \State interchange $A[low]$ and $A[i]$    \State $w \gets i$    \Return $A$ and $w$\end{algorithmic}\end{algorithm}

The sorting algorithm

\begin{algorithm}\caption{QUICKSORT}\begin{algorithmic}\Input An array $A[1..n]$ of $n$ elements.\Output $A[1..n]$ sorted in nondecreasing order.\State \Call{quicksort}{$A$,$1$,$n$}\Procedure{quicksort}{$A$,$low$,$high$}\If{$low < high$}    \State \Call{split}{$A[low..high]$,$w$} \Comment{$w$ is the new position of $A[low]$}    \State \Call{quicksort}{$A$,$low$,$w-1$}    \State \Call{quicksort}{$A$,$w+1$,$high$}\EndIf\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of these pseudocodes in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

def split(A, low, high):
    i = low
    x = A[low]

    for j in range(low + 1, high + 1):
        if A[j] <= x:
            i += 1
            if i != j:
                A[i], A[j] = A[j], A[i]

    A[low], A[i] = A[i], A[low]
    w = i
    return (A,w)

def quicksort(A, low, high):
    if low < high:
        A, w = split(A, low, high)
        A = quicksort(A, low, w - 1)
        A = quicksort(A, w + 1, high)
    return A

In the average case, the time complexity of quicksort is $O(n \log n)$, while in the worst case, it can degrade to $\Theta(n^2)$. Although no auxiliary storage is required by the algorithm to store the array elements, the space required by the algorithm is $O(n)$.

The Closest Pair Problem

\begin{algorithm}\caption{CLOSESTPAIR}\begin{algorithmic}\Input A set $S$ of $n$ points in the plane.\Output The minimum separation realized by two points in $S$.\State Sort the points in $S$ in nondecreasing order of their $x$-coordinates.\State $(\delta, Y) \gets $ \Call{cp}{1, n} \Return $\delta$\Procedure{cp}{$low, high$}\If{$high - low + 1 \leq 3$}    \State compute $\delta$ by a straightforward method.    \State Let $Y$ contain the points in nondecreasing order of $y$-coordinates.\Else    \State $mid \gets \lfloor (low + high)/2 \rfloor$    \State $x_0 \gets x(S[mid])$    \State $(\delta_l, Y_l) \gets $ \Call{cp}{low, mid}    \State $(\delta_r, Y_r) \gets $ \Call{cp}{mid + 1, high}    \State $\delta \gets \min\{\delta_l, \delta_r\}$    \State $Y \gets$ Merge $Y_l$ with $Y_r$ in nondecreasing order of $y$-coordinates.    \State $k \gets 0$    \For{$i \gets 1$ \To $|Y|$} \Comment{Extract $T$ from $Y$}        \If{$|x(Y[i]) - x_0| \leq \delta$}            \State $k \gets k + 1$            \State $T[k] \gets Y[i]$        \EndIf    \EndFor \Comment{$k$ is the size of $T$}    \State $\delta' \gets 2\delta$ \Comment{Initialize $\delta'$ to any number greater than $\delta$}    \For{$i \gets 1$ \To $k - 1$} \Comment{Compute $\delta'$}        \For{$j \gets i + 1$ \To $\min\{i + 7, k\}$}            \If{$d(T[i], T[j]) < \delta'$}                \State $\delta' \gets d(T[i], T[j])$            \EndIf        \EndFor    \EndFor    \State $\delta \gets \min\{\delta, \delta'\}$\EndIf\Return $(\delta, Y)$\EndProcedure\end{algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

import math

def distance(p1: tuple, p2: tuple):
    return math.sqrt((p1[0] - p2[0]) ** 2 + (p1[1] - p2[1]) ** 2)

def brute_force(S):
    min_dist = float('inf')
    for i in range(len(S)):
        for j in range(i + 1, len(S)):
            min_dist = min(min_dist, distance(S[i], S[j]))
    return min_dist

def cp(S, left, right):
    if right - left <= 3:
        return brute_force(S)

    mid = (left + right) // 2
    x_0 = S[mid][0]

    delta_l = cp(S, left, mid)
    delta_r = cp(S, mid + 1, right)
    delta = min(delta_l, delta_r)

    k = 0
    T = []

    for i in range(0, len(S)):
        if abs(Y[i][0] - x_0) < delta:
            k += 1
            T.append(Y[i])

    delta_prime = 2 * delta

    for i in range(0, k-1):
        for j in range (i+1,min(i+7,k)):
            if distance(T[i], T[j]) < delta_prime:
                delta_prime = distance(T[i], T[j])

    delta_prime = min(delta, delta_prime)
    return delta_prime

# S = np.random.rand(200, 2) * 10
S = sorted(S, key=lambda x: x[0])
Y = sorted(S, key=lambda x: x[1])
delta = cp(S, 0, len(S) - 1)

Dynamic Programming

The Longest Common Subsequence Problem

A simple problem that illustrates the principle of dynamic programming is the following. Given two strings $A$ and $B$ of lengths $n$ and $m$, respectively, over an alphabet $\Sigma$, determine the length of the longest subsequence that is common to both $A$ and $B$. Here, a subsequence of $A = a_1a_2,\ldots,a_n$ is a string of the form $a_{i_1}a_{i_2},\ldots,a_{i_k}$, where each $i_j$ is between 1 and $n$ and $1 \leq i_1 < i_2 < \cdots < i_k \leq n$. For example, if $\Sigma = \{x,y,z\}, A = zxyxyz$, and $B = xyyzx$, then $xyy$ is a subsequence of length 3 of both $A$ and $B$. However, it is not the longest common subsequence of $A$ and $B$, since the string $xyyz$ is also a common subsequence of length 4 of both $A$ and $B$. Since these two strings do not have a common subsequence of length greater than 4, the length of the longest common subsequence of $A$ and $B$ is 4.
State Transition Equation:

$$L[i, j] = \begin{cases} 0 & \text{if } i = 0 \ \text{or } j = 0\\ L[i - 1, j - 1] + 1 & \text{if } a_i = b_j, \\ \max(L[i - 1, j], L[i, j - 1]) & \text{if } a_i \neq b_j. \end{cases}$$

\begin{algorithm}\caption{LCS}\begin{algorithmic}\Input  Two strings $A$ and $B$ of lengths $n$ and $m$, respectively, over an alphabet $\Sigma$.\Output The length of a longest common subsequence of $A$ and $B$.\State $L[0, 0] \gets 0$\For{$i \gets 1$ \To $n$}    \State $L[i, 0] \gets 0$\EndFor\For{$j \gets 1$ \To $m$}    \State $L[0, j] \gets 0$\EndFor\For{$i \gets 1$ \To $n$}    \For{$j \gets 1$ \To $m$}        \If{$a_i = b_j$}            \State $L[i, j] \gets L[i - 1, j - 1] + 1$        \Else            \State $L[i, j] \gets \max(L[i - 1, j], L[i, j - 1])$        \EndIf    \EndFor\EndFor\Return $L[n, m]$\End{Algorithmic}\end{algorithm}

Below is the implementation of the pseudocode in Python:

1
2
3
4
5
6
7
8
9
10
11
12

def LCS(A, B):
    n = len(A)
    m = len(B)
    L = [[0] * (m + 1) for _ in range(n + 1)] # 注意下面代码的越界问题
    
    for i in range(1, n + 1):
        for j in range(1, m + 1):
            if A[i - 1] == B[j - 1]:
                L[i][j] = L[i - 1][j - 1] + 1
            else:
                L[i][j] = max(L[i - 1][j], L[i][j - 1])
    return L[n][m]

Question Variant: The Longest Common Continuous Subsequence Problem

The longest common continuous subsequence problem, often referred to as the longest common substring problem, modifies the definition of a subsequence to require continuity. Given two strings $A$ and $B$ of lengths $n$ and $m$, respectively, over an alphabet $\Sigma$, the goal is to determine the length of the longest substring that appears consecutively in both $A$ and $B$. Here, a substring of $A = a_1a_2,\ldots,a_n$ is a string of the form $a_i a_{i+1} \ldots a_j$ where $1 \leq i \leq j \leq n$. For example, using the same alphabet $\Sigma = \{x,y,z\}$ with $A = zxyxyz$ and $B = xyyzx$, the substring $xy$ appears consecutively in both strings, as does the single character $y$, but the common substring $xyyz$ from the subsequence example is no longer valid because its characters are not contiguous in both strings. In this case, the longest common substring is $xy$ with length 2. The problem illustrates a different dynamic programming approach, where the continuity constraint leads to a recurrence that resets the count when characters do not match, distinguishing it from the standard longest common subsequence problem.
State Transition Equation:

$$L[i, j] = \begin{cases} L[i - 1, j - 1] + 1 & \text{if } a_{i-1} = b_{j-1}, \\ 0 & \text{if } a_{i-1} \neq b_{j-1}. \end{cases}$$