Convex Sets and Functions

Suppose $f$ is convex. For $\lambda\in(0,1)$, Equation (1) rearranges to $\frac{f(x+\lambda(y-x)) -f(x)}{\lambda}\le f(y)-f(x)$, and letting $\lambda\to 0^+$ the left side tends to the directional derivative $\ip{\nabla f(x)}{y-x}$, giving the tangent inequality. Conversely, if the tangent inequality holds, apply it at the point $z=\lambda x+(1-\lambda)y$ to both $x$ and $y$, $f(x)\ge f(z)+\ip{\nabla f(z)}{x-z}$ and $f(y)\ge f(z)+\ip{\nabla f(z)}{y-z}$, and take the $\lambda,(1-\lambda)$ weighted sum, whose gradient terms cancel because $\lambda(x-z)+(1-\lambda)(y-z)=0$, leaving $\lambda f(x)+(1-\lambda)f(y)\ge f(z)$. For the Hessian, the tangent inequality along the line $x+sv$ is equivalent to the one-variable function $g(s)=f(x+sv)$ being convex for every direction $v$, which for $g\in C^2$ holds if and only if $g''(s)=\ip{v}{\nabla^2 f(x+sv)v}\ge 0$, that is the Hessian is positive semidefinite.

Let $p$ be the projection of $z$ onto $C$, the unique nearest point, which exists because $C$ is closed and convex and $\R^n$ is complete. Set $a=z-p\neq 0$. By the variational inequality $\ip{z-p}{x-p}\le 0$ for all $x\in C$, we have $\ip ax \le\ip ap$ for all $x\in C$. Moreover $\ip az-\ip ap=\ip a{z-p}=\norm a^2>0$, so $\ip ap<\ip az$. Choosing $c$ strictly between $\ip ap$ and $\ip az$ separates, since $\ip ax\le\ip ap<c<\ip az$ for all $x\in C$.

Pick an interior point $w\in\operatorname{int}C$ and set $z_k=x_0+\tfrac1k(x_0-w)$, which converge to $x_0$ and lie outside the closure $\overline C$. Indeed if $z_k\in\overline C$, then $x_0$ is a proper convex combination of the interior point $w$ and the point $z_k\in\overline C$, namely $x_0=\tfrac1{k+1}w +\tfrac k{k+1}z_k$, which places $x_0$ in $\operatorname{int}C$ because the open segment from an interior point to any point of the closure lies in the interior, contradicting $x_0$ being on the boundary. That principle holds because, picking $r>0$ with $B(w,r)\subseteq C$ and $y'\in C$ near $z_k$, convexity gives $\lambda B(w,r)+(1-\lambda)y'=B(\lambda w+(1-\lambda)y',\lambda r)\subseteq C$ for $\lambda\in(0,1]$, and $\norm{y'-z_k}$ small enough keeps $\lambda w+(1-\lambda)z_k$ inside this ball. Each $z_k$ gives by Theorem 3, applied to the closed convex $\overline C$ (the closure of a convex set is convex, since for $x_n\to x$, $y_n\to y$ in $C$ the combination $\lambda x_n+(1-\lambda)y_n\in C$ converges to $\lambda x+(1-\lambda)y\in\overline C$), a unit vector $a_k$ with $\ip{a_k}{x}\le\ip{a_k}{z_k}$ for all $x\in\overline C$. The unit vectors lie in the compact unit sphere, so by the Bolzano-Weierstrass theorem a subsequence converges to a unit vector $a$. Passing to the limit in $\ip{a_k}{x}\le\ip{a_k}{z_k}$, with $z_k\to x_0$, gives $\ip a{x}\le\ip a{x_0}$ for all $x\in C$, the supporting hyperplane.

Let $x^\ast$ be a local minimum, so $f(x^\ast)\le f(x)$ for $x$ near $x^\ast$, and suppose some $y$ had $f(y)<f(x^\ast)$. Along the segment, convexity gives $f(\lambda y+(1-\lambda)x^\ast)\le\lambda f(y)+(1- \lambda)f(x^\ast)<f(x^\ast)$ for every $\lambda\in(0,1]$, and taking $\lambda$ small enough that the point lies in the neighbourhood contradicts local minimality. So $f(x^\ast)\le f(y)$ for all $y$, a global minimum. The minimisers form the sublevel set $\{f\le f(x^\ast)\}$ at the minimum value, which is convex because $f$ is. If $f$ is strictly convex and $x_1\neq x_2$ both minimised, the midpoint would have $f(\tfrac12 x_1+\tfrac12 x_2)<\tfrac12 f(x_1)+\tfrac12 f(x_2)=f(x^\ast)$, below the minimum, which is impossible, so the minimiser is unique.