Reproducing Kernel Hilbert Space (RKHS)

A very good tutorial of RKHS:

http://sadeepj.blogspot.com/2012/11/tutorial-introduction-to-reproducing.htmlhttp://www.sadeepj.blogspot.com.au/2013/09/a-tutorial-introduction-to-reproducing.htmlhttp://www.sadeepj.blogspot.com.au/2014/05/a-tutorial-introduction-to-reproducing.html

Terminology:

• Vector space - A set endowed with two special operations: addition and scalar multiplication.
• Normed vector space - A vector space with length of vectors defined.
• Metric space - A set (nor necessarily a vector space) where the distance between two points in the set is defined. A normed vector space is a metric space. But the converse is not true.
• Banach space - A complete normed vector space.
• Inner product space - An inner product space, (V,⟨.,.⟩) is a vector space V over a field F with a binary operator ⟨.,.⟩:V×V→F, known as the inner product, that satisfies the following axioms for all vectors x,y,z∈V and all scalars a∈F.

1. Conjugate symmetry: ⟨x,y⟩=⟨y,x⟩*
2. Linearity in the first argument: ⟨ax,y⟩=a⟨x,y⟩ and ⟨x+z,y⟩=⟨x,y⟩+⟨z,y⟩
3. Positive definiteness:  ⟨x,x⟩≥0 where the equality holds only when x=0.

• An inner product space is also known as a pre-Hilbert space

Hilbert space:

• Hilbert space - An inner product space which is complete with respect to the norm induced by its inner product. 

Reproducing kernel:

• Let X be a nonempty set and H⊆CX be a Hilbert space of complex-valued functions defined on X. A kernel k:X×X→C is called a reproducing kernel of H if it has the two properties:
  1.     For every x0∈X, k(y,x0) as a function of y belongs to H.
  
  
  2.     The reproducing property: for each x0∈X and f∈H, f(x0)=⟨f(.),k(.,x0)⟩H

Reproducing Kernel Hilbert Space (RKHS):

• Reproducing Kernel Hilbert Space (RKHS) - A Hilbert space H of complex-valued functions on a nonempty set X is called a reproducing kernel Hilbert space if the point evaluation functional Lx is a bounded (equivalently, continuous) linear operator for all x∈X. 

• Point evaluation functional - The point evaluation functional Lx:H→C at a point x∈X is defined as Lx(f):=f(x).

• Linear operator (Linear map) - Let f:V→W be a function between two vector spaces V,W over the same field F. f is called a linear operator if the following two conditions are satisfied for all x,y∈V and a∈F.

1. f(u+v)=f(u)+f(v)

2. f(au)=af(u)