# Sequential sampling from Gaussian conditional not working

Data Science Asked by Jacob Holm on September 28, 2020

I’m trying to sequentially sample from a Gaussian Process prior.

The problem is that the samples eventually converge to zero or diverge to infinity.

I’m using the basic conditionals described e.g. here

Note: the kernel(X,X) function returns the squared exponential kernel with isometric noise.

Here is my code:

n = 32

x_grid = np.linspace(-5,5,n)

x_all = []
y_all = []
for x in x_grid:
x_all = [x] + x_all
X = np.array(x_all).reshape(-1, 1)
# Mean and covariance of the prior
mu = np.zeros((X.shape), np.float)
cov = kernel(X, X)
if len(mu)==1: # first sample is not conditional
y = np.random.randn()*cov + mu
else:
# condition on all previous samples
u1 = mu
u2 = mu[1:]
y2 = np.atleast_2d(np.array(y_all)).T
C11 = cov[:1,:1] # dependent sample
C12 = np.atleast_2d(cov[0,1:])
C21 = np.atleast_2d(cov[1:,0]).T
C22 = np.atleast_2d(cov[1:, 1:])
C22_ = la.inv(C22)
u = u1 + np.dot(C12, np.dot(C22_, (y2 - u2)))
C22_xC21 = np.dot(C22_, C21)
C_minus = np.dot(C12, C22_xC21) # this weirdly becomes larger than C!
C = C11 - C_minus
y = u + np.random.randn()*C
y_all = [y.flatten()] + y_all


Here’s an example with 32 samples, where it collapses:

enter image description here

Here’s an example with 34 samples, where it explodes:

enter image description here

(for this particular kernel, 34 is the number of samples at which (or more) the samples start to diverge.

I’ve gone through this code so many times that I’m going blind – something must be fundamentally wrong with it but I just can’t see it.

## Related Questions

### Semantic networks: word2vec?

1  Asked on April 29, 2021

### Python to clean miswritten words with repetitive letters such as “wwwwooorrrrddss” to “words”

1  Asked on April 29, 2021 by pythoner

### One-Hot Encoded Matrix Inupt/Ouput for Autoencoder

0  Asked on April 29, 2021 by aslconwnb

### Data Lineage/Traceability in Pipelines

0  Asked on April 28, 2021 by gustav1985

### How to find ‘pre-requisite’ relationships between sentences

0  Asked on April 28, 2021 by talismanic

### Fast AI Lesson 4 – MNIST. Confused about multiplying weights by pixels?

2  Asked on April 28, 2021

### Should kernel size always be a prime number?

1  Asked on April 28, 2021

### Dealing with missing data

1  Asked on April 28, 2021

### How to apply dataset balancing techniques whilst using Pipeline in Sklearn?

1  Asked on April 28, 2021

### Learning Python for Data Science

2  Asked on April 28, 2021 by pythonguy

### Dealing with unseen data/categories in machine learning models for stream data

1  Asked on April 28, 2021 by maeaex1

### How to get summary statistics in Orange?

2  Asked on April 28, 2021 by john-tarr

### Dataset image size and inference speed

1  Asked on April 28, 2021

### Comparing 2 columns from separate dataframes and copy some row values from one df to another if column value matches in pandas

1  Asked on April 28, 2021 by asif17

### What does the phrase ‘underlying mapping’ mean?

1  Asked on April 27, 2021 by feona

### Hindsight experience replay: strategy for sampling goals

1  Asked on April 27, 2021 by nsalas

### Explanation of why Neural Networks are non convex

1  Asked on April 27, 2021

### how to see decision tree when running in anaconda?

2  Asked on April 27, 2021

### Resampling train and test data in R

2  Asked on April 27, 2021 by znoris007

### How do Ioffe & Szegedy obtain the equation $frac{partial text{BN}((aW)u)}{partial u} = frac{partialtext{BN}(Wu)}{u}$?

0  Asked on April 27, 2021