# Pruning input layers and output layers after training model with shared layers

Data Science Asked by Howard Wang on July 31, 2020

I’m trying to train a search and item encoder and this is the model I have

input_search = Input(shape=(40,), dtype='int64', name='input_search')
input_title = Input(shape=(40,), dtype='int64', name='input_title')
input_desc = Input(shape=(40,), dtype='int64', name='input_desc')
input_brand = Input(shape=(40,), dtype='int64', name='input_brand')

embedding = Embedding(input_dim=20000, output_dim=50, input_length=40)
s_emb = embedding(input_search)
t_emb = embedding(input_title)
d_emb = embedding(input_desc)
b_emb = embedding(input_brand)

s = GlobalMaxPool1D()(s_emb)
t = GlobalMaxPool1D()(t_emb)
d = GlobalMaxPool1D()(d_emb)
b = GlobalMaxPool1D()(b_emb)

concat = concatenate([t, d, b])
concat = Dense(128)(concat)
s = Dense(128, name='vec')(s)
similarity = Lambda(cos_sim)([s, concat])

model = Model(inputs=[input_search, input_desc, input_brand, input_title], outputs=similarity)

_______________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_search (InputLayer)       [(None, 40)]         0
__________________________________________________________________________________________________
input_title (InputLayer)        [(None, 40)]         0
__________________________________________________________________________________________________
input_desc (InputLayer)         [(None, 40)]         0
__________________________________________________________________________________________________
input_brand (InputLayer)        [(None, 40)]         0
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 40, 50)       1526100     input_search[0][0]
input_title[0][0]
input_desc[0][0]
input_brand[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_5 (GlobalM (None, 50)           0           embedding_1[1][0]
__________________________________________________________________________________________________
global_max_pooling1d_6 (GlobalM (None, 50)           0           embedding_1[2][0]
__________________________________________________________________________________________________
global_max_pooling1d_7 (GlobalM (None, 50)           0           embedding_1[3][0]
__________________________________________________________________________________________________
global_max_pooling1d_4 (GlobalM (None, 50)           0           embedding_1[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate)     (None, 150)          0           global_max_pooling1d_5[0][0]
global_max_pooling1d_6[0][0]
global_max_pooling1d_7[0][0]
__________________________________________________________________________________________________
search (Dense)                  (None, 128)          6528        global_max_pooling1d_4[0][0]
__________________________________________________________________________________________________
product (Dense)                 (None, 128)          19328       concatenate_5[0][0]
__________________________________________________________________________________________________
lambda (Lambda)                 (None,)              0           search[0][0]
product[0][0]
==================================================================================================
Total params: 1,551,956
Trainable params: 1,551,956
Non-trainable params: 0
__________________________________________________________________________________________________


Items have 3 features – title, description, and brand. I want to use the same embedding layer between all 4 inputs to generate embeddings in the same vector space. However, for prediction time, I would like to remove the 3 inputs for the item, and make the output of the model the embedding vector from the dense layer named ‘vec’ to store the feature vectors. This is what I have tried:

search_model = Model(model.inputs[0], model.layers[-2].output)
item_model = Model(inputs=model.inputs[1:], outputs=model.layers[-1].output)


but I get the error

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_brand_1:0", shape=(None, 40), dtype=int64) at layer "input_brand". The following previous layers were accessed without issue: []


Is there a way for me to share the embedding layer weights between the search and the item features that would allow me to prune certain layers come inference time? Or perhaps create 2 separate embedding layers, where 1 is run through the search and the other is run through the 3 features of the item but somehow keep the weights in both layers the same?

## Related Questions

### Target encoding with KFold cross-validation – how to transform test set?

1  Asked on January 15, 2021

### Sklearn: applying cost complexity pruning along with pipeline

2  Asked on January 15, 2021 by codeman340

### String Values in a data frame in Pandas

4  Asked on January 15, 2021

### What model is recommended: I am using text features in a regression and want to interpret coefficients

1  Asked on January 14, 2021 by thewhitetie

### Are there libraries similar to cleverhans for different types of classifiers?

1  Asked on January 14, 2021

### Look up a number inside a list within a pandas cell, and return corresponding string value from a second DF

1  Asked on January 14, 2021 by donum

### How to create model for selecting a set of categories with a set of attributes?

1  Asked on January 14, 2021

### CycleGAN: Generator losses don’t decrease, discriminators get perfect

1  Asked on January 14, 2021 by marie-m

### Is my data appropriate for Hierarchical Clustering?

0  Asked on January 14, 2021

### What has to be considered when creating a radar chart?

1  Asked on January 14, 2021

### Sudden drop in loss at epoch start, then gradual raise

0  Asked on January 14, 2021 by coobit

### Cross-entropy loss explanation

5  Asked on January 14, 2021

### Can we calculate AUC for deep learning based regression task

1  Asked on January 14, 2021

### PySpark: How do I specify dropna axis in PySpark transformation?

2  Asked on January 14, 2021 by horbaje

### how can i plot probability distribution of my classes in the way below?

0  Asked on January 14, 2021

### Building a machine learning model to predict crop yields based on environmental data

3  Asked on January 13, 2021 by user308827

### product recommendation of a single product based on customer similarity?

0  Asked on January 13, 2021 by corianne1234

### Pearson Product Moment Correlation vs Cosine Similarity For Encoded Text Comparison

0  Asked on January 13, 2021

### Cosine similarity with arrays contaning NaN

3  Asked on January 13, 2021 by user641597