TransWikia.com

How to correctly organize Tensorflow model layers?

Stack Overflow Asked on January 1, 2022

Hellow world!
We are writing our own AI and we struggle to create correct model layer.
What we have to input in our neural network is a list that contains n lists with m tuples

e.x. list = numpy.array([ [[1,2,4],[5,6,8]] , [[5,6,0],[7,2,4]] ])

What we expect to get as a result is either 0 or 1 ( it makes sense trust me )

This is what we have for now:

tpl = 3 # because we have tuples
nl = 2 # number of lists we have
model = tf.keras.Sequential([
# this should be entry layer that understands our list
            tf.keras.layers.Dense(nl * tpl , input_shape=(nl, tpl), activation='relu'),

#hidden layers..
            tf.keras.layers.Dense(64, input_shape=(nl, tpl), activation='sigmoid'),

#our output layer with 2 nodes that one should contain 0, other 1, because we have 2 labels ( 0 and 1 )
            tf.keras.layers.Dense(2, input_shape=(0, 1), activation='softmax')
        ])

But we get the error below:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     58     ctx.ensure_initialized()
     59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60                                         inputs, attrs, num_outputs)
     61   except core._NotOkStatusException as e:
     62     if name is not None:

InvalidArgumentError:  Incompatible shapes: [56,2,2] vs. [56,1]
     [[node huber_loss/Sub (defined at <ipython-input-25-08eb2e0b395e>:53) ]] [Op:__inference_train_function_45699]

Function call stack:
train_function

If we summarize our model, it gives, the following structure:

Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)             (None, 2, 6)              24        
_________________________________________________________________
dense_2 (Dense)             (None, 2, 64)             448       
_________________________________________________________________
dense_3 (Dense)             (None, 2, 2)              130       
=================================================================

Finally,

What we have understood is that our provided data is not compatible with the last layer, so how do we convert the last layer to => shape (None, 2) or what is the right way to work this error out?

One Answer

You can use Flatten() or GlobalAveragePooling1D before your output layer. Full example:

import numpy
import tensorflow as tf

list = numpy.array([[[1., 2., 4.], [5., 6., 8.]], [[5., 6., 0.], [7., 2., 4.]]])

tpl = 3  
nl = 2   
model = tf.keras.Sequential([
    tf.keras.layers.Dense(nl * tpl, input_shape=(nl, tpl), activation='relu'),
    tf.keras.layers.Dense(64, input_shape=(nl, tpl), activation='sigmoid'),
    tf.keras.layers.GlobalAveragePooling1D(),
    tf.keras.layers.Dense(2, input_shape=(0, 1), activation='softmax')
])

model.build(input_shape=(nl, tpl))

model(list)
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[0.41599566, 0.58400434],
       [0.41397247, 0.58602756]], dtype=float32)>

You won't get only 0 and 1 though, you'll get probabilities per class. Also you should shadow the built in keyword list.

Model: "sequential_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_12 (Dense)             (None, 2, 6)              24        
_________________________________________________________________
dense_13 (Dense)             (None, 2, 64)             448       
_________________________________________________________________
global_average_pooling1d (Gl (None, 64)                0         
_________________________________________________________________
dense_14 (Dense)             (None, 2)                 130       
=================================================================
Total params: 602
Trainable params: 602
Non-trainable params: 0
_________________________________________________________________

Answered by Nicolas Gervais on January 1, 2022

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP