Warning: This MADlib method is still in early stage development. Interface and implementation are subject to change.

Contents

This utility function loads model architectures and weights into a table for use by deep learning algorithms. Model architecture is in JSON form and model weights are in the form of PostgreSQL binary data types (bytea). If the output table already exists, a new row is inserted into the table so it can act as a repository for multiple model architectures and weights.

There is also a utility function to delete a model from the table.

Load Model

load_keras_model(
    keras_model_arch_table,
    model_arch,
    model_weights,
    name,
    description
    )

Arguments

keras_model_arch_table

VARCHAR. Output table to load keras model architecture and weights.

model_arch

JSON. JSON of the model architecture to load.

model_weights (optional)

bytea. Model weights to load as a PostgreSQL binary data type.

name (optional)

TEXT, default: NULL. Free text string to provide a name, if desired.

description (optional)

TEXT, default: NULL. Free text string to provide a description, if desired.

Output table
The output table contains the following columns:

model_id	SERIAL PRIMARY KEY. Model ID.
model_arch	JSON. JSON blob of the model architecture.
model_weights	bytea. Weights of the model which may be used for warm start or transfer learning. Weights are stored as a PostgreSQL binary data type.
__internal_madlib_id__	TEXT. Unique id for model arch. This is an id used internally be MADlib.

Delete Model

delete_keras_model(
    keras_model_arch_table
    model_id
)

Arguments

keras_model_arch_table

VARCHAR. Table containing model architectures and weights.

model_id

INTEGER. The id of the model to be deleted.

Examples

Define model architecture. Use Keras to define the model architecture:

import keras
from keras.models import Sequential
from keras.layers import Dense
model_simple = Sequential()
model_simple.add(Dense(10, activation='relu', input_shape=(4,)))
model_simple.add(Dense(10, activation='relu'))
model_simple.add(Dense(3, activation='softmax'))
model_simple.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
dense_1 (Dense)              (None, 10)                50
_________________________________________________________________
dense_2 (Dense)              (None, 10)                110
_________________________________________________________________
dense_3 (Dense)              (None, 3)                 33
=================================================================
Total params: 193
Trainable params: 193
Non-trainable params: 0

Export the model to JSON:

model_simple.to_json()

'{"class_name": "Sequential", "keras_version": "2.1.6", "config": [{"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_1", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "dtype": "float32", "activation": "relu", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 10, "batch_input_shape": [null, 4], "use_bias": true, "activity_regularizer": null}}, {"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_2", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "activation": "relu", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 10, "use_bias": true, "activity_regularizer": null}}, {"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_3", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "activation": "softmax", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 3, "use_bias": true, "activity_regularizer": null}}], "backend": "tensorflow"}'

Load into model architecture table:

DROP TABLE IF EXISTS model_arch_library;
SELECT madlib.load_keras_model('model_arch_library',  -- Output table,
$$
{"class_name": "Sequential", "keras_version": "2.1.6", "config": [{"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_1", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "dtype": "float32", "activation": "relu", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 10, "batch_input_shape": [null, 4], "use_bias": true, "activity_regularizer": null}}, {"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_2", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "activation": "relu", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 10, "use_bias": true, "activity_regularizer": null}}, {"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_3", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "activation": "softmax", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 3, "use_bias": true, "activity_regularizer": null}}], "backend": "tensorflow"}
$$
::json,  -- JSON blob
                               NULL,                  -- Weights
                               'Sophie',              -- Name
                               'A simple model'       -- Descr
);
SELECT COUNT(*) FROM model_arch_library;

 count
-------+
     1

Load another model architecture:

SELECT madlib.load_keras_model('model_arch_library',  -- Output table,
$$
{"class_name": "Sequential", "keras_version": "2.1.6", "config": [{"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_1", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "dtype": "float32", "activation": "relu", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 10, "batch_input_shape": [null, 4], "use_bias": true, "activity_regularizer": null}}, {"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_2", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "activation": "relu", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 10, "use_bias": true, "activity_regularizer": null}}, {"class_name": "Dense", "config": {"kernel_initializer": {"class_name": "VarianceScaling", "config": {"distribution": "uniform", "scale": 1.0, "seed": null, "mode": "fan_avg"}}, "name": "dense_3", "kernel_constraint": null, "bias_regularizer": null, "bias_constraint": null, "activation": "softmax", "trainable": true, "kernel_regularizer": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "units": 3, "use_bias": true, "activity_regularizer": null}}], "backend": "tensorflow"}
$$
::json,  -- JSON blob
                               NULL,                  -- Weights
                               'Maria',               -- Name
                               'Also a simple model'  -- Descr
);
SELECT COUNT(*) FROM model_arch_library;

 count
-------+
     2

Load model weights. To load weights from previous MADlib run, use UPDATE to load directly into the table. For example, if 'model_data' are the weights in the output table 'iris_model' from a previous run of 'madlib_keras_fit()' :

UPDATE model_arch_library SET model_weights = model_data FROM iris_model WHERE model_id = 2;

To load weights from Keras using a PL/Python function, we need to flatten then serialize the weights to store as a PostgreSQL binary data type. Byte format is more efficient on space and memory compared to a numeric array. The model weights will be de-serialized when passed to Keras functions.

CREATE OR REPLACE FUNCTION load_weights() RETURNS VOID AS
$$
from keras.layers import *
from keras import Sequential
import numpy as np
import plpy
#
# create model
model = Sequential()
model.add(Dense(10, activation='relu', input_shape=(4,)))
model.add(Dense(10, activation='relu'))
model.add(Dense(3, activation='softmax'))
#
# get weights, flatten and serialize
weights = model.get_weights()
weights_flat = [ w.flatten() for w in weights ]
weights1d = np.array([j for sub in weights_flat for j in sub])
weights_bytea = weights1d.tostring()
#
# load query
load_query = plpy.prepare("""SELECT madlib.load_keras_model(
                        'model_arch_library',
                        $1, $2)
                    """, ['json','bytea'])
plpy.execute(load_query, [model.to_json(), weights_bytea])
$$ language plpythonu;
-- Call load function
SELECT load_weights();
-- Check weights loaded OK
SELECT COUNT(*) FROM model_arch_library WHERE model_weights IS NOT NULL;

 count
-------+
     1

Load weights from Keras using psycopg2. (Psycopg is a PostgreSQL database adapter for the Python programming language.) As above we need to flatten then serialize the weights to store as a PostgreSQL binary data type.

import psycopg2
import psycopg2 as p2
conn = p2.connect('postgresql://gpadmin@35.239.240.26:5432/madlib')
cur = conn.cursor()
from keras.layers import *
from keras import Sequential
import numpy as np
#
# create model
model = Sequential()
model.add(Dense(10, activation='relu', input_shape=(4,)))
model.add(Dense(10, activation='relu'))
model.add(Dense(3, activation='softmax'))
#
# get weights, flatten and serialize
weights = model.get_weights()
weights_flat = [ w.flatten() for w in weights ]
weights1d = np.array([j for sub in weights_flat for j in sub])
weights_bytea = psycopg2.Binary(weights1d.tostring())
#
# load query
query = "SELECT madlib.load_keras_model('model_arch_library', %s,%s)"
cur.execute(query,[model.to_json(),weights_bytea])
conn.commit()

From SQL check if weights loaded OK:

SELECT COUNT(*) FROM model_arch_library WHERE model_weights IS NOT NULL;

 count
-------+
     2

Delete one of the models:

SELECT madlib.delete_keras_model('model_arch_library',   -- Output table
                                  1                      -- Model id
                                );
SELECT COUNT(*) FROM model_arch_library;

 count
-------+
     3

Related Topics

See keras_model_arch_table.sql_in