LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-16-2019, 12:53 PM   #1
John-Marie
LQ Newbie
 
Registered: May 2018
Posts: 18

Rep: Reputation: Disabled
ERROR when executing a python script


Hi,
I tried to execute a python script but I don't know how to initialize the self variable in python:
This is my code:
Code:
import tensorflow as tf
import numpy as np
import readers
import pre_precessing
from app_flag import FLAGS
from StackedAutoencoder import StackedAutoencoder


def write_and_encode(data_list, tfrecord_filename):
    writer = tf.python_io.TFRecordWriter(tfrecord_filename)
    for label, data_matrix in data_list:
        example = tf.train.Example(features=tf.train.Features(
            feature={
                "label": tf.train.Feature(int64_list=tf.train.Int64List(value=[label])),
                "data_raw": tf.train.Feature(bytes_list=tf.train.BytesList(value=[data_matrix.tostring()]))
            }
        ))
        writer.write(example.SerializeToString())

    writer.close()


def read_and_decode(tfrecord_filename):
    reader = tf.TFRecordReader()
    filename_queue = tf.train.string_input_producer([tfrecord_filename],)
    _, serialized_example = reader.read(filename_queue)
    feature = tf.parse_single_example(serialized_example,
                                      features={
                                          "label": tf.FixedLenFeature([], tf.int64),
                                          "data_raw": tf.FixedLenFeature([], tf.string)
                                      })
    data = tf.decode_raw(feature["data_raw"], tf.float64)
    data = tf.reshape(data, [FLAGS.image_rows, FLAGS.image_cols])
    return data, feature["label"]



def train_input_fn():

    tfrecord_file = "../resources/train_tfrecord"  
    dataset = tf.data.TFRecordDataset(tfrecord_file)
    dataset = dataset.map(parser)

    train_dataset = dataset.repeat(FLAGS.num_epochs).batch(FLAGS.batch_size)
    train_iterator = train_dataset.make_one_shot_iterator()

    features, labels = train_iterator.get_next()

    return features, labels


def parser(record_line):

    features = {
        "label": tf.FixedLenFeature([], tf.int64),
        "data_raw": tf.FixedLenFeature([], tf.string)
    }
    parsed = tf.parse_single_example(record_line, features=features)
    label = tf.cast(parsed["label"], tf.int32) - 1  
    data = tf.decode_raw(parsed["data_raw"], tf.float64)
    data = tf.reshape(data, [FLAGS.image_rows, FLAGS.image_cols])
    data = tf.cast(data, tf.float32)
    return data, label




def write_user_instances_to_tfrecord():
    
    users = ["0"+str(i) for i in range(1, 10)]
    users.extend([str(i) for i in range(10, 17)])
    users.extend(["32", "40", "41", "42", "43", "49", "50", "51"])

   
    instances = []
    for user in users:
        train_data = readers.read_user_files(user)
        for label, instance in train_data.items():
            instances.append((label, instance))

 
    formalized_instances = pre_precessing.extend_to_maxsize(instances)

   
    train_instances = formalized_instances[:100]
    write_and_encode(train_instances, "../resources/train_tfrecord")

 

def main():
    build_stacked_ae("../resources/train_tfrecord")

def build_stacked_ae(path):
    """
    Build the stacked auto-encoder neural network, and evaluate its performance
    :param path: Path to the genetic dataset
    :return: Accuracy of classification of cell cycle phase.
    """
    ############### Stacked Auto-Encoders ##############
    train=train_input_fn()
    ae = StackedAutoencoder(train,5)
    ae.create_autoencoder()
    result = ae.evaluate_autoencoder()
    return result[1] * 100
    print("Accuracy: %.2f%%" % (result[1] * 100))

if __name__ == "__main__":
    main()
The Stacked autoencoder function:
Code:
from __future__ import print_function
import keras
import numpy
from keras.models import Sequential
from keras.layers.core import *
from sklearn.model_selection import train_test_split


class StackedAutoencoder(object):
    """
    Implementation of stacked autoencoder multi-class classifier using the Keras Python package.
    This classifier is used to classify cells to cell cycle phases S, G1 or G2M.
    """
    def __init__(self, features, labels, num_labels):
        self.features = features
        self.labels = labels
        self.auto_encoder = None
        self.encoding_dim = num_labels
        

        # fix random seed for reproducibility
        self.seed = 7
        numpy.random.seed(7)

    def create_autoencoder(self):
        """
        Build the stacked auto-encoder using multiple hidden layers.
        The stacked auto-encoder is then trained and weights are freezed afterwards.
        A softmax classification layer is that appended to the last layer, replacing the input
        re-constructed layer of the auto-encoder.
        :return: Compiled classification neural network model.
        """
        self.auto_encoder = Sequential()
        self.auto_encoder.add(Dense(3000, activation='relu', input_dim=self.features.shape[1]))
        self.auto_encoder.add(Dense(1000, activation='relu'))
        self.auto_encoder.add(Dense(30, activation='relu'))

        self.auto_encoder.add(Dense(3000, activation='relu'))
        self.auto_encoder.add(Dense(self.features.shape[1], activation='sigmoid'))

        self.auto_encoder.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
        self.auto_encoder.fit(self.features, self.features,
                              epochs=10,
                              batch_size=5,
                              shuffle=True,
                              validation_split=0.33,
                              validation_data=None)

        self.auto_encoder.layers.pop()
        self.auto_encoder.add(Dense(self.encoding_dim, activation='softmax'))
        self.auto_encoder.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
        print(self.auto_encoder.summary())

        # Freeze all weights after training the stacked auto-encoder and all the classification layer
        for i in range(0, len(self.auto_encoder.layers)-1):
            self.auto_encoder.layers[i].trainable = False

        return self.auto_encoder

    def evaluate_autoencoder(self):
        """
        Fit the trained neural network and validate it using splitting the dataset to training and testing sets.
        :return: Accuracy score of the classification.
        """
        self.auto_encoder.fit(self.features, self.labels,
                              epochs=10,
                              batch_size=5,
                              shuffle=True)

        X_train, X_test, Y_train, Y_test = train_test_split(self.features, self.labels, test_size=0.33, random_state=self.seed)
        #predictions = self.auto_encoder.predict_classes(X_test)
        #print(predictions)
        #print(self.label_encoder.inverse_transform(predictions))
        score = self.auto_encoder.evaluate(X_test, Y_test, batch_size=5, verbose=1)
        return score
The error is:

Using TensorFlow backend.
Traceback (most recent call last):
File "Classifier.py", line 110, in <module>
main()
File "Classifier.py", line 91, in main
build_stacked_ae("../resources/train_tfrecord")
File "Classifier.py", line 101, in build_stacked_ae
ae = StackedAutoencoder(train,5)
TypeError: __init__() takes exactly 4 arguments (3 given)
 
Old 02-16-2019, 02:45 PM   #2
Corvette
Member
 
Registered: Jul 2017
Location: Missouri, United States
Distribution: Debian 9
Posts: 93

Rep: Reputation: 23
I am not a Python person, but is looks to me like you are passing two parameters - the "train" tuple and "5" -to the constructor/initialization function of StackedAutoencoder. You need to pass 3 - features, labels, num_labels, in addition to the implicit self variable.

Last edited by Corvette; 02-16-2019 at 02:49 PM.
 
Old 02-16-2019, 03:30 PM   #3
John-Marie
LQ Newbie
 
Registered: May 2018
Posts: 18

Original Poster
Rep: Reputation: Disabled
I passed 3 variables: features, labels, num_labels but I don't know how to initialize the self variable.
 
Old 02-16-2019, 04:34 PM   #4
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: Slackware
Posts: 8,724

Rep: Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731
Quote:
Originally Posted by John-Marie View Post
I passed 3 variables: features, labels, num_labels
Uh, no you didn't.

Quote:
Code:
ae = StackedAutoencoder(train,5)
You passed two variables.

Last edited by dugan; 02-17-2019 at 10:16 AM.
 
Old 02-17-2019, 04:24 AM   #5
John-Marie
LQ Newbie
 
Registered: May 2018
Posts: 18

Original Poster
Rep: Reputation: Disabled
No, I entred the ouput of train which are features and labels and as you can see TypeError: __init__() takes exactly 4 arguments (3 given) so I give 3 arguments my problem is how to initialize the self argument
 
Old 02-17-2019, 04:32 AM   #6
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 12,249

Rep: Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788
no, you don't initialize self. It is initialized when you created an instance of that class. self will point to that instance. self is a very first argument passed to know which is the actual instance, but you do not need to take care about it. It is done automatically. So you need actually pass 3 parameters to the constructor. If 3 given that means you passed 2.
 
1 members found this post helpful.
Old 02-17-2019, 04:38 AM   #7
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,680

Rep: Reputation: 510Reputation: 510Reputation: 510Reputation: 510Reputation: 510Reputation: 510
Quote:
Originally Posted by John-Marie View Post
... so I give 3 arguments ...
No, you did not, and
Quote:
my problem is how to initialize the self argument
you do not initialize 'self'. This is done internally by python. You did pass, in fact, two arguments. The output says you passed 3 because 'self' was also passed automatically. Here is some further info:

https://www.programiz.com/article/python-self-why

edit: pan64 was a bit faster.

Last edited by crts; 02-17-2019 at 04:39 AM.
 
Old 02-17-2019, 05:16 AM   #8
John-Marie
LQ Newbie
 
Registered: May 2018
Posts: 18

Original Poster
Rep: Reputation: Disabled
Thanks for your clarification, the problem is solved

Last edited by John-Marie; 02-17-2019 at 05:24 AM.
 
Old 02-17-2019, 05:25 AM   #9
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 12,249

Rep: Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788Reputation: 3788
probably use () : (features,labels)
 
Old 02-17-2019, 10:12 AM   #10
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: Slackware
Posts: 8,724

Rep: Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731Reputation: 3731
Quote:
Originally Posted by John-Marie View Post
No, I entred the ouput of train which are features and labels
That would have looked like this:

Code:
ae = StackedAutoencoder(*train(),5)
When you did this?

Quote:
Code:
ae = StackedAutoencoder(train,5)
You passed the train function (equivalent to passing its function pointer in C), and num_labels. That's two variables.

Last edited by dugan; 02-17-2019 at 10:17 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Crontab not executing my shell script its executing standlaone atulkvtiwari Linux - Newbie 15 12-04-2018 06:25 AM
Problems executing perl script with inputs when executing under other perl scripts TheStr3ak5 Programming 4 04-22-2017 04:07 AM
I got error while installing python-tk python-psycopg2 python-twisted saili kadam Linux - Newbie 1 09-05-2015 03:03 AM
[SOLVED] Help executing shell commands in a Python script zippy4251 Programming 7 09-25-2012 11:20 AM
LXer: Python Python Python (aka Python 3) LXer Syndicated Linux News 0 08-05-2009 08:30 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:07 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration